PyPA mechanism for making "official recommendations"

Perhaps, but then, build backend is by far the easiest one here. Almost by definition you are a more advanced developer at this point, and should be quite satisfied with a decision tree that filters out “need native extensions”, “needs to generate/relocate files on build” and then tells everyone else to choose one of the remaining 4-5 options based on style.

It basically boils down to Project Summaries - Python Packaging User Guide but arranged by category and linked more prominently. I don’t have any problems with letting the backends self-promote in this kind of context - they’re the most motivated to commend themselves to potential users.

But I don’t think it’s a good idea for this ad-hoc group of contributors to vote on “who wins”, which is what any “official” recommendation from PyPA would boil down to. Listing the known backends that all fundamentally work the same is better - or possibly a feature matrix to make comparisons easier.

1 Like

I am worried this would lead to the same issue we have right now of some misguided and ill-informed advice rising quickly to the top for the wrong reasons (example the typical sys.path or PYTHONPATH shenanigans) and then stay at the top forever. Or I am not understanding how it would be different than what we already have now (StackOverflow, and so on). Would that be somewhat moderated / curated by PyPA someone in the know (maybe the owners of said project)?

To repeat, there is no “PyPA” in the sense of a group that can moderate like this. Anyone can create such a project, and moderate or curate it as they wish. I guess it’s similar to the various “Awesome X” projects that exist out there.

The project owners could request membership of the PyPA, which would make it a “PyPA project” - but that wouldn’t (for example) give me (as another PyPA member) any ability to curate the project, unless I asked explicitly to be a maintainer.

1 Like

No, it would be very similar to what we have now, except it would be “official” in the sense that “official” sites would link to it. That kind of thing matters a lot to some people (and doesn’t matter at all to many others).

The main advantage of having it somewhere PyPA members are already active is that they may learn about issues or difficult aspects to their projects they weren’t aware of, or may be able to correct misconceptions. That can be done anywhere, but it’s hard to find on StackOverflow, and even harder to prove your credibility there.

But whether the idea would work basically comes down to who would want to be involved. Though I think that applies to all the other ideas we’ve had here too. There’s only so much value in Paul and I throwing ideas around - either the people who want to write recommendations need to speak up, or the people who want to maintain the chosen winner need to speak up (or someone with another idea could jump in).

1 Like

I was talking about package creation at large, but even for the backend, I would have been quite happy to never find out that there even is such a thing, unless I was trying to solve a specific problem with the default. (But I accept that, by contraposition, I might not be an advanced developer.)

If you’re planning to write code, you need to choose what language to use.

If you’re planning to write a web site in Python, you need to choose which framework to use.

If you’re planning to produce a Python package, you need to choose which backend to use.

This aspect is no different from how all development works. About the only difference is that other ecosystems dictated a single option from the start and never allowed the range of diversity we have in Python. So people asking for one option are literally asking us to crush that range of options. Fundamentally, we don’t like that approach to “community development”, which is why we’re resistant.

1 Like

Thanks, Paul and you have clarified the idea for me now, I understand what you mean.

StackOverflow now has something called “Collectives”. If I understand correctly this would mean PyPA could create its own collective. Honestly I have no idea how exactly it works. I guess the idea would be to put some kind of PyPA stamp on some Q&As. Maybe someone knows more about this and can comment whether or not it could be a good match for our use case here. I also do not know how commited to this they are at StackOverflow. I feel like they have been rather inconsistent lately, adding experimental features then removing them later in a relatively short time span.

Otherwise I guess the GitHub Discussions could work as well, but I do not know how it works from a “moderator/curator” point of view.

In my mind, modernizing packaging.python.org and maybe a couple of cookiecutter templates (like scikit-hep/cookie or cjolowicz/cookiecutter-hypermodern-python) would be good enough. I have started to contribute a bit to packaging.python.org, we’ll see how far I manage to get.

1 Like

This is again the false dichotomy between an informed default option that covers most use cases, and mandating a single option.

I’m labouring the point because it dismays me to see that the current state of things is considered not only acceptable but desirable. How has the packaging UX on average, over the population of all packages, improved from when it was “write a setup.cfg file”, and that was that? I am maintaining a few packages, only to share useful code with others in my field, and having to know anything about and keep up with the packaging meta-development is purely a chore.

1 Like

It’s only a false dichotomy because you believe everyone else will see that the informed default means that they could equally choose another option and be “correct”.

I don’t believe it’s false, because I regularly deal with people who insist that unittest is the only correct way to run tests, because it’s “official” (as well as many other examples across all fields, not just Python packaging).

So my experience says that not everybody will understand the difference between an official tool and a recommended tool. Hence why I’m opposed to having anything that could be mistaken for an official tool (unless it really is the only one under active development and widespread use - pytest seems to be very close to meeting this criteria).

1 Like

Working in a large ecosystem that migrated from unittest to
testtools/stestr ages ago, primarily for things like subunit
attachment serialization in test results, I’m happy to provide
counterarguments that pytest is not the only actively developed
unittest alternative in widespread use.

Simply saying so is enough argument :slight_smile: Even in that space, there’s no universal one tool.

All that said, within a project or within an organisation, you can have one specified tool. e.g. at work I always recommend (and in a few cases, requi… er… recommend really strongly) using pytest, black, and conda. And that’s fine for my limited scope - we also have a style guide (a few, actually). None of that affects anyone outside of the scope that I understand and to some extent, control.

Recommending something at the level we’re discussing here does affect anyone. Ever had someone use PEP 7 or PEP 8 against you? I have. It’s because they mistook it for a global requirement, and not the style guide for a particular project. And that kind of thing is easy to do - let’s not make it easier.

Hello,

First time commenting here but I wanted to leave my two cents.

I think simple “vanilla” cookiecutter repos is what many people need. A cookiecutter repo for single file module, another one for a multi-file package, another one with a C extension, another one with entry point or “instalable scripts”, etc.

Then people can choose which one to start with and modify and merge accordingly. I understand, though, that it can grow very quickly. But at least have a few ones for people to start with the most common situations, where 90% of the people are.

1 Like

Along these lines - what would happen if one of the requirements for being a PyPA project (ie a project repo living in the organization) included a bus factor value (x number of maintainers). is that possible? would that encourage maintainers to work together a bit?

the reason i mention it is it’s a worry of ours that so many of the packaging tools have n=1 maintainers which makes me a bit nervous in terms of sustainability / burn out etc. That then would make the project a bit more community oriented as well. more people can more easily engage with more users, etc.


As for the cookie cutters. i think they are a great idea. but i also think if PyPA recommends ones created by others, they should have some criteria around them being maintained as just like documentation. Otherwise they become dated as tools and approaches evolve.

That’s an even more difficult aspect of PyPA governance than the relatively manageable change I suggested at the start of this thread[1]. The establishment of the current rules (PEP 609) was fairly difficult, as far as I recall, and one of the key factors in getting agreement was an explicit statement that the PyPA will not interfere in the management of individual projects. That would specifically exclude requiring any form of maintainer quota, as it very definitely means that the PyPA has no say in who is given commit rights on any given project. Also, of course, you can’t dictate what people want to work on in any case - you could make me a pipenv maintainer, for example, but it wouldn’t make any practical difference to the sustainability of the pipenv project, as I’ve no interest in contributiing to that project.

We could, of course, change the rules and governance, but I don’t think anyone has the appetite for that (I know I certainly don’t). And in the current climate, with a lot of (entirely reasonable!) outside interest in the PyPA, trying to have a public debate on governance runs the risk of burning people out, and as a result making worse the problems we’re trying to address :slightly_frowning_face:


  1. And you can see how little progress we’ve made on that one… ↩︎

2 Likes

Checking for multiple active maintainers when accepting a project
doesn’t necessarily mean it will always stay that way, so you
similarly need to decide what you do (if anything) when the active
maintainer count falls below the acceptable threshold, and who will
be checking for that and how often.

However, I’ve seen plenty of projects go from lots of active
maintainers to approximately zero overnight, if they were all using
it in the same organization and suddenly business plans changed, so
it can be useful to think beyond simple maintainer count and
consider affiliation diversity as well if you think it’s an
important risk to mitigate. And even then, it’s the sort of thing
that can change over time, I’ve also seen my share of projects go
from being maintained by people from many organizations to only a
few, and then one, and then… none.

1 Like

To note, the one benefit of a project being under the PyPA org to begin with is if worst comes to worst and all the maintainers abandon the project, there exists at least the capability for another trusted PyPA member interested in picking up maintaining it to be added, or other actions taken (of course, they presumably wouldn’t have the PyPI keys unless they were stored as org secrets or there was a PyPI abandoned project request, but they’d at least have the source and the community and could reclaim the project name once sufficient time had passed).

Whether the PyPA would be willing to do that is, I presume, not a settled question, but it’s at least possible. Kicking someone out of the PyPA if they don’t maintain a maintainer threshold is, in a way, actively counterproductive to the goal of reducing long term bus factor as the projects who could most benefit from org rather than personal ownership won’t have it.

1 Like

ok i hear all of you - bad idea on my part :slight_smile: i am still learning the dynamics of this community here and by no means want things to move backwards !! It’s great to better understand what the challenges are.

And also you’re right @fungi - a maintainer team could also step down at any point in time and it would be really hard to in general to support such an idea. ok i (unintentionally) drove this conversation in the wrong direction!

It’s not a bad idea, it’s just that there’s a lot of “been there, done that” sentiment that means people tend to resist reopening topics that have been painful in the past. A fresh viewpoint is very important, and it’s entirely possible we should do things like this, it’s just hard to work out how :wink:

Right, I spend a lot of my time in another large open source
community that does try to take things like active maintainer count
and maintainer affiliation diversity into consideration (also if
you’re into those sorts of sustainability topics, maybe check out
https://chaoss.community/ since they’ve got lots of people who work
on finding ways to measure and analyze such things). I just wanted
to make it clear that it’s not as simple as having a “bus factor”
requirement, and there is necessarily a lot of work involved in
implementing those sorts of policies as well as impact trade-offs
that have to be weighed.

The idea is not bad on its own, but it sounds like it may not be a
fit for PyPA (and would, I’m sure, need a lot of discussion).

Are there any organizations similar to PyPA to draw inspiration from? PyCQA? Some “science Python” communities?