Remove the "Authority" from Packaging

I guess the essence of Antoine’s argument is: if the PyPA changed its name, then users will stop pressuring him to provide wheels for projects where that’s currently unreasonably difficult. I get Antoine’s frustration, but I agree with @dstufft that changing the name is unlikely to actually produce the result Antoine is hoping for.

I am weakly on favor of changing the name though, exactly because it has this history of making people to think of the PyPA as a monolithic organization that is forcing people to do things a certain way. That’s inaccurate, makes people think it’s hard to contribute, and encourages people to treat PyPA volunteers as punching bags for venting their frustrations.

2 Likes

It’s more than a documentation change. Changing the name involves expending effort to communicate that the name has changed. There will be a fair number of people who have only heard the old name, or only heard the new name, and will be confused by the difference. It will involve user education that will likely take years for it to fully shake out. I’ll point to the Cheeseshop / PyPI naming issue, and there is still to this day confusion about what the difference between those two things are (thankfully not a whole lot anymore… but that’s over a decade).

It’s a lot of nebulous “soft” work that isn’t represented in a code diff, but that doesn’t make it any less work.

I don’t think changing the name is going to change that, you’d have to change the fact we have an umbrella organization all together I think, and even then I think it would just spread it out from a nebulous “the PyPA is doing X” to “the pip/virtualenv/manylinux/whatever developers are doing X”. Most likely most of that would end up falling on pip since it’s the tool that people are typically actually invoking, even if the problem they’re having is with some other component.

I’ve been involved long enough to remember what things were like before we had the PyPA acting as an umbrella organization, and those same problems existed back then. The only difference was instead of blaming the PyPA, people blamed “distutils-sig”, people still thought it was hard to contribute, and distutils-sig was basically a toxic wasteland of everyone treating each other as punching bags for venting their frustrations.

I really don’t think there is “one weird trick” to solving those problems, low information commenters, frustrated people, and people willfully ignoring reality are going to generalize that some amorphous group is the one making all the bad choices that are ruining their day (and of course, they’re not a member of that group in their mind-- even if someone else might say that they were). I think the best we can do is try to document what the PyPA is, provide contributing guides that make the onramp as easy as possible, and refuse to tolerate abusive behavior in any of our spaces.

3 Likes

I’ve gone ahead and offered to write some docs about conda-forge and PyPI as suggested https://github.com/pypa/packaging.python.org/issues/367#issuecomment-512003560.

FWIW, I think excellent work has been done by conda-forge/conda and PyPA/PyPI/pip folks. Thank you for the time and effort.

If anyone would like to help draft the document with me, please leave a note in the issue. Thanks.

6 Likes

I’d like to offer an alternative line of thinking. People in general tend to look for an authoritative voice when encountered with a decision, no matter there is one or not; this is human nature, and not really changeable. PyPA, or more specifically, whatever group of people maintaining pip, PyPI etc., is already established as the thing that looks most like the authority people are looking for, and will continue to be considered so. A rename won’t change that fact.

I guess what I’m going with this is that instead of renaming PyPA to fit what PyPA wants to be, it would be more possible to actually create the authority that people want, so PyPA don’t need to fill that expectation. I understand this would not be easy to script (if at all), but it IMO better addresses the underlying issue.

8 Likes

That’s an interesting proposal. Though I wonder what kind of consensus could be reached in said “new authority”. That obviously depends on who is part of it. @teoliphant Any thoughts?

I read it described somewhere as “I don’t have your problem therefore you don’t have your problem.” Maybe this is what people are trying to get at with a name change, which would not in itself change the underlying thing.

2 Likes

I appreciate everyone sharing their views here.

@dholth I agree that it would be good to have clearer rules. If @dstufft or @dustin could comment on this issue, I’ll add some docs to pypa.io:

3 Likes

Note that the “Authority” in PyPA does have real authority behind it: there is a standing delegation from the elected Python Steering Council ceding control of certain kinds of Python Enhancement Proposals to specific individuals that identify themselves as members of the PyPA. (And before that the standing delegation was from Guido as BDFL, ever since we first set the arrangement up back in 2013 or so)

The difference between PyPA’s design perspective and conda is that PyPA is mostly platform neutral - we definitely point folks towards conda when we think it would help them (see https://packaging.python.org/overview/#depending-on-a-separate-software-distribution-ecosystem for example), but we’re not going to start telling people to stop using non-conda Python distributions (and any of the other distribution techniques mentioned on that overview page).

So even though it’s a freely available open source platform, saying “We only support conda users” would be akin to saying “We only support Enthought Canopy users” or “We only support ActivePython”.

We’ve just been successful enough in our cross-platform design goals that folks initially publishing to one particular platform don’t get asked “Please support (some other particular platform)”, they get asked “Please support the default Python ecosystem tooling”.

8 Likes

This is a really important consideration.

For a parallel perspective, we can look at the, well-intentioned but commercial, influence that RStudio is exerting over the R-Language ecosystem, as articulated by Norm Matloff.

It’s not really parallel though. conda is just a package management utility (like rpm, apt, etc.). There is a commercial open source distribution named Anaconda and built around conda. There is also a volunteer-based open source distribution package repository named conda-forge and built around conda.

In theory, any group of people could build their own distribution or package repository around conda. Of course, in practice it represents such a large amount of work that few such efforts will ever be attempted. But, no, using conda doesn’t put you at the mercy of a commercially-developed ecosystem.

1 Like

The parallel I see is that conda presents a bifurcating alternative to pip, in that Conda and Conda Forge packages are separate from PyPI. I recognize that conda and pip can be used together, primarily when a project is initialized with Conda.

The influence of Anaconda, by both the Anaconda distribution and conda package repositories, may not at this point be as significant as the RStudio/Tidyverse influence on R-Language ecosystem, but is significant nonetheless. :grinning:

I’ll say it again : conda is not a package repository. conda-forge is, and it’s maintained by the community.

1 Like

To be pedantic, I suppose I was referring to the Anaconda Repositories, accessible by default with the conda tool, that are maintained by Anaconda, Inc…
https://repo.continuum.io/

Conda Forge repositories must be manually enabled, or explicitly referenced after installing the Anaconda distribution, with:

conda config --add channels conda-forge

The main point I was trying to affirm is the importance that central aspects of a language ecosystem, such as packaging and distribution, be stewarded and by an official governing body (Authority) of representatives from across the community.

1 Like

This is somewhat orthogonal, but it’s one thing I really liked about the charter for the Steering Council: there is a limit as to how many SC members can work for the sane company, I presume to avoid the possibility for the same sort of subtle, unintentional influence on the language as what is described above for R.

Well, there’s no such official governing body, then. The PyP"A" doesn’t consist of representatives from accross the community. It’s just people who coopted one another, and it’s not even obvious how (i.e. I don’t think there are any formal rules as to how someone becomes a member of that body).

1 Like

The PSFmember.org donation page says, “Donation for the Packaging Workgroup” in its title. (PyPI.org > user profile drop-down > Donate)

Does that mean the “Authority” part of the name was dropped in the meantime, or is “Workgroup” just an old name forgotten in the attic?

The page title stroke me immediately today (after visiting PyPI to activate 2FA), so I came here to search for a possible reason. – Why did it strike me? Because I felt the “authority threat” that this discussion is seemingly talking about, too in the (not too far) past.

I feel it’s indeed hard to contribute. It’s a bit like if you didn’t interact with some of the maintainers personally (on conferences) you’ll have a hard time getting your PRs through. There is a strong (male white?) attitude ruling the GitHub issues & PRs landscape. Of course, that’s just a very personal observation, but it sure doesn’t come from pure fantasy.

Let me emphasize that I’m very attached to a positive image “Python” should convey. There should be a welcoming atmosphere in any Python community or workgroup. And yes, names convey an image and certainly can trigger behavioral change.

2 Likes

The PyPA and the Packaging-WG are two separate (but overlapping) groups. The working group is primarily concerned with funding and the disbursement of funds for packaging-related projects, see PackagingWG - PSF Wiki for more details.

1 Like

Personally, I have not had this impression thus far from any part of the Python development community. While there have been a few times I’ve waited a bit long for feedback on PRs or had a few reviews that went unnoticed, the majority of the core devs are happy to help anyone with questions. This includes myself, and I’ve never interacted with any of the core developers in person.

If a PR goes unnoticed for a long period of time and the author reaches out through python-dev, it is usually reviewed. In the majority of cases I’ve seen, the author usually never reaches out (despite this being mentioned on the devguide).

Especially recently, the Python development community has become significantly more diversified. At least in more recent recent years, I’ve never seen anyone treated differently as a result of their ethnicity, skin color, gender, sexuality, etc. This may not be the case across every organization on GitHub, but it definitely has been for Python, at least from my experience.

The difficulties of contributing (from my interpretation) comes primarily from a lack of resources. There’s far more people submitting PRs on a day-to-day basis than there are active core developers and PR reviewers.

Hopefully this will improve in the near future once the Python triage team has a decent number of active members. Having a group dedicated towards PR review and categorization on GitHub will help to reduce the number of unaddressed PRs. This will also allow the core devs to spend less time on the preliminary steps, allowing them to potentially merge more PRs within the same amount of time.

6 Likes

I just took the PyPA at its word that they’re open to contributions. Result: I had a great experience getting started contributing to the PyPA docs. That’s why i’m here now.

I submitted two Issues and a PR that were all enthusiastically received and quickly acted on. In my experience, this openness to a newcomer is rare in large, established open source projects:

3 Likes

Thanks for chiming in @aeros and @dogweather!

W.R.T. the lack of formal rules, yes, the situation has to improve.

PyPA currently operates on an informal-consensus based model (“no one has blocking issues with this, so it’s OK”) for a lot of the things we do. We are working toward a governance model to make the decision processes and membership formal, and more transparent.

1 Like