Remove the "Authority" from Packaging

I’ve gone ahead and offered to write some docs about conda-forge and PyPI as suggested https://github.com/pypa/packaging.python.org/issues/367#issuecomment-512003560.

FWIW, I think excellent work has been done by conda-forge/conda and PyPA/PyPI/pip folks. Thank you for the time and effort.

If anyone would like to help draft the document with me, please leave a note in the issue. Thanks.

6 Likes

I’d like to offer an alternative line of thinking. People in general tend to look for an authoritative voice when encountered with a decision, no matter there is one or not; this is human nature, and not really changeable. PyPA, or more specifically, whatever group of people maintaining pip, PyPI etc., is already established as the thing that looks most like the authority people are looking for, and will continue to be considered so. A rename won’t change that fact.

I guess what I’m going with this is that instead of renaming PyPA to fit what PyPA wants to be, it would be more possible to actually create the authority that people want, so PyPA don’t need to fill that expectation. I understand this would not be easy to script (if at all), but it IMO better addresses the underlying issue.

8 Likes

That’s an interesting proposal. Though I wonder what kind of consensus could be reached in said “new authority”. That obviously depends on who is part of it. @teoliphant Any thoughts?

I read it described somewhere as “I don’t have your problem therefore you don’t have your problem.” Maybe this is what people are trying to get at with a name change, which would not in itself change the underlying thing.

2 Likes

I appreciate everyone sharing their views here.

@dholth I agree that it would be good to have clearer rules. If @dstufft or @dustin could comment on this issue, I’ll add some docs to pypa.io:

3 Likes

Note that the “Authority” in PyPA does have real authority behind it: there is a standing delegation from the elected Python Steering Council ceding control of certain kinds of Python Enhancement Proposals to specific individuals that identify themselves as members of the PyPA. (And before that the standing delegation was from Guido as BDFL, ever since we first set the arrangement up back in 2013 or so)

The difference between PyPA’s design perspective and conda is that PyPA is mostly platform neutral - we definitely point folks towards conda when we think it would help them (see https://packaging.python.org/overview/#depending-on-a-separate-software-distribution-ecosystem for example), but we’re not going to start telling people to stop using non-conda Python distributions (and any of the other distribution techniques mentioned on that overview page).

So even though it’s a freely available open source platform, saying “We only support conda users” would be akin to saying “We only support Enthought Canopy users” or “We only support ActivePython”.

We’ve just been successful enough in our cross-platform design goals that folks initially publishing to one particular platform don’t get asked “Please support (some other particular platform)”, they get asked “Please support the default Python ecosystem tooling”.

8 Likes

This is a really important consideration.

For a parallel perspective, we can look at the, well-intentioned but commercial, influence that RStudio is exerting over the R-Language ecosystem, as articulated by Norm Matloff.

It’s not really parallel though. conda is just a package management utility (like rpm, apt, etc.). There is a commercial open source distribution named Anaconda and built around conda. There is also a volunteer-based open source distribution package repository named conda-forge and built around conda.

In theory, any group of people could build their own distribution or package repository around conda. Of course, in practice it represents such a large amount of work that few such efforts will ever be attempted. But, no, using conda doesn’t put you at the mercy of a commercially-developed ecosystem.

1 Like

The parallel I see is that conda presents a bifurcating alternative to pip, in that Conda and Conda Forge packages are separate from PyPI. I recognize that conda and pip can be used together, primarily when a project is initialized with Conda.

The influence of Anaconda, by both the Anaconda distribution and conda package repositories, may not at this point be as significant as the RStudio/Tidyverse influence on R-Language ecosystem, but is significant nonetheless. :grinning:

I’ll say it again : conda is not a package repository. conda-forge is, and it’s maintained by the community.

1 Like

To be pedantic, I suppose I was referring to the Anaconda Repositories, accessible by default with the conda tool, that are maintained by Anaconda, Inc…
https://repo.continuum.io/

Conda Forge repositories must be manually enabled, or explicitly referenced after installing the Anaconda distribution, with:

conda config --add channels conda-forge

The main point I was trying to affirm is the importance that central aspects of a language ecosystem, such as packaging and distribution, be stewarded and by an official governing body (Authority) of representatives from across the community.

1 Like

This is somewhat orthogonal, but it’s one thing I really liked about the charter for the Steering Council: there is a limit as to how many SC members can work for the sane company, I presume to avoid the possibility for the same sort of subtle, unintentional influence on the language as what is described above for R.

Well, there’s no such official governing body, then. The PyP"A" doesn’t consist of representatives from accross the community. It’s just people who coopted one another, and it’s not even obvious how (i.e. I don’t think there are any formal rules as to how someone becomes a member of that body).

1 Like

The PSFmember.org donation page says, “Donation for the Packaging Workgroup” in its title. (PyPI.org > user profile drop-down > Donate)

Does that mean the “Authority” part of the name was dropped in the meantime, or is “Workgroup” just an old name forgotten in the attic?

The page title stroke me immediately today (after visiting PyPI to activate 2FA), so I came here to search for a possible reason. – Why did it strike me? Because I felt the “authority threat” that this discussion is seemingly talking about, too in the (not too far) past.

I feel it’s indeed hard to contribute. It’s a bit like if you didn’t interact with some of the maintainers personally (on conferences) you’ll have a hard time getting your PRs through. There is a strong (male white?) attitude ruling the GitHub issues & PRs landscape. Of course, that’s just a very personal observation, but it sure doesn’t come from pure fantasy.

Let me emphasize that I’m very attached to a positive image “Python” should convey. There should be a welcoming atmosphere in any Python community or workgroup. And yes, names convey an image and certainly can trigger behavioral change.

2 Likes

The PyPA and the Packaging-WG are two separate (but overlapping) groups. The working group is primarily concerned with funding and the disbursement of funds for packaging-related projects, see https://wiki.python.org/psf/PackagingWG for more details.

1 Like

Personally, I have not had this impression thus far from any part of the Python development community. While there have been a few times I’ve waited a bit long for feedback on PRs or had a few reviews that went unnoticed, the majority of the core devs are happy to help anyone with questions. This includes myself, and I’ve never interacted with any of the core developers in person.

If a PR goes unnoticed for a long period of time and the author reaches out through python-dev, it is usually reviewed. In the majority of cases I’ve seen, the author usually never reaches out (despite this being mentioned on the devguide).

Especially recently, the Python development community has become significantly more diversified. At least in more recent recent years, I’ve never seen anyone treated differently as a result of their ethnicity, skin color, gender, sexuality, etc. This may not be the case across every organization on GitHub, but it definitely has been for Python, at least from my experience.

The difficulties of contributing (from my interpretation) comes primarily from a lack of resources. There’s far more people submitting PRs on a day-to-day basis than there are active core developers and PR reviewers.

Hopefully this will improve in the near future once the Python triage team has a decent number of active members. Having a group dedicated towards PR review and categorization on GitHub will help to reduce the number of unaddressed PRs. This will also allow the core devs to spend less time on the preliminary steps, allowing them to potentially merge more PRs within the same amount of time.

6 Likes

I just took the PyPA at its word that they’re open to contributions. Result: I had a great experience getting started contributing to the PyPA docs. That’s why i’m here now.

I submitted two Issues and a PR that were all enthusiastically received and quickly acted on. In my experience, this openness to a newcomer is rare in large, established open source projects:

3 Likes

Thanks for chiming in @aeros and @dogweather!

W.R.T. the lack of formal rules, yes, the situation has to improve.

PyPA currently operates on an informal-consensus based model (“no one has blocking issues with this, so it’s OK”) for a lot of the things we do. We are working toward a governance model to make the decision processes and membership formal, and more transparent.

1 Like

It’s probably worth pointing out that the PyPA is a collection of projects, not of individuals1. So in one sense, the answer to @pitrou’s question is “Be a maintainer of a PyPA project”. As the PyPA explicitly does not have control over member projects’ governance, the rules on who is a maintainer for a particular project are at the discretion of that project.

1 I refrain from commenting on whether that’s obvious to people outside of the PyPA, and I don’t want to get into a debate on whether that’s the way it “should be”… :slightly_smiling_face:

1 Like

Yep yep.

All of these details are things that benefit from a clear document to point to saying: “this is how PyPA works” .

And, that’s literally the PyPA governance discussion. :upside_down_face:

2 Likes