I think it is reasonable to say that there’s concensus (or at least a general vibe) that we’re hitting the edges/limits of the current governance model for Python “packaging” and there’s some need for evolution here. This has come up in the various long thread discussions.
Let’s do an informal brainstorm: post your wishes about what perfect (or very good) Python packaging governance should look like.
Some ground rules, to keep this productive:
This is about the decision making process and NOT about specific decisions we make.[1]
Post one item per message.
If someone has already posted something you want, “vote” for it using the “heart” mechanism.
If someone has posted something you don’t like, hold that thought for now. We’re brainstorming asynchronously and this is going to work only if we’re disciplined about not shooting down ideas here.
Avoid discussion of pros and cons of specific ideas. I intend to start a separate discussion for those at a later date (or, you can open one; but please be mindful of splitting the group’s attention).
For example, “don’t use the PEP process” is reasonable. But, revisit PEP NNN isn’t. ↩︎
Change: Special case “default” projects (i.e. setuptools and pip) and explicitly include major changes within them as on-topic for the decision making process.
During discussion of this PEP, it became clear that changes to pip’s UX are not controlled by PEPs as proposed.
The fact that “should this be a PEP” was a question, about an ecosystem affecting change that was a proposal for change, in the default installation tool that ships with the language, doesn’t feel right; even though I understand the history here.
Change: Have the same explicitly named group make non-technical decisions (eg: providing oversight in funded roles/projects) as well as technical ones. This doesn’t need to be formalised but at least have the PSF on board with the idea of engaging with this group when seeking/receiving funding for packaging related roles.
Effect: Funders have a body closer to the development processes than the PSF to engage with. The community has a clear point of contact that isn’t PSF’s ED or staff to flag ideas/concerns to.
Packaging governance explicitly, and actively, supports the existence of, and interoperability with, other packaging ecosystems outside of its control, like conda, Linux distros, Homebrew, etc.
This deliberately avoids the question of what precisely is governed by the packaging governance model, which is a separate question (which I’ll therefore post separately).
Governs everything that interacts with PyPI. The core distribution mechanism covered by “Python packaging” should be PyPI (or a successor to PyPI), with other distribution methods being independent (but supported, see my previous point).
Note that “governs” here does not necessarily mean “manages”. The PyPA currently “governs” various projects, but explicitly does not have a say in day to day project maintenance. However, it could mean something stronger[1] if the projects involved agree.
This is a major social issue, of course. Unlike the current PyPA, I’m explicitly saying that “not being under the packaging governance model” should not be a credible position for a tool that wants to be considered a mainstream packaging tool to take. It’s going to be very hard to make this work, and will involve a lot of compromises and negotiation. But if we’re not going to do this, I don’t see that we can credibly describe what we have as “packaging governance”.
Could you please elaborate a bit more on this one. I don’t understand what explicitly include major changes within them as on-topic for the decision making process means in this context…
Change: Prioritize the approach “standardise existing behaviour then do follow up improvements” instead of “create a new stardard that is not backwards compatible with existing behaviour (or at least hard to make backwards compatible with)”
Effect: avoid or minimise “growing pains” in the ecosystem.
Change: Improve the process for reaching consensus or (at least reaching consent) for new standards/PEPs in a way that volunteers feel motivated to implement the change and avoid standards/PEPs that still feel controversial/sour.
Effect: Motivate the community to work towards a truly shared vision and minimise the risk of something like the one reported in the following thread to happen again (I am highlighting below 2 important comments that I feel summarise the problem):
Change: Protect the decision making process against ballot stuffing.
Currently, every PyPA committer has one vote (in PyPA committer votes). The PyPA is composed of PyPA projects. However, PyPA-as-a-body has no control over who gets added as a committer to a PyPA project.
In other words, the every-vote-is-equal process is susceptible to a project adding way-too-many committers and resulting in over-representation for that project.
Change: Explicitly decouple governance from tool recommendations.
Effect: Make it possible for official documentation to recommend projects that aren’t subject to “packaging governance”, and avoid giving preferential treatment to projects that are.
(Note: This is irrelevant if “Govern everything that interacts with PyPI” is adopted, but if that is rejected, this is a possibly less controversial version).
Change: Require re-application for membership from every project.
Effect: Gives every project an opportunity to re-evaluate whether they are willing to agree to the new governance, and gives the governing body a choice over who to accept. Allows the governing body to set the tone of what it means to be a “packaging project” without historical baggage or accidental bias.
Change: Extend the scope of the term “packaging” to include building/publishing standalone applications, on the one hand, and informal “sharing of scripts” on the other.
Effect: Moves away from the current “everything must be a library” model.
Change: Stop requiring governed projects to live in a single GitHub organisation.
Effect: It’d allow us to resolve CI slowness issues during bursty development (due to a shared Actions runner bucket across all projects) and help with ensuring complete access control to a project (eg: automating releases would allow any admin to make the release of a project, which is part of why pip doesn’t use trusted publishers). It does complicate certain other aspects though notably around money via GH Sponsors and communication around what projects are a member project – but we’ll figure something out.
Change: Establish a formal agreement with the SC and core devs for “power sharing” with regard to packaging tools in the core distribution. This would cover ensurepip, venv, and the Python launcher/CLI, as well as PEPs like the removal of distutils and PEP 582 (local packages directory).
Effect: Allows the packaging community to take a more holistic approach on “how people use Python”, and avoids the community’s current impotence as soon as a proposal needs core or stdlib changes.
Change: Establish a formal agreement with the PSF, to not only have funds available for packaging improvements but also have a team that actively is looking to disseminate opportunities and recruit programmers to carry out the improvements that the community deem necessary.
Effect: Speed up the implementation process for the backlog of fundable improvements, and avoid having necessary changes that never get implemented because of the lack of available programmers.
↩︎
