Hi @smm. I believe that you have answered my specific question in the draft PEP. Thanks.
When I asked about accountability, it was to move forward discussion on whether the new governance falls under the Python Steering Council, the Python Software Foundation, or as a separate, independent entity.
Has there been an updated version of the PEP that I missed? Where is it available?
I assumed it was the same doc that was linked in the initial post from @smm just updated on their GitHub account.
Do you mean this: https://github.com/s-mm/peps/blob/packaging_council/pep-9999.rst
I donât see any updates to that since the date of the original post on this thread.
All my updates have been in the first post in this thread. I hope you were referring to the draft in that post.
@smm â as a quick note, I think many people might have missed that youâve made edits in the initial post [1] â from what I understand these also donât get sent out to users of the mailing-list mode.
Please would you be able to post the current version as a new reply? (Potentially with a short summary of changes made since this thread was opened).
A
I certainly did and Iâve been trying to follow this thread âŠď¸
The latest version of the draft is available below. I have tried to answer the questions raised in a previous post-
The new edits are in the Abstract and Rationale sections.
PEP: 9999
Title: Python Packaging Governance
Author: Shamika Mohanan
Sponsor:
Status: Draft
Type: Process
Topic: Governance
Content-Type: text/x-rst
Created:
Abstract
========
This PEP describes the Python Packaging Governance for
all tools that are included as part of the PyPA. The Packaging
Council will govern all PyPA tools, represent the community
that supports these tools and act as a representative of PyPA
tools. From here on, this Packaging council will be called the
Python Packaging Authority (PyPA). This PEP will supersede
[PEP 609](https://peps.python.org/pep-0609/)
and will cover all aspects of Python Packaging Governance.
Rationale
=========
One issue facing Python Packaging is that there is a lack of a
comprehensive vision and strategy for Python Packaging tools. While
`PEP 609 <https://peps.python.org/pep-0609/>`_
formalized the governance of tools included under the PyPA
umbrella, this governance does not extend to setting long-term goals
for PyPA tools. In order to ensure PyPA tools remain relevant in the
future and cater to user requirements, it is important to set a vision
and strategy for Python Packaging and PyPA tools. It is hoped that
the council will focus on the Packaging ecosystem, define the
vision and work with the community to develop the strategy.
Attempts at defining the vision and strategy have failed so far.
The main motivation for this PEP is to create a body with the necessary
powers to define the short-term and long-term goals for Python Packaging
and to support the community in setting the strategy to achieve the goals. This
governing body will represent PyPA tools when working with any other tool or group.
As such, the mission of the initial council will be-
* Define the short-term and long-term vision and strategy for Python Packaging
* Approve PEPs that support the Packaging strategy
* Identify solutions to few (1-2) problems identified in the
[Python Packaging survey](https://drive.google.com/file/d/1U5d5SiXLVkzDpS0i1dJIA4Hu5Qg704T9/view) and marshall resources to implement the solutions
We can expect the initial council to pick up a few problems
outside of this list that they feel is important to the community.
With each council, this mission will change depending on
pressing issues and how the vision and strategy has changed.
The mandate and powers of the PyPA have been drawn up such that they
are not too restrictive. Certain issues such as authority over maintenance
and improvement of installers, build backends, Warehouse, and other PyPA
software are not in the scope of the initial council. This could be added
the future if all stakeholders agree to this change. It is expected that
the mandate of powers of the council will grow as long as it can show that
it is working in the best interest of the Packaging community. This
council will be accountable to the Python Steering Council.
Specification
=============
Python Packaging Authority
--------------------------
Composition
~~~~~~~~~~~
The packaging council will consist of 5 members.
Mandate
~~~~~~~
The packaging council will-
* Set vision and strategy for Python Packaging and PyPA tools (defined below)
* Support, encourage and facilitate the maintenance and improvement
of the quality of all PyPA tools
* Represent PyPA tools as a group when working with non-PyPA tools,
PSF, Python Steering Council and third-party entities such as operating
system package managers. The council will start discussions and engage
with stakeholders and the Packaging community and represent the PyPA interests.
* Arbitrate on any packaging related issue brought to the council. This
should be done as the final resort.
* Improve diversity of the Packaging community. The council should work
with the PSF to introduce programmes aimed at improving diversity of contributors.
Powers
~~~~~~
Packaging Governance
++++++++++++++++++++
* Accept or reject PEPs within the scope delegated by the Python Steering Council
* Delegate parts of their authority to other subcommittees or processes
Packaging Assets
++++++++++++++++
* Authority over `packaging.python.org <https://packaging.python.org/en/latest/>`_
* Maintain advisory role for any issue related to packaging docs
* Authority over PyPA GitHub org
Stakeholder Management
++++++++++++++++++++++
* Act as a liaison with any packaging-related Python or non Python
technical group when specific tool representation is not appropriate.
For example, any group representing supply chain security.
* Act as a liaison with the PSF and the Packaging Working Group to obtain
funding and any other areas of shared interest
* Act as liaison with the Python Steering Council
Packaging ecosystem and community
+++++++++++++++++++++++++++++++++
* Maintain ownership of PyPA tools. The council will not interfere
with a project's day-to-day operations but rather govern
the community that supports these tools. The PyPA can require
projects to implement accepted PEP standards. In such a situation,
the PyPA should liaise with project maintainers and the PSF to
secure resources to implement accepted PEP standards when necessary.
* Recommend projects for adoption into PyPA by vote. External projects can self-nominate to
be included under the PyPA umbrella.
* Expel PyPA project/individual
Scope: It is expected that the Packaging council will consider the Python Packaging
ecosystem holistically and improve all aspects of Python Packaging rather than just
one project. This includes any tool and workflow related to the PyPA and PyPI. The
council should work towards improving user experience of all packaging tools under
its purview and this might include deprecating certain tools and developing new tools
depending on requirements. While it does not have any authority over non-PyPA tools,
it is expected that the council makes decisions that benefit the entire Python Packaging
ecosystem. This scope includes all aspects of packaging and distribution. It is expected
the council will work on long-term goals such as improving interoperability with non-PyPA
packaging tools and improving packaging UX.
To use its powers, the council votes. Every council member must either
vote or explicitly abstain. Members with conflicts of interest on a particular vote
must abstain. Passing requires a strict majority of non-abstaining council members.
Elections
~~~~~~~~~
Packaging Voting Body: The Packaging Voting Body (PVB) includes the following:
* PyPA committers
* Affiliate project voters
PyPA committers can nominate either non-PyPA projects or individual contributors
who do not contribute to a PyPA project to be added to the PVB. These voters will
be called affiliate project voters.
* Any voter nomination has to be seconded by another PyPA committer
* External projects nominated by PyPA committers will get to appoint a representative
for each ballot
The initial seed of voters will include all committers for the existing non-archived
public projects under the `pypa` organisation of GitHub.
The PyPA has to maintain a list of eligible voters and their project affiliation. This
list should contain the names of PyPA committers and affiliate project voters.
This canonical list should be maintained by the PyPA with access available to
all Packaging Voting Body members. This list should not share personal
information publicly. It is the responsibility of all PyPA projects to
ensure that the names of any new committers are added to the list of eligible voters
and any communication spaces for the PyPA.
For the initial election, this voter list will be maintained by the PSF.
Adding any new voters and subsequent due diligence is the responsibility
of PyPA committers. Once the new council has been elected, ownership of the
Packaging Voting Body membership list and the PyPA-voters mailing list will
be transferred to the PyPA.
PyPA elections will be held in three phases
* Phase 1: Packaging voting body members nominate affiliate project voters.
Affiliate project voters are added to the list of eligible voters.
* Phase 2: Candidates advertise their interest in serving. Candidates must be
nominated by a PyPA voting body member. Self-nominations are allowed.
Candidates need not be a PyPA committer.
* Phase 3: Each PyPA voting body member can vote for zero or more of the candidates.
Voting is performed anonymously. Candidates are ranked by the total number of votes
they receive. If a tie occurs, it may be resolved by mutual agreement among the candidates,
or else the winner will be chosen at random.
Each phase lasts one to two weeks, at the outgoing councilâs discretion. For the
initial election, all three phases will last two weeks. The election for all
subsequent councils will start in the 12th month since the previous council election.
The election process is managed by a returns officer nominated by the outgoing
Packaging council. For the initial election, the returns officer will be nominated
by the PSF Executive Director.
The council should ideally reflect the diversity of Python Packaging contributors
and users. PVB members are encouraged to vote accordingly.
Term
~~~~
A new council is elected once every year. Each councilâs term runs from
when their election results are finalized until the next councilâs term
starts. There are no term limits. Generally, each council memberâs term
should last 12 months with one exception described below.
Vacancies
~~~~~~~~~
Council members may resign their position at any time. There could also be
situations that council members have been removed from the council via a
vote of no confidence.
Whenever there is a vacancy during the regular council term, the council
may vote to appoint a replacement to serve out the rest of the term. In
such a situation, this council seat will be up for election at the next
council election as long as the replacement has served 12 weeks or longer
in the council. Therefore, the longest term for any council member will be
15 months before that seat is up for election.
If a council member drops out of touch and cannot be contacted for a month or
longer without prior notice, then the rest of the council may vote to replace them.
Conflicts of interest
~~~~~~~~~~~~~~~~~~~~~
In order to avoid any appearance of conflict of interest, at most
2 members of the council can work for any single employer.
In a council election, if 3 of the top 5 vote-getters work for the same
employer, then whichever of them ranked lowest is disqualified and the
6th-ranking candidate moves up into 5th place; this is repeated until
a valid council is formed.
During a council term, if changing circumstances cause this rule to be
broken (for instance, due to a council member changing employment), then
one or more council members must resign to remedy the issue, and the resulting
vacancies can then be filled as described above.
Ejecting PyPA committer
~~~~~~~~~~~~~~~~~~~~~~~
The Packaging Council or PyPA member may initiate a vote to
eject a member from the PyPA committer body. A council member
or PyPA committer can put forward a proposal and call for a vote
on a public PyPA communication channel. A PyPA committer vote
is triggered when a PyPA committer (not the proposer) seconds
the proposal.
The proposal will be put to a vote on the
`PyPA-Committers <https://mail.python.org/mm3/mailman3/lists/pypa-committers.python.org/>`_
mailing list, over a 7-day period. Each PyPA committer and council member
can vote once, and can choose one of +1 and -1. If at least two
thirds of recorded votes are +1, then the vote succeeds.
Ejecting PyPA project
~~~~~~~~~~~~~~~~~~~~~
The Packaging Council or PyPA member may initiate a vote to eject
a project from the PyPA. A council member or PyPA member can put
forward a proposal and call for a vote on a public PyPA communication
channel. A PyPA committer vote is triggered when a PyPA committer
(not the proposer) seconds the proposal.
The proposal will be put to a vote over a 7-day period. Each PyPA
committer and council member can vote once, and can choose one of +1 and -1.
If at least two thirds of recorded votes are +1, then the vote succeeds.
A project can also choose to leave the PyPA. If a project is leaving the
PyPA or has been ejected from the PyPA, it is the responsibility of the council
to support the transfer of the GitHub repository out of PyPA to a personal repository.
Ejecting an affiliate project voter
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Any Packaging Voting Body member or council member may initiate
a vote to eject an affiliate voter from the Packaging Voting Body.
A council member or PyPA committer can put forward a proposal and call
for a vote on a public PyPA communication channel. A PyPA committer vote
is triggered when a PyPA committer (not the proposer) seconds the proposal.
The proposal will be put to a vote on the PyPA-voters mailing list, over
a 7-day period. Each PyPA voting body member can vote once, and can choose
one of +1 and -1. If at least two thirds of recorded votes are +1, then the
vote succeeds.
Vote of no confidence
~~~~~~~~~~~~~~~~~~~~~
Any PVB member or Packaging Council member can publicly call
for one or more Packaging Council members to be removed from the Council
via a vote of no confidence.
The vote of no confidence should be called on a project communication
channel and should be seconded by another PVB member.
The vote lasts for two weeks. PVB members can vote for or against the
removal. If at least two thirds of voters express a lack of confidence, then
the vote succeeds.
If the vote of no confidence is for a single member, the council member is
removed from the council and the vacancy is filled as described above. If
the vote is for the entire council, the council is dissolved and a new election is held.
PyPA committer
--------------
Role
~~~~
Similar to the Python core team, the PyPA committers is a group
of volunteers who maintain and support PyPA tools. They have
authority over the Python Packaging infrastructure, the Python
Packaging GitHub organization and repositories, the bug tracker,
the mailing lists, IRC channels, etc.
Prerogatives
~~~~~~~~~~~~
PyPA committers may participate in formal votes,
typically to nominate new PyPA projects,
and to elect the Packaging council.
Membership
~~~~~~~~~~
Any Packaging project can request PyPA membership.
A PyPA member can put forward a proposal to add a project
to the PyPA and call for a vote on a public PyPA communication channel.
This proposal must not be opposed by the existing maintainers of the
project. A PyPA committer vote is triggered when a PyPA committer
(not the proposer) seconds the proposal.
The proposal will be put to a vote on the PyPA-Committers mailing list,
over a 7-day period. Each PyPA committer can vote once, and can choose
one of +1 and -1. If at least two thirds of recorded votes are +1,
then the vote succeeds.
Once a project has been added to the PyPA organization, the project
falls under the purview of the PyPA and will be required to meet the
guidelines as set by the PyPA.
As Packaging contribution requires support and time, it is the
responsibility of the Packaging Council to ensure there are sufficient
support mechanisms in the form of (but not limited to) mentorship, internship
and fellowship to support and guide new PyPA contributors. The Packaging Council
may work with the PSF to establish such programmes.
Acknowledgements
================
This PEP is based on `PEP 13 <https://peps.python.org/pep-0013/>`_
which in turn is based on a Django governance
document authored by Aymeric Augustin.
Given that thereâs no diff (and even if there were, diffs arenât that easy to read) would you mind answering those questions here, so that people can see your responses without having to re-read the whole PEP and try to spot what changes were made?
I think the way that people expect to see the changes is in commits to the repo, where they can see them rendered, see the diffs, and so on. This also ensures that all the changes are tracked as part of the repo. Itâs neither robust nor convenient for readers for the changes to be made via edits on a web forum post, where (due to the constraints of this forum software) they are not presented in a readable format.
For what itâs worth, when I click on the âeditedâ button at the top of the post, I get a history of the post with each revision and changes highlighted. It requires some scrolling around to find things, but itâs not that opaque.
edit: specifically, the unified HTML version allows you to scroll through the whole post and see changes. The side-by-side version fails to synchronize scrolling in the embedded text snippet.
Hereâs a Git diff from the last one: s-mm/peps#1
All HTML diff views look quite broken. Only clicking âRawâ helps. Apparently itâs a known problem with discourse.
Also, diffs just say how the text changed. Iâm asking for a bit more than that - some of @smmâs thinking around what the intention is behind the changes. In programming terms, the spec for the change, rather than just the code diff.
Iâve not been able to follow the discussion here super closely due to time constraints, but I wanted to weigh in a little bit with my thoughts.
The tl;dr here is that I think what we need is a decision making body that has the following properties:
Which I think should generally take the shape of an elected council, with a mission that is something like:
Own, Define, and Improve the Python Packaging Story and Toolchain.
I think their broad powers should stem from being the ultimate owners of the definition of what the Python packaging story is, including ownership of the default set of tools. The assumption is good faith would of course have to come from us, giving them broad powers without trying to rules lawyer specifics about what we expect to see from them or what they are and are not allowed to do, and the escape hatch would be some sort of no confidence / removal procedure.
Thereâs a lot that goes into why I think this is the correct thing, and I guess Iâm turning into an old man because I feel compelled to tell a story about âback in my dayâ here, but I think the easiest way to understand why I think this is to understand the perspective that Iâm coming from.
Iâve been involved in Pythonâs packaging ecosystem for⌠well over a decade at this point (and of course many people have been involved longer), and the world that existed when I first got started was very different than the one we have today, but in many ways I think the same underlying problem still exists and is still plaguing us today.
The packaging landscape back in the early 2010s was very different. The historical âpackaging toolâ was distutils, which is where the interfaces that still exist today originally got defined. However, distutils was closer to Make than a package manager, so we had setuptools which had brought in easy_install, the ability to define dependencies, locate them on PyPI, download them, and install them. This had come out of the needs of a single project at the time, and attempted to be a drop in replacement for distutils (with a bootstrapping story of "stick this other .py file in your sdist and import it and run it).
Many projects had shifted to using setuptools and easy_install, but the way that setuptools/easy_install worked from an end user point of view wasnât how many people wanted to work, so virtualenv and pip came along as a competitor to easy_install and gained popularity.
Eventually maintenance of setuptools had dropped off, and itâs problems (and of course the problems of distutils) were becoming more and more apparent and problematic. However, since maintenance had dropped off, it was difficult to get any changes made to setuptools, so it eventually got forked into âdistributeâ which further split the ecosystem between distutils, setuptools, and now distribute. There were also things like numpy.distutils and such which existed, but were much less widely used.
There was of course also PyPI, which was also quite different back then, but mostly in ways that donât matter here.
Now, thatâs the rough state of the tooling in the early 2010s (thatâs not every single tool, but those were the major players), but a very important missing element here is that all of these various tools were being built and maintained by completely different people, but they all were very interrelated to each other and depended on each other (including down to implementation details) which made the entire toolchain fragile and hard to change.
Worse than that though, is that the landscape that these tools existed in was very adversarial. At the time discussion around packaging largely took place on two mailing lists, distutils-sig and catalog-sig, which technically were separate lists but for our purposes weâll just lump them both together as âdistutils-sigâ.
The distutils-sig mailing list was, at the time, widely considered one of the most toxic places in the Python community. Iâve done a lot of reflection as to why it was the way it was over the years, and I think you can distill it down into:
This created a situation where nobody was happy with the status quo, but opinions were very split on what the right path forward was, with no clear mechanism for resolving this stale mate. So every time someone had an idea of how to improve something, the process looked something like:
In this landscape, a few packaging PEPs had managed to make it through (some of which we eventually did adopt and start to use!) but they were mostly ineffectual, because there was nobody had ever agreed to use the PEP process for this purpose, so the maintainers of those tools largely just ignored those PEPs.
So this is our world, no single tool, in fact multiple competing tools, no communication or ability to coordinate between them, no decision making process besides unanimous consent, which was all but impossible to get, and the only thing resembling a decision making process that could be used, the PEP process, is being wholly ignored.
What ended up breaking this cycle was a a combination of a few things:
That brought us to where we are today, which (and Iâm biased of course), is such a significant improvement over the world of of the early 2010s in ways that are hard to fully articulate, but which still has a lot of the same fundamental problems. Or rather, in my opinion, still has the same fundamental problem.
See the problem back then was we didnât have a mechanism for making broad, ecosystem wide decisions, and well, thatâs still our problem today. Weâre in a better position today than we were, because we have a process for making some kinds of these decisions, but other kinds weâre still limited to the âpost your idea, people comment in support or opposition, the discussion peters out and goes nowhere because thereâs no decision making process besides unanimous consent, which weâre never going to getâ experience.
I personally think the standards process weâve settled on has been very successful, and I think in large parts thatâs because it roughly follows the properties I laid out in the start of this post:
At the time, bringing all of these projects together and getting any agreement on standards was an ambiguous, and ambitious mission.
The powers of the two PEP delegates are also very broadly defined here, we have âAnything to do with PyPI or the Repository APIâ and âAnything to do with interoperability standardsâ. We never sat down and tried to narrowly iterate what @pf_moore or my powers were in those roles, or even exactly what either of us would attempt to accomplish by our decisions in those roles, there was just an assumption that we were both reasonable people who cared about this ecosystem, and we would generally do our best to make good decisions. Ultimately though, weâve always had an escape hatch incase that wasnât true, projects could refuse to implement something or we could get the standing delegation removed.
Even now, thereâs nothing thatâs stopping us from just saying that these broader, non standards questions that weâve run into donât just fall under one of the existing delegations, except that the people involved are all reasonable people who generally agree that that is further than the general consensus of what these roles are currently, so we donât do it.
There is an important, but subtle bit of power dynamic at play here too. Itâs not a mistaken that I am the standing delegate for PyPI PEPs and Paul is the standing delegate for packaging standards PEPs. Iâm one of the core developers for PyPI (and for a time, I was the core developer for PyPI) while Paul is one of the core developers for pip. A lot of the implicit power that resides in our current decision making process stems from that, chances are really high that both PyPI and pip are going to implement and/or enforce whatever decisions we make, because in most cases the people making those decisions are also in the position to enforce them.
This bit of subtle power dynamic I think is a big part of why the system we have was able to exist and actually work (remember, there were other packaging PEPs that largely got ignored prior to all of this, and I think thatâs in part because those PEPs lacked this power dynamic), and I think that itâs important that we recognize that, and try to extend that to whatever new system we come up with (if we do in fact come up with something).
But I also think there is a bit of a double edged sword here with that, because I think we need to be careful that we donât create a world where people have extra incentives to avoid making hard decisions. To use myself as an example, if someone came up with a PEP that completely replaced the idea of PyPI with something else, that is something where I would be incentivized to say no, even if saying yes is the right thing, for a variety of very human reasons.
That was a lot of words, but hopefully it provides at least an interesting, if not useful perspective.
To reiterate from above, and provide some more detail, hereâs what I think we want:
I think that singular, long standing delegations work reasonably good for technical decisions, but they have problems with larger questions. Weâre somewhat limited to a direction that either myself or Paul are generally OK with, even if weâre wrong and the community thinks we should be going in a different direction.
Moving to an elected council means that ultimately the community is in charge of the direction we take packaging, through the nature of being able to elect people to the council who want to shape things in the way the electorate wants.
Ambiguous goals and broad powers means that we donât have to try and flesh out all of the answers right now. I see questions like:
As the wrong level of detail for what we put into the governance, because if we could define those already, we wouldnât actually need this at all, but also because the answers for those things may change over time and ambiguity here allows the council to evolve over time to meet the needs of people in a decade from now.
In other words, all we really need to define is that we want this council to have the powers to decide on these kinds of things, and then let the voting and council set the exact questions that need answered.
I think it is important that the council âownsâ the default tooling, whether they are involved in the day to day maintenance of it. This is probably one of the âscarierâ things in this, but I think that people tend to worry about the worst case possibilities too much. What does it mean for the council to own a project? It means that ultimately they can override decisions that the day to day maintainers make, including removing those maintainers.
Of course we live in the real world, and these tools are largely maintained by volunteers, so there are limits to what the council can dictate, but I donât think we need to actually sit here and try and come up with some sort of legal document or exhaustively document every possible way this fundamental contradiction might expose itself. The council are not employing these maintainers nor are these maintainers slaves to the council, so at the end of the day if the council wants something done, and a maintainer doesnât want to or canât do it, the council will have to find another way to make it happen, but the key thing is that they have the power to make it happen.
I donât want to pick on setuptools, because I donât think that the recent situation was exactly setuptools fault [2], nor do I think the particular problem is nearly egregious enough to warrant something particularly aggressive from the council, but just as an example with the recent issue around name normalization. If we had the council at that point, they could have simply wrote their own PR to setuptools to bring it in line with the standard as it existed now, and merged it against the wishes of the setuptools maintainers.
Of course, doing anything against the wishes of the maintainers risks angering, and maybe even losing those maintainers, but I donât think that we need to spell that out, because I think the protection against that is the assumption of good faith in the part of the council, and the ability to remove them if they are violating that good faith.
I think that we can assume that anyone elected to this council will be aware of that, and we expect them to generally use this power rarely, if ever at all, but I think it is important that they have this power, because I think thatâs in part the power dynamic that the rest of their softer power flows from (much like how currently that implicit power exists through nature of Paul and myself positions within pip and PyPI).
I think it would be reasonable to explicitly call out that we expect the council to operate by building consensus and incentivizing behavior rather than top down dictation wherever possible, so called âsoftâ power, and using their âhardâ power sparingly, as a method of last resort.
In my mind, I think the hardest question we have to answer right now, because think all the other really hard questions should be left up to the council, is who are the voters who vote on the council, because that is going to have a large impact over who has the power to set the direction of the ecosystem.
In theory I would love it if we had fully open elections, anyone who cares enough can vote. Unfortunately in practice open elections on the internet donât really work, and you need some mechanism for defining a voter roll.
My second preference was that we use something like the PSFâs contributing members, where people can self certify as meeting some requirements, but Iâm told that managing that process for the PSF is a nightmare, and logically it does essentially boil down to open elections unless you have someone continuously verifying all of the certifications since anyone can self certify, whether they actually meet the requirements or not.
Where I personally ended up landing (and this may not be the right thing either!) is something similiar to the CPython Core Developer process, which is basically âan existing core dev nominates you, with a time limited vote amongst the existing members, and if a majority votes yes, then youâre a voting memberâ. I think we would be able to just explicitly say that anyone who âparticipates or is involved in building the packaging ecosystemâ would be eligible, which we would define to include things like people who are core devs of packaging tools (whether official or not), people who participate in the packaging discussions on discourse, people who educate users about packaging tools, etc.
Itâs up in the air whether being a core dev for a âdefaultâ tool would imply automatic voting member or not, I suspect it doesnât really matter because I doubt weâd get many No votes for anyone who is actually involved, so it may be simpler to just not imply automatic voting rights and that it just means youâre eligible to ask to be a voter.
Of course we would need an initial list of voters to seed the member list with, and I would say we can just use the existing list of PyPA committers for that, maybe with a period of time before the first election for non PyPA committers to get a chance to ask to be members and get voted on.
One last bit in this already way too long post, I think it may be a smart idea to go about this in a slightly different way.
Right now, the process for migrating from where we are, to this hypothetical new world is to essentially âweâre going to redefine what the PyPA isâ.
I honestly think thatâs technically fine, but I think there may be some optics and political reasons to switch that around, and instead say "weâre going to dissolve the PyPA, and create a packaging council [3]", and projects currently under the PyPA will need to move to be their own projects, and if the council decides to define a default toolchain, they will need to either create it or work with the tools they want to adopt to own it.
This has a few nice benefits:
However, it does have one big risk:
I think that the council will have to live in reality where there currently is a default toolchain, whether they agree with that toolchain or not, which means there are a set of tools that should exist in the council right from the start if we want this all to work (PyPI for instance is a big one, likely pip as well, probably need a build backend or two too).
The question then becomes, if we decide we want this, do we setup all of the machinations for electing the council, elect them, etc, and then risk having the de facto set of âdefaultâ tools decide not to accept council ownership? Or do we try and define what the âinitialâ de facto toolchain is, and make sure that those projects are OK with it as part of moving forward.
Personally I would suggest defining a minimal set of tools that make up the initial de facto toolchain (PyPI and pip almost certainly, and we would need to survey what build backends are being used but I would guess setuptools is still the most widely used one and makes sense, but thatâs just a guess). Make sure those tools are OK with being owned by the council, and then saying everything else is TBD by the council.
Over time, if the council decides that those tools arenât the right tools to continue to be the default, then they will be able to set that direction, sponsor or adopt a different tool, and deal with the process of how to migrate a community, and then ultimately retire (likely with the option for the existing maintainers to take over ownership outside of the council if they want to).
That was a lot, sorry 
Hopefully folks found it interesting and/or useful!
And I donât care what the proposal was, because at this point distutils-sig had gotten so adversarial that every change was controversial to someone, so every idea had some people arguing against it. âŠď¸
I think it was basically a confluence of locally reasonable decisions that created a bad situation where consensus had diverged without giving setuptools a chance to weigh in, and so they weighed in after the fact by refusing to implement a change to a standard. âŠď¸
I donât care what we actually name it, just something new. âŠď¸
This is an excellent summary, and overall I agree with most of what you say. However, I do think that the question of âownershipâ remains the one fundamentally hard problem here.
The issue is simply that there is no one thing to âownâ. The Python Steering Council owns CPython, which is a single thing (although even there, the SC doesnât have direct override authority, they can say âCPython will do Xâ but the core devs still need to implement that). I canât think of any other equivalent group to the packaging council. Thereâs no âPython Web Councilâ, just project specific groups like the Django board. Thereâs no âPython ML/AI Councilâ. Thereâs no board with overall authority for âscientific Pythonâ.
So weâre trying to create something brand new here, which is a council with overriding authority over multiple projects, and the council decides which projects. Itâs simply not going to be possible for the council to
Own, Define, and Improve the Python Packaging Story and Toolchain.
if they have to exclude significant solutions in the packaging area (such as conda and poetry) because those projects arenât willing to be âownedâ by the council[1].
Much as I have a personal dislike of conda, I think itâs entirely reasonable for the council to decide that conda is a key component of the packaging toolchain (for certain classes of use). So how would that square with the mandate to âOwn⌠the Python Packaging⌠Toolchainâ?
To give another example, I think itâs quite possible that the council will want to define the scope of Python Packaging to include application deployment. A key tool in that area is PyInstaller. How would the council make a statement that âPyInstaller is the key part of the packaging toolchain for building standalone applicationsâ without asserting ownership over PyInstaller? And if that is OK, why canât pip, PyPI and all the various other projects that are part of the toolchain, also decide to not accept council ownership? After all, what do they gain from being owned by the council?
Thinking of the packaging council as beling âlike the SCâ is a huge mistake. The SC was created by the Python core developers, the exact group that the SC were going to have authority over. In contrast, the packaging council will be created by the packaging community (or the voting body, however thatâs constituted) to have authority over a somewhat-unspecified group of projects, who have no means of voting directly over whether they even want to be governed by the council. And the council will have a percieved authority which will be completely undermined if they canât exercise the authority that the voters think it has.
Iâm assuming based on previous history that conda wonât agree to such ownership, and Iâm taking poetryâs unwillingness to be a member of the PyPA as a similar indication. But even if the details change, the point remains - the council canât define the complete packaging story if important projects donât want to participate. âŠď¸
Certainly the most significant thing the council could own is the default shipped tools, and the power of official docs/websites.
Is that a strong enough incentive for tool authors to be open to standards/syntax/workflow council decrees?
Cpython makes you sign a CLA when contributing code. It would be understandable if the Packaging council said âwe want to make your tool the default tool, but hereâs the requirements, for stability and future maintainability.
I donât know. I get the feeling that conda wouldnât agree. And no-oneâs even thought to ask tools like Beeware or PyInstaller. And why would tools like poetry and PDM that have chosen not to be part of the PyPA choose differently when considering whether to accept council authority (a far more significant commitment, if I understand the proposal)?
Speaking as a pip maintainer, I doubt weâll get a choice (pip will be expected to be under the council, to an extent that I donât think would make it credible for us to say no) but I honestly see no benefit to us from being owned by the council. And yet, itâs not like the council could choose not to make pip the default installer at this point.
And in any case, I think itâs an extremely bad idea for the council to be refusing to support or recommend the best tool for users simply because that tool isnât willing to transfer ownership to the council. Conda users are already frustrated enough that the official Python packaging documentation doesnât give conda sufficient prominence. Telling them âwell, itâs because conda wonât agree to be owned by the councilâ isnât going to help reduce friction in the packaging community 
How would being owned by the council provide this? It doesnât gain us additional maintainers[1], or get us any resource we donât already have access to. This feels like itâs just optimism without anything concrete to back it.
At least, not ones who have established credibility with the project. âŠď¸
For âtoolsâ (with a fuzzy definition), the situation doesnât read all that different from making the functionality of certain packages part of the Python standard library (e.g dataclasses). If the Council decided that e.g. PyInstaller-like functionality should be part of the Python packaging toolchain, it should incorporate that into the Python packaging tool(s). Everything that is third-party remains third-party, itâs the creation and maintenance of an official first-party tool or tools that would be new (even if they would largely agree with current PyPA tools).[1]
For âenvironmentsâ such as conda, I imagine there is ample scope for making the integration with the Python packaging tools more natural. You could imagine efforts to make conda more interoperable with pip, defining the standards for non-Python dependencies (e.g. libqt5 and so on), setting rules around vendoring on PyPI, etc. I donât see how the mere existence of conda would be an argument against the points laid out so carefully above.
If âcreating a single packaging experienceâ is the goal of the Council. âŠď¸
Despite being one of the maintainers of a build back-end I didnât pay attention to the discussion about Python packaging governance. However, this post form @pf_moore leaves me a bit worried.
I donât understand why there is the need to create an entity that âownsâ the packaging tools. In a broad sense, there was such an entity already (in the form Steering Council as the body having decision power to what is part of CPython, and distutils was part of CPython) and the process to get rid of it (removing distutils from the CPython distribution, so effectively removing Steering Council decision power over it) just concluded. I donât see why an identical structure needs to be created under another name and with a much weaker mandate.
I though that what was being discussed was a way to get all the players in the field to agree on how to move the packaging standards forward, not to create a body that will âownâ solutions. I donât see any push back from authors of maintainers of packaging tools in implementing the PyPA standards, thus I donât see the reason of creating an entity whose constitution seems designed to force these standards on reluctant maintainers. On the other hand, the the PyPA standard documents are not in a great shape, which makes implementing the standard a matter of guessing (and ultimately conforming to what setuptools has been doing since the beginning of time, when applicable).
I have the impression that defining âownershipâ of the projects is a way to decide that the standard is defined by how it is implemented by the tools âownedâ by the Python Packaging governance entity. But this is just repeating the distutils saga again, under slightly different circumstances.
I really hope that the direction for Python Packaging governance is revised toward something more a consensus based decision model around the definition of standards, hopefully with a strong focus in making the existing standards much more strictly defined, rather than something based around the idea of âcode ownershipâ. I am in favor of reference implementations, but these need indeed to be just a reference, not the only blessed solution.
I donât think itâs an entirely new thing in the software world, and I donât even think that itâs a new thing in the Python space.
Yes, the SC primarily officially owns CPython, but that ownership gives them tons of power that allows them to apply requirements to other related projects. For instance, mypy isnât owned by the SC, but the fact that the SC owns the typing PEPs through their ownership of CPython means that thereâs weight behind the SC decisions.
On the rust side, they have the âdevtools teamâ, which is broken up into smaller teams like the âCargo teamâ, the âclippy teamâ, etc. So the individual projects are owned by the overall dev tools team (which is itself owned by the rust project), and theyâve all agreed to use their RFC process for everything, not just standards but pretty much all major decisions.
So thereâs a few things about this, and I donât have a perfect answer, but a few random thoughts:
I assume PyPI / pip would need to be the âdefaultâ for now anyways, it would be up to the council what to do from there. If PyPI/pip wanted to not be owned by the council, I think both projects get that choice, but if either say no, it effectively means that the idea of ownership within the council is DOA, and we just simply canât include that.
I think that this is a mischaracterization or misunderstanding of what is being proposed and the reasons for these discussions.
We already have a process for defining standards and it works reasonably well. If the solution to the problems weâre having were just to define another standard, then this discussion wouldnât exist.
The problem weâre running into is that the only thing we have a process for decision making with is standards, so weâre forced to try and turn everything into a standard, even if it doesnât make sense for it to be, and anything that canât be hammered into a standards shape we just sit here and endlessly discuss with no forward movement. Thus the governing body is designed to provide a means to make those much larger or vaguer decisions.
I also think that the idea of code ownership is a relatively small part of the overall idea, and it could be done without it. However, I do think we lose something without it. The key thing is we effectively already have this in the current system, we just have it implicitly. PyPI and pip tend to just implicitly act as the enforcement mechanism, which in large part stems from the fact that Paul and Myself are the default decision makers for those PEPs, and also can ensure that . The code ownership aspect of this idea, is, in my opinion simply formalizing that existing implicit relationship.
I do not think that ownership has anything to do with trying to decide standards through implementation defined standards vs actual standards. Nor do I think that it means that weâre repeating the distutils saga (a and I donât think the distutils problems had anything to do with ownership). Nor do I think that anything in this means that there will no longer be consensus based decision making.
The ultimate question becomes, when we canât come to a consensus, what do we do? Right now the answer is that we frustrate everyone involved by long, never ending discussions that will never go anywhere where everyone agrees that something should happen, but nobody agrees on anything else.