PEP 772: Packaging governance process

Packaging Coordination
1

On behalf of my co-authors @Deb and @pradyunsg, we are finally happy to announce PEP 772, which proposes to establish a more formal governance process for Python packaging. Our thanks to the many folks who also provided valuable insight and feedback during the drafting process, and the PEP editors who reviewed the PR.

Packaging is a crucial element of the Python ecosystem, and arguably one of the biggest factors contributing to Python’s success. It’s also incredibly complex, touching libraries, services, third party tools, and more. The packaging landscape is ever evolving, and there is a need to establish this more formal decision making process. PEP 772 builds on PEP 609, and the Packaging Working Group, and is modeled in large part on PEP 13, the governance document for the Python language.

We look forward to your feedback.

29 Likes
2

Thanks for working on this! On the whole the proposal looks good and well thought out.

About adding a new member:

Members are added to the Packaging community by a simple majority vote by the current membership. Quorum for adding new members is 50%.

A vote to add a new member is triggered when a Packaging community member calls for one publicly on an appropriate communication channel, and another Packaging community member seconds the call within two weeks.

The vote lasts for two weeks. Packaging community members vote for or against.

Two weeks feels a long time, compared to the one week vote for Python core membership (PEP 13):

It is granted by receiving at least two-thirds positive votes in a core team vote that is open for one week and is not vetoed by the steering council.

One week feels about right for core team votes (and still feels a long time when the vote is about your own potential membership!), perhaps also use one week for Packaging community votes?

1 Like
3

This is an incredible step forward to make Python Packaging move forward.

Thanks to everyone involved! I thumbs up everything proposed here :ok_hand:

3 Likes
4

I appreciate the work that’s gone into this so far! Great to see this coming together. Just want to re-surface the one concern that @zanie raised in the PEP PR itself, which is that this list of initial members would (on the surface) omit many of us working on uv:

  • PyPA members: Anyone with the triage bit or commit bit, or at least one project in the PyPA organisation.
  • Packaging workgroup members: Anyone who is listed on the Packaging WG charter will be moved into the Packaging community.
  • Interested core team members: Any Python core team member who is willing to participate is welcome.
  • Wider community members: Non-profit organisations that participate in packaging or working with new packagers. For example, PyOpenSci, NumFocus, Django, are encouraged to initially nominate up to seven members by sending an email to [todo].

For example, I wouldn’t qualify under any of these bullets. Neither would @konstin, who was the original author of maturin, which is an important packaging tool but is not under the PyPA umbrella and is not stewarded by a non-profit. So, I’m mostly looking to understand whether the criteria is structured this way intentionally or whether it should be amended to accommodate those of us working on OSS tools outside the PyPA and outside of a non-profit :slight_smile:

8 Likes
5

Thanks for opening this! I concur with the sentiment that this looks good overall, and I think it’s a huge step in the right direction for packaging.

I have questions re: two responsibilites:

  • Have broad authority over the Python packaging standards and Python Packaging User Guide, that are maintained on https://packaging.python.org.

Am I correct in interpreting this to mean that the Packaging Council will effectively replace the current sponsor/PEP-Delegate system for packaging PEPs?

That makes a lot of sense to me if so (and is a reasonable change, IMO), but it might be worth spelling out explicitly, since there’s some ambiguity around how PEP approval is delegated for packaging PEPs (the current PEP language says that the SC will delegate decision making, but it’s also de facto delegated per PyPA Specifications — PyPA documentation).

  • Establish processes for making binding decisions regarding packaging standards, tools and implementations as well as for considering ecosystem-wide changes.

Out of curiosity, are there binding decisions that you (or others) envision the Packaging Council making, outside of PEP acceptances themselves? I can think of a few categories of things that could be useful to have binding decisions on[1], but I’m curious if there were specific things you had in mind!

  1. Such as officially “blessing” tools beyond their PyPA affiliation, or independently declaring tools as fully compatible with relevant PEPs. Both of these could be valuable as formalizations of e.g. the existing CPython-ensurepip-pip relationship. ↩︎

1 Like
6

So glad this is finally happening!

You missed a unique opportunity though – the group should really be called Python Packaging Cabal, to continue the trend I set last year to make all groups use a different group noun (Typing Council, C-API Workgroup, and Editorial Board).

Or you could use another group noun, e.g. Pack (the PPP!).

22 Likes
7

Initial membership in the Packaging community will include anyone who has taken the time to formalise their participation in the Packaging community

I assume this is a submission process and not automatic? Is there an idea yet on how this submission process be communicated to the wider community (those not closely following PEPs)? And who is running it?

I wouldn’t qualify under any of these bullets. Neither would @konstin, who was the original author of maturin, which is an important packaging tool but is not under the PyPA umbrella and is not stewarded by a non-profit. So, I’m mostly looking to understand whether the criteria is structured this way intentionally or whether it should be amended to accommodate those of us working on OSS tools outside the PyPA and outside of a non-profit :slight_smile:

The language here is a little confusing the bullet point is “Wider community members” but then immediately describes it as “Non-profit organisations”, if it’s meant to be limited to such organizations I think the bullet point should be reworded to reflect that, if it’s not meant to be limited it should give more examples with an “etc.”

On that note, there are many for-profit organizations, besides Astral, I would be quite happy to see members paticipate in the packaging governance process (e.g. Anaconda, JFrog, Azure, AWS, etc.).

In order to maintain a reasonable expectation of quorum, failure to participate in Packaging Council elections for two consecutive council elections automatically removes a person from the list of voting members, until they re-submit their intention to resume their participation to the Packaging Council in writing.

Unless I’m missing something, there’s no listed way for someone to remove themselves as a member except by not voting in Packaging Council elections?

7 Likes
8

I’m very much against the use of this word. ‘Cabal’ has a problematic etymology that links the Jewish mystical tradition of Kabbalah with an idea of a powerful secret political group that is involved in nefarious conspiracies, which is by itself a centuries-old deadly antisemitic trope. The word has a long history of being used as a racist dog whistle, and even in a completely innocuous context (such as the title of the packaging group) it has the potential to make people uncomfortable.

I know that not that many people are aware of this history, and certainly don’t mean to imply it was used in your post with any harmful intent. Still, I think it’s important to steer clear of such terms, at least in professional settings.

To give a few sources: the American Psychological Association’s ‘Inclusive Language Guide’ has a paragraph about this word under the heading ‘Avoiding microaggressions in conversation: Culturally appropriative and pejorative language’. It is also included in the American Jewish Committee’s ‘Translate Hate’ glossary.

5 Likes
10

Looks good to me! Thanks for working on this.

2 Likes
11

Your feedback is noted. Let’s not derail the thread, please.

For context, this was a joke based on the long-standing usage of the term on the Internet. The Haskell package manager is called that. The Python testing group that introduced mock under the late Michael Foord used to be called that.

Sure, names evolve. The Python Secret Underground, which emphatically does not exist, used to be called the Python Cabal, which also, equally emphatically, did not exist.

We’ll keep this in mind. But let’s not forget context here. And most importantly, let’s stay on topic.

12 Likes
12

I’m going to try to do a few things in my responses to this thread.

  • I will be keeping notes of the changes and suggestions so that I can discuss with the other authors and post occasional summaries, knowing how easy it is for DPO threads to get overwhelming
  • I’m going to respond as a co-author, usually with my own thoughts. Where the co-authors have discussed them, I’ll try to note that
  • While writing this PEP (and while the PR was being reviewed), there were points we were less certain about, and were hoping could be discussed here to reach consensus
  • Obviously if this PEP reaches SC approval phase while I’m in office, I will abstain from the vote.

PEP 772 tries to model PEP 13 where it makes sense, and I think the one week vote works well in that context. The way PEP 772 is written today, it could be up to 4 weeks to accept a new member (two weeks for seconding, two weeks for voting). That does seem like too long to wait. We can keep the two week seconding, since that gets short-circuited as soon as that second arrives (and I suspect in practice, it’ll be nearly instantaneous). A one week vote here makes sense to me, unless there’s a good reason to stick with the two week vote, in which case, the PEP should explain the reasoning.

Thanks @charliermarsh and @zanie for lifting that topic from the PR to this thread.

My personal opinion is that any packaging community membership that doesn’t represent and include your participation would be incomplete. You’re all clearly “wider community members” even if you’re not associated with a non-profit. On the flip side, I think there’s the usual concern about commercial backed members overwhelming volunteer OSS contributors[1]. Maybe the “seven member” rule is enough to alleviate those concerns. It’s a delicate balance, so getting the wording right is important. Do you have any suggestions for such wording? What would folks think about just removing the words “Non-profit” from this bullet point? Does that strike the right balance?

Yes, that’s the intent.

PEP 772 in the section on Delegations says:

The Python Steering Council is expected to delegate decision making to the Packaging Council for PEPs related to the Python packaging.

I can see a couple of clarifications that could be added here.

  • Explicitly state that SC approval of PEP 772 includes transfer of standing delegations for both existing Packaging PEP-Delegates to the Packaging Council.
  • Update the PyPA documentation to reflect the new order.

Would that cover it?

I think these are areas of responsibility that will be worked out by the PC once it’s established. We deliberately didn’t want to go into excessive detail in the PEP so as to leave room for the PC and the community that elects it, to decide the scope.

Personally, I think the PC is in a somewhat different position than the SC here. The SC has mandate, responsibility, and authority to decide “what is Python”, i.e. the language, the CPython implementation, and the stdlib. The PC has in some sense a more complex relationship with standards and projects under the PyPA umbrella[2]. E.g. it can’t really mandate how independent workflow managers (e.g. Hatch, PDM, Poetry, uv) must work. In my mind the PC’s most powerful influence is in consensus building and PEP pronouncements, but I also personally think that packaging is so much more important to Python’s success today than its ever been, so a more active stewardship and coordinated, focused direction, and clearly established priorities are crucial[3].

So, rename PEP 772 to “The Python Packaging Pack Pact”? :smiley:

  1. and to be clear, I am not casting aspersions here! ↩︎

  2. perhaps it’s more akin to the SC not having direct influence over alternative implementations of Python ↩︎

  3. I guess that’s obvious, otherwise I wouldn’t have co-authored this PEP! :smiley: ↩︎

6 Likes
13

Admittedly, not so much. I think we need to nail this down logistically, so feedback is welcome.

PEP 13 says:

in order to provide the general public with a reasonable idea of how many people maintain Python, core team members who have stopped contributing are encouraged to declare themselves as “inactive”. Those who haven’t made any non-trivial contribution in two years may be asked to move themselves to this category, and moved there if they don’t respond. To record and honor their contributions, inactive team members will continue to be listed alongside active core team members; and, if they later resume contributing, they can switch back to active status at will. While someone is in inactive status, though, they lose their active privileges like voting or nominating for the steering council, and commit access.

PEP 772 may need similar language.

There’s also an open ticket on the core-workflow tracker which proposes to be more proactive about removing the commit bit, and thus core dev privileges, for inactive members. This is largely driven by security concerns, but does have implications for voting rights. I don’t know if that same model would work for the packaging membership, but I do think it’s useful for quorum and such to keep the voter rolls accurate. In the core dev case, effectively inactive members can very easily request reactivation, no questions asked.

1 Like
14

I assumed that the intent of that section was mostly about initial “seeding” of the community, and the normal route for new members was nomination. So to an extent, it makes little difference in the longer term (assuming no-one is expecting a bad-faith attempt to block membership on the part of the initial community members). With that in mind, I’m more than happy (as a PyPA member and therefore a member of the community by default) to nominate a bunch of the Astral people, and just bypass the problem that way.

But equally, I’m fine with removing “non-profit” from that section. I assume that any nominations under that section will be reviewed somehow before approval, so it’s not like we’re leaving things open to abuse? Or is it purely “say you’re interested and you’re in”?

5 Likes
15

Yes, thanks!

1 Like
16

Yes.

Cool. One reason to still remove the “non-profit” words would be to include other commercial representatives who aren’t as plugged in as the Astral folks. E.g. JFrog/Artifactory representatives. But maybe that’s also okay since we talking about the initial seed.

Who would do the review of the initial membership? The SC? You and Donald as the current delegates? The PyPA? No reviews? I don’t have an answer so looking to the community for opinions!

4 Likes
17

A few comments, mostly formal things:

Packaging community

Responsibility

Packaging community members participate in formal votes to elect the Packaging Council.

I would use the term “voting members” instead of “community members”, since the community around packaging is far larger than the number of people you probably expect here, e.g. pretty much any PyPI package author could be called a community member, since they all create and upload packages.

Initial membership
…

  • Wider community members: Non-profit organisations that participate in packaging or working with new packagers. For example, PyOpenSci, NumFocus, Django, are encouraged to initially nominate up to seven members by sending an email to [todo].

This part is not clear. Are those non-profits you listed allowed to nominate 7 members each, or 7 members in total ?

The section should also clarify how these initial members are selected (beyond the ones already pinned down in the PEP), i.e. what is the process for nomination, who does the final selection ?

As already mentioned in the discussion, I too would remove the “non-profit” limitation from the list. “uv” and “conda” are both backed by companies, not non-profits, and Python has generally always embraced companies wanting to take part in the community (it’s one of the reasons we are as popular as we are nowadays).

Adding a new member

Members are added to the Packaging community by a simple majority vote by the current membership. Quorum for adding new members is 50%.

Getting that quorum is going to be much harder than achieving simple majority, esp. as the number of voting members grows. I would not define a quorum requirement.

Also note that we have not had good experience with this scheme in the early PSF, which used a similar scheme. It took ages to get the membership to a reasonable size.

A possible better way is to have the board / council vote in new members, since that’s a much simpler process than to have annual/regular voting members votes. Removal already works in the same way, so why not use the same approach for adding members ?

An alternative would be to use the PSF self-certification process for contributing members, given that the packaging community is fairly large.

Motions by voting members / PSC

I would add a way for the voting members and the PPC to call for votes of the membership. The PEP currently does not include this possibility, but there may well be situations where the PPC is not clear about a certain direction, or where the voting members would like to push for a certain change to a policy, e.g. a change to this PEP.

Possible ways to enact this is by e.g. allowing PPC / membership motions to be put on the annual election agenda (see e.g. the EuroPython Society Bylaws for such a method) or by establishing a way to call for a vote as necessary (see the PSF Bylaws Section 3.3 for an example) or both.

Aside: PEP 13 also doesn’t spell out this possibility. Something we should probably consider as well.

4 Likes
18

Thank you @barry @Deb and @pradyunsg for this important PEP and step in Packaging’s future growth. Users will benefit :smile:

As a PEP editor who read many drafts of the PEP, the team really took all feedback seriously and collaborated as questions arose. While there are likely small things that will still be tweaked, the PEP is very well done.

6 Likes
19

Hmm, good point. I don’t have an answer to that either. I think it’s probably only nominations under the “wider community members” point that has any potential to be problematic. Maybe we won’t even get any (apart from Astral)?

How about a PyPA vote, as that’s our current process? I’d hope this is all purely a formality, so I don’t want to make it too onerous.

One other thought that isn’t spelled out in the PEP - would the list of community members[1] be finalised before the PEP is submitted for approval, or before the initial council vote? In other words, when do we switch from the “initial community membership” process to the ongoing “Adding a new member” one?

  1. which I assume will be maintained somewhere public. Who will do that, by the way? ↩︎

20

@barry I agree with Marc-Andre’s points on:

  • replacing “community members” with “voting members”
  • adding a brief additional statement about seeding the members makes sense.
  • add some language to permit business participation to cover uv / pixi / others.
  • Quorum of 50% could be changed to 50% approval of participating voters in the election
4 Likes
21

Doncerning the “Wider community members”: I think, the right to nominate up to 7 representatives (each?) should be awarded to the developers or maintainers of software packages in the python package space that are available under a free and open source licence (as opposed to: stewarded by “non-profit organisations”). It might make sense to specify that these projects should commit themselves to follow the interoperability specs (packaging.python.org; PEPs).

Since the PEP proposes that election is to be managed by a Returning Officer, it would make sense that that Returning Officer is responsible for voter registration, i.e., collecting nominations for the “Wider community members” (cf. the [todo] mark in the PEP) and self-nominations of the “Interested core team members”. The Returning Officer would also set a deadline for the composition of the initial members that ends before the nomination phase of the first election. After the election of the first Packaging Council, that Council takes the responsibility for the continuation of the list of members of the Packaging Community.

Since PEP 772 replaces PEP 609 (see intro), I suppose that there will be no more PyPA Committer Votes, like for adding projects to the PyPA organization in github and removing them. Maybe the PEP should acknowledge that such votes are not necessary anymore since the assignment of a project to the PyPA organization in github ceases to have effects on voting rights under the new PEP.

Besides those points, I think the proposal is very nice and thougt trough.