Draft update to Python Packaging Governance

… of course, one of the problems (with the draft PEP, not with this suggestion) is that the council is, even if not “the PyPA” as such, the owner of the PyPA projects - while still being the “steering council for packaging”. That’s a rather bad situation (and could even be described as a sort of “institutional conflict of interests”) as there’s a clear pressure to have the “standard packaging tools” and the PyPA tools be the same thing - even when a particular PyPA tool may actually not be the right choice.

I don’t think there’s a good answer to this, we’ve had many hundreds of posts discussing precisely this issue and haven’t come to a good answer, much less a consensus.

Just throwing something out of left field here.

Would it make sense to “re-assign” what PyPA stands for to be less authoritative?

E.g. (strawman) Python Packageing Amalgamate where we combine our ideas about packaging

2 Likes

“Python Packaging Alliance” has a nice ring. Or “Python Packaging Association”, sort of like a football association. Every year the worst performing packages get relegated. :joy:

1 Like

Hello @smm,

Thank you for taking the time to write a draft PEP. I appreciate the work that everyone puts into packaging. Since I was unable to attend the packaging summit at PyCon, I have some general questions and thoughts based on my experience with open source governance and structure.

I believe, as others have brought up, that a new organization should have a very distinct name from the PyPA to reduce confusion and establish a distinct mission.

Some general questions which the Python community will need to collectively answer clearly initially are purpose, mission, and governance home (PSF or ??) of a new group.

  1. What specific problems would the additional governance group address? [Purpose]
  2. Why would a new governance group help address those problems? [Purpose]
  3. What type of problems would the group hope to address (initially limit to 2-3 key problems)? [Mission]
  4. What relationship does the new group have in relationship to the existing Python Packaging Authority (PyPA)? [Mission and Scope]
  5. Who is this new group accountable to as stakeholders? [Governance home]

If these are clearly documented already, please do let me know where. Clarity on these points will help make any new group and its interaction with PyPA be successful. Having a clear purpose, mission and scope upfront will help with communication, clarity, and focus.

Warmly,
Carol

8 Likes

Another thought - what will the approval process be for this PEP? Presumably it will have to be decided on by the SC, not the packaging PEP delegates. And I assume that this discussion thread is intended to look for some sense of consensus from the packaging community as a whole. But is there a process for existing stakeholders (individual PyPA projects and the existing PEP delegates) to provide separate feedback independent of the general discussion? And, following on from a point I’ve made before, if, say, an individual PyPA project is fundamentally opposed to this PEP, how would they register that position, and how would that opposition be taken into account - before it reaches the point where they have to withdraw from the PyPA?

1 Like

As in private feedback? Or a separate thread?

I would assume they would speak up here.

1 Like

If this PEP affects the governance of PyPA (but it might not if we decide to decouple them), I suspect it should also be subject to a vote. According to PEP 609:

PyPA committer votes are required for, and limited to, the following kinds of proposals:

Updates to the Governance/Specification Processes

Proposing changes to how the PyPA operates, including but not limited to changes to its specification and governance processes, and this PEP.

1 Like

Few of your questions have been briefly answered in the rationale and scope section of the draft. I will add more clarity in the draft to answer each question. I’ll let you know once it is ready for review.

1 Like

I expect that this PEP will undergo a two step process for approval:

  • A PyPA committer vote, since this changes how PyPA projects are governed.
  • An SC review and approval, since this changes the PEP decision making process’ standing delegations.

Supercede. This should be made explicit and clear in the PEP. One of the bits of feedback I had is that the PEP needs to be clear that it’s replacing the existing governance and be a self-contained reference to all the processes.

Yes. This should be made explicit and clear in the PEP.

5 Likes

I completely agree that “supercede” needs to be more prominent in the PEP. @smm Please consider adding to the first paragraph of the PEP. It wasn’t clear to me that supercede was expected in the original draft. Thanks!

1 Like

Today is the last day that this thread will be alive right?
Did we manage to clarify all the points of the proposal, or should we extend the expire date a bit more?

I suspect that regarding some points that we debated here and are still open, the author’s final proposal is presented in the document as it is, and then (after this thread is closed), it would be up for the PEP delegate to go through the proposal and require changes if they think is necessary. Is that supposition correct?

Which brings to question, who would be the PEP delegate for this proposal, and how does that will interplay with the process highlighted by Pradyun?


I would also like take this opportunity (since it might be my last post in this thread), to highlight again some of the points on the current PEP that I (in principle) disagree with:

  • Committer rights on projects should not be subject to a general vote. I believe that each project should have the capacity to manage themselves and delegate rights/responsibilities in a way that suit their workflow.
  • The wording in The PyPA can require projects to implement accepted PEP standards is problematic. Maybe a better way of putting this would be a variation a long of the lines of The PyPA should liaise with project maintainers and the PSF to secure resources (not only funds, but maybe more importantly, programmers) to implement accepted PEP standards when necessary.

But there is also another whole can of worms to be debated, e.g.:

  • What happens if a project has many more committers than another project? Is it an intentional “design choice”/“feature” of the governance system that this project “has more representativity”? (it might be…)
    • How does this dynamic affect the relationships between the projects? (e.g. what does it mean for “re-packagers” to have more or less representativity than “build-backends”, or what does it mean for “installers” to have more or less representativity than “workflow tools”?) Is that going to bias/affect PyPA in the long term? Is that intentional/desired?
  • How does the new governance actively address the consensus problems as captured by the short hint in Donald’s comment? Is simply adding a group of “elected benevolent co-dictators for mandate” going to actually solve the problem?
    • What is the effect of imposing a decision with the basis of “elected legitimacy” on a group of voluntary workers that disagree with the decision?
2 Likes

I hope the thread is extended, as I don’t believe all of the points made here have been addressed. I hope that @smm doesn’t intend to let this thread close, make changes based on what’s been said, and then start another discussion on a revised PEP (or worse, submit the revised PEP without further review). Because that seems to me to be wasteful of people’s time and energy - much better (IMO) to have a more interactive discussion where the PEP is updated as points are made, because that way people can see that their points have been recognised, and feed back immediately (while the details are fresh) if the response doesn’t meet their expectations.

Regarding approval of the PEP, I think @pradyunsg gave the only reasonable approach:

But again, I’d like to see a discussion that ends in at least a reasonable level of consensus before that process occurs. And it seems to me that this thread in particular is nowhere near reaching consensus.

3 Likes

Thank you very much @pf_moore , I agree.

@smm , @davidism would it be possible to extend the duration of this thread considering what was discussed above?

Yes, this thread can be extended. But I don’t see an option to do it.

1 Like

Maybe the moderators can remove the time limit on the thread? As a general point, I think time-limited threads are a very bad idea for discussions like this, as there’s no way of knowing how long it might take to get consensus, and without consensus there’s no real way forward.

5 Likes

I agree that many issues are still unresolved.

This is something that had been nagging at me as well. Lucky for us, I don’t think anyone’s going to try to game the system by suddenly giving commit privs to a bunch of people to stuff the ballot box. :slight_smile: But it still seems like some distortions could arise. In particular it could mean that projects that are important to the packaging ecosystem, but are maintained by a small, overworked team, could wind up with less voice than might be warranted.

This gets at what I see as the crux of the issue, which is: who is this packaging council for and what substantive actions will it take? In my mind I have two possible answers to this.

One possibility is that the packaging council is for management of packaging projects. In this case the question of “imposing decisions” becomes very relevant. I’m skeptical that such imposition will be very successful. So in this case it’s unclear to me what meaningful actions the PyPC might take. Maybe it can approve or reject packaging PEPs, but it can’t make anyone follow them.

The other possibility is that the packaging council is for messaging to Python users. In this case the council’s main task is not to manage the projects themselves, but rather to distill the work on those projects into documentation and recommendations that helps users navigate the landscape of tools. My perception is that this also could be fraught, both because there may be reluctance to “pick winners and losers” and because, even if such reluctance could be overcome, it’s unclear there’s consensus on what to recommend.

However, it at least seems in the realm of possibility to me that a packaging council could get to a point where it would release a document that says “Tools X, Y and Z exist to do such-and-such. We recommend X for the following reasons. Here is a guide on how to use it. Watch this space for future updates.” At any rate it seems more plausible than “Tools X, Y and Z exist to do such-and-such. The way X does it seems to be the best. Therefore, we’re telling Y and Z to start doing it that way.”

So the second view (namely, PyPC is for user-directed messaging) seems more doable to me. And under that view the council’s main task would be to create such documentation and make clear recommendations (which might include, for instance, saying “We don’t recommend Tool Z because we standardized on PEP 12345 and Tool Z has not adopted that”).

Because of that, I’m uncertain as to the practical impact of the sections of the PEP about liaising with or managing tools, requiring implementations to do this or that, etc. In the end, anyone maintaining a tool who doesn’t like what the PyPC says can just ignore the PyPC, fork the project if necessary, and do whatever they want. The only real power the council would have is to say “This project is officially recommended and that one isn’t.” So maybe that should be foregrounded more in the PEP.

1 Like

This topic was automatically closed after 21 days. New replies are no longer allowed.

The key thing I’d like to see (whether here or on the original thread) is some sort of response from @smm on the comments made, explaining what her view is and how she intends to address the points in the PEP - specifically including details and not just “the PEP will be updated to reflect this”.

I think that any other discussion is probably pointless unless @smm engages more actively - I know I’ve mostly given up on posting further as it feels like shouting into the void…

6 Likes

“I want to discuss more” is nice to say, but as you’ve observed it doesn’t appear that more discussion is actually having an effect at this point.

I’m not sure how, but you may want to coordinate more with the PSF about the Packaging Project Manager role. Here was the original post Python Software Foundation News: The PSF is hiring a Python Packaging Project Manager! If their job description or duties didn’t match the help packaging maintainers needed, you should probably send that feedback to the PSF so they can amend their job description and tasking/reporting processes in the future.

That said, given how packaging discussions seem to go in general, you may need to accept that no one will be able to fill the roll that everyone else thinks they want the way they all think they want it. At some point, someone will need to make concrete decisions rather than just continue discussion forever.

1 Like

FWIW, the funding for that role has run out and @smm is no longer doing this as a job. It can be seen that she’s also no longer listed as PSF staff.