I am not sure if this is the right venue to discuss this, but since the original decision to leave decisions on the interoperability standards to the PEP process, a requirement was added that all PEPs originate from or be sponsored by a Python core dev, which means that even Dustin, the BDRFN of the PyPA, needs to get a core dev to sponsor this proposal, which rubs me the wrong way a bit.
Obviously it’s not a huge deal because there’s a good amount of overlap between PyPA and CPython core dev (I dunno who would self-identify as being “in the PyPA”, but there’s I think at least 7 or so Python core devs involved in the PyPA), but it does feel like an unfair asymmetry that I can propose packaging PEPs without a sponsor but Pradyun and Dustin (and many others) cannot. I think at the very least there needs to be a process by which PyPA members can be empowered to make these proposals without a sponsor: I am ambivalent as to whether that is automatic upon joining any PyPA project or whether there’s a slightly higher bar - but it cannot be as high as “is a core dev of CPython”.
Once we decide who we, as the PyPA, want to be able to propose interoperability specs, the ways forward I see are:
Use essentially the same PEP process, but in a different namespace (e.g. PPEP - Python Packaging Enhancement Proposals) and with our preferred rules for who can propose them.
Continue using PEPs, but get a ruling adding the PyPA’s “allowed list” to the list of people who can sponsor (at least) PyPA-related PEPs in the main namespace.
Move to our own process (possibly one where decisions are made by people elected by the PyPA rather than by core devs).
I have listed them in order of desirability from my perspective. I think in the long run we may want to move to #3, because increasingly core dev and PyPA are two distinct groups with distinct needs, but in the short-to-medium term, taking advantage of the fact that the SC exists and is willing to decide on (or delegate decisions about) packaging topics to ease the transition from “loose collective” into “well-governed meta-project”. I favor a new namespace over the main namespace (#2) because I think it’s a better way to organize our proposals and because it makes a transition to something like #3 the easiest.
We’ve discussed this before, and I think the result was that anything “approved” by whoever in the PyPA has that job counts as the endorsement of a core developer.
Since PEPs here aren’t going to be a burden on python-dev or steering council time, the same goal is achieved.
I’m not clear what you mean by this, what is “whoever in the PyPA has that job”? Is this a rule that is already in existence, or are you saying that the previous discussion was that we should propose that?
As it is, in order to propose this PEP, Brett asked Dustin to get a sponsor. From what I can tell, he was entirely right to do so according to the current rules of proposing PEPs. The proposed PEP says that decisions will be made by PEP, so I think it should be changed to either recommend some parallel process that doesn’t depend on Python core devs being involved or we should propose in the PEP that PyPA core devs should also have the right to sponsor packaging-related PEPs.
Thanks for raising this @pganssle. I think this might be worth breaking into a separate thread but I’ll reply here for now.
Would it make sense to use a combination of #1 and #2? I.e., add a Track: Packaging header to all packaging-related PEPs, and consider PyPA members committers (which will be well-defined with PEP 609) equivalent to core devs w/ regards to sponsorship for that track?
Aside from all the concerns listed above about #3 with regards to duplicating effort and process, I’d also be concerned that any non-PEP process wouldn’t be perceived as having the same weight/authority as the PEP process by the wider community.
I’m having a hard time parsing this sentence. Are you talking about BDFL-Delegates?
Agreed. The role of a sponsor involves helping the PEP author work through the PEP process, so it’s distinctly more than just sanity-checking PEP ideas. And while approving PEP topics is fine as a role for a packaging BDFL (or whoever @steve.dower had in mind) being default sponsor for those PEPs is a lot to ask of one individual.
Do understand this was one of 13 PRs I was working through in that repo so I didn’t stop and think of this as a packaging PEP and how that might shift who could be consider a sponsor or who is empowered to be considered a “PyPA developer” who knows the PEP process well enough to not need a sponsor to help guide them through it.
But the key thing here is to understand that the sponsorship requirement is to make sure people who are submitting PEPs have someone to walk them through the process and make sure the PEP is in good shape to take that burden off the PEP editors. I would have no issue to change the sponsorship requirement to not apply to people who have already authored >=2 PEPs or something.
So to reiterate, this isn’t about python-dev gating everything, it’s to make sure the jobs of the PEP editors doesn’t suck due to bad PEPs. If there’s a way to tweak things so that works with PyPA then that’s all I’m after as a PEP editor.
@pganssle I’d thought of this and it wasn’t a major concern to me TBH. That said, I’m in favor of improving the situation here.
However, I don’t want to tackle this as part of the initial version that we formalize. That’d make it more than “document the status quo as a starting point” which was an intentional choice to minimize what all we try to “fix” in this initial version. Voting fits in because that’s compatible with what we do already.
We can (and should!) make improvements like this, after we formally document what the processes are currently – for the same reasons that one should separate the moving code and refactoring/improving the moved code into separate commits.
With this in mind, I think @dustin’s proposed Track: Packaging would be a good way to tackle the issue.
@dustin – I think we should raise the bar to PyPA committers since I feel giving out triage bit isn’t a very strong signal of someone’s familiarity with the processes and expectation. At least, not as strong as having the commit bit.
So I was thinking about this last night and I think it might be time for packaging to have its own separate process. For instance, @pradyunsg I’m sure thinks that saying PyPA committers can sign off and sponsor PEPs is simple enough, but that means I have to keep track of who that is on top of core developers and to make sure people who come in proposing PEPs are proposing them in the proper category (which isn’t even officially a thing in PEPs to begin with so that’s yet another deviation from the current PEP process).
I think if the PyPA is going to function as an independent body it might be time to just shift over and manage their own standards process. So I say let’s get distutils out of the stdlib and just hand everything package-related to the PyPA from that perspective and have a cleaner separation. That way the PyPA can have Python Packaging Standards or something and it can be managed in its own way without everyone trying to force things into a single process which doesn’t quite fit for anyone.
I’d not thought of that – and it’s a very valid concern.
I do think we can use automation to take care of this (get the sponsor email/name, if sponsor is in python/<core-developers> or Track == "Packaging" and sponsor is in pypa/<committers-team>).
If we can solve a problem with a GitHub bot, I’d much prefer that we do that, instead of trying to create and manage an entirely new process. Would that help make PEP editors’ job easier (which I definitely don’t want to make more difficult)?
That said, I’ve argued that we don’t have a significant problem in my previous post.
Not entirely as we now have a GitHub action that must be kept working no matter what just for this use case. Not that there isn’t any other group asking to use a forked process as there is no other group; it is just the core devs and the PyPA.
Sure, but that’s because you’re asking me to manage the current process to meet your needs.
Honestly, the more I think about this the more I think the PyPA should be run as a separate group with their own process. You can fork off what python-dev does, but the fact that discussions happen entirely separately and the core devs mostly have no idea what’s going on over here shows that things are already run very separately. And to be perfectly frank, I don’t want people thinking that core devs have anything to do with packaging and instead the PyPA is in charge of that because it causes people to not know who to talk to about what.
I have no problem if the power delegation continues to come from the steering council, but I think at that point everything is devolved into being different. Heck, there’s now a separate summit for packaging at PyCon! I think it’s just time to accept the PyPA has “out-grown” being under python-dev in any way and just needs to start operating on its own officially instead of unofficially as it does now.
I will also say @ncoghlan strongly disagrees with me on this. I believe he would rather have PEP 609 come up with a clear delineation of who the PyPA would accept as sponsors.
(This is an additional thing that we’re discussing on top of the existing process, which is intentionally out of scope of the PEP – can one of the moderators please move the discussion on PyPA’s use of PEPs + sponsors discussion into a new topic?)
If the choices are many PyPA members needing sponsor for PEPs vs moving to a separate process entirely, I strongly prefer the former.
To be clear, I don’t think status quo is a problem. The fact that I’d need a sponsor for PEPs that I propose, is not a major concern for me personally.
There’s > 5 core developers who are also PyPA committers. They’re all reasonable people. If that count reduces substantially, we can look into ways to address that problem – I’d prefer that we deal with it when it’s a more real issue.
If we’re at a point where no core developer (regardless of whether they’re a PyPA member) is willing to even sponsor a packaging-related PEP, we probably have a different problem that’s manifesting as no sponsor for said PEP. In the worst cases, we’d still have the SC to escalate to if things go south.
As for folks who are PyPA+CPython core developer, it’s fine that they don’t need a sponsor? They’re already wearing two hats, both of them come with responsibility of keeping the process from not loosing effectiveness and I’m willing to trust them on that front.
I’ll echo this sentiment.
I think that a separate process would also be sub-optimal, if we plan to continue calling PyPA an “authority” (see here) and we should to prevent unnecessary churn like that.
The main reasons I advocate strongly for PyPA to continue using the PEP process are so that folks outside Python core development and the PyPA can continue to ignore the fact that these are distinct-but-overlapping groups, and so that we don’t get into “Which process should we use?” debates for proposals which affect, and need to be reviewed and approved by, both groups (e.g. the past PEPs to add the ensurepip module, or the likely future PEP to make setuptools the reference implementation of the distutils API and switch it to an ensurepip style bundling model).
I think of it as being similar to the different working groups within IETF: the internal processes may vary a bit between groups, but the general approach and the communication to the outside world via the RFC process is broadly consistent.
There are certainly improvements we could make on that front (e.g. I like the idea of more explicitly separating language standards from packaging standards in the metadata, especially if we did it in a way that let us better distinguish CPython implementation proposals from Python language proposals), but I view those as process maturation issues, where PyPA and the typing SIG are marking a trail that should make it easier for other related-but-separate groups to consider using the PEP process for publishing their interoperability specifications (e.g. PyCA, PyCQA, perhaps a future ratification of the ASGI spec).
Edit: note that when PyPA first started using the PEP process, the packaging PEPs could be identified based on a Discussions-To entry that referred to distutils-sig. Similarly, typing PEPs usually referred to typing-sig. That system doesn’t really work any more, as most PEPs these days are pointing to specific discuss.python.org forum threads, regardless of which interest group is managing them.
Up to this point I know all the people who would qualify as a non-core sponsor. Now if people trust me to know who the PyPA committers are then it’s not more work. Otherwise people need to either trust people aren’t lying that the person they listed is a PyPA committer or I need to know who to ask for verification.
I don’t think we’d need make it an ad hoc thing. Instead, I’d suggest adding a list of specific people that are willing and able to serve as packaging PEP sponsors to the standing delegation document in the steering council repo (beyond the folks that are also core developers).
Way back when we originally started discussing formalizing the PyPA governance discussion, I advocated for moving the PyPA standards process out of the PEP process, for basically all the reasons listed here. I believe I even explicitly called out the fact that since the PEP process was targeting the Python language, and in many cases, more specifically the CPython implementation of said language, that there was always going to be a bit of a square peg, round hole scenario. I called out a risk that the Python PEP process would evolve in a direction that made it more difficult for the PyPA projects to continue to use it.
It appears that as the Python governance has shifted, that the PEP process is in fact drifting further away from a process that is wholly workable for the PyPA. While we can continue to try and force it, I think it’s a much saner idea to establish our own processes.
edit: Note that, in this situation – we need the mentoring that’s part of the PEP Sponsor role. Even in a dedicated PyPA process, we’d still need that part but all PyPA folks would be allowed to be sponsors (vs a small overlapping subset). If folks think that’s the issue here (we have someone who’s PyPA but not CPython core dev who can mentor), that’d be an argument in favor of adopting a dedicated, separated process.
I believe he would rather have PEP 609 come up with a clear delineation of who the PyPA would accept as sponsors.