tl;dr; I’m planning on being stricter in the future requiring packaging PEPs to follow the existing processes. My goal here is to reduce the number of standards we have in an “agreed, but not yet implemented” state. I’d appreciate feedback.
Recent packaging PEPs have prompted me to think more about the current standardisation process. In particular, I’m concerned that by focusing on standardisation as an end in itself, we’ve ended up in a situation where we spend more time discussing principles than actually implementing functionality, and that hurts us when our specifications end up impractical to implement, or unimplemented simply because packaging projects don’t have the resources to implement them.
To that end, I think we should tighten up how we develop packaging standards.
We currently develop standards via the core Python PEP process. That has some rough edges, because the process wasn’t designed for developing standards the way we use it, but it does an adequate job. It’s heavily dependent on the PEP delegate, and I’m conscious that as a result, people who are not happy with the way @dstufft or I have performed the role may have differing opinions, but I’m not proposing that we stop using the PEP process. If someone does want to propose that we stop using the PEP process, PEP 609 outlines how to do that - basically, it would need an alternative proposal, which would have to be put to a PyPA vote.
I’m going to assume here, though, that we retain the PEP process, and focus on how we can improve the way that process serves our needs. To an extent, this is going to be me saying “this is what I will be doing in future when I’m responsible for approving a PEP, if people think it’s unreasonable then let’s discuss the options”. But I hope it will also serve as a starting point for people (and in particular people likely to be writing PEPs) to think about what they want from the standards process, and feed back how they think we could improve things.
In the PEP process, the SC acts as the final arbiter of any proposal. They step in when there’s no PEP delegate, or if there is a need to override the delegate’s decision. Neither of these has ever happened in practice for a packaging PEP, and in reality, it’s not clear that the SC would want to be involved in packaging PEPs, as they don’t have much background in the area (maybe @brettcannon could comment here). I’ve been sort of hoping we never need this option, to be honest. But it is there, should we need it.
The packaging user guide includes a section which contains (in theory) all of the packaging specifications. In practice, we’re still migrating a lot of the older specs to that area, and frankly it’s a lot of work that doesn’t add a huge amount of value (until we need to update the spec), so it isn’t really progressing particularly fast.
The full details of the process for maintaining the standards documentation is here. In particular, it requires the following for all PEPs defining new standards:
Such proposals must be accompanied by a pull request to the Python Packaging User Guide repository, against the PyPA Specifications section, that adds a new subsection defining the purpose of the new specification and the role it plays in the wider Python packaging ecosystem.
I plan on strictly enforcing this requirement for any new PEPs that I approve, as we’ve not been doing that with recent PEPs, which means that the specifications section of the PUG is gradually becoming less useful as a reference, rather than more.
I’m not sure what “defining the purpose and role of the new specification” is meant to mean in this context. Historically, we’ve documented the actual specification in the PUG, and the process for making changes seems to assume that’s what happens, so I’m taking that view. If anyone disagrees, I encourage them to get the process changed to clarify (I assume such a process change would be a proposal plus PyPA vote, as usual, but maybe it could just be a PR to the process document? I’m not sure who would make that call). If no-one disagrees with my interpretation, I will at some point propose a clarification to that effect myself.
The documentation for the PEP process itself, PEP 1, explicitly states that a successful PEP should include a reference implementation:
Reference Implementation – The reference implementation must be completed before any PEP is given status “Final”, but it need not be completed before the PEP is accepted. While there is merit to the approach of reaching consensus on the specification and rationale before writing code, the principle of “rough consensus and running code” is still useful when it comes to resolving many discussions of API details.
That requirement is less approprate for packaging PEPs, where standards typically impact many tools and there is no one place where a “reference implementation” makes sense to add. However, IMO it is still an extremely useful requirement, as it ensures that the proposal is usable in real-world code, and that there are no major problems with implementing the spec.
Again, I plan on being stricter in the future on expecting this requirement to be followed. While it may not always be possible to implement a major specification in a key project like pip or setuptools as part of the process of developing a PEP, I will expect to see at least a library or a standalone tool that implements the spec.
If the authors of a particular standard want to omit the reference implementation, they should explicitly note that in the PEP, and include their reasons for not providing an implementation. I’ll happily confirm in advance of the final approval whether I have a problem with their reasons, and I assume any other PEP delegate would be willing to do so as well. In general, though, an implementation should be included - not providing one should be the exception, not the norm.
In general, a PEP will get accepted or rejected. I’m not a great fan of “provisional” status, in particular I don’t think it works well for interoperability specifications (where “might change in incompatible ways” essentially translates as “doesn’t provide interoperability”…) So while provisional status, as defined in PEP 1, is a valid outcome for the PEP process, personally I tend to avoid using it.
There’s not much else to say here, although it’s worth noting that some projects, like pip, work on the principle that new functionality should be standardised before it gets included. There’s a nuance here, in that technically, “standardised” means "marked as
Final", and acceptance doesn’t immediately imply the PEP is Final. In reality, the acceptance may include follow-up tasks that must be completed before final status can be recorded (including the standard in the PUG, completing the reference implementation, etc). Ultimately, though, it’s up to individual projects what policies they set.
One thing the PEP process doesn’t cover is how we transition to the new standard. This can involve a lot of complexity, deprecating older legacy mechanisms, interpreting legacy data in terms of the new standard, timescales for dropping non-standard approaches, etc. As a result, the packaging ecosystem is held back by the “long tail” of legacy data, often in unmaintained or obsolete packages.
I propose that we add a requirement for a “Transition Plan” section in new PEPs, that explicitly discusses this sort of issue. It would include such items as:
- Plans for how we let projects know that they are no longer standards-compliant, and explaining what they need to do to update.
- Conditions under which it is considered reasonable for tools to drop support for legacy approaches.
- Discussion of how projects can support the new standard alongside legacy approaches.
- Deprecation timescales, setting expectations for how long it should be before tools can reasonably assume that they will only have to deal with projects compliant with the PEP.
Covering these questions as part of the PEP avoids situations where the standard works in an “ideal world” where everything is compliant, but is unimplementable in the real world of legacy data and unmaintained but still useful projects.
OK, that’s it. Sorry for the huge wall of text - I didn’t mean to go on this long, but I wanted to cover the whole process in one place. Thanks to anyone who’s made it all the way through.
I welcome comments or suggestions on what else we might do to make our standards process more effective, more streamlined, and ultimately more effective, both for PEP authors and for end users.