Why not? It’s my view that modularity is one of the better ways to improve efficiency of organisations. It sets up boundaries which allow smaller groups to focus on improving specific verticals.
If the PEPs are targets of discussion which the SC have final say over, which groups are not under the SC’s purview?
Agreed.
Modularity is great for making sure that organizations can properly scale and efforts are undertaken by the right domain experts. However, the results of that work should ideally end up in a central knowledge base, so that everyone involved has the chance to easily get a full picture of where the organization is heading – including the Steering Council, who’s main role is to provide guidance for that direction.
The PEP index is our natural knowledge base for managing Python standards, governance, designs, architectures and changes to all these.
Breaking this up into separate silos would not help in clarifying where Python is heading. Well, at least IMO.
This is new but we have already this for packaging, release and typing PEPs, and can add more:
It’s linked to in the intro of the PEP 0 homepage.
Excellent start 
I would make it a bit more prominent, though, and put the topic listing right before the category listing (if possible):
Another idea to enable additional ways of categorizing PEPs (this also goes into the direction of what @barry-scott had suggested):
Add topic tags to the headers of the PEPs and add more sub-indexes to the topics section based on these, e.g. for PEP 248 and 249 these would be:
Topics: standard, database
I think that’s a more flexible way of describing what a PEP targets than using prefixes.
Again, we almost agree, but again: I don’t think it’s useful to mix change proposals (with motivation, rejected ideas, etc.) with the things they’re modifying. It’s one thing to get a picture of where we’re heading (recently approved PEPs?), but another to know where it is (active informational/process PEPs?)
I don’t think the PEP repo is a good knowledge base for the latter.
The living documents are already in separate places (documentation&code, PyPA standards, HOWTOs, etc.), suggesting that they don’t work well as PEPs. Others should be moved, IMO. (Having release schedules be unstructured documents isn’t great.)
Also: PEP numbers work reasonably well for change proposals, but for finalized features with a wider audience, “DB-API v2.0” works much better than “PEP 249”.
The PyPA specs index or the devguide looks much closer to what I’d call a “knowledge base” than PEP 0 with Topics. Can we build something like that for other living documents?
Or of course we can double down in making living documents be first-class citizens in the PEP repo. I think that’s the worse option. It makes “the PEP process” we have for proposals hard to explain. If both kinds of documents are named PEPs, people will continue to conflate them.
Agreed.
Internet standards are born @ the IETF through the RFC process. Every public networking standard worth talking about that’s been developed over the past 4+ decades arose out of the RFC process. Now, clearly, those documents achieve far more and are written with much loftier goals than simply disseminating a request for feedback.
PEPs are proposals because the author or authors are proposing the document be considered for inclusion in the collection of defining specifications for All Things Python™. And every PEP that’s accepted enhances the greater Python ecosystem in some way, simply by embodying “explicit is better than implicit”.
An accepted PEP is no longer a proposal of anything. It’s been codified as standard language/library/community practice/policy/procedure. Which is exactly what had been proposed when it was initially submitted.
The problem here (which we’ve hit repeatedly with packaging specifications) is that if the PEP is the spec, then there’s no real way to change the spec - PEPs are supposed not to change once Final. RFCs have the same issue, with (for example) the email specification being covered in a chain of RFCs, all building on each other. I don’t know about anyone else, but the only number I know is still RFC822…
The analogy is more real than many might imagine/remember. @guido and I (and the rest of Pythonlabs) worked for CNRI back in the day, which also ran the IETF meetings, and so we were very familiar with the RFC process. I was looking for a lightweight way to help Guido make decisions about change proposals by collecting everything about a particular change into a singular document, so something akin to paired down RFCs made obvious sense. I came up with the name “PEP” and backronymed it into Python Enhancement Proposal.
And the PEPs have served the community very well. I love the PEP process, even if I think it can be improved :)
Anyway, looks like the discussion has run its course, and is now repeating arguments. (My own last post is full of them. Couldn’t help myself in the moment, sorry for that!)
I’ve learned that Informational & Process PEPs have their place. So, it would be worth it to document them better – make the first-class citizens.
I suggest that PEP 1 should be restructured to clarify which parts only apply to standards-track PEPs (which is most of the text).
I also suggest that standards-track PEPs should link to the living standard or documentation before they’re marked Final. If the PEP itself is the living standard, it should be Informational/Process. I’ll suggest that PEP editors watch out for this from now on. (There can be exceptions, but they should be made intentionally.)
By default, an Informational PEP is essentially a personal blog post, but that’s not clear to everyone. IMO, we should that each Informational PEP explicitly says who’s responsible for it and how to change it.
It could be good to have another PEP Template, with sections that you can consciously omit, for Informational PEPs rather than Standards-Track ones. Rough draft:
And even with all these changes, I’d encourage people to put content in other places if the RFC-style “immutable document” model isn’t helpful.
In the Standards-Track PEPs, we might note that for Typing/Packaging-style PEPs, the Specification section might be copied as-is to start a living document, and writing it with that in mind may save work.
Does that make sense?
I’d go further and say that (for Packaging PEPs at least) the specification section should be copied as-is[1] to create the living document, once the PEP is approved.
Into packaging.python.org ↩︎
I disagree with such a characterization. Informational PEP documents are much more than just a personal blog post. They become part of Python’s history and are an essential part of what defines Python, its culture and its ecosystem. This is why we have them and make them part of the PEP index.
The wording in PEP 1 is a bit misleading in this respect and should probably be more clearly defined. The description puts too much emphasis on dismissing such PEPs as irrelevant, without focusing more on the importance and intent of these PEPs as guidelines for the Python community (guidelines per se can be ignored, but it’s often better to follow them).
Overall, I think this was a good discussion and highlighted a few shortcomings in the PEP index documentation. Let’s work together to improve those areas.
Sorry, I was unclear there. I agree – PEP1 wording is too weak. And this suggestion of mine doesn’t go far enough:
The DB-API PEP should say it’s maintained by the DB-API SIG and represents SIG consensus. But the PEP also implies the DB-API SIG is the authority on the issue, which is kinda obvious here.
That would mean PEP 672 makes me an expert on Unicode – I don’t have all the answers, I’m not the best expert, but I did the research and I volunteer to be the person to talk to.
I guess that’s the missing piece. Publishing an Informational PEP is like putting your name on the experts index.
+1!
I agree that informational PEPs should include some section explaining how they came to be and whether authority is claimed or not. The author of a PEP is likely going to be an expert on the topic in question, but making all this more explicit is a good idea.
For the DB-API PEPs: Those documents existed long before we had PEPs, so they don’t include any such explanation. But I can add something along those lines, once we have a standard way of expressing this. The DB-SIG is the authority on those standards. How much this means depends on whether the community accepts this authority or not, but given the wide support and acknowledgement of the standards by the database module authors, I suppose, it’s accepted [1]
We should also include some wording which explains whether an informational PEP is a living document or a fixed standard or something in between. Essentially documenting how changes to a PEP document are going to be handled.
[1]: I believe it would be worthwhile having more such API standards as the DB-API, the crypto block cipher API, crypto hash API and WSGI API. I can see that some of these are forming in the ML space with e.g. .fit() and .transform(), same with a subset of the Pandas dataframe API, etc. Such standards make it easier both for users and module authors, since it simplifies the learning curve and reduces design overhead.
Actually, nowadays I think it’s even better to provide ABCs/Protocols, having the document look more like API documentation and managing the “standard” as a library.
Or as a stdlib module, if there’s stdlib code that implements the standard.
I would presume that the code is not the most important thing here – it’s the semantics, which often are hard to capture in an ABC or Protocol, but which are important for the users. E.g. what should we think of a 2-dimensional object that implements __len__ in one dimension and __iter__ in the other?
Definitely, the standard should still be important part. But having the standard as a PEP means you’ll probably not even think of adding code.
Providing an ABC implementation could be part of a standard documented in a PEP, but I don’t think it can replace a PEP. Standards typically need more background information, higher level semantics and explanations of edge cases, etc. than just a reference implementation.
On the topic of fragmentation:
I just learned today (at PyData Global) about an initiative to come up with data API standards: https://data-apis.org/ IMO, it would be great to at least get some kind of informational PEP into the index to point people to such standards (with or without adding the standards themselves).
Yeah, that’s been in the works for a while now; AFAIK from some of the folks involved (albietly some time ago), they’re mostly focused on the core array API, I’m not aware of concrete work (yet) outside of that. Meanwhile, there’s also “Scientific Python”'s SPECs, which has a lot of institutional funding but has yet to really pick up the same level of community traction, plus individual project-specific indices like NumPy’s NEPs and Jupyter’s JEPs.
While I’m all for increased collaboration between core Python and scientific Python, being a (seemingly rare) member of both communities, I’m not sure a PEP would be the right place to link to these sites/consortium—unlike DB-API, these standards don’t directly impact CPython or anything shipped with it, it could be viewed as implicitly endorsing the projects or standards bodies within, and it would need to be kept up to date and maintained, which is a challenge given there’s been a lot of change in this space just in the past year or so (both SPECs and DataAPIs didn’t even publicly exist a year or so ago).
