I actually don’t want to touch the workflows for core developers or any other SIG. I’d rather that someone else step forward, say it’s useful to them, and then take point on the relevant concensus building discussions with the relevant group.
I’m happy to have a design that accomodates for extensions in the future, but I don’t want to block on trying to establish that this is useful for other groups. As it stands, this can be a self-contained change that benefits a well-defined subset of users, of the PEP process, without affecting others’ workflows. I’d like to keep it that way and get this resolved in a reasonable timeframe.
Relatedly, I don’t think there’s any need to try to establish concensus at python-dev or whatever for something that PyPA would benefit from in the PEP process.
If the PEP editors are OK with such a change, I think this can get landed with small changes to PEP 1 and the Packaging PEPs. This does not need to bubble up all the way to the SC, if PEP editors and PyPA members are in agreement. If we don’t reach an agreement, this can bubble up to the SC.
I think you’re more likely to upset people and get push back by trying to make the change that way than you are by posting up front to say “We’re proposing to make a change to the PEP rendering process that emits a sub index with just the packaging PEPs, these are the mechanics of how we’re planning to make it work, here’s how it could be expanded to other PEP categories in the future if folks want to do that”.
I do agree that actually generalising the process and formalising the idea of standards track categories in PEP 1 can wait, but PEP rendering is a shared resource and it’s courteous to keep other users of that resource informed when making significant changes to it.
That’s part of why I favour keeping the category information in parentheses inside the existing Type field: it’s far more defensible as a rendering pipeline enhancement that doesn’t require PEP 1 and PEP template changes the way the addition of a new field in the PEP header would.
I definitely agree here. I don’t think there will be much resistance or lengthy debate to a narrowly-scoped, technically-focused proposal, but I think its much less likely to elicit a negative reaction to just be up front with it and get a formal okay than go ahead and make the change unilaterally. In the mean time, I can get a PR ready; I expect technical and content review of that and any further discussions here will likely take longer than getting a quick SC signoff anyway.
Actually, its in fact less clean conceptually, more complex to implement and more potentially disruptive to the existing uses to change the syntax of the existing Type header than to add a new header tailored to this purpose, as well as less flexible, extensible and forward-compatible with potential future changes/applications. This is why after careful analysis of the various facets this would touch, I mapped out the planned implementation above with a separate Track header.
To summarize, the current Type header and its valid values is used/referred to a bunch of different places in the checkers, processing code, PEP index, JSON API and PEP 1/12/template, all of which would have to be reviewed and many/most of which would have to be carefully modified to accommodate and handle the new, more complicated Type [(Track)] syntax (which is before even considering the actual core changes needed to generate different index pages).
By contrast, adding a new field only requires a few very specific and localized changes to PEP 1/12/the template, a simple new linter check and adding it to the desired packaging PEPs. It should require essentially no code changes, aside from that needed anyway to generate sub-index pages—which is a fair bit simpler not having to extract a field within a field. It also makes it trivially easy to not actually show the field in the header table (to further minimize initial impact), doesn’t require further parsing in the structured API, and is generally easier to extend to accommodate future additions/changes.
I think we’re both thinking the same thing and disagreeing about the wording.
Well, whatever syntax we go, we’re gonna need changes to PEP 1 to reflect that as being valid syntax.
I literally can’t make such a change unilaterally and do not intend to either.
My plan is to file an issue on the PEPs repository with a specific design proposal for the change and a summary of this discussion. I’ll then direct the “how do we implement this exactly” discussion from here into that issue, and then collaborate with whomever responds there to get this implemented.
I’m not planning to email typing-sig or python-dev or some other mailing list to be like “hey, this is happening and could be useful to you, please chime in” – if someone else wants to do that, please feel free to. If, during the discussion, this turns out to be controversial and something that this needs SC-level descretion, I’m happy to file an issue for that as well.
My plan is to file the issue for this before Monday. If you think that’s a bad idea, please say so before then.
If someone thinks this needs to more discussion on whether such a separation would be useful for the Packaging PEPs, please say so.
So far, basically everyone has agreed that this would be useful and most of the discussion at this point seems to be about the mechanics of implementing this.
We might want to be careful on page design so as to avoid peps.python.org/pep-0000/packaging being seen as a canonical source for anything, especially given the efforts to move specifications to authoritative homes.
A small concern in the grand scheme of things, though.
That’s a valid concern. I’m hoping once we have a dedicated page for these PEPs, we can make it hav a clearer introduction for these as well as link to the packaging.python.org page for the “current state” while also describing those PEPs as the initial design and change proposals.
Having a clear list of these PEPs will also make it easier to migrate them to packaging.python.org (we’d basically have a checklist) and I think reducing the work that whomever picks up the task of migrating them would need to do is a good thing.
I’d still hope that the canonical URL for (for example) PEP 517 would be https://peps.python.org/pep-0517/, even if the specification in that PEP gets migrated to packaging.python.org and the PEP itself is also accessible from a dedicated “Packaging PEPs” area. I have shortcuts and utilities that build URLs from the PEP number, and I’m sure others do too.
Good point; if we’re going to have a dedicated, bespoke page, we would want to take the opportunity to link the package specs site on said page and make clear the role of the PEPs versus official specs. Though, the URL path would be something like peps/python.org/track/packaging, rather than having pep-0000 in it.
Long term, a much more flexible and functional and less “canonical” looking approach would be to instead of statically generating all the pages, just store the PEP index as a CSV or JSON and use D3 to dynamically load and filter them client side based on the user’s query params. But that’s obviously a much more far-reaching (if also much more generally useful) change.
Yeah, the canonical URLs to the PEPs themselves wouldn’t change at all, there just would be a new peps.python.org/track/packaging index page that specifically lists just the packaging PEPs.
You might be interested in our new (experimental) JSON “API”; you can get the canonical URL to any PEP via just requests.get('https://peps.python.org/api/peps.json').json()["517"]["url"]. You can also access all the other header metadata there (right now as just string values, but the infra is in place to parse and expose it in a more structured format). If and when Track: is implemented, you could get all the Packaging PEPs like this:
peps = requests.get('https://peps.python.org/api/peps.json').json()
packaging_peps = {pep.get("Track") == "Packaging" for pep in peps}
The “API” is still experimental and not officially documented, so I wouldn’t rely on it yet, but just wanted to give you and others a heads up.
That rationale makes sense - I hadn’t considered the fact that adding extra text would confuse anything looking specifically for “Standards Track” in the Type field.
OK, so looks like we have concensus on Track and that this is a good idea (given the extensive discussion on “what’s a good way to make this happen”).
I’ve filed an issue for adding such a page on the PEPs repository:
The proposal I’ve put up there is to have peps.python.org/track/packaging page and a Track: Packaging header. If there’s more implementation-related discussion items, please flag those on the issue!
If there’s any “actually, we shouldn’t do this”-style concerns, I’d appreciate if folks flag that here!
Proposals for new interoperability specifications should be formulated and submitted as new Standards Track Python Enhancement Proposals in accordance with PEP 1.
