PEP425 Python Tag for Pyston

Hello
I’m one of the developers of the Pyston Python implementation and have started
work on adding it to manylinux.

That work and the feedback I received made it clear it’s a good time to start discussing
getting a short PEP425 Python Tag assigned for Pyston (like pp for PyPy, ip for IronPython,…).
I suggest using pt for Pyston but we are open to other combinations.

The project is under active development by Anaconda and v2 (which supports Python 3) is available already since October 2020 (the older discontinued Python 2 compatible project much longer).

From the sidelines, this sounds desirable. I wonder if it might be possible to modify PEP 425 so that it defers to the packaging project under the PyPA as a single source of truth for accepted python tags. The PEP could mention those that were current at the time of its last update as examples, and provide a link to the well-defined list of current values in the packaging code.

Not totally related, but it might be nice to also allow extending platform.python_implementation() which currently is documented as returning only CPython, IronPython, Jython, and PyPy.

In principle I’m -1 on this, because it moves us back to the position of having the standard defined by a single implementation. I think we should probably maintain the definitive list in Platform compatibility tags — Python Packaging User Guide

3 Likes

Pyston developer here – specifically re source of truth, I think we’re already pretty far into the world where Python packaging is defined by the PyPA projects and not by the CPython docs, so I’m +0.5 on moving it.

I’m not suggesting it get added to the CPython docs, but to the PyPA specification standards, which are documented at the link I gave. We’ve been working for a long time now to create standards that are defined independently of any specific tool, and while PEP 425[1] is a little vague on some details, IMO that’s a reason to tighten the spec up, not to make a particular implementation the definitive version.


  1. Which hasn’t been completely migrated to the specification page, but again that’s because of lack of people to do the work, not lack of intention. ↩︎

2 Likes

Apologies then, I should have said the PyPA docs. I just personally feel like in this case if there is a discrepancy between the implementation and the docs, the docs will be changed, which makes the implementation the source of truth. For example even modern systems ship with very old versions of PyPA tools, so we had to try to have compatibility with these old versions even though I think they are technically non-spec-compliant now.

I can’t speak for anyone else, but personally that’s absolutely not what I’d want to happen. The docs would be the source of truth, and implementations would have to follow. Certainly, if there’s a good argument for changing the docs we’d do that, but just “because a particular library implements a different list than the docs has” wouldn’t be one.

1 Like

I’m with Paul in terms of preferring standards over code.

But I’m also wondering why you specifically want a short name? I honestly would move away from them or come up with a spec on how to ask the interpreter what it should be called for wheel tags (although turf wars over some short name would be “fun”, hence wanting to move away from them entirely).

While I have none of the authority and a small fraction of the experience of Paul and Brett, as a Python user, package author, avid PEP/specification reader and now packaging PEP writer, I’d also strongly support defining the tags in a single, managed source of truth (the PyPA spec site) independent from digging in the source code of a particular package, even if is the de-facto standard implementation.

I think we are all in agreement that it would be preferable if the spec was authoritative, but unless someone migrates the world to recent packaging tools then we are de facto in a “the real spec is what people use” world.

As to why we’re looking for a two-letter name, I forget the exact details, but some current packaging tools require the usage of a two-letter abbreviation. We got away for a while without one but to support this particular tool we either need a two-letter name or to modify the tool.

Personally I agree with the last point: I think the packaging tools should ask the interpreter for metadata, instead of each interpreter needing to patch the packaging tools. Things have been moving in the opposite direction though.

A few cases which I can think of right now:
manylinux: Pyston gets installed into /opt/python/pyston38-pyston_23_x86_64_linux_gnu while other interpreters have shorter names like cp38-cp38, pp38-pypy38_pp73
cibuildwheel: uses them in CIBW_BUILD: Options - cibuildwheel
wheels: have much longer filenames

While the items I mentioned above are no real problems I definitely run into cases where code assumes a two letter code (unfortunately just can’t remember right now where it was :-/) one can likely fix this assumptions but I assumed it would be a better idea to ask for a two letter abbreviation like the other implementations have.

Any chance you can dig that up? Otherwise totally understand if you simply don’t know.

I understand what you mean here - it’s easy to hit weird glitches, fix them up as best you can and move on, ending up with just a vague unease that “things might go wrong” without being able to remember why.

But honestly, I’d much rather if it were possible that we tightened up the specs we have, made sure all of the tools people are writing/using are following the specs, and worked out migration strategies for moving people off no longer supported approaches. In fact, when PEP 425 was developed, we had very little experience with questions of alternative implementations. So I’d much rather treat this thread as a chance to improve the standard with some real-world experience, than to try to blindly force things to fit the current version of the spec.

In any case, though, PEP 425 doesn’t require abbreviations - it says that “major implementations” have abbreviations, but not that every implementation does. While I’d be the last person to say that PEP 425 is a shining example of precision, a tool that doesn’t support anything but 2-character Python tags definitely isn’t following the PEP. So in actual fact, working on the assumption that Pyston needs a 2-character abbreviation simply rewards tools that aren’t implementing the existing standard properly.

2 Likes