PEP 780: ABI features as environment markers

Packaging Standards
42

How is that any different? The PEP will still be required to include details of how to compute the correct values, which will be “use the value from sys.abi_info” in Python 3.15+, and in Python 3.14 and older, will specify the logic to use. We’re already into uncertain territory, as @steve.dower pointed out, because the calculations will differ in different platforms and Python implementations. How would you calculate the value of debug in GraalPython 3.12, for example? Remember, these are fields we want to guarantee are present, so “omit the field” isn’t an option.

And it does nothing to help interpretation of fields that aren’t defined in the spec.

IMO, we need to be very clear that this is an interoperability spec we’re defining. And you can’t be interoperable without having an agreed meaning of the flags you’re proposing to add. The only reason we even have standards is to ensure that users don’t have to deal with implementation-defined behaviour. And by saying “it’s up to the implementation to provide” you are explicitly requiring implementation-defined behaviour :slightly_frowning_face:

I’m sorry, but I think this proposal has suffered enormous scope creep, and we should cut it right back. The original request was purely for a marker for free threading - a simple yes/no question. The discussion led to the idea that we could have a framework for “features” that covered both free-threading and any other features we might add in the future. That is more than we need right now, but seems like a sensible extension. The idea is that features are either present or not, and until they are available via the “ABI features” list in sys, they aren’t present. We have to do some patching up for free threading, because that was added before the mechanism was introduced, but that’s fine.

Up to this point, I’m fine with things.

But then the sys.abi_info discussion happened, with the core developers taking that feature in a different direction, which was not focused on packaging. I don’t have a problem with that, but no-one seemed to be taking an interest in ensuring that the final form of the information still addressed the needs of the packaging use case. So we’re now in a position where PEP 780 is trying to force sys.abi_info into a form that can work as a packaging marker while still answering the original use case, “how do we make a dependency conditional on free threading?”

All of the complexity we’re discussing now is for things where there is no identified packaging use case.

IMO, we need to go back to basics.

  1. A marker for free threading can be "free_threaded" in sys_abi_info.
  2. To provide extensibility, any other boolean value in sys.abi_info will also be exposed in sys_abi_info, with the abi_info key being present if the value is true, and not present otherwise.
  3. Non-boolean values in sys.abi_info are ignored[1].
  4. We’ll need to add some backward compatibility, providing rules for how we decide if we should add free_threaded to the marker in CPython versions where sys.abi_info is not present. But we only do that for free threading, in all other cases we mandate that the sys.abi_info value will be the single source of truth, and must be added at the same time a new feature is introduced[2].

As far as I am aware, this satisfies all known use cases for ABI features as environment markers. I’m not interested in theoretical use cases - it’s more important to me that we provide support for free threading, which people need.

In the interests of full disclosure here, I may or may not be PEP delegate for packaging by the time this PEP comes up for approval. If I’m not, I can’t say what the packaging council will be willing to accept. But if it is me that ends up pronouncing on this PEP, please consider the above as my official position as PEP delegate. I’m open to being persuaded to change my view, but any arguments will need to be rooted in actual real-world use cases, the same as the free threading case, and not theoretical arguments about what some project “might” want to do.

  1. In implementation terms, sys.abi_info is scanned, and any key with a value of boolean True (the exact value, not a “truthy” value like 64) is added to the sys_abi_info marker. ↩︎

  2. We can’t enforce that, because it’s up to the core developers. But we can define how packaging handles things if the core devs don’t follow that requirement. ↩︎

1 Like