There has probably already been a discussion about this, but I couldn’t find it.
Is there a standard trove classifier for specifying that a package supports the new free-threading builds?
I cannot see any in Classifiers · PyPI
Or is it tacitly intended that the user should check the available wheels and see if there’s one for the free-threading ABI?
Long-term I don’t think we want a classifier for this, but as a transitional feature it may help.
Of course, I’m pretty sure once we add one it can’t be removed again, right? So do we want to have a wart like this around forever?
(Bear in mind that the plan is either for free threading to become the default/mandatory or it will be removed entirely. There’s no world where we have two parallel builds forever.)
🧵 Free-Threaded Wheels works without a classifier, so I think checking the download file names is clear enough.
You already have to do that to distinguish pure Python wheels from those with platform specific binaries.
FWIW it may be helpful to expand PyPI’s “friendly names” for ABI tags to show something more user-friendly for free-threaded wheels.
For example, looking at numpy · PyPI they’re currently rendered as “CPython 3.13t”.
If anyone is interested in contributing that, warehouse/warehouse/utils/wheel.py at main · pypi/warehouse · GitHub is where the relevant code lives.
I just wanted to clarify that this note is about the separation of the free-threaded build being definitely temporary.
Even in the future where that build becomes the only build, it’s plausible that it will retain its conservative “single core fallback mode” indefinitely (so importing an extension module that relies on the GIL will just turn off the free threading support rather than failing entirely).
trove-classifiers maintains a list of deprecated entries, see here:
and preventing them from being used for new uploads:
Edit to add: This is not advocating one way or another, rather stating factually about the mechanics that exist today.
Although checking a classifier would be easier than this sort of thing:
I’d be mildly in favour if adding the classifier allowed the checking code to be simplified, but instead it would just need to deal with additional states without eliminating any.
Status quo:
Adding a classifier means all of those come in two variants (with or without the classifier) rather than allowing the file list to be ignored completely.
That said, the existence of a dedicated free threaded wheel just means “importable on free threaded builds”, it doesn’t say anything about whether the package actually runs in free threaded mode without enabling the GIL.
So maybe there would be value in a classifier after all, to distinguish “it works properly without enabling the GIL” from “it can be installed from a wheel on a free threaded client” (the summary site would then need a 2nd pie chart for the classifier check).
Yes, I can see value in this. It’s similar to how I view Python version classifiers: if I see a 3.13 classifier on a package, I know the maintainer has tested on 3.13 and is confident enough to slap the classifier on it.
Likewise, a free-threading classifier could indicate the maintainer believes this package can be used on a free-threading build, whether it has free-threaded or pure Python wheels.
40 posts were split to a new topic: How to get an extension module working w/ free threading?
To recap:
3.13t tagged wheel may be inconclusive
(*) this sounds to me like an unlikely case: if you’re building a package for free-threading with a separate build step, are you really not going to make the effort to mark the module with Py_MOD_GIL_NOT_USED?
I thought that the issue could also be simply solved with a GitHub-style badge (kudos to the ppl working on the py-free-threading guide! 
).
You cannot use it to filter search results on PyPI of course, but it may just do the trick.
Hello @oscarbenjamin 
Your case seems interesting.
About a possible free-threading trove classifier, what are your thoughts for python-flint?
In my head, your situation sounds like an experimental phase in which you wouldn’t want to publish your package with a free-threading classifier, even though you want to provide cp313t-tagged wheels.
So that you would be making it available to try, but you’re not
Would you agree?
My phrasing was misleading:
I meant that setting the slot was intended but overlooked, which is why it sounds to me unlikely to be stumbled upon.
I very rarely look at or update the trove classifiers for the projects that I maintain and I never look at the trove classifiers for other projects. From my perspective trove classifiers are basically always irrelevant except that it annoys me that they exist for projects that I maintain.
I certainly would not set a trove classifier for this. What I would do is document somewhere what the situation is with text that explains what is safe and what is not.
I expect that to begin with we will provide free-threaded wheels that are thread-safe but only with caveats around exactly how they are used. In other words if you use certain features in multiple threads without GIL then you may get segfaults. The documentation will explain which features are safe or not using text that does not fit in a trove classifier.
So the fact that a package has free-threading wheels does not directly imply that it supports free-threading.
There seems to be a necessity for something more explicit, like a classifier or a badge.
There are different cases here:
This means that you can’t install on a free-threading build from a wheel. It may or may not be the case that building from source will result in a package however:
Py_MOD_GIL_NOT_USED flag set in its extension modules after building.Py_MOD_GIL_NOT_USED gets set.Another possibility is:
However caveats abound:
Py_MOD_GIL_NOT_USED flag set in its extension modules after building.(As an example, python-flint’s test suite does not have any tests that involve Python’s threads at all: it has never been relevant to consider them before.)
A third possibility is:
Py_MOD_GIL_NOT_USED set and so CPython will run without the GIL after importing the package. Downstream users can simply use the package as they already have done without worrying about thread-safety at all.However a fourth possibility is:
Note an important caveat about the third and fourth cases:
Many projects that upload wheels to PyPI are concerned about storage limits so a decision to upload wheels for any given OS/ABI etc is affected by how large the files are and expectations around how likely they are to be used. Some projects may test the free-threaded build in CI to confirm that it works but then may choose not to upload any wheels to PyPI. Currently a decision to upload cp313t wheels would double the size of the wheels for CPython 3.13. There are also other things like Linux aarch64, Windows on ARM etc that threaten to more or less double the number of wheels in other ways.
You can discuss having a classifier or a badge etc but I don’t see how you can reliably use that to distinguish all of the different cases above. Note that the most likely scenario here is actually the second case with the most likely caveat being:
No badge or classifier is going to help you if you don’t read the docs about what can be used safely without the GIL.
To quote from the guide linked above:
For NumPy, we are generally assuming users will not do pathological things like resizing an array while another thread is reading from or writing to it and do not explicitly account for this. Eventually we will need to add locking around data structures to avoid races caused by issues like this, but in this early stage of porting we are not planning to add locking on every operation exposed to users that mutates data. Locking will likely need to be added in the future, but that should be done carefully and with experience informed by real-world multithreaded scaling.
The situation for python-flint will likely be identical: don’t resize a polynomial that is shared between threads. Having thought about this a bit more I realise a more problematic case which is that the elements of some python-flint matrices can live on the heap and so potentially setting an element of a matrix is not thread-safe:
M[0,0] = large_integer # may trigger realloc
I will want to test this sort of thing in a multithreaded scenario before uploading wheels so that at least I have some idea how likely this is to be a problem.
So the situation will be that we upload wheels with the cp313t flag and the Py_MOD_GIL_NOT_USED but you have to read the docs to know which operations can be used safely in a multithreaded context. In future this may change and it may be that non-thread-safe operations are deprecated or it may be that per-object locks are introduced so that those operations become thread-safe at least in the “no segfaults” sense.
The most useful information for downstream users here is not a classifier or a badge but a link to the place in the docs where the status of testing, thread-safety, new/deprecated APIs, future plans etc can be found.
In my view, it’s entirely likely that even in the longer term, “safe for use in a free-threading mode as long as you read, understand, and follow the guidelines given in the documentation” will remain the norm. After all, locking has an overhead, and high performance libraries could quite reasonably decide to offer the user a choice between slower, thread-safe APIs and faster, single-threaded APIs.
I’m getting the feeling that we’re discussing about much too fine-grained issues.
A package may have a lot of functionality, and it is unfeasible to discuss each possibility.
Many features are likely to remain unsafe for concurrent use.
For instance, PEP 703 explicitly retains the thread safety guarantees that were in place before the GIL was removed, and it doesn’t either add nor remove any.
So that, for instance, my_int += 1 is unsafe both in the free-threaded and in the default build.
The fact that there is no GIL doesn’t mean that everything must now be thread safe.
I think this is a point that should be remarked.
No one is expecting all features of all libraries to become thread-safe.
The standard threading.Lock is going nowhere; it is normal to expect that the user has to take care of concurrency issues.
The question is whether or not their programs need to change when they update their dependencies to use the free-threading build.
I think that most free-threading users will not expect their dependencies to function reliably with an experimental interpreter.
On the other hand, I think they’ll want to know whether or not some work had been done in the library to support free-threading, which is where the usefulness of a classifier/badge comes in.
I want to refrain from providing precise details as to what this classifier means: it will surely mean something specific to every package.
Nevertheless, it can have a generic meaning too: yes, you can go ahead and use this library even with this experimental version of CPython.
It signifies intention. (That doesn’t mean there are no bugs.)
Also, there are many pure-Python libraries out there.
We all know that they can be used from a free-threading interpreter too, because we know the principle.
Does every Python programmer out there know this?
Furthermore, to know that a package is pure-Python, you need to have some knowledge about the implementation of the package, so you need to go and check.
(Or you can check in the Download Files section of PyPI and check the tags; again, the same question, does everyone know the principle?)
A classifier/badge makes this immediately obvious.
Only if you actually look for it, which I think is rare. There is almost never a reason to go to the pypi website and look at the classifiers on a package. You just pip install and it finds the package or not. If not, my first stop is the documentation for the project, not the pypi page (sometimes that’s where the docs are, but not often).
At the risk of adding work to the packaging tools, I think the most visible and useful way to communicate this would be when install fails. E.g. instead of “couldn’t find any package for 3.13t”, a message more like “there are wheels for 3.13 but not 3.13t, this package does not yet support that variant” would short-circuit the discovery process somewhat.
Of course, a lot of people don’t bother to read error messages…
Also, let me point out that this topic is about a trove classifier, and not concurrency issues in general.
Okay fair enough. Much of the rest of the thread has been removed now (presumably still accessible somewhere else).
The question we can come back to though is this: what should it be expected to mean if a package has the trove classifier set (or not)?
The OP says:
Is there a standard trove classifier for specifying that a package supports the new free-threading builds?
It is unclear to me how to define what this means in a way that is both helpful and understandable from the perspective of someone who might be looking at the trove classifiers. Possibly that is because I am looking at the state of things right now rather than how they might be in say 5 years time.