Create a "Python Threading" category

Discourse Feedback
1

I’m hoping the admins might be interested in creating a new “Python Threading” category with the following proposed description:

Discussion of multithreaded Python programming, thread safety, and multithreaded parallelism.

This overlaps somewhat with the free-threading tag but also is broader and encompasses questions about multithreaded python in the GIL-enabled build as well. There’s also overlap with the Python Help category of course, but I’m hoping the description fits questions that are coming from library maintainers and app developers rather than questions coming from all Python users.

Why a new category?

Over the past year I’ve been working full-time on helping community packages support free-threaded Python. As I explained in the PEP 779 discussion, I’m very bullish on future prospects for free-threaded Python and I think it’s clear that free-threading will eventually be fully supported.

However, the PEP 779 discussion also showed there’s a lot of confusion and trepidation around supporting free-threading, and also some confusion about what guarantees the GIL provides and how to write thread-safe code in pure Python, let alone the complexities that native extensions introduce.

Of course it’s no surprise that multithreaded Python is getting more interest with the advent of free-threading, but that means that longstanding thread-safety issues and common thread-unsafe practices and patterns will need to be discussed. Code that up-until-now never anticipated multithreaded use will need to be updated. That means that this topic will be getting more attention for at least the short and medium term as more and more people experiment with multithreaded parallelism.

Me and my team have been working on the free-threaded Python guide which we hope will answer many questions people have about free-threading and be a spot to accumulate community know-how.

While a spot for the community to read and contribute documentation is great, I think we also need a moderated space for direct communication between developers and among the community. While chat like discord or zulip might also be needed, I think the searchability and moderation tools in discourse are perfect for what I have in mind, and indeed a new category in the CPython discourse seems like the perfect spot for this. If this was 10 or 15 years ago, we’d probably start a mailing list :slight_smile:.

10 Likes
2

I’m afraid it will on the contrary attract a large number of user questions of the kind “how do I make my code parallel?”.

If you want to discourage user questions, you should make it explicit in the description, for example:

Discussion of thread safety issues and techniques for library authors. This is not a forum for user questions such as “how do I make my code parallel”. Those questions should be asked on Python Help - Discussions on Python.org.

The category title should probably be more specific as well.

3

I was trying to follow the examples set by the descriptions for the other categories, which are all one sentence. That said, I see the core dev category has a similar parenthetical suggesting to look elsewhere.

Maybe something like:

Discussion of thread safety and multithreaded scaling for library authors. (usage of threading should be discussed in the #users::category category, new ideas for Python to Ideas).

Following Async-SIG, Threading-SIG would also work, I think. My main aim in the naming and description language is to not make this category specific to free-threading.

4

New category added to the forum:

7 Likes