Nope, most of my work is single-threaded, and single-process. A lot of Python code runs just fine as a single-process application with no threads or other concurrency. Writing command line utilities in Python is a clear example of a case where performance drops are often problematic (people want their command line tools to be fast).
I don’t know how to build an application using free threading. What would that mean? Would I need to do my own locking (something I can currently mostly ignore because of the GIL)? Would I have to think about race conditions myself? I’ll absolutely agree that because Python has the GIL, no-one has really been thinking about effective, user-friendly support for parallelism in Python until now. I’d love it if nogil gave us that. But right now, everyone is focusing on existing threading models - and if all we get from nogil is the ability to use current thread models with CPU-bound Python code, then people (like myself) who don’t already have problems because of the GIL will continue to struggle to see the attraction.
It would be really nice if someone could give the case for nogil in terms of the sorts of opportunities it would give for new programming techniques, or improvements that would benefit code that nobody currently thinks of as “needing threads”. For example, if
map() could transparently parallelise the calculation, that would be a genuinely interesting benefit. But I don’t know if that’s reasonable to expect, or if the conditions needed to ensure thread-safety for such an operation would prevent it being transparent.
Agreed. It would be a shame if funding (and hence Sam’s involvement) dried up because we (or rather the SC) are still trying to work out how to tackle the transition and logistics issues. But conversely, it’s rather sad if the funding/commitment is only for doing the technical work, and doesn’t cover working on ecosystem issues like transition (or at least factor in the time it will take for others to handle those issues).
But we only have limited information here, and I don’t want to second-guess the constraints. I just hope the project doesn’t fail because technical and ecosystem work can’t be co-ordinated somehow.
Are you saying that if a package is pure Python, then it won’t be impacted by nogil? That’s great if it’s true, and I don’t have the technical understanding of the nogil work to know for myself, but isn’t it necessary for any code that might get called in a free-threaded application to at least be aware of the implications? Suppose I had a library that contained this function:
total = 0
for n in l:
total = total + n
who ensures (whether by auditing or explicit locking) that the input list isn’t being mutated while the function code is running? If
sum_list can’t make that assumption, doesn’t it need a lock? Or am I misunderstanding nogil totally?
But that will result in a split ecosystem. People using python-with-nogil won’t be able to use certain packages, which haven’t been upgraded to be nogil-aware. That sounds very like the Python 2-3 transition, where you could find that some packages you rely on only supported Python 3, while others only supported Python 2. That’s precisely the sort of ecosystem impact that I imagine the SC is concerned about.