Hi,
can time.sleep(0) be used to simulate yield like behavior in threads? is there any other way to yield threads in a multithreaded program?
has the behavior of time.sleep(0) changed from earlier versions of python like 3.6? your help would be highly appreciated.
time — Time access and conversions — Python 3.11.3 documentation has a note on what changed in 3.11, though if might not be relevant to you.
1 Like
Thank you terry. I went through this change set, it has nothing related to time.sleep(0) behaviour on Unix systems.
With python there are two layers of software that are interestin.
There is the python threading logic and the OS threading.
Python only allows one thread at a time to execute python code.
To have python “yield” to another thread needs you to call a function
that will release the GIL and do its action and then acquire the GIL again.
sleep.time(0) is implemented to do exactly that.
You can also “yield” to another python thread by doing I/O.
The os.sched_yield() will call the linux sched_yield() function that will
cause the thread to be put at the end of its scheduling queue (according to
man sched_yield), but does not release the GIL.
Note: pthread_yield() on linux is a call to sched_yield() and not used in python.
Depending on the exact requirements you have either use sleep.time(0) or os.sched_yield().
1 Like
I’m curious though, when do you need to yield in a thread? Threads should be preempting each other as needed. Can you show code that requires explicit yield points?
thank you, Barry.
I am trying to compile python with POSIX user threads Chris so any simple multiprocessing test case with tight loop would hang.
Starting with Python 3.10.8 and 3.11.1, os.sched_yield() (source link) has been updated to release the GIL. Unfortunately this change wasn’t documented or noted in a changelog.
sched_yield() is a POSIX system function. It’s not specific to Linux. But it’s not really for general use. It’s expected that a realtime, deterministic scheduling policy is set for the process (Linux also allows setting the policy for individual threads), such as FIFO or round robin. On Linux, the default scheduling policy (SCHED_OTHER) is dynamic based on the process nice value. The man page states that “[u]se of sched_yield() with nondeterministic scheduling policies such as SCHED_OTHER is unspecified and very likely means your application design is broken”. On Linux, a realtime scheduling policy must have a non-zero scheduling priority, which is limited to privileged processes, per RLIMIT_RTPRIO or CAP_SYS_NICE.
1 Like
Thank you Eryk, I found this information highly valuable in my case.
To clarify the change on Windows, time.sleep() was modified to use a waitable timer (high resolution, if available) for a nonzero duration . However, time.sleep(0) is still implemented to release the GIL and call WinAPI Sleep(0). This yields the processor to another thread that’s ready.
1 Like
Doh! My git clone of cpython was damaged and failing to git pull or report errors.
That’s a nice improvement.
1 Like