It looks the issue is not in the multiprocessing module itself, but Python’s import machinery somewhat changed.
Following files reproduce similar error.
main.py
from concurrent.futures import ThreadPoolExecutor
import threading
def f2(arg):
print("enter f2", threading.get_ident())
try:
from sub.sub1 import func
func()
print("leave f2", threading.get_ident())
except:
print("err f2", threading.get_ident())
raise
with ThreadPoolExecutor(max_workers=2) as e:
ret = e.map(f2, range(2))
for x in ret:
print(x)
sub/init.py
# blank
sub/sub1.py
def func():
from .sub2 import VAR
sub/sub2.py
import time
import threading
print("begin sub2", threading.get_ident())
time.sleep(1)
VAR = 100
In Python 3.8, the second thread blocks until the first thread completes importing sub.sub2 module. But in Python 3.9, the second thread fails soon.