Possible regression on multiprocessing's resource tracker in 3.13

Core Development
1

Hello!

I’ve found something that started to break in 3.13, and I want to confirm that it is a bug (so I open an issue) and not something weirder (maybe in my code).

This is a simple script to expose the problem. It creates a portion of shared memory, then forks. The child writes in that shared memory. The parent waits for the child to finish, reads the shared memory, do the proper cleanup, and then shows the result.

import os
import struct
from multiprocessing import shared_memory


type_size = struct.calcsize('B')
shm = shared_memory.SharedMemory(create=True, size=type_size * 2)

pid = os.fork()
if pid == 0:
    # child
    values = [1, 2]
    offset = 0
    struct.pack_into("2B", shm.buf, offset, *values)
    exit(0)

# only the parent will reach here
os.wait()

# get produced value and cleanup the shared memory
result = struct.unpack("2B", shm.buf)
shm.close()
shm.unlink()

print(result)

It works just fine in 3.12:

$ python3.12 badtrack.py
(1, 2)

But in 3.13 (and 3.14, and 3.15 just compiled from main):

$ python3.13 badtrack.py
Exception ignored in: <function ResourceTracker.__del__ at 0x784fa13d0540>
Traceback (most recent call last):
  File "/usr/lib/python3.13/multiprocessing/resource_tracker.py", line 84, in __del__
  File "/usr/lib/python3.13/multiprocessing/resource_tracker.py", line 93, in _stop
  File "/usr/lib/python3.13/multiprocessing/resource_tracker.py", line 118, in _stop_locked
ChildProcessError: [Errno 10] No child processes
(1, 2)

Note that the script still ends “successfully”, but there is that ChildProcessError in the middle. The weird thing is that the error happens when closing the resource tracker, when it does waitpid(self._pid), being that _pid the process id of the tracker itself… it looks it finished before, so it’s gone at that moment.

This is 100% reproducible in my machine (Ubuntu 24.04, kernel 6.8.0-79-generic, x86_64 architecture).

Of course, calling SharedMemory with track=False makes the problem go away (but this workaround is not possible when using ShareableList).

Thanks!

. Facundo

2

I tried it on my Mac. I can’t get it to work even back to 3.12. I modified it slightly to demonstrate why:

type_size = struct.calcsize('B')
print("type size:", type_size)

shm = shared_memory.SharedMemory(create=True, size=type_size * 2)
print("sh mem bufsize:", len(shm.buf))

The output:

type size: 1
sh mem bufsize: 16384
Traceback (most recent call last):
File “/Users/skip/tmp/badtrack.py”, line 26, in
result = struct.unpack(“2B”, shm.buf)
struct.error: unpack requires a buffer of 2 bytes
(python313) ~/src/python/py3.10% /Users/skip/src/python/py3.10/Lib/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked shared_memory objects to clean up at shutdown
warnings.warn('resource_tracker: There appear to be %d ’

It seems the minimum buffer length is 16kb on my machine?

3

@smontanaro thanks for trying it.

Docs say (#TIL): “Because some platforms choose to allocate chunks of memory based upon that platform’s memory page size, the exact size of the shared memory block may be larger or equal to the size requested.”

Anyway, we just need to get part of the buffer when unpacking:

import os
import struct
from multiprocessing import shared_memory


type_size = struct.calcsize('B')
chunk_size = type_size * 2
shm = shared_memory.SharedMemory(create=True, size=chunk_size)

pid = os.fork()
if pid == 0:
    # child
    values = [1, 2]
    offset = 0
    struct.pack_into("2B", shm.buf, offset, *values)
    exit(0)

# only the parent will reach here
os.wait()

# get produced value and cleanup the shared memory
result = struct.unpack("2B", shm.buf[:chunk_size])  # buffer size may be larger than indicated
shm.close()
shm.unlink()

print(result)

That should do it.

Thanks again!

. Facundo

4

Looks like a bug to me. It’s also broken in 3.12.10 (but not 3.12.9).

5

Thanks, @facundo that solved the buffer size problem. Now it works for me with this:

Python 3.12.9+ (heads/3.12:969631aec9b, Mar 12 2025, 05:54:09)

but fails with this:

Python 3.12.11+ (heads/3.12:aaca85949ae, Jul 11 2025, 09:01:29)

That’s not as narrow as @colesbury’s observation, but serves to support the observation that something broke on the way to 3.13.

6

Thanks @colesbury @smontanaro , this is indeed general and not something weird in my corner.

I wonder if the situation is that it should not wait for that pid to be gone (i.e. remove the waitpid as the process will always end before), or there are some cases/platforms/race conditions where the process is still there, and needs to be waited.

Probably the best to do there is just wrap it with a try/except ChildProcessError: pass like the same code does in other corner.

I’ll open a bug.

Thank you again!

7

The issue: Regression on multiprocessing’s resource tracker in 3.13

8

When using multiprocessing facilities, please don’t call os.fork directly but use the Process class instead.

(and if you do use os.fork - in which case you are really on your own -, you should not call exit in the child but os._exit)

1 Like
9

Thanks for the tip!