Multiprocessing with spawn and environment variables

Hey folks,

When using spawn as the multiprocessing start method, is there an easy way to persist the environment variables (as a snapshot) from os.environ into the new processes?

We have an old case where fork was used and this all worked out, but now with spawn we lose the environment variables changed in the parent python process when the child starts up. I’m sure I could save them and pass them as a function parameter or something but that seems like something that someone may have already solved for.


1 Like

Answering my own question:

Something else is probably wrong. The environment variables should pass on their own. Example:

from multiprocessing import Pool, get_start_method
import os

def print_test():

if __name__ == '__main__':

    os.environ['TEST'] = 'hello world'
    with Pool() as p:

    os.environ['TEST'] = 'hello again'
    with Pool() as p:

gets us:

hello world
hello again

So something else is afoot.

What was the problem at the end? I am running into something similar.

In my case, spawned subprocesses are about to load a c++ module but this doesn’t see the environment variables. The parent process and the subprocesses do see them.

I don’t fully remember. I think it had to do with the Pool being started before the environment variable was set. (The env var has to be set before the spawn/fork happens).

1 Like

Ok. Thank you. In my case the environment is set before the Python interpreter even starts.

1 Like