C++ module in child process not seeing environment

Python Help
1

Mock example below.

I have an application that spawns some child processes.
They all need to load a module libAAA.so written in C++ .
Before python run.py the LD_LIBRARY_PATH contains the path to the module and libraries of which libAAA.so depends.

However, the child processes are not able to find the libraries of which libAAA.so depends.

  • If I put workers=1 below, everything is fine.
  • If I put libAAA.so, and all the libraries that this uses in some place like /usr/lib/, while keeping workers=4, it also works.

I need to use multiple workers and not pollute /usr/lib/.

What is an appropriate way to load accomplish that?

# run.py
import os
import time
import uvicorn

from typing import Union
from fastapi import FastAPI
from pydantic import BaseModel
from libAAA import aaa


class ExamplRequest(BaseModel):
    x: int
    y: int
    problem: str
    delay: float


app = FastAPI()



@app.post("/compute")
def read_item(request: ExampleRequest):
    time.sleep(request.delay)
    x = request.x
    y = request.y
    print(f'{os.environ.get("LD_LIBRARY_PATH")}')
    # Use aaa
    return {"problem": request.problem, "x": x, "y": y, "sum": x + y, "pid": os.getpid()}

if __name__ == '__main__':
    uvicorn.run(
        "run:app",
        host='127.0.0.1',
        port=8000,
        reload=False,
        workers=4,
    )

Likely the example can be simplified even further, spawning the child processes by hand. I believe uvicorn uses the multiprocessing module. But just in case, here it is as a FastAPI app, which is what I am actually using.

2

Without knowing the processes involved its hard to guess why the env is not passed around.

You might be better off, as you found, installing your .so into /usr/local/lib (not /usr/lib as the os owns that location).

You could also configure an additional lib directory for the system to search. (Forgot the config place in /etc off the top of my head).

1 Like
3

This post can be closed.

We found it. Somewhere inside the libAAA.so the LD_LIBRARY_PATH was being explicitly set to something else.

1 Like