I work on cocotb, a framework for testing simulated HDL code, which is written in C++ and Python. Our framework is loaded by a running simulator application and a Python interpreter is embedded.
Recently, a contributor pointed out an issue with the RPATH on our libraries. They contain hardcoded paths to the libpython used to compile the extension modules in our project. This works fine if you build the package in the environment you use it in. However, this is untenable for binary distribution (something we haven’t done yet). His issue has to do with pip reusing the wheel in another conda environment after the original conda environment was destroyed.
How should projects that have extension modules that embed a Python interpreter into some other process source libpython into the library load path?
Just the one’s I’ve thought of
- RPATH pointing to system libraries absolutely (current solution): broken
- RPATH pointing to system libraries relatively: I don’t think we can make the assumption that the site-packages are installed into the same directory structure as the python install, so probably not feasible
- LD_LIBRARY_PATH: not handled automatically by virtualenv or conda right now, so it’s on the user, which is not great…
- RPATH pointing to a libpython shipped with the package: sounds just awful to manage