I have been reading discussions here about issues similar to mine, however was not able to find something more concrete.
Basically my problem is, I have a bigger project written in C (libraries, executables). In there also are Cython based and manually written extension modules for CPython. These function as a kind of glue between the C world and Python world.
The whole thing is build with CMake, so during a run CMake first builds the required libraries, then generates the Cython C code and starts a PEP517 frontend, which calls backend and that create sdist, call CMake again and builds the wheel.
Now, the sole idea behind the Python module is to allow accessing the C symbols from Python. And I want to support the ability for the user to use multiple Python environments and version, to allow them to recompile the module if they want so. Given the fact that Python is just a smallish part of the overall project, it makes no sense to distribute the shared library (.so, it’s a Linux project) in the wheel, but it should be distributed via normal, distribution specific channels (DEBs, RPMs etc).
So, is there a way, was there a discussion or proposal to somehow specify in the wheel that the module depends on some shared library and that the installer should make sure the required dependencies can be found by the loader in a standard manner (commonly via where the ld would have normally looked)?
There have been many discussions, but unfortunately nothing implemented. It’s not clear how this would work, given that there’s so much inconsistency across systems in how they distribute shared libraries, track binary compatibility, etc.
I’d recommend you to publish a wheel containing the shared library, but make the build system able to just use whatever is on the system instead. This way users that pip install will have evreything working, and distro packagers or users that want to build from source can use the library that is on the system.
I understand. It probably has many facets I cannot fully comprehend at the moment. On the other hand, the binary wheels are already system specific…
If I take it correctly, I would then need some wrapper around the compiled extension module which would check for the existence of linked library and fail with some exception if it does not exist. Hmm. (Because there is no pre-installation hook for wheels. [And it would be probably security issue anyway.])