(Cross posting from Assistance building pygmo2 wheels · Discussion #1368 · pypa/cibuildwheel · GitHub)
I am collaborating with pygmo2, a Python & C++ under the European Space Agency Github, to try to build and publish manylinux wheels. At the moment the project is only publishing conda packages, the existing wheels got outdated, and no source distributions are on PyPI.
I made significant progress on https://github.com/esa/pygmo2/pull/110 creating a custom Docker image based on
quay.io/pypa/manylinux2014_x86_64 that includes all the dependencies. However, I’m hitting a wall now.
The project does not use a conventional procedure with a Python metadata file (
pyproject.toml, or anything else), but rather a custom
CMakeLists.txt that shuffles the Python files around:
For this reason, to even start cibuildwheel I’m doing
touch pyproject.toml, but of course this ends up producing a hollow, pure-Python wheel:
cibuildwheel: Build failed because a pure Python wheel was generated.
Trying to compile the code inside
CIBW_BEFORE_ALL fails because Python3 is not found:
CMake Error at /opt/_internal/pipx/venvs/cmake/lib/python3.9/site-packages/cmake/data/share/cmake-3.24/Modules/FindPackageHandleStandardArgs.cmake:230 (message): Could NOT find Python3 (missing: Python3_INCLUDE_DIRS Python3_LIBRARIES Development Development.Module Development.Embed) (found version "3.11.0")
I get that probably I shouldn’t try to compile the code against a specific Python version in
CIBW_BEFORE_ALL, and that cibuildwheel will take care of that. But I couldn’t find any technical details in the documentation on how the
manylinux Docker images work, and the “how it works” section of the docs is a bare flow chart with not much explanation.
In summary, I’m a bit lost on how to proceed. I’m wondering if it’s technically possible at all to build the wheel with the current architecture (in which case I’d love to have some guidance), or rather if it will be difficult or impossible and I should first try to standardize the build process using something like scikit-build.