I’m trying to figure out the best packaging strategy to distribute an external binary alongside my Python package.
I’m working on a Python wrapper around an old complex Fortran software that I compile with gfortran. The compiled executable is small (<1 MB), but I need to support multiple platforms (Linux, macOS, Windows), which means building a different binary for each platform.
Currently users need to install the external Fortan software manually. In my Python wrapper I call the external software by relying on an environment variable pointing to the executable. But this approach is not very robust as it can lead to mismatches between the version expected by my library and the version installed by the user.
Ideally, I would like a simple pip install mypackage to provide the correct binary automatically for each platform, which would significantly improve the user experience.
I am wondering what’s the recommended approach is in the Python packaging ecosystem for this kind of use case. Would it be acceptable to bundle platform-specific binaries in wheels? Should these binaries be included in the source repository, or only added at build time?
I don’t think there is such a thing as a best practice here but one thing to aim for if you can is being able to pip install your package from an sdist and it’ll compile the executable from source. That’s useful for niche platforms, older glibc/macOS versions, repackagers, anyone who’s choosy about compiler flags or debug symbols, …
To add to this and the above scikit-build link, there’s also meson-python and meson. The idea here is you’d set up a meson build for the dependent binaries and make your Python project have a build dependency on meson-python.
That’s how SciPy manages its fortran dependencies. But also SciPy depends on fortran extensions, not independent binaries, so it’s a little different.
But in general I agree, you want to include the sources for all your native dependencies and build them if they aren’t available as binaries at build time.