For some packages, we really need to have different builds (producing different wheels), with different build dependencies.
My motivation is in particular about the fluidfft and fluidsim packages and we need to be able to build with and without Pythran and with and without MPI and mpi4py.
I don’t enter in the details of why we need this because it’s going to be long to explain (I can do it if it is useful).
The solution we choose was to build differently depending of the result of
import mpi4py and
import pythran with try/except statements. It is far from optimal but it works. Of course, it forbids the use of isolated build and therefore pyproject.toml. Note that it is similar to how conda works (the package installed by a command depends on what was installed in the environment).
However, I feel that soon having a pyproject.toml will be kind of mandatory to stay mainstream and get good behavior with many Python tools.
Unfortunately, all examples I see with pyproject.toml are very simple and I don’t see any mechanisms to specify options to build the same package in different ways (potentially with different build dependencies) so that it produce different wheels. For example, in conda the fftw package (https://anaconda.org/conda-forge/fftw/files) is build as fftw-3.3.8-mpi_openmpi, fftw-3.3.8-mpi_mpich, fftw-3.3.8-nompi, etc.
Could it be possible to support things like
pip install fluidsim # build with pythran as build dependency (or get the wheel if available) pip install fluidsim[purepy] # wheel without extension and with no MPI support pip install fluidsim[pythran, mpi] # pythran & MPI
It seems to me that currently the brackets in
pip install commands are not at all usable for this usage.
I tried to read PEP 517/518 but I don’t understand how they apply to our needs.
Note also that mpi4py does not upload wheels on PyPI (as far as I understand it is for good reasons because mpi4py needs to be recompiled for different MPI implementations). I know that it was a problem few months ago to add it in
[build-system] requires, but maybe it’s no longer a problem?
I hope someone here could give me some good advice.