Selecting variant wheels according to a semi-static specification

Packaging
100

Jumping to a slightly different spin on this idea, how far did we ever look into having custom/plugin-provided environment tags?

Rearranging Oscar’s original example (and going back to claiming one package name for each variant), imaging installing python-flint and having these to choose from:

python_flint-0.6.0-cp312-cp312-win_amd64.whl
python_flint_neutral-0.6.0-cp312-cp312-win_amd64.whl
python_flint_x86_64_v3-0.6.0-cp312-cp312-win_amd64.whl
python_flint_x86_64_v4-0.6.0-cp312-cp312-win_amd64.whl

Obviously we grab the one named python_flint-0.6.0.... But what’s in there? Basically just metadata:[1]

...
Install-Requires: python_flint_x86_64_v4==0.6.0; cpuversion == 'x86-64-v4'
Install-Requires: python_flint_x86_64_v3==0.6.0; cpuversion == 'x86-64-v3'
Install-Requires: python_flint_neutral==0.6.0; cpuversion != 'x86-64-v4' and cpuversion != 'x86-64-v3'

Then the trick is to make cpuversion available. I don’t think we’ve yet come up with a way to do any of our proposals without installing something extra first, but while that currently looks like a big deal, I imagine there’ll eventually be one popular tool and distros will just start bundling it. IOW, no reason to block an idea.

I get that having multiple package names isn’t as convenient as silently selecting the right one, but it also sounds like silent selection will either not work or not be silent. Splitting up into separate packages does help stay under PyPI’s quotas though, which ought to be very attractive for these kinds of packages (if you put all CUDA versions under one name, you’ll probably only fit one version at a time on PyPI).

Using environment markers as the selector mechanism also seems likely to be flexible enough. Otherwise we’re going to invent a new flexible-enough system, which will slow adoption time fairly drastically compared to reusing an existing one (though marker extensions will take time to get out there too I suppose).

  1. IIRC, there’s a self[extra] version of this that would work too, but I don’t know the details well enough to write it up. ↩︎

1 Like