Would it be such a bad thing to have the selector package but let it do its work at runtime rather than during a pip install? I know this is against the status quo but say we had a pip install-able bootstrap package which doesn’t contain CUDA but instead contains some initialize_cuda(version) function. Dependent code can call the function as boilerplate at the top of their code and the function will, the first time, download the appropriate libraries into somewhere external to the Python environment (and writable!) like ~/.cache/python-cuda/{version} then subsequently just return immediately.
The benefits would be:
- The bootstrap package is free to be non pure Python and can therefore call the magic C++ APIs to figure out what hardware it has available (as before)
- Pip/packaging/importlib/PyPI don’t need to know or care what’s going on with GPU tags nor invent a new tag standard to accommodate them
- The CUDA binaries don’t need to live on PyPI
- The boostrap process will do the right thing and rerun itself if someone swaps out their GPU for an incompatible one
- If CUDA do ever find a way to slice their binaries up so that you can have the bits you need without dead weight or compatibility code (at least I’m guessing that’s why they’re currently so unbelievably huge) then the bootstrap package should be easily adjustable to the refined DLL selection process
- This package should be narrowly scoped enough to be comfortably maintainable by a small group of people who know their way around the world of GPU compatibility
- If multiple versions of a CUDA-dependent package use the same CUDA version then you get a nice bandwidth saving since you don’t need to redundantly re-download CUDA (I’m somewhat biased on this one since, in my rural world of sheep and hay fever, I only get ~500kB/s bandwith)
- End user packaging tools (e.g. PyInstaller/Nuitka/py2exe) don’t then immediately inherit the problem of having to process these new GPU tags with the inevitable request that packagers will want to combine the contents of multiple CUDA wheels into one fat package so that it’s not locked to the same hardware as whoever built the application (I’m very biased on this one since I co-maintain PyInstaller and I already want to crawl into a dark corner as soon as I hear the word GPU)
It does mean that something like pip freeze isn’t giving you the full picture anymore though…
(Apologies if this is all very naive)