That is difficult if there’s any kind of dynamic resolution. Part of what has been percolating for me is whether it makes sense to have the hardware implementation metadata be separate from the python package metadata. This is swimming into implementation, so take it with a grain of salt, but here goes:
If you allow there to be a kind of placeholder python package that takes care of installing appropriate hardware support when it is itself installed or otherwise “activated”, then you would not necessarily need to capture the hardware support in an environment spec. If you can factor out the hardware-specific parts, then you leave the environment flexible to be instantiated on other hardware. For the sake of reproducibility, you’d still capture the hardware packages, but by keeping them separate from the “normal” packages, you’d maintain the ability to pre-generate dependency graphs - they’d just stop at stubs for the hardware implementations.
Is this a high priority goal of the community?
I hope so! It is a high priority at NVIDIA. I’ll do what I can to help make it happen.
Or is install-time arbitrary code execution done for the foreseeable future?
Realistically, I think this is likely. The good news is that it’s not arbitrary code execution in setup.py. The PEP 517 build backend approach puts the redirection logic into a wheel that can be statically inspected without running it. It also means the redirection logic is centralized and shared among many packages, instead of being copied and scattered across all the packages.
In the absence of a PEP 517 build backend to do redirection, I think we’d probably be stuck with vendor-provided metadata plugins to pip and other package managers. The thing I really like about the build backend approach is that it obviates the need for the user to do any kind of pre-installation to set things up or activate some behavior.
This said, if you need fully resolved package graphs and you can’t tolerate having stubs for hardware implementations, I don’t see any way to completely avoid dynamic behavior at some point in the process.