What to do about GPUs? (and the built distributions that support them)

Welcome back! Your laziness is totally understandable, at least until the kid(s) starts sleeping through the night. You may find my summary post helpful as a starting point, but there’s a ton of good discussion that you’ll eventually need to read.

That’s the way that NVIDIA is doing things now. Looking forward, it seems like there is a general desire to avoid sdists as much as possible, so we need to find a different way. The different way most likely means more environment markers recognized by installers. Because it is desirable for these markers to not be hard-coded into install tools, most (all?) designs here include some kind of component that users must install, which would provide custom environment markers (and maybe other metadata).

Conda represents system state metadata with “virtual packages”: Managing virtual packages — conda 24.7.2.dev36 documentation

These behave the same as packages and participate in solving environments. These started out as implementations in conda itself (conda/conda/plugins/virtual_packages at 82bcb12633cbb3fd0c0837c6f8fc89a5918d0c7e · conda/conda · GitHub), but a plugin mechanism was developed later: Plugin mechanism for virtual packages · Issue #10131 · conda/conda · GitHub

Pip might be able to take this approach, but pip would probably need to take all constraints in an environment into account before this would work (Implementation variants: rehashing and refocusing - #63 by pradyunsg)

2 Likes