A proposal for sdist build complexity signaling, providing user agency

It definitely feels like a gray area to me. Imagine instead that nvidia-stub gathered up a bunch of information from the build and/or host environment, and passed that as a JSON file to some build farm, which pulled the source from somewhere, spun up a container, and built the exact ABI matching library on demand. Is that any less or more gray?

Are we implicitly assuming [build-system] means “build the thing locally on my machine”?

I think it violates user expectations, and if not the letter of the specification, then it certainly does things that certain policies exist to prevent right now. nvidia-stub basically exists to do an end-run around pypi not allowing pep508 dependency links to dependencies that aren’t on pypi.

1 Like

Is it really violating things any worse than the build frontend downloading the backend automatically, along with any other specified dependencies (e.g. from get_requires_for_build_wheel()?

They could easily enough list all the wheels as build dependencies and then serve up the one that matches. It’d be a huge waste of time and bandwidth, but if it avoids “violating” expectations, it might just keep them trying to actively produce a better experience rather than just forcing them away from the PyPI-based ecosystem entirely.

3 Likes

I’m not saying it’s a bad solution, and I don’t think it’s something I would advocate breaking. I’m only pointing out that the situation for what an installer can and can’t assume is not necessarily fully found within the lines of the specifications.

That said, arguably yes from the perspective of either reproducibility based on artifacts on pypi or from the security argument that was made requiring dependencies to be hosted on pypi, but the solution probably shouldn’t be to break this use case, but to document it as possible (so that those reviewing dependencies understand this without needing to have deep knowledge of packaging) or relax the requirements on pypi since they aren’t enforceable reasonably anyhow so that this kind of solution stops being necessary.

1 Like