And I don’t have any insight on what “large” is in this case.
For them, sure, but since I’m doing all the work right now it isn’t a blocker for me. And I’m not suggesting something could come about later, but I need to start somewhere and my needs are explicitly met by restricting myself to wheel files, so I’m managing scope creep by worrying about the simplest requirement to get what I need out of this.
It really depends on whether you want a lock file upfront to be usable across platforms, or if you’re okay amending new platforms as necessary to the lock file? For instance, does the lock file need to work on Windows, manylinux, macOS, etc. all at once in one go, or can you start with your platform and add others as people come forward with the appropriate details for their platform (e.g., wheel tags, etc.)?
I’m personally not aware of anything requiring a PEP to make this work. What are you specifically thinking of?
To start, I think there’s some over-assigning of the term “reproducibility” to what I was proposing. To be clear, my goal was to make sure users got the same wheels downloaded and installed when using a lock file with the same markers and wheel tags. So there was never any expectation that the files themselves would be exactly the same simply because stuff in the
scripts/ directory will end up with their shebang rewritten.
There’s basically three use cases I had in mind when I wrote the PEP.
One is security/consistency. For work we download wheels and install them for distribution within VS Code extensions (e.g., the Black extension ships with a copy of Black). We need to make sure that what we use in CI is the same as what gets shipped ultimately to the VS Code Marketplace (so version pinning). But we also have to do our due diligence as much as we can to make sure no one “pulls a fast one” and substitutes out some file for something that is completely different (that’s where verifying hashes comes in). We also have a desire to have lock files be platform-specific as we can ship extensions that are platform-specific, e.g. ship the accelerated version of Black on macOS, Windows, and Linux for Python 3.11 - 3.7 and then fall back to the pure Python support on other platforms.
Two, consistency when your dev and production environments are different. This is for the macOS or Windows developers who push to a Unix production environment. Being able to all agree to what you are developing against locally is consistent as well as how you all agree with what is in production takes a lock file that can handle different platforms appropriately (e.g., pip-tools doesn’t do this because it creates its resolved list of packages and versions for the current platform and then fakes cross-platform support by listing every single wheel possible when you specify
--generate-hashes; that doesn’t guarantee any platform-specific stuff on other platforms is included).
Three, is simply avoiding “it works on my machine”. Beginners do not always understand that they may have accidentally installed something different than their friends did even if things are pinned down to the version as you may have gotten the pure Python wheel and they got the accelerated version. And try as they might, not all packages will nail the compatibility story perfectly (Python’s own stdlib can’t even pull that off). So having a lock file that can differentiate down to the wheel means it’s potentially easier to diagnose when there’s drift in what was installed.
The people who really care will cache the wheels they use and restrict their installers to just pulling from the personal index/cache, so they effectively cut out PyPI and any potential external source of bytes. I have also used pip-tools with
--generate-hashes, stripped out the hashes for any files that don’t pertain to me, and then use
pip install --no-deps --require-hashes --only-binary :all: -r requirements.txt (and this is what I have pip-secure-install · Actions · GitHub Marketplace · GitHub do and thus what we use at work).