Well, one is that there’s no actual necessity that the current project is being locked. For example, I could imagine wanting to lock the “documentation build environment” for my project, which installed things like Sphinx, etc., but did not install the project itself.
Hashing “the inputs” to the lock is a reasonable thing to do, but you can’t be sure what those inputs were unless you record them in the lockfile as well.
Sure, if PEP 735 is accepted, then dependency-groups should be hashed as well. Is that what you meant?
Why do lock files need any interaction with the keys in pyproject.toml? It’s about specifying an environment, not a project.
No. Imagine a tool (called lock-requirements, for example) that generated a lockfile. lock-requirements sphinx pygments is a perfectly valid request to create a lockfile that pins specific copies of Sphinx and Pygments, and it references nothing from pyproject.toml (or anywhere else in the project).
Poetry uses a hash of some keys from pyproject.toml to allow it to validate the freshness of the lock file relative to what is in pyproject.toml. This allows it to catch common bugs where a user updates base requirements in pyproject.toml but forgets to relock the lock file.
So it comes down to what responsibilities we are trying to solve with this proposal.
I guess a generic solution to this would be to save a copy of the locking tool’s input in the lock file, instead of just the hash.
I feel though with PEP 735, it’s not unreasonable for people to quickly write
[dependency-groups]
docs = ["sphinx", "furo"]
into pyproject.toml before running the locking tool. (I guess the tool would then create a separate file for that environment? pylock.docs.toml?)
That’s my personal expectation.
FYI Frost wrote a blog post about how lock files work in PDM.
For what it’s worth, I think it’s more than this. When you run poetry install, Poetry looks at your pyproject.toml to determine the root dependencies, then traverses the lockfile to figure out what to install. If you comment out all of your root dependencies and run poetry install without changing the lockfile, Poetry won’t install anything. In other words, Poetry requires both the lockfile and the project manifest.
IIUC, Cargo is similar, you need both the Cargo.toml and the Cargo.lock to do anything.
At the lockfile standard format specification level, the answer that makes the most sense to me is “ways to determine the ‘freshness’ of the lock file are locking tool specific, and hence appear in the [tools] table rather than in a standardised field (or fields)” (since there’s no one standard input format for the locking process, there can’t be a standard way to check if the resulting output is up to date).
On the installer side, a generic installer will have to trust that the lockfile is up to date - checking that the lockfile is fresh before using it will necessarily be a request made of the locking tool rather than something an installer can do on its own.
Closing the topic so it doesn’t change as I write a draft PEP (which you shouldn’t expect any sooner than June thanks to parental leave).