I know you are, but others like @davidism aren’t so thrilled if the outcome is “yet another file to keep track of” if we can help it.
Thanks for the segue… ![]()
While I was trying to fall asleep last night (yes, this whole endeavour consumes way too much of my mental energy), I started to think about how this compared to pyproject.toml and why that seems to be successful (and please don’t argue with me if you think it isn’t; that would be a distraction for this topic). And I thought about the three things pyproject.toml gives you:
[project]allows for some tool portability which also helps to prevent lock-in.[project]allows for some data analysis most of the time (but withdynamicand[tool]there are escape hatches for tools that need them which technically hurt this and point 2, but people seem generally okay with the trade-off).[build-system]allows for programmatic understanding of how work with a source tree, in this case to create a wheel.
I think the mistake I have made is trying to meet all three points with lock files. I tried to make this happen with the separation of lockers and installers. But the split ends up hampering lockers so much that they can’t really operate very much outside of the spec and that prevents lockers from filing in gaps or innovating. And that leads us to the current situation where no one is getting what they want from the PEP because there’s always that next use-case to support.
So what if we stop trying to separate lockers from installers, instead letting tools write lock files with some tool-specific info that no one else might be able to work with? That tosses out point 1, but I don’t know if tool lock-in matters that much for something that’s meant to be written by code and not by hand.
You can still have point 2 where basic information can generally be available, maybe even in most cases, but in some instances some data will be tool-specific. But based on our experience with dynamic and [tool] in pyproject.toml it doesn’t seem to be a massive concern that you can’t always access all data upfront statically (Brainstorming: Eliminating Dynamic Metadata - #110 by ncoghlan notwithstanding).
Point 3 is possible if we make lock files document what tool to install and how to run the tool that created them (think [build-system] but with a CLI API). And this doesn’t necessarily need to be extensive and cover all use-cases, but I’m sure we can think of some common use-cases for what should be specified in the API. And thanks to lock files not meant to be hand-written we don’t have to worry about being too verbose or anything.
Does reframing things from this perspective and ditching the locker/installer dichotomy hold enough promise to suggest we could still maybe get that “any and all scenario” lock file dream to come true? I would bring back allowing for multiple lock files as exporting with no [tool] tables would be like having a standardized requirements.txt file for when that’s critical to someone. The PEP would be changed as necessary so that there was flexibility at all key points for tools to tweak things as necessary when they needed it while covering the common scenarios.