To be blunt, this is a terrible argument. PyPI is not a secure system, so if you’re that tight on security, you should not be installing from PyPI anyway, but should be using a (curated, security vetted) mirror. And if you’re doing that, you can “easily” (I appreciate that when it comes to corporate security, nothing is “easy” 
) host lockfiles for the applications you want to support as well.
I appreciate that “we can’t just install from a random URL on the internet” is a real issue in some corporate environments, but I don’t think the Python packaging standards should be designed to work around such ill-advised policies.
Also, if this is your concern, another option is for PyPI to host lockfiles as well as wheels. Then tools could install from the lockfile, without needing that file to be included in the wheel. This better separates the two use cases and makes it easier to audit the lockfile (you don’t need to download and unpack the wheel). I’m not specifically proposing this option, just pointing out that you’ve not really thought through the possibilities, but rather you’ve thought of one specific approach and leapt straight to proposing it as a standard.
Actually, I see that you did consider this option, as “Distribute lock files alongside wheels, similar to .METADATA”. But frankly, your downsides seem pretty weak. It feels like you simply went looking for reasons to reject this option, rather than giving it serious consideration.
Hosted lockfiles don’t have to be mutable. If they were hosted on PyPI, for example, they wouldn’t be mutable. If they were hosted on github, they could be referenced with an explicit commit ID, and hence would be immutable.
If you publish lockfiles in an insecure manner, they are insecure. This is hardly surprising.
To be perfectly honest, my concern about harm to the ecosystem is largely a “gut feeling” based on my years of experience with packaging. That’s not a very good answer, though, so I’ll try to articulate my concerns more concretely. Please understand, though, that these are simply my best attempts to describe my concerns - point by point rebuttals won’t help here, you need to look at the underlying points I’m trying to express.
First of all, the big problem with standards is that once we have them, getting rid of them is next to impossible. If we don’t get the design right first time, we’ll be stuck with something suboptimal. The best example we have of this is the wheel format itself. When we developed it[1], we thought that it was clear and pretty future proof. But we have found over the years that there are limitations, and migrating to a new format is far harder than we expected. To the extent that there’s a significant amount of work going on in the wheel-next project looking at how we can implement a smooth transition to a new version. This proposal isn’t as major as the wheel spec, but on the other hand, it feels like it’s not been thoroughly thought out, so it would be very easy for us to end up with a standard that needs revision or replacement, and that simply isn’t an easy thing to do. Better to wait until we have a better understanding of the problem before committing to a design, IMO.
Also, describing the proposal as “optional” is naive, I’m afraid. Once anything gets standardised, there’s a serious social pressure to use that solution, and not to look at alternatives or to innovate any more. So even an optional standard stifles progress, by locking us into a design.
My second big concern is that even after all the work that’s gone into packaging, we still don’t have a good understanding of how people package applications. We don’t know the constraints they work under, or the use cases they have. That’s a huge gap in our knowledge, and in my view, until we have a much better appreciation of these points, we’re simply not equipped to know what a good solution would even look like. I don’t want this to sound like some sort of perfectionist “we can’t do anything until we have the ultimate solution” demand, it’s not that at all. But even incremental improvements need to fit into the overall strategy, and we don’t have a strategy yet, and I’m not even sure we have enough information to formulate one.
It’s worth remembering that the wheel format was designed for publishing Python packages, not applications. There are significant limitations in capabilities that make wheels inappropriate for many applications. There’s no “post install script” capability, which would be needed to (for example) register the application with the system (start menu and registry entries on Windows, and similar things on Unix). Similarly, there’s no customisation of uninstalls - it’s just a simple “delete all the files that got installed and hope that’s enough”.
Honestly, if we are serious about distributing applications like this, I suggest that we create a new format, designed explicitly for applications. This “pyapp” format could include the application code and a lockfile that describes any dependencies. It could have additional metadata to support things like start menu registration, file type associations, additional cleanup to be done on uninstall, etc. I’m not personally convinced such a format would be the right solution, as opposed to integrating with existing application managers like Windows Store, chocolatey, homebrew, apt and dnf - but it seems like a much more reasonable approach than forcing (part of) what’s needed for application installation into the wheel format.
Basically, though, this proposal just feels like a “quick fix” rather than a well thought out evolution in how Python packaging supports application deployment.
I’ll also make some specific comments on your new proposal.
Most Python projects (certainly libraries) support use with a range of versions of their dependencies. That’s what the dependency specifiers are for, and it’s necessary if the project is to be installed alongside other projects in the same environment. Locking dependencies is only feasible if you only ever intend to install the project in a dedicated, isolated environment. That’s not the normal situation for the vast majority of Python projects.
This proposal only targets the minority of projects that are designed to be installed into a dedicated environment, rather than alongside other projects. Maybe that’s a reasonable subset of projects to consider, but your motivation suggests this is useful for a much broader range of projects than it is.
These are all use cases for locking, but they don’t demonstrate any need for lockfiles to be included in a wheel. If anything, they suggest that lockfiles can be useful even for projects that don’t include them in the wheel, meaning that the proposal isn’t the right solution.
This suggests a misunderstanding of lockfiles. A lockfile doesn’t “lock a package”, it describes an environment, which typically contains many packages. If I do pip lock requests sympy, I get a lockfile which describes an environment containing both requests and sympy. I find it hard to reconcile the fact that you can do this with the view of lockfiles that you seem to hold.
??? So this isn’t a proposal to distribute applications that can be installed via pipx install app? But that’s precisely what you originally claimed it was.
You’ve not given any reason why the lockfile has to be included in the wheel, rather than being published separately. And as I pointed out above, by requiring the lockfile to be in the wheel, you exclude use cases where someone wants a locked installation of a package that doesn’t include a lockfile in the wheel.
The second option is a huge demand on build backends, as you’re requiring them to implement a resolver. IMO, this is an unreasonable thing to expect.
What makes you think that testing is done with the exact requirements in the lockfile? There’s nothing in the proposal that ensures that’s the case. Most projects test with a matrix of configurations, and there’s no reason to assume that this matrix can be captured in a single lockfile.
Without seeing your original, it’s hard to be sure, but overall I don’t think the LLM improved anything. The proposal is in my opinion too wordy, and the arguments given are plausible-sounding but weak, and don’t really stand up to scrutiny (all of which are common problems with LLMs in my experience).
and I was involved in that, so this is first hand knowledge ↩︎