First, I’m inquiring about the status of Pipfile.lock file. My understanding is that it has been in a discussion phase and remains there. pipenv is the primary implementation as mentioned in the project README. However there is a section regarding eventual implementation into pip:
pip will grow a new command line option, -p / --pipfile to install the versions as specified in a Pipfile , similar to its existing -r / --requirement …
Is integration with pip still a feature that is being explored?
Second, I realize there is focus on pip’s new package resolver. Is a lock file (or a similarly more robust mechanism to reproduce pip dependencies) part of that vision?
If implemented as a separate command (i.e. pip resolve instead of pip install --dry-run), this can be done even before --use-deprecated=legacy-resolver is dropped because the two wouldn’t overlap. But someone needs to actually put in the effort to make the command happen, and everyone seems to be focusing on other things right now. Either we wait for someone to put up volunteer work, or have to put up a plan to recruit someone to do it.
This is less obvious. Commenting as a pipenv maintainer, the current lock file format is likely too limiting to be useful for pip. It at least needs to grow more flexible grouping strategy. And as a pip maintainer, the benefit to pip resolve using a new lock file format is also not significant enough over requirements.txt.
Speaking for myself only, I would like to revive my Structured, Exchangeable lock file format design for this. It is now clear the new format will not happen as a standard because it’s not viable to design something that fits all tools’ needs, but the format will be an improvement to both requirement.txt and Pipfile.lock, and interoperability would be less of a problem with PEP 650. But again, the improvements may not be significant enough to convince pip maintainers as a team to adopt the new format, and status quo may win because it is already implemented and wildly used (even though poorly designed, ill specified, and has weird implementation edge cases).
I appreciate the response. I’m glad to see activity on the pip resolve idea.
My hope is that a robust lock file comes about soon. I have not had luck with reproducing conda environments that are mixed with pip packages. There are a few solutions that aim at conda packages, but avoid pip packages (likely because of the complexity):
@uranusjr@pylang Some questions and some comments - 1) What is the status of this? 2) How can the community put priority around this? I’ve been diving into the python packaging ecosystem, and, compared to Rust’s Cargo, there is a lot to be desired (if you use cargo, you’ll immediately understand). I understand it’s an open source project and people only have so much time. Requirements.txt doesn’t seem like an adequate solution for building, testing and publishing in a pipeline with different dependencies (Pipenv answers this) in today’s world, and I hope you can understand appreciate that statement.
What is it going to take to put a stake in the ground for the community to say we need something better and to phase out requirements.txt? It would make a heck of a lot of sense to integrate the locking feature with pip, and perhaps for Pipenv and pip, replace requirements.txt and the Pipfile in favor of the pyproject.toml file in use for front end, instead of just the backend, see: Quickstart - setuptools 65.3.0.post20220826 documentation (you’re likely well aware of that documentation)
Put some serious thought around this:
deprecate requirements.txt over several years
extend pyproject.toml for front end requirements instead of just the backend build system
implement a locking system like Pipenv uses
Declare and implement building packages within 1 toml file like poetry does - this is obvious and powerful
I think any reasonable person would be very hard pressed to assert that implementing the above wouldn’t provide a radically better package management solution for the language. If you have the time, please read this and prioritize this, this has the potentially to be incredibly impactful in a very positive way for many, many people.
I think if you dig deeper in the conversations that were made around existing proposals you’ll find that many people are unhappy/don’t like how either pipenv or poetry handles this. So there’s a lot more effort involved here than you imply of just adopt x and roll with it.
We definitely don’t need any more encouragment/request/push on how important this is or how much time we should spend on this. There’s no shortage of that! What we do lack though is time and effort from volunteers (you could be one of them!) that are willing to spend a lot of time and effort to do this. We need to design a system that addresses all concerns and then implement it (which implies a lot of systems). We’ve put a lot of serious thoughts (and effort too) on this topic already, but it’s still far from being good enough or adoptable.
Sorry, just saw this. I don’t have much say on the status or trajectory of this topic. AFAIK, PEP 665 was that latest official development on this.
I think the closest generic option to a lock file is a hashed and pinned requirements file generated via pip-tools. That was the same tool Pipenv used under the hood. Coupled with the new pip resolver, I believe there has been some light progress in this area, but pip-tools is not a perfect solution.
My impression from the last round of debates is that many people are still not clear on what problem a lock file (or lock-file standard) solves. Perhaps some are not convinced that there is a problem. To me, at least one problem remains - if I make an environment today, how do I reproduce the same environment years from now?