File locks. I haven’t really come across anyone who does package locking like we have.
Give me a minute.
The problem is unless I create a tool that people are willing to use that implements this PEP then I won’t get that “real-world stress testing”. And the other issue is the pre-existing lock files weren’t proposed as a PEP, along with their creators admitting there were design choices they would re-evaluate.
As for examples, the code at GitHub - brettcannon/mousebender at pep should work for per-file locking if you want to try out other examples. Admittedly, I had to write my own resolver to make that PoC work, so I can’t promise it to be bug-free.
So are you hinting you want to propose it as a PEP once you have released it and gotten real-world experience with it?
It’s based on PDM which is based on Poetry, so I don’t think it’s that radical. Really the biggest difference is the flattening of the package/file list instead of following a graph so reading diffs is more straight-forward. And it has been changed based on feedback from e.g., Randy, so while it might be different it isn’t without input from pre-existing implementers.
I was actually planning on updating the PEP to say entries need to differ by name, version, and marker requirement to be as flexible as possible about selecting the right scenario.
The PEP says you have to choose one, but basically it’s there because previous feedback related to needing sdist support flowed into, “I need all possibilities supported as I don’t know what it will take for a user’s platform to get a working wheel”.
Change that to past tense and you’re right.
Give me a minute and I have an idea that may simplify things a bit.
I’m not sure what you mean by a “revision” compared to a “Git SHA”. What’s the difference?
And the PEP asks for the “commit ID” which “MUST be immutable”, which means Git SHA commit.
Yes.
So, I’ve been thinking about how to simplify per-file and per-package locking and melding them together more as well as supporting multiple groups of dependencies, i.e. locking for all extra combinations for a pypproject.toml
in a single file (although some are asking for more files; I’ll never win).
An idea – and I stress this is an idea and I won’t move forward with it unless people show appropriate interest – is the following:
- Drop
[[file-locks]]
and[[package-locks]]
(and related other bits) - Introduce
[[supported-environments]]
to represent what environments the lock file explicitly supports- It’s optional, so that if you leave it out it’s best-effort support for your environment
- Finding the appropriate thing to install follows per-package guidance in the PEP
- This is effectively assuming per-file locking is simple enough to not need to be as much of the focus of the PEP and is essentially a simplified scenario of per-package locking
- Drop
dependencies
and introduce[dependency-groups]
- Much like PEP 735, table of named groups of dependencies
- The default group is the empty string as it sorts to the top no matter what
project.dependencies
plusproject.optional-dependencies
named with brackets, extra names are normalized, and extras as sorted, but otherwise list the complete list of dependency specifiers, effectively flattening the extras with the required runtime dependencies- If PEP 735 comes to fruition then they just slot right now based on the dependency group name
- Can either follow Alyssa’s advice and normalize with markers by adding a
group
marker variable, or introduce agroups
/part-of
/applies-to
/bikeshed-name-ater array in[[packages]]
of dependency group names if the package version applies to only a select set of groups
And here’s the PEP example converted (due not you will need to scroll the text box to see the whole thing):
version = '1.0'
hash-algorithm = 'sha256'
[[supported-environments]]
name = 'CPython 3.12 on manylinux 2.17 x86-64'
marker = ""
wheel-tags = ['cp312-cp312-manylinux_2_17_x86_64', 'py3-none-any']
[[supported-environments]]
name = 'CPython 3.12 on Windows x64'
marker = ""
wheel-tags = ['cp312-cp312-win_amd64', 'py3-none-any']
[dependency-groups]
'' = ['cattrs', 'numpy']
[[packages]]
name = 'attrs'
version = '23.2.0'
multiple-entries = false
description = 'Classes Without Boilerplate'
requires-python = '>=3.7'
dependents = ['cattrs']
dependencies = []
direct = false
files = [
{name = 'attrs-23.2.0-py3-none-any.whl', origin = 'https://files.pythonhosted.org/packages/e0/44/827b2a91a5816512fcaf3cc4ebc465ccd5d598c45cefa6703fcf4a79018f/attrs-23.2.0-py3-none-any.whl', hash = '99b87a485a5820b23b879f04c2305b44b951b502fd64be915879d77a7e8fc6f1'}
]
[[packages]]
name = 'cattrs'
version = '23.2.3'
multiple-entries = false
description = 'Composable complex class support for attrs and dataclasses.'
requires-python = '>=3.8'
dependents = []
dependencies = ['attrs']
direct = false
files = [
{name = 'cattrs-23.2.3-py3-none-any.whl', origin = 'https://files.pythonhosted.org/packages/b3/0d/cd4a4071c7f38385dc5ba91286723b4d1090b87815db48216212c6c6c30e/cattrs-23.2.3-py3-none-any.whl', hash = '0341994d94971052e9ee70662542699a3162ea1e0c62f7ce1b4a57f563685108'}
]
[[packages]]
name = 'numpy'
version = '2.0.1'
multiple-entries = false
description = 'Fundamental package for array computing in Python'
requires-python = '>=3.9'
dependents = []
dependencies = []
direct = false
files = [
{name = 'numpy-2.0.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl', origin = 'https://files.pythonhosted.org/packages/2c/f3/61eeef119beb37decb58e7cb29940f19a1464b8608f2cab8a8616aba75fd/numpy-2.0.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl', hash = '6790654cb13eab303d8402354fabd47472b24635700f631f041bd0b65e37298a'},
{name = 'numpy-2.0.1-cp312-cp312-win_amd64.whl', origin = 'https://files.pythonhosted.org/packages/b5/59/f6ad30785a6578ad85ed9c2785f271b39c3e5b6412c66e810d2c60934c9f/numpy-2.0.1-cp312-cp312-win_amd64.whl', hash = 'bb2124fdc6e62baae159ebcfa368708867eb56806804d005860b6007388df171'}
]
Tomorrow is BC Day so I won’t be reading any responses, and I have stuff going on Tuesday and Wednesday, so there’s a chance I might not reply for a few days.
I will wait to see if this idea gets more or less support than the PEP as it stands. At that point I will take stock and decide what next steps are to unblock progress towards a conclusion (potentially with a poll).