PEP 751: lock files (again)

Packaging Standards
23

Exactly. Regarding one file vs. multiple files @ncoghlan @DanCardin:

With multiple files you need one file for each combination of extras and groups. That does not scale well with many extras/groups.

Poetry resolves all (root project) extras and dependency groups into one lock file so that you can choose at install time which extras and dependency groups to install from the lock file. Extras can be expressed via markers so that one is easy but for dependency groups you need something else. Currently, Poetry does not lock markers and groups but resolves again with a “lock file repository” at install time, but triggered by Bretts initial draft I worked on Lock markers and groups by radoering · Pull Request #9427 · python-poetry/poetry · GitHub so that we will be able to adopt a standardized package-lock lock file with less effort.

@brettcannon Not sure, if this should be addressed now or later but package-lock might be influenced by PEP 735 – Dependency Groups in pyproject.toml | peps.python.org If you want to lock all groups (like Poetry already does), you need a package.groups to lock which groups include the package and package.marker can be different per group.

For better understanding: In Lock markers and groups by radoering · Pull Request #9427 · python-poetry/poetry · GitHub (in Poetry’s format) it can look like this if the marker is the same for all groups:

groups = ["github-actions", "test"]
markers = "python_version < \"3.11\""

or like this if they are different:

groups = ["main", "github-actions", "test"]
markers = {
    main = "os_name == \"nt\"",
    github-actions = "sys_platform == \"win32\"",
    test = "sys_platform == \"win32\"",
}

(Real world examples from Poetry’s own lock file.)

If there is no standardized way to define groups, we can still put it in the per-package tool section. We can put a dummy marker (union of all markers?) in package.marker and the per-group markers into the tool section. (Of course, that would mean that other installers, which do not consider Poetry’s tool section, may install additional packages, which are not required for a specific group in a specific environment.)

We have that, too. This could be put in the (per-package) tool section. However, that means that the lock file is more bound to a specific installer, i.e. other installers would ignore it…

I think such information can be put into the (global) tool section. It is not relevant for the installer, is it? Thus, it should not even hurt to be tool specific.

4 Likes