# Help me shake out pip-lock 🔒

**URL:** <https://discuss.python.org/t/help-me-shake-out-pip-lock/107232>\
**Category:** Packaging\
**Tags:** packaging-help, package-installation\
**Created:** [May 7, 2026, 5:58pm UTC](https://discuss.python.org/t/help-me-shake-out-pip-lock/107232 "2026-05-07T17:58:50Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![bernatgabor](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/bernatgabor/32/3003_2.png) [@bernatgabor](https://discuss.python.org/u/bernatgabor)\
**Post date:** [May 7, 2026, 5:58pm UTC](https://discuss.python.org/t/help-me-shake-out-pip-lock/107232/1 "2026-05-07T17:58:50Z")

</div>

Hi! 👋 I’ve been building a new top-level command for [`pip-tools`](https://github.com/jazzband/pip-tools) called `pip-lock`. It produces a [PEP 751](https://peps.python.org/pep-0751/) `pylock.toml` from the inputs `pip-tools` already understands (`pyproject.toml`, `requirements.in`, raw arguments) and adds [PEP 735](https://peps.python.org/pep-0735/) `[dependency-groups]` parsing for the new command. The output is a cross-platform lockfile that any [PEP 751](https://peps.python.org/pep-0751/) installer can consume.

The work lives on PR [#2380](https://github.com/jazzband/pip-tools/pull/2380) and is ready for real-world testing. pip 26.1+ ships [`pip lock`](https://pip.pypa.io/en/stable/cli/pip_lock/) and a [`pip install -r pylock.toml` / `--lockfile`](https://pip.pypa.io/en/stable/cli/pip_install/#cmdoption-lockfile) reader, both labeled experimental. uv ships [`uv export --format pylock.toml`](https://docs.astral.sh/uv/reference/cli/#uv-export) on the writer side and `uv pip install --lockfile` on the consumer side. With several independent implementations in the field at different points on the maturity curve, comparing outputs across them is a cheap way to find the spec’s corners; that’s what I’m asking for help with.

## What I’d love help with

Try `pip-lock` on a project you already maintain. Any datapoint helps; you do not need a project that matches a particular shape. The shapes below stress code paths I want to exercise, and a small `requirements.in` is as useful as a monorepo with conflict groups. What matters is that the inputs come from a project I did not test against, instead of from the few projects I happened to pick.

- A library or application with several extras and dependency groups. Bonus points for projects that already declare [`[tool.uv].conflicts`](https://docs.astral.sh/uv/concepts/projects/config/#conflicting-dependencies) for mutually exclusive variants (multiple lint pins, GPU/CPU builds, alternative TLS backends, …); the same shape works in `[tool.pip-tools].conflicts`.
- A monorepo that resolves for several platforms and Python versions at once. The marker-driven cohort scan does the most work when the matrix is non-trivial.
- A project with VCS dependencies pinned to commit SHAs.
- A project with a non-trivial [`[build-system].requires`](https://peps.python.org/pep-0518/) you want captured in the lock alongside the runtime graph.

## How to try it

```console
$ pip install --pre 'pip-tools @ git+https://github.com/jazzband/pip-tools.git@refs/pull/2380/head'
$ pip-lock # universal lock for the project
$ pip-lock --check # CI: fail when the lock drifts
$ pip-lock --dry-run # preview without writing
$ pip-lock --all-extras --all-groups # the maximal lock
$ pip-lock --no-universal # current platform only, fastest

```

The full README walkthrough lives in the [PR description](https://github.com/jazzband/pip-tools/pull/2380) and the [`Example usage for pip-lock` section of README.md on the PR’s branch](https://github.com/gaborbernat/pip-tools/blob/pep751/README.md#example-usage-for-pip-lock).

## How `pip-lock` relates to the other PEP 751 producers

Three other tools write `pylock.toml` from a Python project today: `pip lock`, `uv export --format pylock.toml`, and `pdm lock --format pylock` (also `pdm export -f pylock`). They make different trade-offs, and `pip-lock` makes a fourth set. Here is the feature surface at a glance:

| | `pip lock` (pip 26.1) | `uv export` | `pdm lock` | `pip-lock` (this PR) |
| --- | --- | --- | --- | --- |
| universal lock default | no | yes | yes | yes |
| PEP 735 groups | no | yes | yes | yes |
| declared conflicts | no | yes | yes | yes |
| multi-variant emit | no | [PR #14728](https://github.com/astral-sh/uv/pull/14728), unmerged | no | yes |
| resolver runtime | Python | Rust | Python | Python |
| status | experimental | stable | stable | proposed |
| reads `pip-tools` inputs | yes | no | no | yes |

[`pip lock`](https://pip.pypa.io/en/stable/cli/pip_lock/) is an experimental subcommand in pip 26.1+. It uses pip’s resolver, reads `requirements.in`-style inputs plus `pyproject.toml`, and writes a single-environment `pylock.toml` for the interpreter pip runs under. The command has no native cross-platform mode and no universal lock across multiple Python versions in one invocation. [PEP 735](https://peps.python.org/pep-0735/) dependency groups and declared conflicts are not on its surface yet. The reader side ([`pip install --lockfile pylock.toml`](https://pip.pypa.io/en/stable/cli/pip_install/#cmdoption-lockfile)) is also experimental in the same release.

[`uv export --format pylock.toml`](https://docs.astral.sh/uv/reference/cli/#uv-export) produces `pylock.toml` from a [uv](https://github.com/astral-sh/uv) project. The export is single-resolution: every package picks one version, and markers describe which environment each candidate fires on. The “multi-use” PEP 751 export that emits multiple version variants for one package under disjoint markers is in motion upstream as [astral-sh/uv#14728](https://github.com/astral-sh/uv/pull/14728) and has not landed in a public release. On the consumer side, [`uv pip install --pylock`](https://docs.astral.sh/uv/reference/cli/#uv-pip-install) and [`uv pip sync --pylock`](https://docs.astral.sh/uv/reference/cli/#uv-pip-sync) read PEP 751 today; they emit an experimental-feature warning unless the user passes `--preview-features pylock`.

[`pdm lock --format pylock`](https://pdm-project.org/en/latest/reference/cli/#pdm-lock) and [`pdm export -f pylock`](https://pdm-project.org/en/latest/reference/cli/#pdm-export) ship in [PDM](https://pdm-project.org/). PDM had cross-platform locking before PEP 751 existed (its team co-authored the spec), so the producer is one of the more mature ones; the [tracking issue](https://github.com/pdm-project/pdm/issues/3439) for full PEP 751 support closed in mid-2025 and bug fixes through 2026 have closed most of the corner cases. Choose this if PDM is the project’s workflow root.

`pip-lock` (this PR) is the new third command in `pip-tools`, alongside `pip-compile` and `pip-sync`. The default lock is universal across the project’s `requires-python` and the built-in platform matrix; a marker-driven cohort partitioner collapses target environments that share a dependency closure into one resolver invocation, then dispatches the cohorts to worker processes for parallelism. [PEP 735](https://peps.python.org/pep-0735/) dependency groups are read for this command (the existing `pip-compile` and `pip-sync` siblings do not parse `[dependency-groups]` yet; that work is tracked separately in `pip-tools` issue [#2062](https://github.com/jazzband/pip-tools/issues/2062)). `[tool.pip-tools].conflicts` mirrors uv’s shape; the difference is that `pip-lock --all-groups` accepts the request and emits one `[[packages]]` entry per conflicting variant under disjoint markers instead of refusing, so a single `pylock.toml` carries every variant a downstream installer can pick from with `--group <name>`. Resolution runs in Python on top of pip’s resolver, so wall-clock times will lag uv; the design goal is to produce a correct [PEP 751](https://peps.python.org/pep-0751/) lockfile from the same authoring inputs `pip-compile` already understands, including the multi-variant case. Choose this if the project already uses `pip-tools`, if the multi-variant emission is what you want, or if you want a producer that lives inside the `pip-tools` workflow.

## Cross-checking with pip and uv

Once you have a `pylock.toml`, install it the same way every other PEP 751 lockfile gets installed:

```console
$ pip install --lockfile pylock.toml . # pip 26.1+
$ uv pip install --lockfile pylock.toml . # uv

```

A divergence between any two of `pip-lock`, `pip lock`, and `uv export --format pylock.toml` is interesting evidence: sometimes a real bug, sometimes a spec ambiguity that needs sorting out between the implementations, sometimes a deliberate design choice on one side. The bench is triangulation, not an oracle. When you find one, please reach out to me with whatever you have to hand: the `pyproject.toml` (or a reduction if you can produce one easily), the `pylock.toml` files produced by each tool, and the exact command line you ran on each side. The size of the reproduction does not matter; the existence of the reproduction does.

## What I’m looking for feedback on

The first bucket is **correctness bugs**. Anything that produces a lockfile one of the implementations rejects, picks a different version of a package than the others given the same inputs, or fails the disjointness check on a project where uv is happy. A divergence on a real project’s lockfile that survives the design-difference filter (project-as-package, conflict variants under markers, no-universal scoping) and reproduces against the latest commit on the PR is a strong signal. If something looks wrong, it probably is; please reach out to me.

The second bucket is **usability** : error messages, defaults, and the intuitiveness of the CLI surface. When a same-name marker collision raises, the witness environment plus the conflicts hint should tell you what to change in `pyproject.toml`. When `--uploaded-prior-to` rejects a stale pip, the error should say what version is needed. When `--platform some-typo-arch` falls through to the synthetic-os-arch fallback, the marker emitted should match what you wanted. If any of those messages leave you guessing, the text is a doc bug; flag it. The same goes for the README itself: the deeper documentation under `Example usage for pip-lock` on the PR’s branch covers the realistic flag combinations, and I would love feedback on which scenarios still feel ambiguous.

The third bucket is **performance**. `pip-lock` will not be uv-fast: uv resolves in Rust against a tightly engineered native solver, while `pip-lock` runs in Python on top of pip’s resolver, so an order-of-magnitude gap is the floor. The cohort partitioner is what keeps the universal mode tractable; it collapses target environments that share a dependency closure into a single resolver invocation:

```plaintext
   17 platforms ┌─ resolve once ─┐
   ────────────── ┌─ cohort 1: 3.10 envs ─────────────►│ (parallel) │
   × │ └────────────────┘
   5 python versions │
   ────────────── partition cohort 2: 3.11 envs ──────────► resolve once
   = 85 target envs by marker ... ...
                       equivalence
                          └─ cohort N: distinct dep closure ──► resolve once
                                                                  ─────────────
                                                                  N invocations
                                                                  (N << 85)

```

As a rough datapoint, a recent run on [`datamodel-code-generator`](https://github.com/koxudaxi/datamodel-code-generator) `main` (124 packages, conflicts declared, `--no-default-groups` baseline, single laptop sample) measured `uv lock` at around 460 ms; `pip-lock --no-universal` with a warm cache at around 1.2 s; `pip-lock --no-universal --rebuild` (cold cache) at around 3.4 s; and `pip-lock` universal across 17 platforms and 5 Python versions (85 target environments) at around 10.3 s with the cohort partitioner doing the work. Those are single-sample numbers on one machine and will vary with project shape, package server latency, and matrix size. The interesting numbers are the comparative ones from your project: the matrix dimensions, the `Locking for N platforms x M python versions` line printed at the start of a run, the wall-clock time, and the cohort count after partitioning. Datapoints from larger or more conflict-heavy projects are the input I need to prioritise the next round of optimisation.

## Where to send feedback

Two channels: this thread (replies here, or via direct message here) and the PR itself at [https://github.com/jazzband/pip-tools/pull/2380](https://github.com/jazzband/pip-tools/pull/2380) for review comments, line-level questions, and anything that pairs naturally with a diff. For background on the installer side, pip’s PEP 751 support is documented at [https://pip.pypa.io/en/stable/news/](https://pip.pypa.io/en/stable/news/) and uv’s writer/reader pair lives at [https://docs.astral.sh/uv/concepts/projects/sync/#exporting-the-lockfile](https://docs.astral.sh/uv/concepts/projects/sync/#exporting-the-lockfile).

Thanks ahead for any help you can provide getting this right from the get-go. 🙏

## PS. Why `pip-lock --all-groups` does not refuse a conflicting request

If you set `[tool.pip-tools].conflicts` on a project that declares mutually exclusive groups (a CPU build vs. a GPU build, several alternative TLS backends, several pinned linter variants like `black22` / `black23` / `black24`) and then run `pip-lock --all-groups`, `pip-lock` accepts the request and emits one `[[packages]]` entry per conflicting variant, each carrying a marker that fires only when the user requests that group and _not_ the others. uv refuses the same request with “you asked for a contradictory combination”. Both behaviors are PEP 751 conformant; they reflect different defaults about who decides at lock time.

uv’s refusal is better when the lockfile’s audience is one install scenario. A team locking for “the production deployment” or for a single CI job passes `--all-groups` thinking “include everything”, forgets about the declared conflicts, and uv’s error catches the mistake before it becomes a confusing install-time failure on whichever variant happens to win. The same applies to a [tox](https://tox.wiki/) or build-matrix repo where each job has its own pinned variant: the author already knows which group each job wants, runs `uv export --group <variant>` per job, and the refusal reinforces “lock per variant” as the right shape. Audit-first teams that want `pylock.toml` to be a 1:1 bill of materials for a specific install scenario also benefit from uv’s strictness, because three `[[packages]]` entries for `black` with disjoint markers in one file, even when formally correct, take longer to read at a glance than three separate lockfiles each holding one version.

`pip-lock`’s accept-and-emit-variants is better when the lockfile’s audience is several install scenarios served from one artifact. A library or framework publisher who wants downstream consumers to pick a variant at install time without re-running the resolver gets that with one `pylock.toml` and three different `--group` flags on the consumer side; the uv approach forces the publisher to ship N separate lockfiles or pushes the resolver work onto every consumer. A monorepo whose deploy pipeline supports `--group cpu` and `--group gpu` (or `--group postgres` and `--group sqlite`) from one canonical lockfile committed by SRE benefits from “lock once, install many ways”; generating N lockfiles per variant means N commit churns whenever any unrelated dependency moves. Dockerfiles with multiple stages benefit too: the build stage runs `pip install --lockfile pylock.toml --group dev` while the runtime stage runs `pip install --lockfile pylock.toml --group prod`, both reading one authoritative file instead of juggling `pylock.dev.toml` and `pylock.prod.toml`.

The simplest heuristic: if the lockfile’s audience is _one_ environment (a service deployment, a CI job that already knows what it wants), prefer uv’s strictness; failing loud on contradictions catches forgetfulness early. If the audience is _several_ environments served from the same artifact (a publisher, a monorepo with several deploy targets, a Dockerfile with several stages), prefer `pip-lock`’s variant-per-marker approach. The underlying resolver math is the same in both tools; the surface contract for `--all-groups` plus declared conflicts is what differs, and the right contract depends on who consumes the lock.

---

<div class="post-metadata">

**Author:** ![notatallshaw](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/notatallshaw/32/10861_2.png) [@notatallshaw](https://discuss.python.org/u/notatallshaw)\
**Post date:** [May 7, 2026, 6:54pm UTC](https://discuss.python.org/t/help-me-shake-out-pip-lock/107232/2 "2026-05-07T18:54:47Z")

</div>

Thanks a lot, I’ll be checking this out tonight.

I’m about to make an annoucment in the same space, but coming at the problem from a different direction. I don’t suppose you’ll be at PyCon US to discuss?

---

<div class="post-metadata">

**Author:** ![bernatgabor](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/bernatgabor/32/3003_2.png) [@bernatgabor](https://discuss.python.org/u/bernatgabor)\
**Post date:** [May 7, 2026, 7:47pm UTC](https://discuss.python.org/t/help-me-shake-out-pip-lock/107232/3 "2026-05-07T19:47:33Z")

</div>

I’ll be at PyCon US 2026.

---

<div class="post-metadata">

**Author:** ![cjames23](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/cjames23/32/32376_2.png) [@cjames23](https://discuss.python.org/u/cjames23)\
**Post date:** [May 7, 2026, 8:53pm UTC](https://discuss.python.org/t/help-me-shake-out-pip-lock/107232/4 "2026-05-07T20:53:11Z")

</div>

I will try to test this out over the weekend. I am sure we can also catch up at PyCon US on anything. We are working on hatch producing pylock.toml per environment and preparing to release that soon so this falls into I very much want to test this before we ship for hatch.

---

<div class="post-metadata">

**Author:** ![cburroughs](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/cburroughs/32/35223_2.png) [@cburroughs](https://discuss.python.org/u/cburroughs)\
**Post date:** [May 8, 2026, 9:21pm UTC](https://discuss.python.org/t/help-me-shake-out-pip-lock/107232/5 "2026-05-08T21:21:49Z")

</div>

For your cross-checking, if you need another comparison point I wanted to add that [GitHub - pex-tool/pex: A tool for generating .pex (Python EXecutable) files, lock files and venvs. · GitHub](https://github.com/pex-tool/pex) supports both producing and consuming PEP 751 lock files.