`poetry add`, but for PEP 621?

I am personally very guilty of trying to generalise things as soon as possible, and I agree with your assessment; this is mainly just editing TOML in a CLI-friendly way, it doesn’t need to be shared between projects. I’d be tempted to ping @ofek and see whether he’d be interested in this kind of CLI for Hatch. I know that in the long run, the missing feature of Hatch (and many of these tools) is a standardised lockfile format. However, I could see this being added in a non-locking manner to start with.

Just dumping my thoughts at this point; I wonder what would happen if Hatch adopted PDM’s lockfile as a temporary interrim. Much of the machinery should be re-usable, i.e. the interface for a lockfile resolver. However, I am certain that Ofek doesn’t need any more work to be dropped on him, so perhaps he’d prefer to just wait until we’ve settled on what a lockfile should include.

I hadn’t even considered this as a locking operation, simply a way of adding a dependency (which then gets resolved on install).

If you want a locking tool that isn’t part of a “full workflow” solution, you should probably look at pip-tools, which I understand already fills this niche pretty well.

Sure. I don’t think it is a locking-only feature, but I am of the understanding that the locking aspect is a blocker for Hatch, so I assumed that is why Hatch doesn’t yet have this API. This might be a logical jump too far; perhaps @ofek doesn’t want Hatch to have this kind of API! I’ll let him speak to that.

Yeah, my first thought was just the “naive” functionality of adding a dependency, without any consideration for lockfiles (I’m a happy user of pip-tools :slightly_smiling_face:).

If this is something that Hatch grows in the near future, that’d likely be enough to convince me to move most of my company tools over to it!

So, this is tied to locking in that the operation that is actually performed with add differs depending on the presence of a lock file.

That said I do plan to add this by PyCon (maybe the week of for extra visibility) along with the introduction of locking plug-ins (the first of which will wrap pip-tools) and the concept of workspaces which will be useful especially for monorepos.

I was trying to wait for Brett to write that lock file proposal but it doesn’t seem like that will happen soon and now people are considering turning pip into an everything tool which I view as deleterious, so timelines are forcing my hand lol


Having a command line tool that could manipulate PEP 621 dependencies would be an enormous booster for downstream redistributors such as Fedora.

We often need to patch/sed dependencies (e.g. when we tested that a particular software works with an older version of a library that upstream decided they no longer support but we cannot update it yet or when we remove optional test dependencies). Having the ability o do this in a semantic way instead of hacky sed replacements would be very nice.

# remove coverage-related test dependencies
pyproject-tool remove --extra test pytest-cov codecov

# allow a lower version of packaging before we can update it to 23 in Fedora
pyproject-tool remove 'packaging>=23'
pyproject-tool add 'packaging>=21'

Yeah, don’t wait for me. I am moving forward still, just not at lightning speed (unless someone else wants to take over pushing WebAssembly and specifically WASI in Python forward? :wink:).

1 Like

This is an innocent sentence with a large amount of complexity behind it. That is because TOML is supposed to be human-centered (“A config file format for humans”), and so you absolutely have to be able to maintain things like comments in the toml file correctly.

This is not a trivial thing to do, because when you just .load() the toml file and then .dump() it again with an additional dependency, you’ve lost all those comments, plus other things that people have opinions™ about (order of sections, inlining of tables, etc.).

It’s a much larger space than just reading a toml, and the main reason why the 3.11 stdlib-implementation comes without writing capabilities (see e.g. this thread). This info might be out of date, but a year ago, only tomlkit supported style-preservation (and perhaps unsurprisingly, this is what poetry uses; indeed it’s written by the same people).

Adopting something like this IMO requires either:

  • having tomlkit as a dependency everywhere (yet more bootstrapping problems)
  • getting a style-preserving TOML parser/writer into the stdlib

Hmm, why would this complicate bootstrapping? I don’t see it being part of a build backend or the barebones installer. Note that backends with a frontend counterpart do separate these (compare poetry with poetry-core, hatch with hatchling, flit with flit-core, etc.).


Yeah, I envisioned this as being frontend-only; plus, it looks like tomlkit is pure Python with no dependencies, so vendoring it wouldn’t be the worst if absolutely necessary (and is well-trodden w/r/t breaking cycles in other Python packaging tools).

Oh just FYI when I mentioned I’d be adding commands for adding dependencies and therefore modifying TOML although that implies the use of tomlkit I plan to write my own parser that only supports modifying arrays because I am deeply discontent with the bugs I’ve encountered in that library and I would not want to expose users to that nor implement a postprocessing step to fix style.


I plan to write my own parser that only supports modifying arrays

That’s not as simple as it sounds since you still need to retain the complete document format when you write it back, even if you modify one part of it.

because I am deeply discontent with the bugs I’ve encountered in that library and I would not want to expose users to that nor implement a postprocessing step to fix style.

Good thing tomlkit is open source and is accepting contributions. Feel free to open issues about these bugs and even contribute pull requests to fix them, it will avoid duplicate efforts since we have a common goal in the end :slightly_smiling_face:


On another note, I plan a partial rewrite of tomlkit to solve some known issues around formatting and how nodes are stored internally. I don’t have an ETA since I don’t have a lot of time to dedicate to it at the moment but hopefully it will come soon.

Fyi, I opened a request for this in Hatch a while back. Glad to hear it’s likely to become reality. Feel free to add Hatch-specific contributions there:

1 Like

Oh for sure that will be a multi-day effort :slightly_smiling_face:

If I have time I might do that indeed! In my mind writing logic for a very specific use case sounds like less time investment than getting up to speed on a code base.

FWIW, part of the power of poetry add is that it checks whether the new dependency is compatible with the existing dependencies and will let you know immediately whether it is compatible. A poetry add that only modifies the file and doesn’t check for compatibility is not really very useful.

I do like the idea of a smaller utility that just does the one thing.

1 Like

Agreed that checking for incompatibilities is valuable, but IMO there’s value even without that – one separate advantage to something like poetry add is minimizing context switches (I don’t have to open a separate editor buffer, jump to the right TOML table, etc.).

IIRC that’s what cargo add is limited to: pretty much all it does is modify the TOML, and subsequent commands are expected to actually perform the resolution (and fail, if appropriate.)

No, it also performs resolution and modifies the lock file.


My error! In that case this would indeed by unusually limited relative to other tools in the space without dependency resolution.

1 Like

There is demand: