# PEP 735: Dependency Groups in pyproject.toml **URL:** \ **Category:** Standards\ **Created:** [November 20, 2023, 9:50pm UTC](https://discuss.python.org/t/pep-735-dependency-groups-in-pyproject-toml/39233 "2023-11-20T21:50:35Z")\ **Posts on this page:** 1\ **Showing post:** 35
**Author:** ![brettcannon](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/brettcannon/32/34895_2.png) [@brettcannon](https://discuss.python.org/u/brettcannon)\ **Post date:** [November 22, 2023, 1:44am UTC](https://discuss.python.org/t/pep-735-dependency-groups-in-pyproject-toml/39233/35 "2023-11-22T01:44:56Z")
> [@sirosen](#): > > I think the question that needs answering here is “what is the behavior of a tool which implements PEP 735 if the data is extended with a future spec that the tool does not implement?” If it’s backwards-compatible then it ignores it, else it errors out. I don’t think we want to prematurely optimize here. > [@h-vetinari](#): > > I think it boils down to the fact that `pyproject.toml` (including `project.dependencies`) was once meant first and foremost for wheels, but has clearly outgrown that role. We should acknowledge that, instead of twisting ourselves into pretzels to cling to a v1 API which unforeseeable(!) evolution has shown to be inadequate. That’s going back to [Projects that aren't meant to generate a wheel and `pyproject.toml`](https://discuss.python.org/t/projects-that-arent-meant-to-generate-a-wheel-and-pyproject-toml/29684) and the whole reason that conversation was started. We can always revisit that, but people here already complained that it was too long and meandering, suggesting a lack of consensus to begin with. > [@sirosen](#): > > Just to share a couple of quick thoughts on this (I’ll probably have more to say later), I had an early idea for dependency groups which looked something like this: > > ```auto > [[requirements.groups]] > name = "test" > is_extra = false > deps = ["pytest", "coverage"] > [[requirements.groups]] > name = "mysql" > is_extra = true > deps = ["sqlalchemy"] > > ``` > > But I nixed it because I thought it was so much more complicated to explain I agree. I don’t want to try and explain that to a beginner who just wants to do the right thing and right down what dependencies the code in some zip file has. I think one of the reasons requirements files have worked is because there isn’t much ceremony to them and they map to what you specify for `pip` and other installers to install. > [@pradyunsg](#): > > On the actual proposal: Isn’t the idea here basically the one discussed extensively in [Adding a non-metadata installer-only `dev-dependencies` table to pyproject.toml](https://discuss.python.org/t/adding-a-non-metadata-installer-only-dev-dependencies-table-to-pyproject-toml/20106)? It’s similar to my strawman, although I proposed a bit of requirement syntax for referring to other dependency groups and a solution to how to write stuff down for PEP 723. > [@pradyunsg](#): > > The syntax distinction between `.[test]` vs `[test]` is extremely subtle, especially since it’s “just a dot”. I agree it may be easy to overlook. > [@pradyunsg](#): > > With that, we’d basically bless the extras-based model as a good way to declare such dependencies since you can then do: > > ```auto > [project.optional-dependecies] > test = ["pytest"] > > ``` > > and then being able to use pip like: > > ```auto > pip install --only-deps .[test] > > ``` > > This will avoid needing to introduce a new concept at the standards-level, recognise the fact that this is something that people are already doing _and_ enables the existing workflows without needing a substantial review of what the UX would need to look like for a completely new named-concept. But this goes back to the “`[project]` is only for wheels” discussion since that doesn’t work for the web app scenario. > [@sirosen](#): > > For my own edification, “Draft” means it has been merged but still has `Status: Draft`? That’s what I take this to mean. Correct. > [@pf\_moore](#): > > And in particular, I think the point that some people currently use extras for this type of situation needs to be addressed. As one of those people, it’s honestly to keep my dependencies together and in a consistent format. Using the web app example of run, test, lint, and docs dependency groups, I _could_ have the run requirements in `[project.dependencies]` and then have a `requirements-test.txt` that consisted of: ```auto . pytest ``` and a `requirements-lint.txt` that had: ```auto ruff ``` But that does physically separate stuff a bit and in two different formats. And on top of that it’s pip-specific and not necessarily as discoverable trying to figure out how to run tests or build docs from an sdist as using extras. But the fact they are extras which makes them a public “API” for my wheel is actually an annoyance I’m just willing to live with. > [@pf\_moore](#): > > That’s why I much prefer standardising existing practices, and in particular, why I prefer to strictly scope dependency groups as an[[2]](#footnote-128667-2) attempt to standardise (a subset of) requirement files. Ditto. Plus requirements files have shown to be flexible and simple enough to meet a lot of needs. > [@pf\_moore](#): > > We’re back to the idea that projects that build into a wheel are in a sense fundamentally different from ones that don’t[[1]](#footnote-128681-1). And we don’t yet have any form of consensus over how we deal with that. I just had a thought in response to Pradyun’s point about `pip install --only-deps`. In that instance we are letting `pyproject.toml` just record some _stuff_ with some semantic meaning, but we are letting a tool choose how to use that information. Or put another way, with `--only-deps`, pip is saying it’s willing to ignore that something is meant for a wheel which won’t be installed and effectively skip the whole “this is a wheel” aspect of `pyproject.toml`. Up until this point, though, we have all operated under the assumption that everything in `[project]` is for _some_ wheel, while `--only-deps` suddenly ignores that fact. What happens if we take this a bit farther? The reason [`pyproject.toml` requires `name` and `version` for `[project]`](https://packaging.python.org/en/latest/specifications/declaring-project-metadata/#specification) is because [core metadata requires it](https://packaging.python.org/en/latest/specifications/core-metadata/). But what if we said that build back-ends are the ones that require `name` and `version` because they need it for core metadata generation? We can still say that `name` can’t be dynamic for discoverability purposes, but we _could_ drop the requirement that `pyproject.toml` consumers **must** enforce that `name` always be there. Thus the requirements of the core metadata is enforced by build back-ends and not by the specification of `[project]`. This then lets us use `project.dependencies` and `project.optional-dependencies` as-is since the installer chooses to view that data in isolation from the rest of `[project]`. Whether everything in `optional-dependencies` infers `dependencies` could be left up to the installer and the user since we are giving tools more leeway in choosing how to interpret the information (i.e. some flag which tells the installer to skip `project.dependencies` when specifying something in `project.optional-dependencies`). It doesn’t change how you validate those keys in `pyproject.toml`, it just lets tools choose how to use the information. That then puts us back to being after a way to control what’s visible as an extra _when_ a build back-end interprets the contents of `[project]` from a wheel-building perspective (i.e. how does the user communicate to the build back-end what is an extra and what’s just a group of dependencies for some other tool to work with?). Is this basically just back to allowing for more uses of `[project]` and broadening it outside of wheels? Yes, but at least in my head this is a different angle to view the same outcome. --- _[View the full topic](https://discuss.python.org/t/pep-735-dependency-groups-in-pyproject-toml/39233)._