PEP 735: Dependency Groups in pyproject.toml

[brettcannon]

@pf_moore

My call for use cases was intended to say, let’s see what people are doing with what the packaging ecosystem already provides, and look at where the pain points are and whether a standard could help.

So what do you feel has been established solidly enough by the ecosystem to warrant standardizing so we are not specifying the same thing in a bunch of different ways in the context of this PEP? And then can we design it to be simple enough to use, while being flexible enough to build on top of as we establish more widely accepted practices as time goes on?

We have talked about trying to standardize at least the idea of what requirements files represent as an easy way to write groups of dependencies. But we have also talked about how some of the concepts and features supported by requirements files are very specific to pip and not at all standardized. So what exactly do we consider “widely accepted via convention/use” enough to warrant standardizing how everyone should write the same thing down now instead of waiting longer to standardize or never standardizing?

If you were asking me, I would say we agree as a group that the following features are widely established and used:

1. Grouping a set of dependencies together such that each dependency can be expressed in a way to cover the features allowed by Dependency specifiers - Python Packaging User Guide (I’m purposefully not specifying a table-based or string-based format as that’s not the point of this response); this includes what Frost pointed out in PEP 735: Dependency Groups in pyproject.toml - #170 by frostming of different requirements based on markers
2. A way to reference other groups of dependencies (i.e., -r in requirements files)

I think everything else as specified by Requirements File Format - pip documentation v23.3.1 could be viewed as out of scope for now if we don’t feel there’s enough agreement around ubiquity/use of a feature.

I do think we should design a solution for the two features above that allows for future expansion, e.g. editables if Ofek or someone else doesn’t come up with a workspace solution that’s more powerful than editables, but I also don’t think we have to bake that support in now. This is why my initial proposal at Projects that aren't meant to generate a wheel and `pyproject.toml` - #173 by brettcannon which inspired this PEP was rather simplistic in terms of feature scope as I thought it had a high chance of reaching agreement on the feature set (I’m ignoring the format question).

@ofek

I might be missing some context but how is 723 blocked on this?

I don’t think it’s blocked by this PEP as much as potentially informing PEP 723.

Hidden to avoid going OT as it's not critical to this PEP, but left here to explain why this PEP might inform PEP 723 ...

If this PEP gets accepted, then there’s a [dependency-groups] table. To me, that suggests we either open up project.dependencies by dropping the name and version requirements for [project], or we add a top-level dependencies key. There’s a third possibility of allowing folks to mark a dependency group as the default, but that feels like purity over practicality to me as that’s probably more verbosity/complexity than is required for that use case.

If this PEP gets rejected, it just means we still have to decide how to represent the runtime dependencies in pyproject.toml that still works for single-file scripts for PEP 723.