PEP 735: Dependency Groups in pyproject.toml

Packaging Standards
52

Just to provide another user story: At work, we use a template for Python projects to provide a consistent developer experience across projects. Many of our developers do not mainly program in Python and don’t care about which of the many tools in the Python ecosystem we use. We therefore abuse extras to provide a workflow where we can tell people “If you change something, run tox run -e lint/test/docs/build and make sure that the output is green”, while still maintaining all dependencies in one place. Setting up a development environment usually just means running pip install -e .[dev] (dev contains lint/test/…). This leads to a very nice developer experience but requires installing the package with all its runtime dependencies, although we just need to install build to build a wheel.

This PEP looks very promising to me but I am not clear on a few things.

  1. From my understanding, three types of dependencies have to be considered:

    • required runtime dependencies (project.dependencies)
    • optional runtime dependencies → “extras” (project.optional-dependencies)
    • non-runtime dependencies → “dependency groups”

    Why should the last type of dependencies be specified in requirements.packages instead of project.non-runtime-dependencies (actual name tbd)? What’s the difference between requirements and dependencies? I’d find it very unnatural to explain to someone why these things have to be treated so differently.

  2. I’m not sure if it is the right time to start specifying the object definition, like {spec = "numpy>1"}, as it is not clear yet, what the required fields are. The PEP could say that requirement specifiers can only be strings for now, but may be extended to objects in the future. Tools should warn that they don’t support object notation but must not error out.

    The object notation is required in case requirements.packages replaces project.dependencies and project.optional-dependencies in the future, right? That seems like a massive migration effort and using something like project.non-runtime-dependencies could prevent the need entirely. However, this assumes that the project table can be used for things that are not strictly related to building distributions, as discussed in Projects that aren’t meant to generate a wheel and pyproject.toml.

2 Likes