From Hyneck’s blog post, you can compose an extra/optional dependency from another extra key:
[project]
name = "my-pkg"
[project.optional-dependencies]
tests = ["pytest"]
docs = ["sphinx"]
dev = [
"my-pkg[tests,docs]",
"pdbpp",
]
I took a quick look at PEP 685 and PEP 508 and couldn’t find anything that I understood to document this behavior. The post claims it was released with pip 21.2 and I’ve found nothing in their change log nor the current documentation.
PEP 621 talks about project metadata in pyproject.toml. That PEP also provides a link to the Python packaging docs which are kept up to date with any changes to the specification. The description of optional dependencies is here: Declaring project metadata — Python Packaging User Guide
The feature I’m asking for is a lot more specific than that.
This feature allows you to add your own package’s extra as a dependency of another extra, hence recursive optional dependency.
[project]
name = "my-pkg"
[project.optional-dependencies]
...
dev = [
"my-pkg[tests,docs]"
]
I don’t really see what’s so special here that needs documentation. It’s not a specific feature, it’s just a consequence of the fact that you can specify any package as a requirement (including your own package), and you can specify package extras in requirements.
Well it’s special because having a dependency that is your own package is ambiguous. A package installer or the generated metadata would have to resolve a few special cases:
my-pkg[extra] : This should disambiguate to the current package if matches project.name and don’t look for an existing package on PyPI.my-pkg[extra]==x.y.z: What about this? Should it now look for a package on PyPI? What if x.y.z matches project.version?I don’t think there is any “ambiguity” here since it is impossible in Python to have two versions of the same package installed in the same environment (without conflicts, anyway). Dependency resolving is “just” solving a set of constraints having the form “package A version X depends on package B version Y”. Mathematically, a self-dependency is no different. So, for example, a dependency on my-package[foo]==1.0 in my-package 1.1 is simply an unsolvable constraint and will prevent my-package 1.1 from being used, just like my-package 1.1 is unusable if it depends on libA 1.0 and libB 1.0 where libB 1.0 depends on libA 2.0.
Fundamentally there’s nothing wrong with it, but if I had to implement a installer or a resolver, then this is a special case I have to be aware of when parsing the package metadata. Clearly some code was necessary to add this behavior in pip (version 21.2 as Hyneck claims). Anyways, we can agree to disagree else we go into this rabbit hole too much.
Okay, so I’m convinced there is no documentation of this specifically. I think it’s needed. I will take this request elsewhere.
Wonderful! I was hoping that you amongst others were far aware of this already.
Should this be documented only in the installer’s docs (e.g., pip and hatch) or should this be documented more explicitly in the core metadata (i.e., how to interpret Require-Dist if it includes Name as dependency)?