Pip offers the ability to use constraints files. Some key points from the docs:
Including a package in a constraints file does not trigger installation of the package.
… purely a way to specify global (version) limits for packages.
I was wondering if it would be possible (and desirable…) to implement something similar for setuptools. What I have in mind is a section in pyproject.toml that allows us to specify dependency constraints, for example:
[project]
# ...
dependencies = [
# foo depends on bar>=1.0.0
"foo>=2.0",
]
constraints = [
# we do not depend directly on bar (only via foo),
# but we want to impose this constraint for security reasons
"bar>=1.0.1",
]
Our package (library) depends on the foo package, and foo depends on the bar package. Now bar has had a security update, but foo has not picked this up yet. We want to make sure that the secure version of bar is installed with our package, even though foo still allows the insecure version.
Our current solution is to add bar>=1.0.1 to the dependencies section. However, this feels a bit awkward because we do not “depend” on bar directly. Moreover, if foo drops its dependence on bar in the future, the bar package would still be installed with our package, even though it is no longer used.
This has probably been discussed before, but I have not been able to find anything similar here.
But if you feel that it’s important that users use a specific version, then your project does seem to implicitly rely on it somehow.
If it’s important for foo to not ship an insecure dependency then I would work with them to update their dependency to one that’s secure. Or work with bar to yank their insecure release. Otherwise, Python packaging in general takes the view that what gets installed is ultimately the end user’s concern, so having individual packages start to restrict isn’t really built into the workflow.
As I understand it, the best practice sentence you’re referring to is meant for <=, ~= or == pins in libraries (which make it difficult for users to upgrade), rather than requiring newer versions.
The “specify too much” problem is not specific to constraints; there are packages that specify hard dependencies too narrowly as well—there’s a weak but steady stream of people on pip’s issue tracker requesting the ability to override dependencies from a package. So far our stance has been a. you should trust the package developers, b. please work with them to loosen the requirement, and c. fork it if you must. So if we follow the same rationales, I don’t see why we can’t add constraints to package metadata. Looking forward to a PEP for this
Your use case is reasonable. However, as you said, the constraints file is mainly used for global dependency constraints, such as excluding versions with security vulnerabilities. In this case, it is less ideal to store this configuration in pyproject.toml , which is a project-level configuration file.
I can think of a more concrete use case: optional features. pip, for example, detects whether keyring is installed, and provide extra functionalities if its presence is detected. But this introduces some bad interactions when a keyring makes backward incompatible changes, since we don’t know if the user has the correct keyring installed. The traditional way to deal with this is by parsing package metadata and only provide the functionality if the version satisfies, but that also means the user has to take extra care and install a compatible version—which is not declared anywhere—if they that extra feature. This new metadata would resolve that during the installation phase, and a compatible keyring would be installed automatically if possible. (Runtime detection is still needed since it’s still possible for an incompatible version to be installed, but I don’t think there’s a good way around that.)
I’m not sure why normal extras aren’t sufficient for that. It doesn’t work in pip’s case, but that’s because pip doesn’t want to install keyring, just use it if it’s already installed (and constraints don’t help with that, you need the runtime check).
Specifiers in extras don’t have any effect when they are not selected. I guess if extras become hints it’d fulfil the same need, i.e. the resolver considers them even when they are not selected by the user, if another package does request packages included in the extra.
I still don’t see the “optional features” use case you’re suggesting though. The example of keyring with pip doesn’t work, because installing pip never installs keyring, and if it’s already installed constraints don’t have any effect…
