I have a private package (in GitLab Registry) for my work, which has two dependencies: pandas and SQLAlchemy among others. Now due to recent updates, it can work when pandas<2 and SQLAlchemy<2, or when pandas>=2 and SQLAlchemy>=2. v1 of any one package is not compatible with the v2 of the other.
My question how do I specify such a dependency condition in dependencies section of pyproject.toml. Other dependencies of my package, or other dependencies of other codebases for which my package is a dependency can lead to pinning of either of the two packages. Based on the aforementioned constraints, I want to restrict the final solved version for other package.
I’ve tried specifying both as <2, or specifying both as >=2. Of course that works, but I’d like to use the new format whenever possible, and sometime pinnings caused by other dependencies are forcing the conflict. Can someone please suggest on how to define dependencies constrained on each other?
Dependency configuration isn’t dependent on whether it is specified in pyproject.toml, setup.py, or any other file. It is primarily dependent on the ability of the eventual installer (pip or any other tool) to understand the dependencies and arrive a compatible combination of package versions.
I don’t think pip has the ability to understand cross-package dependencies in this way. If pandas doesn’t express a dependency on SQLAlchemy itself, you’ll have to pin all of the dependencies more tightly and disallow usage of versions <2.
I am quite certain. The issue happens when pandas get resolved to <2, >=2 is fine. pandas<2 did not have a specific bound on SQLAlchemy (see below), so pip install does the latest by default and then it fails because of this.
Would it work to make an empty package with v 1.0 depending on 'pandas <2', 'SQLAlchemy <2', and v 2.0 depending on 'pandas >=2', 'SQLAlchemy >=2', and then have your real package depend on this metapackage? You might also want to specify >= 2, <3 while you’re at it.
That’s clearly not very convenient, but if you really want pip’s dependency resolver to be able to select either of those pairings, I think it should achieve that.