Keeping type stub package versions in sync

Typing
, ,
1

When maintainers release PEP 561 stub-only packages, they sometimes adopt the same release package versions as the target and sometimes use calver. Regardless, one would like to keep these in sync so that the type annotations are not wrong.

We constantly find ourselves in situations where our lock file forces different versions of package and stubs:

pip freeze | grep "paramiko\|protobuf\|psycopg2\|pytz\|pyyaml\|requests"
paramiko==3.3.1
protobuf==4.25.1
psycopg2-binary==2.9.9
pytz==2023.3.post1
requests==2.31.0
types-paramiko==3.4.0.20240205
types-protobuf==4.24.0.20240302
types-psycopg2==2.9.21.20240218

Some discrepancies are worse than others. Is there a (relatively frictionless) way to ensure that for every resolved package, we have the correct set of stubs?

There is some documentation on typeshed that offers 3 approaches, but as stated, they all have their issues. Is this still the status-quo or has the situation improved?

1 Like
2

Ideally we’d have a metadata field that can tell with which packages a particular stub package is compatible. Since we don’t have such a field, typeshed currently encodes the upstream’s package version in the stub version + a calver field to represent the stub package version.

3

Duly noted. Our problem is that we use Poetry to manage dependencies and so we specify the abstract dependency as

[tool.poetry.dependencies]
foobar = "^x.y.z"

[tool.poetry.dev-dependencies]
types-foobar = "^x.y.z"

There is no guarantee from this that the resolved versions that are encoded in poetry.lock have the property that version(foobar) ~= version(types-foobar) (modulo calver compatibility). It seems like in order to achieve this, we’d have to programmatically update the lock file and modify the version of types-foobar.

But this seems very dangerous? Not only are you explicitly subverting the resolver, but both typeshed packages and bespoke stub packages (e.g. pandas-dev/pandas-stubs) have ways for the stub package to specify abstract dependencies (requires= in METADATA.toml for typeshed and a fully specified pyproject.toml for bespoke). In theory, overwriting the version post-resolve can introduce a version conflict.

In the case of pandas, the situation is even more complicated in that the stub package can depend on specific ranges of other stub packages, so it is also possible (in theory) that enforcing version(x) == version(types-x) for all x is not a valid solve.

4

The other thing making this more complex is that typeshed will usually only follow the minor version and not the patch version, so it’s technically only necessary to pin the first two version numbers to be in sync for many stubs packages, otherwise you will miss out on patch versions (i.e. bug/security fixes).

The only time typeshed will start to follow patch versions for a package, is once that package introduces a visible change in a patch version, that will cause stubtest to fail. The setting can change at any point, so at the transition points it will be difficult to figure out which versions are safe to pin to.

For poetry specifically there appears to be a plugin somebody has worked on: GitHub - agonzalezl/stubborn: A Python library that detects version mismatches between code and stub dependencies.

Although it doesn’t really seem to take typeshed’s METADATA.toml into account yet to e.g. detect how tightly the packages need to be pinned together in order to guarantee parity, it just has a fixed list of pairings. For the stubs coming from typeshed it would certainly be possible to build a tool that applies appropriate heuristics to find a more optimal solution for the version pins.