Having build dependencies available as install dependencies

There is actually a need for this (for example, packages like torchvision need ==torch-version because of lack of ABI stability), but not just for a custom == specifier.

Another common use case is >=. For example, every package that uses the NumPy C API (including via Cython) needs a runtime dependency of >= numpy-version-at-build-time. This is annoying enough to express that pretty much no package gets it right. Conda-forge has a special pin_compatible feature for that, which is extremely useful whenever you’re dealing with libraries with a C API/ABI.

I’d say that longer-term it would be great to have at least the == and >= use cases be part of the standard versioning constructs in pyproject.toml that every packaging tool understands. It may make sense to have an initial implementation in one packaging tool only first though. I wrote a longer summary of the needs at UX for constraining build-time and runtime dependencies · Issue #29 · FFY00/meson-python · GitHub. If anyone would like to weigh in on desired syntax for this, either here or on the issue, that would be very helpful.

It could be a different custom hook for every build tool, or the same syntax that every build tool can implement support for. The latter seems preferable in the long term.

1 Like

Well, this is a fairly different requirement than what OP has, wherein they need the version to match at runtime when the dependency isn’t declared.

Both cases do share a thing though: they’d be solved by adding a runtime dependency in the relevant package, in an appropriate manner.

From the docs (Build variants — conda-build 3.22.0+17.gac864127.dirty documentation):

By default, with the pin_compatible(‘package_name’) function, conda-build pins to your current version and less than the next major version.

(it’s a bit more customisable, FWIW)

Theoretically, this use case is something that can be represented already; by adding numpy >= {version in build environment} as a dependency during the build of the package.

I’ll concede that it’s functionality that isn’t implemented as of today in an accessible manner by any of the build backends today, but I also don’t see the need for every build backend (and more importantly, frontends like pip) to implement new functionality to support this usecase. For example, sphinx-theme-builder has no need for such semantics.

This does seem like something a back-end could do with dependencies listed in dynamic today if the support was added. My guess is Hatch could take a plug-in, but that’s because I assume Hatch has a hook for anything. :wink:

2 Likes

I mean… there’s a bunch of discussion about designing a solution for this in…

Yes, that is fair. Let me rephrase: it should be optional to implement, but for the backends that do implement such functionality, it seems preferable for them to all use the same syntax.

3 Likes

It might even make sense as part of extensionlib (designed as a generic extension building library/interface).

/cc @ofek

2 Likes