How to pin build dependencies

Python Help
1

I think it’s widely considered to be a good practice to pin your dependencies in a file like requirements.txt, Pipfile.lock, poetry.lock, etc. The problem I’m having is it doesn’t seem possible to pin build dependencies.

For example, my project depends on pyyaml which has no runtime dependencies but build dependencies on setuptools, wheel, and cython. Regardless of whether I use pip-compile, poetry, or pipenv, none of these tools lock versions of these build dependencies even if I’m on a system where a pre-compiled wheel is not available and these build dependencies must be installed.

As a result, sometimes when new versions of transitive build dependencies of my project are released, installs of my project break despite my attempts to pin everything to working versions. This happened last night with a new version of cython and has happened in the past with new versions of setuptools. Is there a good way to pin down all transitive build dependencies of my project to avoid problems like this?

(There’s an additional problem of pip not respecting flags like --constraints when using PEP-517, although my project has a temporary workaround for that by using the PIP_CONSTRAINT environments variable. See Enable again build isolation with proper pinning of build dependencies by adferrand · Pull Request #8443 · certbot/certbot · GitHub. )

1 Like
2

Did you try to pin your deps in the .toml file? Quoting from Brett Cannons intro to pyproject.toml:

[build-system]
requires = ["setuptools >= 40.6.0", "wheel"]
build-backend = "setuptools.build_meta"
3

There isn’t in a per-dependency way. Best thing to do is to cache your built wheels.

4

The problem isn’t my project’s build dependencies, it’s the build dependencies of all the software my project depends on at runtime.

This might be a better question for another channel such as Packaging - Discussions on Python.org or Mailman 3 Distutils-SIG - python.org, but is there any interest or progress in changing this? I personally think this is a worthwhile thing to do for stability and consistency.

Unfortunately caching all of the built wheels isn’t very feasible for me as my project supports a wide variety of environments and installation methods so caching everything across all environments is a lot of work.

5

Won’t know until you ask. :slight_smile:

6

I created Pinning build dependencies.