PEP 725: Specifying external dependencies in pyproject.toml

BuildRequires: cmake(GTest) and BuildRequires: gtest-devel are not equivalent. Right now they happen to both install the same package. That may or may not change in the future. The name cmake(GTest) satisfies the requirement for cmake’s find_package(GTest). The additional provides make the lives of package maintainers easier. They don’t need to locate the right package for a cmake requirement. The system can figure out the right RPM package for the maintainer.

RPMs provide additional names to decouple the package name from its content. A lot of provides, requires, and build requires are automatically injected by RPM macros and the build system. The cmake and pkgconfig provides are not in in gtest.spec. They are added by the RPM build system.

$ dnf repoquery --provides gtest-devel
cmake(GTest) = 1.13.0
cmake(gtest) = 1.13.0
gtest-devel = 1.13.0-3.fc39
gtest-devel(x86-32) = 1.13.0-3.fc39
gtest-devel(x86-64) = 1.13.0-3.fc39
pkgconfig(gtest) = 1.13.0
pkgconfig(gtest_main) = 1.13.0

Right now the output of the tool is RPM specific and only useful on RPM based distros (Fedora, RHEL, OpenSUSE, CentOS). I’m not familiar enough with other Linux distros to assess their needs. I’m open to extend the tool if there is a use case in the Python packaging ecosystem. The main use case are wheels that link to system libraries. The information from the tool lets us automatically install RPMs and detect ABI conflicts.

First of all, thank you for doing this. It definitely makes sense to have a better way to specify this information. A few thoughts and a little Gentoo perspective from the first reading:

Concrete vs. virtual packages

I feel like defining what constitutes a virtual vs. concrete package may be a bit debatable. For example, to bring the example from the PEP:

[external]
build-requires = [
  "virtual:compiler/c",
  "virtual:compiler/rust",
  "pkg:generic/pkg-config",
]

Firstly, from Gentoo perspective pkg-config would actually qualify as a “virtual”. Gentoo currently uses virtual/pkgconfig to express the dependency on the pkg-config executable, and it is provided by pkgconf. So from this POV generic/pkg-config feels a bit like specifying the compiler as generic/cc :-).

Secondly, I feel like we are going to need a way to clarify the specific virtuals. The discussion so far mentioned C standards — however, there could be other kinds of compiler incompatibilties. Most importantly, on Windows some packages may only build with MSVC (but not GNU-style Clang), while others may not support MSVC.

Package variability

I’m sorry if I missed this, but so far the discussion here has been focused around the possibility of some bits of packages being split by distributions. However, another possible aspect is that different distributions and/or users may be building the same baseline packages with different build options.

To go with perhaps the most expressive example possible, let’s consider ffmpeg. It literally has tens of build options related to various codecs. The Gentoo media-video/ffmpeg package has (as of 7.0.2) over 100 USE flags that users can use to customize which features are included. Conda’s ffmpeg package gives the choice between lgpl and gpl variants. Other distributions provide some fixed subset of all these options. Some unconditionally skip “nonfree” stuff, FWIU.

While obviously I don’t expect us to be able to be able solve this fully, perhaps it’s worth mentioning that this may require addressing at least to some degree.

Implied runtime dependencies

It’s not a big deal but it feels a bit inconsistent that the PEP simultaneously assumes that “build” and “host” requirements can imply runtime dependencies, yet doesn’t permit (or I think it doesn’t, to be honest this section is a bit unclear to me) Requires-External to contain additional dependencies.

It feels more logical to me that if a tool were to inject a runtime dependency from a “build” or “host” requirement, that dependency would end up being added to Requires-External.

2 Likes

The first thought that comes to mind is to leverage the qualifiers part of the PURL spec, but there are no recommendations for build variants or “features” in a package. I’ve seen some projects using qualifiers for e.g. affected architectures or distros. I don’t think we can reasonably encode those in a standardized way so… should this be deferred to documentation or loosely-defined “annotations” metadata that can be shown in error messages?

1 Like