PEP 771: Default Extras for Python Software Packages

Packaging Standards
46

Thanks for your work on this PEP @trobitaille and @jonathandekhtiar. It looks like a net win to me to have this functionality.

I’d like to request some improvements to the Motivation section, because not all examples there seem relevant, in particular for the “multiple backend” part. There are three bullet points in that section:

  • Qt. This use case is clear, and all concrete packages listed as examples (kivy et al) are for Qt.
  • BLAS/LAPACK. This case isn’t so clear to me, because normally those are shared libraries are linked against directly and then vendored in the wheel by auditwheel & co. It could work in theory, but I’m not aware of any concrete package that can depend on either OpenBLAS or MKL in practice. Do you know of any? If so, I suggest adding them. If not, I suggest removing this bullet point.
  • FFT libraries. Same - it’s possible in principle, but there is no package given as an example. The case isn’t too strong, because the functionality provided by both scipy.fft and pyfftw can also be obtained from numpy.fft - with roughly equal performance since numpy 2.0, so I suspect anyone with this use case will anyway have a numpy dependency and therefore use numpy.fft instead of an extra like this.

The other comment I have on the multiple backends use case is that it would be more naturally covered by the ability to specify a logical OR between dependencies. That is both more clear and avoids pitfalls with cases like pip install pkg[other-extra] dropping the default backend. I suspect someone brought this up before, but I don’t see it in this thread nor in the Rejected ideas section. Did you consider this?

The “recommended but not required” use case seems much more broadly applicable. That said, the tensorflow example isn’t worked out and probably not ideal - the build variant support in wheels that has been heavily discussed in other threads is a much more appropriate solution for this than moving an [and-cuda] extra to a default extra. That just swaps CUDA support from opt-in to opt-out, and many users will now get very large downloads unnecessarily before realizing they now need the [cpu] extra - I don’t think that’s an improvement. In this section, I suggest removing the tensorflow example and adding some new examples of packages where the benefit is more clear. A bunch of people in this thread already said “I’d use this for X”, so it shouldn’t be too hard to do.

6 Likes