raises hand I’ll do my best to summarize. Although I might not be the perfect candidate to do it, at worst we’ll have a Cunningham’s Law moment. ![]()
This is my understanding of the issues / debate items, editorial comments at the end:
- This is similar to recommended packages in apt and dnf, and those mechanisms haven’t worked well historically.[1] We may be about to repeat their mistakes without learning key lessons.
- For package consumers who really want a minimal install tree, this makes things harder. In particular, the concern here is inclusion of default extras in second- and third-order dependencies, which can’t be controlled by the package consumer. Possible bad outcomes include more disk and network usage, package version conflicts, increased security risk exposure.
- Package maintainers who add a default extra are changing the install behavior of their package. This can be a breaking change (e.g., if it holds back the version of a second-order dependency) and needs to be socialized properly as a package design decision with nonzero impact.
- Most packages already fail to test scenarios which their metadata claims to support (e.g., testing your maximal Python version with your minimal supported dependency versions). This mechanism encourages packages to have two supported install modes, with and without the default extra, and we need to socialize the testing requirement this implies for maintainers.
- This pattern encourages the use of extras in general. If extras have a bad design, this doubles down on that bad design.
- Default extras can lead to inconsistency across distribution channels, as each redistributor (e.g., linux distro packages) may make a different decision about whether or not to include the default extra.
- The current discussion does not sufficiently explore the alternative of publishing multiple distinct and related packages. Why are packages which provide applications + libraries not satisfied to publish a library package and separate application package?
A couple of these are “this whole thing is a bad idea”. We can acknowledge downsides and keep them in mind. We may be able to mitigate some of the harms which are foreseen by large/uncontrollable install trees, but I don’t see all of these as criticisms which can be “addressed” per se because some are fundamental. I’d like us to get more clarity on these so that we can at least understand why “recommended packages” don’t work and see if the same logic applies to Python.
@bwoodsend wrote some good content about this in the other thread which I want to reread.
Changes to install behavior and testing requirements are social issues. Your perspective on it will vary based on how optimistic/pessimistic you are about package maintainers doing a Good Job. At a minimum we need good guidance and documentation. There will be mistakes in how packages declare their dependencies. I do not find this argument compelling when it presupposes that it is categorically impossible to ever guide users towards proper use of the feature.
Partly, I find “this feature is a bad idea inherently” getting mixed with “proper usage is way more complex and expensive than everyone seems to think”. But if there is such a thing as proper usage and it’s wildly complicated, I’d like to see someone take a crack at defining that proper usage so that we can understand why it’s so hard to use well.
On splitting packages in particular, I think we can do more to understand/explain why this doesn’t work for all packages. We have enough smart and hard working people in the conversation that if this were really an equivalent alternative, we would be simply talking about ways of making that workflow better for maintainers. Many of us don’t want to split our packages – for my part, I think it makes the redistribution problem even more confusing/worse. Presumably there are other reasons for this? I can guess at some, but I emphatically reject “I don’t want the work of doing two releases” as sufficient – there’s something more here, since we are all generally willing to put in effort when it improves things for our users.
Here I’m just echoing what’s been said. I have never used “recommended” package features of my linux package managers, so I have no idea about this. ↩︎