Well, I do like to keep things simple, and I don’t like overkill
Basically, my view is that project developers (and I include myself) have a strong tendency to assume that “their” complexity is essential and unavoidable, and tools must clearly support their use case. However, Python packaging is struggling under the weight of the complexity we’ve accumulated over many years, and I strongly believe that what we need is simplification, not further complexity. We should be spending time looking at our core concepts and models, and considering how to make them simpler, better defined, and more manageable and composable, and not just loading more and more features on an ill-defined and adhoc base.
This is not to say that we can’t support additional use cases, or that we can’t grow, but we should do so by looking at our underlying model and questioning whether it really supports the needs of today’s community. That’s why I think we should be considering whether we "need something that’s conceptually slightly different than extras”, as I mentioned above. And it’s why I think that @steve.dower’s question, why can’t you ship two projects, is important here. Is it not possible that what you really need is for the tools and ecosystem to make it so that handling two projects is more manageable, rather than trying to “fix”
By pushing the complexity back onto the tools, you’re adding overhead to the packaging infrastructure, making it more fragile, harder to maintain, and more prone to failure. When projects like pip are maintained by a team of less than 10 people, all of whom are working on a totally volunteer basis in their spare time, that’s adding a huge risk to your projects. So looking at it that way, maintaining a single package also has an impact, and that impact isn’t just on your projects, but also on other projects which are competing with you for limited tool developer resource.
This is going more in the direction I’m thinking of - but I’m not sure if you’re intending it as a full replacement to extras, or just as a way of restating your proposal using different terminology. It will need a lot more thinking through if it’s to replace “extras” as a concept, so I reserve judgement on it until I see a full proposal (which would need to include a discussion of how we migrate existing projects off extras - just adding another approach to the mix won’t simplify anything). @steve.dower’s selector packages proposal is another idea that may be worth looking at here.
Why should we agree to that? Why not have
project-cpu, and users can install whichever parts of the functionality they want? From a user perspective, that seems far more straightforward to me. Of course, it’s more work for the project to structure itself to make the various parts intuitive for the user. And can we all agree that it’s making things simpler for users is the ultimate aim here? (That’s not rhetorical, this discussion is about making development easier, but it seems to me that we’re just changing which developers have to invest effort, rather than looking at what works best for the users).
Sorry - this got a bit beyond the scope of a single change to how extras work. But I do think the proposal exposes a lot of the issues