(Maintainer of Briefcase (part of the BeeWare suite) which is either a user of packaging tools, or a packaging tool itself, depending on your definitions)
This thread has done a good job at highlighting that packaging is a complex problem for a variety of technical and historical reasons. However, IMHO, the biggest issue that exists isn’t technical - it’s communication.
For me, the most revealing part of the statement that “there are too many tools, and users are not sure which ones to use” isn’t that there are too many tools. The important part is that unless you’re knee-deep in discussions about packaging, it isn’t currently clear which of those tools should be used - and, the fact that it isn’t clear which one to use is, at least in part, the cause of there being too many tools.
As an example - the packaging.python.org tutorial was updated 6 months ago to use Hatch in its examples. This would seem to indicate a significant signal from the PyPA that Hatch is a “new default”; but I’m not aware of any formal statement of that intent. What does this decision mean for Setuptools (the old default)? Should it be considered deprecated? Should existing projects migrate to Hatch (or anticipate a migration in future - and if so, on what timeframe)? What does the introduction of Hatch as a new default mean for Flit, which is also a PyPA managed tool? How does this decision fit into the longer term vision and plans of PyPA members?
Whatever the outcome of the technical aspects of this (and future) strategy discussions, I’d suggest that the process of communicating that strategy - and communicating progress towards the desired future state - is just as important (if not more so) than the strategy itself. This is especially important given it’s going to take months or years to converge on that future state.
I acknowledge that the PyPA is more of an “accumulation of interested parties” rather than a formal standard-setting body. However, even an informal statement declaring a vision and desired future state for Python packaging would be invaluable in terms of setting community expectations, generating convergence towards that desired future state, and guiding the contributions of those who might be inclined to help get there.
Perhaps I’ve missed the point, and the purpose of this discussion is determine what the strategy and vision should be, prior to publication as a PEP (or whatever instrument is appropriate). If that’s the case, consider this a hearty endorsement of that plan. However, if it isn’t, above everything else, I’d advocate for clear articulation of whatever the final vision happens to be.