Python Packaging Strategy Discussion - Part 1

[h-vetinari]

@abravalheri

Going back to “there is only one blessed tool” feels like a throwback… If this was the goal since the beginning, the community could have saved time/money/energy if everyone worked together to fix/improve/rebuild existing tools instead of splitting the efforts.

Is it respectful of the work people have put in?

I mentioned this in the other thread:

@h-vetinari

I think the individuals fighting for improvements in [python packaging] are heroes in their own right, though I don’t think the situation is worth celebrating. […] The core issue is that packaging is an afterthought also on the language level, and it shows.

To my mind, it is not a question of lack of respect for the respective maintainers, who’ve done a fantastic job in very challenging conditions^[1]. It may be my biased view, but in view of the scope of the problems to solve, as well as the lack of deeper language integration of packaging, the interoperability PEPs were the only halfway realistic path forward – no single project could reasonably hope to take on the responsibility of serving the entirety of the Python ecosystem by itself (without systematic support, i.e. language commitment).

One the one hand, having competing solutions is great for innovation, but horrible for duplication of work. And as the survey shows, users don’t exactly appreciate that decentralized and fragmented approach. We may yet get to have our cake and eat it too, if indeed we manage to hide all those different tools behind a unified interface, and I think it would be a large improvement, even though I doubt we can avoid those interfaces leaking implementation-details of the backends quite heavily.

In any case, if there were a drive towards a more centralised solution, I certainly would not see this as disrespectful towards those who have gotten us as far as we are now. I get the emotional investment in something one has spent a long time working on^[2], but ideally, we should be able to uncouple the design decisions going forward from previous efforts (especially if we can agree to remove/lift/change some constraints^[3] that all-but-forced certain decisions at the time).

---

1. huge amount of responsibility for a thankless task that makes people scream loudly if anything breaks ↩︎

2. and I certainly won’t claim that I don’t occasionally fall prey to that as well ↩︎

3. talking generally, not alluding to any specific one here ↩︎