Building distributions and drawing the Platypus

So I use pip wheel extensively, either for creating local caches of wheels or for building my own from an sdist. And that dual use would suggest it may need to change one day, even though it works.

With the primary piece of building a wheel now being installing its dependencies though, I don’t see how or why we’d split it out from pip completely. There are plenty of ways to build one, but the only reliable way is to install the correct dependencies first, and while pip can’t quite do that yet people are still going to use it and packagers are going to have to deal with the limitations.

Uploading is another matter, but again, if you need pip to get twine, you need pip to upload and we may as well bundle it. Presumably pip has to be able to do nearly everything to pull from authenticated feeds as twine has to for pushing, so why have a separate configuration file to get out of sync?

None of this precludes other tools from doing the job, but since pip already covers the download/unpack side, having it also do the pack/upload side makes it more obvious that they’ll be compatible.

1 Like

No. As Brett already pointed out, there’s pep517. And the whole point of PEP 517, and most of the interoperability standards work we’ve been doing, is to make it so that alernatives to pip can be built. It’s an explicit goal of the standards work to not focus on pip as the only front end tool, and an explicit policy of pip to implement standards-based functionality, so that people don’t get locked into implementation-defined behaviour.

While I don’t see anything wrong with that idea (after all, who would argue against being able to do anything with “minimum friction”?) you appear to also be assuming that writing packages is the best solution for everyone. And that’s where I think you’re wrong.

Many Python users don’t write packages, and never will. Instead, they write single-file scripts, plugin code for applications, data science notebooks, etc. To give a specific example, a significant proportion of the people I know who use Python in my organisation, only use Python’s packaging infrastructure to consume 3rd party libraries. They never package their own code, and only ever share it in constrained situations where wheels, PyPI, and pip are essentially irrelevant.

Now it’s not entirely clear that this makes much difference when we’re talking about “building distributions”, which is the topic of this thread. But I think it’s extremely important to keep a sense of perspective on what we’re discussing. Building packages is a relatively minor part of the Python packaging ecosystem. Consuming packages is far more significant. And many, many Python users have no interest in building packages, and rightly so - we shouldn’t force building packages on them as a solution to their problems (even if the problems they have are related to deployment, dependency management, etc. - wheels and PyPI aren’t, and shouldn’t be treated as, a “one size fits all” solution, IMO).

2 Likes

Coming back to this today, it’d be useful to see how much of this can be done with poetry today.

I personally don’t have the time to take a deeper look into this right now.

1 Like

Moving python-build to PyPA is a discussion about taking the route of going down option 3 (see original post for what this thread was for).