Hi everyone - this whole thing is my fault! I only just heard about this thread.
If community consensus is that this is a bad pattern I’ll note that on the go-to-wheel repo and other places I’ve talked about this (I don’t know if taking the project down is called for, but I can also archive the GitHub repository). If this results in an official PyPI policy I’ll link to that prominently as well.
If there’s a benefit to the Python community to having something distributed on PyPI, then that’s OK. If there’s not really a benefit to the Python community and someone is just using PyPI as an alternative to Homebrew, then that’s something that would probably be discouraged.
This is the key question for me. I’ll try to make the case for why I think this is a benefit to the Python community in most cases.
I first noticed this pattern of distributing binaries via PyPI back in 2022 - initially with ziglang and then with the revelation that the enormously popular playwright quietly bundles an entire node.js executable in its own package!
The more time I’ve spent with this pattern the more I’ve grown to like it.
I want to be able to distribute software systems I’ve written in Python for other people to use. Sometimes that means I need them to have extra tools available, like ffmpeg (did you know PyPI hosts static-ffmpeg these days?) or Deno (I built an experimental Python library depending on that a few weeks ago).
Being able to pull those in as regular dependencies solves so many problems. I can release a script that turns a YouTube livestream into a GIF and drop static-ffmpeg in as inline script metadata and everything just works!
~5 years ago the general consensus I saw from the wider technology community was that Python packaging was a big mess. Today I don’t think that reputation holds at all: I think Python’s packaging story has never been stronger, and a great demonstration of that is that PyPI now increasingly acts as a distribution platform of choice outside our own community.
We’re not going to solve cross-language cross-platform package management with this.
I think we’ve come surprisingly close!
I was looking at what it would take to use NPM to distribute a Go binary the other day and it was gnarly - you pretty much need a package that identifies your platform on first run and downloads the necessary binary on demand.
My current intuition is that the pattern is a net positive for Python overall.
-
It helps promote the Python ecosystem - anyone who gets pip or uv up and running just took their first step to joining the Python community.
-
It demonstrates how exceptional Python’s packaging has become.
-
It makes binary tools available as Python dependencies. I see this as the strongest argument in favor - it unlocks an entire world of things that can be turned into an executable.
-
The per-project overhead looks modest to me - half a dozen additional wheels per release likely well under the 100MB limit (my largest Go binary wheel is 11.9MB).
-
On the moderation overhead side it’s not clear to me if this changes the challenge PyPI already faces: we already have policies for handling malicious or illegal Python packages. The big risk would be if this dramatically increases the volume of packages being uploaded to PyPI - the Homebrew alternative Donald mentioned, only worse because Homebrew has an approval step for first publishing and PyPI does not.
The people maintaining Python package wrappers are often not the same people maintaining the upstream non-Python project, so if they disappear and stop updating the wrapper packages then users who relied those are left in the lurch and may not realize they’re running outdated copies
That was one of the reasons I designed go-to-wheel - it’s intended to run in CI such that any release of the Go package can be automatically shipped to PyPI as well, as seen in this GitHub Actions workflow. I should add that to the documentation!
My overall position on this is that I hope we resolve that this pattern is an appropriate use of PyPI that supports the Python community and upholds the mission of the PSF - but if consensus falls the other way I’ll do what I can to undo the damage and make it clear in the right places that this isn’t an appropriate way to distribute software.