Publish Linux installer on python.org

Ideas
101

If a single guy can do it alone and for free on GitHub actions, with all the support guarantees this entails, and if a wide slice of the ecosystem is adopting it over what the official project is doing, then… the market has spoken? The segment of users this serves don’t care about that.

They are not asking for you to become a distro and handle gnarly packaging security updates for all upstream dependencies.

They are asking for a uniform, static and simple to install Python installation for development environments that makes bootstrapping projects consistent and easy. Something which has never been easy and consistent before.

A lot of this discussion has been the same fairly tired, similar calls for inaction that are so common here and elsewhere:

  • what about if users just didn’t do this?
  • what about this scenario I just thought of involving a really specific Linux distribution nobody uses?
  • what about the 10-man dedicated squad we need for this?
  • what if the world was going to end if we don’t ship a OpenSSL patch in 48 hours but evil smelly static linking caused slowdowns and then we all died because silly users refuse to use ancient Debian packages :rage:

To reiterate and refocus:

Large parts of the ecosystem would rather use unsigned binaries from a completely random guy on GitHub than use the official project binaries.

That’s a problem. What can be done to fix this?

Ship the same thing he’s shipping, with the same support policy and the same turn around time as present. The market is happy with that. Fork the damn repository and go from there.

I mean, we’ve got tens of thousands of words written in this thread discussing Linux packaging, the differences and the options available. That… speaks for itself.

And the discussion seems to be rooted in a fantasy world where Linux users do not just smash their way through a “pyenv/asdf install” process and forget about it until next time they are compelled to update, or just shove everything into a Docker container, like everyone else does.

4 Likes
102

Another way to look at this though is that those users also don’t seem to care that it’s not “official” Python. Or maybe it’s just that what people do do doesn’t necessarily tell us what they wish they could do. :slight_smile:

The original post in this thread said (emphasis added):

Because of this it is kind of like internal secret within the more advanced and experience users that instead of installing Python via the operating system on Linux they should just use tools such as pyenv or asdf. This however it’s nowhere documented on python.org

To my mind it’s much easier to solve that problem of telling people about ways to install Python than it is to solve the problem of maintaining a way to install Python. At the most basic level, external options like the standalone builds (or, at least as usefully, conda) could simply be mentioned on the Download page as ways to get Python on Linux (or other OSes).

3 Likes
103

Funny you mention this, the single guy couldn’t do it alone and just this week has handed stewardship over to a company with full-time employees.

9 Likes
104

What’s already happened: the binaries start coming from a company that (hopefully) will sign them, and users will decide whether or not they trust that company.

Anaconda and others have been doing this for many years already. Some people have decided they don’t trust them and won’t use them, others have decided they trust them more than the upstream projects. There’s no problem with choices being available.[1]

My question for you would be, why is the core developer team more trustworthy to you than Astral, or Anaconda, or Red Hat, or ActiveState, or any other company that builds and distributes exactly what you want? Bear in mind that we don’t necessarily even know the legal names of all committers, let alone having the kinds of background checks and legally-acknowledged relationships these companies have with their employees.

  1. I mention Anaconda mainly because they only do Python distribution. ActiveState, Red Hat, Canonical, etc. distribute other things that you may need to use. ↩︎

3 Likes
105

Any reason why we can’t go ahead and put this see pyenv/conda/deadsnakes/astral message on the downloads page and see to what extent people stop asking for Linux installers?

2 Likes
106

Because the core team wants to improve the state of Python packaging for everyone, whereas all of those entities want to improve the state of packaging for their customers.

Astral is the weird exception there, and if we’re happy outsourcing the “improve the state of Python packaging for everyone” to them then that’s an OK outcome I guess but it’s a shame.

That’s a very good point! However the focus here is still on distributing this to end-users, which is missing the mark: It’s providing something that tools can distribute to end users. The project didn’t help provide something that turns out to be hugely beneficial, so someone stepped in and did it themselves.

The difference between this and other channels and the standalone project is that it requires a lot non-trivial patches and work to even get it to build like this, which the author tried and failed to upstream. So in this regard it seems that he succeeded despite us rather than because of us. That’s pretty scary.

Funny you should mention this, because if you dig into why he couldn’t do it alone you’ll find similar themes that others have brought up:

I could never justify the time investment to upstream a lot of my python-build-standalone work. I made some attempts. But it always felt like I was swimming against a heavy current and the talk to meaningful action ratio was too high. The payoff would be there.

I’m not sure attempting to use Gregory’s inability to continue the project alone as some kind of comeback rather than what it is - something to be ashamed of and a failure on the projects part - is a good look.

Rather than look at the wider changes to the landscape the discussion here is about creating an installer for Linux? Is that really a good investment of time and resources, or is it just building something for a increasingly shrinking subset of users (yet over-represented in discussions) whilst ignoring the fairly clear and consistent signals that indicate this isn’t something most users see value in, and apparently offering no support to developers solo working on solutions that are valuable to users?

1 Like
107

Also make sure to include Debian, Ubuntu, Fedora, Arch etc, etc. in the list.

2 Likes
108

I’m sure none of us are treating it as anything other than a concrete example of the fact that this isn’t a trivial task. I can give you all the estimates you like, based on my own experience, and if those are more convincing that we can’t just do it than a real example of someone who tried, I will.

What kind of wider changes would you propose that we consider instead? (I’m not disputing your assessment of the value of a pre-built distro-independent binaries for Linux - I agree they’d be low value for a lot of users, and would go further and say they’d be an active distraction for most. But I’m genuinely interested in what you think would have greater impact here?)

4 Likes
109

I’d just like to point out that they aren’t random artifacts; as far as I know, these are the only builds of python that are actually reproducible by traditional definitions of that term, and the build process is visible, auditable, and there are checksums available. Those concerned with trusting process have plenty of means not to worry themselves about it.

As for trusting people with the process, people choose who they trust, and we can only hope that people make good decisions there because it’s often transitive in nature, and it doesn’t take much to cause a real problem.

These builds aren’t going away, and more and more people are considering statically linked python as the better option. I can ship this on a container that’s entirely distroless with no dynamic linking to be abused, minimizing the attack surface to exactly what the application exposes, and rebuilding python with updated dependencies isn’t hard if it were to come to that, but the decreased attack surface also means it’s less likely some random vulnerability even affect the container at all.

I’m not going to say it’s the correct option for every use of python, but the static vs dynamic linked argument is not as clear cut for security in the direction of dynamic as some people argue.

2 Likes
110

Indeed, he goes into more detail in the first link I shared, listing many very good reasons for not having time: working full-time, getting married and becoming a father, taking up exercise, frustration at Python packaging, discovering Rust and excitement at the new systems-level engineering it unlocked.

But we don’t need justification, anyone can stop contributing to open-source for whatever reason, and that’s their decision to make.

I don’t see anything to be ashamed of: he created a successful project, found himself unable to dedicate the substantial time needed from his diminishing free-time, and has gone one step further and found capable stewards to continue the work.

The point is it’s a big project – the CI has nearly 300 jobs and takes many hours to complete a single build, using some 6.5 days of total CPU time! (here’s just a Linux build) – and I’m trying to push back against the suggestion that it’s simple thing that a single guy can (or should) do alone.

5 Likes
111

Personally (and I know you weren’t asking me) I’d like to see changes to improve the supply chain picture around workflow tools (which are immensely popular, so we can’t simply dismiss them as niche usage). Specifically:

  1. Bring the relevant patches, currently carried by the build-standalone project, into the core. This means the builds come from the source published by python.org, so the provenance is clear. Astral plan on working on this so that’s mostly just a matter of being receptive to the idea.
  2. Acknowledge somewhere on python.org that this sort of distribution is needed, and document our recommendation for how tools like uv and hatch should get their builds. Hosting the builds on python.org would be the cleanest solution here, again because it simplifies the supply chain (“everything comes from the official Python site, which we trust”) but linking to a “blessed” 3rd party project would be sufficient, as well (IMO).

What’s not good enough, IMO, is leaving every user to either do the research themselves, or blindly assume that a Python interpreter installed on their behalf by a tool that (not unreasonably) treats where they get the binaries as an internal implementation detail[1], is safe to use. I think that as the Python core dev team, we have a responsibility to ensure that we are comfortable with the typical ways users get their Python binaries (assuming they follow our advice!).

  1. after all, if they document it, changing it becomes a user-facing breaking change :slightly_smiling_face: ↩︎

12 Likes
112

I think anything that helps external tooling install a correct, working and completely isolated version of Python without any user intervention would have the greatest impact. I don’t think we live in the “users should visit python.org and click download” era anymore - the primary distribution channels are indirect ones via third party tooling - so make it as simple as possible for third party tooling to procure, configure and set up Python for users. Let them then make it as simple as docker run python:3.12.

We might have got some wires crossed around pre-built distro-independent binaries: they would add value for a lot of users, but not if we tie it to any system paths, require any kind of installer to execute, depend on any system packages or we try and offer it directly to users. Nobody is visiting this ungodly page and working out what they need to download, then googling how to use tar and extracting it manually to their home directory.

So offer it directly to tooling. This is what python-build-standalone nailed - there’s an API for listing releases, the names are stable and a download URL can be constructed in a couple of lines of code:

$ curl -L https://github.com/indygreg/python-build-standalone/releases/download/20241016/cpython-3.10.15+20241016-aarch64-apple-darwin-pgo+lto-full.tar.zst | tar xvf -
$ ./python/install/bin/python -c 'import this'
The Zen of Python, by Tim Peters

Before this there was pyenv, and despite this compiling everything needlessly, being rather inscrutable, annoying to setup, and needing a bunch of implicit system dependencies it was a huge success for the simple reason you could run pyenv install 3.10 and just have it work.

I see your point, and I agree that it’s not a trivial project, but the fact remains that he did do it alone - past tense - and without any upstream support. We’re not discussing a proposal or a MVP prototype: it’s had over 70 million downloads so far and a non-trivial part of the modern packaging ecosystem depends on it.

I’m just trying to push back on the idea that it’s some unnatainable goal: if someone external to the project is able to do this without support then I fail to see how it would be harder to do within the project with support, by using the existing work as a blueprint and integrating some of the required changes.

100% agree with everything in this reply, it hits the nail on the head. Especially the last bit.

1 Like
113

[…] as far as I know, these are the only builds of python that are actually reproducible by traditional definitions of that term, and the build process is visible, auditable, and there are checksums available.

This is not true. I don’t know about other distributions, but openSUSE works very hard to be the most reproducible Linux distro out there. We have sent numerous patches (and still keep some in our packages, when they were not accepted upstream as trivial) to make our packages completely reproducible, and of course all our processes are completely open.

114

Off-topic question:

Which reminds me, does anybody have a slightly less crazy version of .github/workflows/ for just one build of one version on one Linux platform for a brief CI checking? I have tried to simplify the directory from python/cpython in GitHub - openSUSE-Python/cpython at opensuse_3.6-gh-action and so far I have failed spectacularly, because it is just a way too complicated I assume.

115

Please open a new topic in Python Help, this thread is long enough already :slight_smile:

116

Seems this is new since I last checked, thanks for the info. I did some browsing of other major distributions to see if anything else major has changed here, but openSUSE’s build that’s designed to work on openSUSE and the prior mentioned build that attempts to work everywhere appear to be the only reproducible python builds with actual use. Nixos doesn’t appear to actually claim reproducibility here[1]. Fedora isn’t claiming reproducibility under the current definitions, but it’s closer than I was aware of too [2]

Where this seems to remain though for the purposes of users is one of the following:

  1. Their distro provides an optimized and security patched build of the python version they need. (good outcome)
  2. Their distro doesn’t provide the version of python they need. (not great)

In the case of the distro not providing it (or the user not knowing to try to get it from their distro first) there are a few other paths:

  • They then get a well reviewed and reproducible build from tooling like uv, hatch, pdm with the aforementioned standalone build. (good outcome)
  • They get it from a 3rd party that packages for that distribution (good outcome)
  • They build it themselves (good outcome)
  • They don’t get the version of python they need (bad outcome)

There are also some potential pitfalls in various options both distro provided and not which may not be obvious to users[3]

The overall goal of any installer on python.org should probably be to cover when the distro doesn’t provide what the user needs because not every user of python is the kind of developer familiar with building packages from source, the use of python encompasses all kinds of developers, and we should want it to be easy for users to get the best outcome for their needs.

I think there’s also a potential tooling win here if we can get enough consensus on which use cases distros aren’t serving (yet?) (factually, not considering aspirational goals of the distro), so that if someone is installing python3.12 with the intention of using it in a non-system capacity on openSUSE vs doing the same on a distro that doesn’t even provide it, the outcomes are based on what options are available for the user’s intended use, though I do worry that such tooling might require more complexity and possible something like additions to sitecustomize to make it more obvious where a specific python version came from if an installer is allowed to pick a source.

  1. NixOS appears to be using a different definition in that “builds should be deterministic”, someone more familiar with NixOS and Nix pkgs might be able to point to this being enough or point out that I missed something here, but I was under the impression that python’s build process isn’t “pure” and therefore while deterministic isn’t reproducible under Nix (I don’t see them carrying patches to make the build “pure”, and they don’t make a reproducibility claim publically for python.) ↩︎

  2. Fedora’s RPMs are signed, and it is not a detached signature. A more permissive definition of reproducible such that the extracted components excluding attached checksums and signatures are reproducible would probably allow claiming it with only minor further changes ↩︎

  3. Without rehashing all of the above thread: System python patched for system use and not suitable for general development use, yet exposed to users, standalone python builds may not adhere to systemwide policies in some cases, building python yourself is easy to end up with both an unoptimized build, and one that is missing features or security policies. ↩︎

1 Like
117

From a distro point of view, folks on Fedora generally don’t encounter many problems getting compatible Python runtimes (the RH Python team maintain parallel installable builds for all upstream supported Python versions, and usually whatever’s upstream of any still supported versions of RHEL).

Where the standalone builds shine is when you want cross-platform portability more than you want deep integration with the underlying platform.

Thus their popularity with project management tools and app distribution tools, where separation from the underlying platform is desirable rather than a problem.

3 Likes