History Q: Why not version-named python executables on Windows?

Title is basically it. Why did we decide to create py instead of generating pythonX.Y.Z.exe, pythonX.Y.exe, pythonX.exe named executables?

Similarly, if we have py, why doesn’t it exist on other OSes? I’m trying to understand what was different between the OSes that led to this difference.

Part of me wonders if it makes sense to add both mechanisms to all OSes for parity. (Or pick one and centralize)

I’ve seen GitHub - brettcannon/python-launcher: Python launcher for Unix but it’s not official.

3 Likes

The py launcher was originally created to solve a very Windows-specific problem, namely that you can’t associate different things with the same file extension. On Unix-like platforms, you can have a wide variety of shebangs that distinguish whatever you like (version, install path, venv, etc), but on Windows, there has to be just one executable associated with “.py”, and it has to be able to handle everything. Hence the launcher, which parses shebangs and then invokes the appropriate interpreter.

I don’t know why there aren’t executables like python39.exe but I suspect it’s mainly just that that wouldn’t solve enough problems. Sure, you could explicitly invoke that from the command line, but double-clicking a script wouldn’t be able to use that, so a lot of people would still be out of luck; which means we need the py.exe launcher; which means the launcher can be used to invoke the particular interpreter from the command line, too. But that’s just me guessing.

4 Likes

In practice its been Windows that has been the pain point as @Rosuav explained.

All the non-Windows are Unix OS, I think, and seem to not have the issues that come up a lot on Windows. If there is a strong enough desire for a py command on Unix OS, as you found, there is a candidate implementation. But I am not hearing that desire being voiced.

The PSF’s app distributions of Python that can be installed from the Microsoft Store support versioned app-execution aliases such as “python3.exe” and “python3.12.exe”.

For the full distributions from the PSF’s website, this can be managed manually by creating versioned “python3.exe” and “python3.Y.exe” symlinks beside each “python.exe” that’s found in PATH, as determined by where.exe python.exe. For example, in the CMD shell, change to the directory of the Python 3.12 installation, and execute the command mklink python3.12.exe python.exe.

In the system environment-variable editor, make sure that the directory of the version you want to run by default for the python and python3 commands is first in PATH. This works generally best only if all installations in PATH are either system installations (i.e. all users) or user installations, or at least if there all only added to either the system or user “Path” value. The issue is that when the system creates PATH for a user environment block, it appends the user “Path” value to the system value. Thus the system value always take precedence.

The .py file association should remain set to the “Python.File” progid that the installer sets up for the py launcher. The new implementation of the launcher has an updated implementation of “#!/usr/bin/env” shebangs. The new implementation looks for versioned “python*” binaries in PATH before falling back on registered installations. The old implementation always used registered installations.

1 Like

Thanks folks. This is all good and useful info. I never really realized that py parses shebangs. I rarely see shebangs on python scripts on Windows, but definitely on the Nix. I never really thought to try it on Windows with py.

I always thought of py as just a way to pick which python via -3.X

It still makes me wonder if parity between OSes would be nice to have. I’ve seen websites with docs and Windows and Linux tabs for examples just to change python3 to py.

Then the idea of teaching that difference to newer users seems a bit confusing compared to ‘use py everywhere’.

Though I guess we might be too far down the path we’re on to look back now.

1 Like

There are two major architectural differences.

The first is what @Rosuav pointed out. py actually parses shebangs, which would otherwise be meaningless on Windows.

The second is how system-wide installed applications are organized. On Linux, the default expectation is that your executables are side by side, directly in folders like /usr/bin, and then the corresponding “support” files are in corresponding places like /usr/lib. But on Windows, the default expectation is that each application, with all its support, gets a separate subfolder within C:\Program Files.

That causes two major problems. First, it breaks tools that can’t handle spaces in paths - you could argue that those are already broken, but it does end up causing a lot of complaints. But that isn’t really relevant for our purposes.

More importantly, it means that each new application is not automatically put into a place that’s on the PATH - because PATH (on both Linux and Windows) specifies exact folders to check, not folders to search. Putting C:\Program Files on PATH wouldn’t help (and isn’t normally done); PATH needs a separate entry for each new application.

So, installers are tasked with modifying PATH to add the new application each time, if they so desire. And there’s a not-super-generous total length limit on PATH (as I understand things, this is 4096 bytes used to store UTF-16 text - typically documented as “2048 characters”, but, you know, surrogate pairs…).

Before py was added for the Windows releases, Python installers did modify PATH by default, and also included the aliases you describe. They would sort of work, in that python3 would find the python3.exe in the first folder on PATH that contained one (corresponding to the most recently installed 3.x Python); and in particular it would not find a 2.x Python, even if you installed one later, because those installations simply didn’t have that alias. But this still means potentially having multiple useless (shadowed) python3.exes lying around - not very elegant. And it still doesn’t solve either of the other problems (PATH pollution and shebang ignorance).


In case this leaves you wondering, “why make a py for Linux, then?” - that’s addressed, IMO, by the Windows documentation:

Unlike the PATH variable, the launcher will correctly select the most appropriate version of Python. It will prefer per-user installations over system-wide ones, and orders by language version rather than using the most recently installed version.

Linux can “select the most appropriate version of Python” natively (this is referring to shebang parsing). But even tricks like #!/usr/bin/env python need additional work to “prefer per-user installations”. py can keep track of them by other means besides the PATH. I believe (although I can’t test this right now) that it also allows overriding the script’s shebang preferences with the py command line.

“Ordering by language version” is moot on Linux, because there’s only one python.exe or python3.exe to find, and it has already been configured to alias a specific minor version. But for example, on my system I have 3.8 that came with the OS, and 3.11 that I compiled from source and alt-installed into /usr/local/bin; python symlinks to python3, which symlinks to 3.8, whereas a hypothetical py would presumably default to 3.11 (assuming it knew about that installation). That would especially be useful on Linux because of the system dependence on Python - users would have an option to access the more recent version easily, while the system could use a system-tested version by just requesting python and not needing to specify.

1 Like

This has bothered me for a long time. I have multiple versions of python installed on my windows PC and a lot of times I want to specify a specific python version on my command line (the same way I’m used to do on linux) in order to “play” abit with the interpreter and test certain things and I find annoying that I cant do it the same way.

I found myself literally creating my own “python3.X.exe” so ill have a single way to do so instead of a separate way between linux and windows. Is it stupid to do so? Probably :stuck_out_tongue:
But I find it easier to do so than to use py.exe every single time I want to test something on the fly.

1 Like

I find that its easier to use py -3.10 to test a 3.10 specific then setup all the PATH infra to allow python3.10 to work.

I feel that the py launcher was a good idea, sort of the first step towards a manager-first Python install. Ultimately, being able to specify just the version of Python isn’t really enough: you may want to specify the whole environment (installed libraries, etc.). This is similar to what can be achieved with conda run. I think the way forward is not to try to create a bunch of differently-named executables, but to created user-named environments, which then can be explicitly specified as part of a particular execution task.

1 Like