Windows Store package to become installer/launcher?

Ideas
1

So I expect many are unlikely to realise, but when we distribute CPython on the Windows Store I’ve been releasing each version as its own app. The model of all app stores is that you don’t get to control installation, and so users (virtually) always have the latest version of an app and cannot install side-by-side copies. Due to how CPython’s compatibility works,[1] the only way to make things properly correct is to have a separate app for each major version (shown in the screenshot below).

image
image1814×1272 294 KB

Clearly, this is far from the least confusing way to give people Python (with the exception of people who already understand our release model and are quite happy to be able to choose which versions they get). So I’d like to propose a new app that includes the ability to install specific Python versions on demand (note, not proposing any changes at all to the regular installer - only to what users get from the Windows Store, which already has some subtle differences).

A significant constraint is that the vast majority of Store package installs come from people who have typed python or python3 on a clean Windows machine. These are presumably users who know they want to run Python, but don’t know anything except that command. Anything we install from the Store has to replace these commands so that these users can re-run their command and be successful.[2] We also can’t modify PATH automatically - users have to be taught to do that themselves.

So, my proposed interface is basically this: a single Python app that provides python.exe and python3.exe commands (as today), where the python command is actually a launcher rather than the interpreter itself.

This launcher would have the following subcommands:

  • python run ... - essentially the same as py.exe ... today,[3] and the run is optional (as in, if no subcommand is specified, the rest of the command is passed directly to whichever real Python runtime is selected). If no suitable runtime is available, it gets downloaded automatically
  • python install [version] - manually acquire a particular version
  • python list - list installed versions/paths (with filters, JSON, etc.)
  • python tool ... - do a pipx-like operation using the current/latest runtime
  • python help ... - get better help than just -?, maybe search/link to doc pages?

[Edit]: Added python help subcommand

Any other thing that looks like a subcommand would be treated as a script filename, as it is today, so it’s only these few commands that would change behaviour. It is very unusual on Windows to create scripts without the .py extension, so I have no concerns about this change (also, python .\tool is a very easy way to ensure you get the file .\tool rather than the subcommand). The python3 command is interpreted like python run ... and so other subcommands are not supported (and one day it will refuse to launch a Python 4, but those days are very far away still).

Worth re-noting here that I am not proposing any changes to the regular installer or the regular python.exe you get when you build from source. This is only for a new Windows Store package, which can only really be installed manually by a user.

Because we can’t directly modify PATH, users won’t get additional python3.13.exe commands etc. when new versions are installed. However, because we’ll directly control the install (both python install and python tool), we can make them available in a single directory that users can add to their PATH if they want.

Venv is still incredibly important, and the idea is for anything more complex than a simple script or basic interactive use, users will python -m venv ... and activate that, giving them the usual interface and behaviours. All that’s changed is the command to create the venv in the first place.

There are a number of other technical choices that flow from here (e.g. where are the installs kept, etc.), but in the hope of keeping the discussion a little bit focused, I’m going to defer those. There’s more than enough in this much proposal to keep everyone busy for now.

So, thoughts on the proposed design?

  • python.exe installed from the Store acts as a launcher/manager rather than direct to a specific runtime
  • python3.exe installed from the Store acts as a launcher (but not a manager)
  • Specific runtimes are downloaded/extracted on demand/request
  • All operations can go via python.exe, or the user can modify their own PATH to get more direct aliases
  • python tool ... subcommand for “just install and run this tool however”
  • python -m venv ... still recommended for serious users (i.e. practically everyone here)

  1. Specifically, we cannot silently upgrade users from e.g. 3.13 to 3.14, which is how it would work if we only had a single app in the Store. (Edit: Added later due to discussion feedback) ↩︎

  2. Guess it’s worth noting that Microsoft owns those pre-installed stub commands, and only point to a python-dev owned app out of respect. They could point it to their own app if they decide it’s better for users. ↩︎

  3. Providing py.exe from this app doesn’t meet the “replace the stubs” constraint, but the entry point could be provided as well. It wouldn’t be the same as the current py.exe though. ↩︎

4 Likes
2

I understand the need for this, and I think it’s a good solution. But I think we’d be missing out on a huge opportunity if we didn’t support PEP 723 (inline script metadata) in this tool. Given that PEP 723 includes the ability to specify the required Python version, having a common launcher with the ability to access and install multiple officially supported versions would be a big win here.

So basically, I’d like to see the functionality of pipx run incorporated into this launcher (both pipx run script for PEP 723, and pipx run toolname as python tool toolname).

4 Likes
3

This is good proposal, but I don’t like this part:

Typing python should really give you the python.exe interface, regardless of how you install Python. Anything else is just confusing.

Why not use a different name, e.g. get-python or install-python ?

4 Likes
4

Because the expectation is set (and demonstrated, as I mentioned in the post) that the person installing this app is already trying to run python. We have to provide that command, or else they’ll fail, and as we’re no longer installing a specific runtime from the Store, it needs the functionality to get something.

Just running python will still get into a regular Python prompt, though (probably after jumping through some kind of console-based downloading UI). Choosing a particular version would require python run --<some option>.

Works for me. I think we can fairly safely assume this is going to be used by the exact audience that was intended for, too.

2 Likes
5

Then it’s probably better to call that new app “python-installer” or something similar, so that people run python-installer because that’s what they expect to get.

Alternatively, the installer called python should enter into a special interactive mode explaining that it’s not the real Python interpreter and ask the user for commands. But I really don’t think this is a good idea.

If we go down this route, people will expect the same sub-commands and interface on other systems and we’d end up just shifting problems, rather than fixing them.

4 Likes
6

I’ll be honest, I wouldn’t be against incorporating PEP 723 support into the core interpreter…

3 Likes
7

We’ve spent a decade (more?) trying to convince people to run py instead of python and that hasn’t worked. So I’m going to need a lot of persuading to introduce another new command.

2 Likes
8

I think this has the potential to cause more confusion if this operates from the python command but is specific to the windows store installation. The user will probably expect these commands to always ‘work’ when typing python and will find out they don’t if anything ends up replacing this special python on their user PATH (or if they need to switch to another operating system as mentioned before).

Also, the proposed install/list/tool features are things that would be useful in general, so it would be a shame to have them locked to only one specific mode of installing Python on one operating system.

I can’t speak for anyone else but I’ve used the py command in the past to specify which version of python to use to build a venv but never really stuck with it as it’s both unavailable[1] on other platforms and for some reason it chooses prereleases[2] by default if it finds them.

  1. or at least the closest equivalent is not included with Python by default ↩︎

  2. and the experimental free threading builds ↩︎

1 Like
9

I use the py command all the time.

I almost exclusively use Windows, which might explain the difference

This, I agree, is a dreadful default. But rather than stopping me using py, it stops me installing prereleases (which is a shame, I’d like to test them, but not as things stand).

10

I’m fairly constantly hopping between windows, macos and linux for different purposes so it’s nice to have things that are consistent between the systems.

I think part of the difficulty might also be that if I was going to be writing a tutorial of some kind, I’d use the python command over py because that means I don’t need to write separate instructions specifically for Windows.

11

This is way offtopic, but for Unix you need to say python3. We had this debate (extensively!) with pip, and came to the conclusion that you have to have platform specific instructions if you don’t want people to misunderstand :slightly_frowning_face:

1 Like
12

Do you still need to specify python3? I don’t think I’ve ever specified that but I’ll admit that as an Ubuntu user when in the linux space, I’ve always treated the OS provided Python as something for the system to use and for me to ignore.

Edit: Yes, on removing all the other options from PATH this is the case. Although the OS python I do get with python3 is also rather outdated (hence ignoring it in the first place).

13

I agree that python and python3 should only ever invoke a Python interpreter, for consistency. Anything else is confusing. MS store Python already has a few behaviour quirks as it is.

However, the installer as described sounds like a good thing, and it would make sense that an attempt to execute python on a system without it installed should then point to download that installer.

Because this seems to be to be agonizingly close to what some people have wanted for a long time: an official Python equivalent to rustup that can bootstrap, install, and manage non-system Python installations. It’s one of the things that pyenv, rye, and now uv (maybe others too) implement because there’s user demand. So why not just go the whole hog and make that tool (not saying you specifically, I mean the Python team/community), and for all OSes rather than purely for the Windows store? I think a lot of people would be extremely happy to see that.

Couldn’t running python on a clean Windows then result in pythonup being downloaded and installed, and during installation a single default interpreter is simultaneously installed to a sensible location x, and the installer prints messages to the CLI informing the user 1) that pythonup can be run in future to manage the Python installation; 2) that python-31x.exe has been installed to x and made the default interpreter run by python.exe; 3) how they should add x to the PATH?

pythonup could also install pip, and should there one day be an officially sanctioned Python project manager along the lines of poetry, uv, hatch, pdm etc. it could also be installed by pythonup, just like rustup installs not just rustc but also cargo.

1 Like
14

Something that replaces the user’s default Python install (presumably the user themselves, or else another application’s installer that has a bug) is a risk, certainly.

But as has been pointed out, the python command already doesn’t work consistently across platforms. And I really don’t see the benefit in creating yet another command that won’t work consistently across platforms - only the losses, when we still need the Store’s python.exe to do more than “just run Python”.[1]

The inconsistency is inevitable. I’d love to be able to have at least one of the inconsistent experiences be a good one.

  1. Repeating, there is no Python runtime installed with this new app. It has to locate and/or install one and then launch that instead. ↩︎

15

Apart from the changed name, that’s what I’m proposing (and I think changing the name is unhelpful, given the reasons above).

We don’t control installation this well. The app is essentially a single zip file that is installed for all users on the machine and managed by the OS.

We can put an interpreter in there,[1] but if the user ever relies on it we can never upgrade it or they’ll break. So the python.exe itself has to be an installer/launcher, not the actual interpreter.

Unfortunately not, this isn’t a feature of app store installs. The best we can do is print an error when the user tries to run python again and make them change their behaviour. This is unpopular (I could find the angry tweets I got last time I tried this, but I suspect those people all quit Twitter and deleted their accounts).

Again, we get exactly one implementation of python.exe in this context. So this becomes very complicated for users to control. It’s actually the current setup, but the UI for choosing which version is python.exe is pretty terrible, and the Windows team has refused suggestions to improve it.

Only the fact that they should. We can’t provide reasonable steps to do it in CLI output (a web page is okay, because we can update it whenever with separate flowcharts for changing these settings based on your Windows version, or at least based on what the user sees when they start following the steps).

  1. And I think I’d at least put a packaged copy of the latest release in there, so that it can be installed on demand without network access. ↩︎

16

I just think it would be weird to deliberately have python run something other than what every single guide on the internet currently says is run by it unless it’s a universal change across all platforms. And a user only needs to be guided to run pythonup once, to get python.exe installed (or very occasionally to upgrade), whereas they’ll want to run an interpreter all the time.

That’s a shame. I think the alternatives that you state would be good enough though (download and cache a packaged interpreter for later installation, and give an instructive error if python is run after installing pythonup but before actually running pythonup). And people don’t need to change their behaviour really, because once things are set up python will work fine, it’s only one time that they need to be guided to run pythonup or whatever the installer is called. Surely people can’t get angry about that, otherwise they’re essentially complaining that they have to install Python at all, that it isn’t installed and ready to go on all Windows systems, which is up to Microsoft.

Microsoft’s guide seems to reckon that installing from the MS Store automatically adds Python to the user PATH, so I’m missing why it would be a bad thing for a hypothetical pythonup to similarly add whatever interpreters it installs/manages to the user’s PATH for them (by default at least)?

Alternatively, if you are very wedded to having this new installer/wrapper invoked by python, why not add the same design to all distributions of Python? Because the design seems to be liked. As you said, in your design, once there’s a suitable Python interpreter on the system then python <script> has the same end result as now.

Also, where would the new installer download runtimes from? Wouldn’t this require official distribution of interpreter binaries? There’s a reason rye and uv use python-build-standalone binaries, right?

Anyway, those are just some thoughts, I’m only a naive user. But it felt to me like with your root problem you’re making a further good case for adding that rustup-like experience in an official way… :grin:

3 Likes
17

Weird perhaps, but entirely unavoidable here. The only thing to discuss is whether to artificially restrict the launcher and give users more errors than help. In my proposal python.exe <anything other than the subcommands> would behave exactly like a normal interpreter[1], and since Windows users practically never have Python scripts without .py extensions, those subcommands are not going to conflict with anything.

That “one time” is what requires python.exe to not just be an interpreter :slight_smile: Either it’s always just an interpreter, in which case users get “command not found” errors until they figure out the new name, or it’s something a bit special.

Windows Store apps can specify global aliases for their entry points, which are static executables contained in the package. That’s how Python gets onto PATH - because it’s a static entry point to the current app. There’s no functionality for dynamically creating global aliases (again, Windows team rejected that suggestion), so the best we can do is create our own directory and ask users to add it to PATH themselves. (I won’t shift on automatically modifying PATH settings. It’s unreliable and unsafe, and we won’t ever do it by default. In this case, we can’t even do it not-by-default.)

Linux used to do it (they now do it via python3), and I don’t want to make changes to the legacy Windows installer because it will break people. The only other distribution we control is the macOS one, which would likely benefit from a similar change, but is different enough from the Windows one that consistency isn’t such a big deal. I believe that it’s also not as popular as our Windows distribution, with many (most?) macOS users preferring Homebrew, which does its own builds. On Windows, tools like Chocolatey and Winget use our distribution.

For Windows it’s very easy - they’re already distributed on Nuget in a nearly suitable package. The python-build-standalone project is needed for Linux and macOS where relocatable binaries are not standard, and the work has not been upstreamed, but for Windows it already works with our standard builds.[2] But others are thinking about making that work for those platforms, so I’m sure it’ll come one day.

Appreciate your contributions!

  1. Apart from a few technical matters such as process handles, native debugging and potentially stream redirections. ↩︎

  2. We don’t distribute them this way because it’s so easy to mess up “installing” a zip file that it’s more harmful to users than helpful. Getting registration, shortcuts, file associations, and default install locations is a real benefit. Nuget has more conventions around packages of files that help ensure the right users are using it. ↩︎

1 Like
18

My guess as to a common way this might happen would be a user following instructions that indicate the user should install from python.org and select add python.exe to PATH. I don’t know if conda or some other scientific distribution can also add its own python.exe to PATH?

1 Like
19

Do you still need to specify python3? I don’t think I’ve ever specified that but I’ll admit that as an Ubuntu user when in the linux space, I’ve always treated the OS provided Python as something for the system to use and for me to ignore.

On my Debian systems (up-to-date “unstable” version) and Ubuntu (latest LTS), there is still no /usr/bin/python by default, only a /usr/bin/python3. It has an optional package available called python-is-python3 which exists solely to install a symlink from /usr/bin/python to python3.

In essence, during the Py3K transition when many distros shipped both 2.x and 3.x versions side-by-side, /usr/bin/python referred to 2.x for backward compatibility purposes. In order to not break systems which continued to install their own old 2.x version even after packages for it were dropped from the distribution, the current 3.x packages don’t automatically take over supplying /usr/bin/python.

20

Yes, I did some looking into this after and found much the same thing. So take it as more a statement of disbelief that this is still the state, while I do now understand why it was during the 2 to 3 transition.

That said once you’re in a venv this stops being an issue as python should always point to the venv at that point…

Which does lead me to raise the question of, would this windows store “python” also replace the python command when a user is inside a venv? I see this as another place where this could be surprising if the commands suddenly didn’t work.

1 Like