Pre-PEP: A Python Installation Manager for Windows

[pf_moore (Paul Moore)]

I’ve gone point by point through your comments. But the important thing is that my expectation is that the installer should install pymanager and the default Python interpreter, all as part of the initial install. See below for an expansion on this point. If that happens, the rest of the user experience flows quite naturally.

@steve.dower

I’ll bundle up all the comments about uv (and conda, and yes, conda is effectively equivalent to uv here) and reply to them all at once.

I’m a strong +1 on the point you make that recommending another tool like uv or conda is effectively abandoning the idea of providing binaries. But unlike you, I’m very definitely against that option. Not just because of the disruption it would cause, but also because I feel that having an authoritative source of binaries is important.

@steve.dower

after installing PyManager, should python install and launch Python on first run? Or should it give an(other) error pointing to pymanager and/or activating a venv?

My answer here is “neither of these”. In my view, installing PyManager should also install Python, so that after instaling PyManager, python should run the newly-installed interpreter.

In trying to understand why you framed the question the way you did, it occurred to me that maybe MSIX installers are like wheels, and can only unpack files and set up shortcuts, with no capability to do “post-install actions” like running the newly installed app. If that’s the case, then the problem is that you’ve completely missed the fact that most participants in this discussion have no idea what the constraints are on the new MSIX installer. And in that case, I think you need (both here and in the PEP) to more clearly explain the constraints we’re working under, and why we have to accept them^[1].

If “Installing PyManager also installs an initial Python interpreter” isn’t an option, then I agree we’re in a bad place (because Microsoft hijacked the python command for this usage, not because of a decision we made). In that case, I think that python installing and launching Python on first run is the best we can make of a bad job. But I think it should be very much a one-off thing, with the initial run of python doing the install and then fixing the app alias to point to the interpreter, so that the long-term effect is the same as if Python had been installed when the manager was.

@steve.dower

should running pymanager install and launch Python on first run? Or should it give an(other) error telling users what all the commands are?

IMO, pymanager should be just that - the manager. It shouldn’t run Python (unless we add a pymanager run subcommand, but why bother with that?) Running it without a subcommand should provide usage help - just like other commands that use subcommands, such as git, do.

Having a separate manager is a common pattern - rustup, for example, is similar. I think this is a perfectly fine user experience, probably even better than having a weird “the manager can also run the language” mixture of responsibilities.

@steve.dower

If we decide python points to the other command, then after installing a global runtime, should PyManager’s python launch that runtime? Or should it point to pymanager/venv still? (to encourage users to move to a single command unless they’ve explicitly activated)

(Added while reviewing)
Re-reading this, I now think that you’re hinting at what I said above - the python app alias initially points to some “other command” (not pymanager itself, but maybe a special “mode” of pymanager) which installs the runtime. If that is what you mean, then once it’s done the install, it should redirect the app alias to point at the new runtime. That’s sort of like saying “PyManager’s python should launch the runtime”, but I don’t think there’s a need for an explicit "PyManager’s python after that initial run.

(My initial comment)
I don’t know what “other command” you’re referring to here. In my model, python always points to the command that runs the Python interpreter (specifically the interpreter that the user expects, which means “the default system one unless a venv is active, when it should be the active venv”).

If, by python, you mean “the Windows AppAlias thing”, then I don’t understand enough about what constraints there are on what AppAliases do to be able to answer that question. Once again, I think you underestimate how much people understand the Windows side of this proposal.

My understanding is that the AppAlias can change what it points to - otherwise how would installing a new runtime change the version that python runs? And my understanding is that a python.exe on PATH overrides the AppAlias. Putting those two facts together, I would say that the AppAlias should point to the installed Python that’s marked as the “default”. That way, it behaves exactly as the core interpreter (because it is the core interpreter), and activating a venv replaces it with a python command that behaves the same way (by the design of venvs). If any of that isn’t possible, you need to explain why the AppAlias mechanism prevents it - both here and in the PEP.

@steve.dower

If we decide python launches global runtimes, should it support -V: and shebang processing for users to select? Or should we only allow those on pymanager?

It should not support -V: or shebangs, because the core interpreter doesn’t. The pymanager command should not support those either, because it’s a manager, not a runner (again, if you want to allow pymanager run then that could be an exception).

The py command should support these things, as it currently does. And with that in mind, installing pymanager should also install py, which behaves functionally exactly the same as it currently does. In some ways, py could be viewed as equivalent to the possoble pymanager run command I’ve mentioned a couple of times, but I see py as the most important one to include (for backward compatibility reasons) with pymanager run as an optional extra.

@steve.dower

If we decide python can have version selection, should it also support auto-install? Or should we error in that case and require manually using pymanager?

No, it shouldn’t. python -V:3.12 is an error according to the Python documentation (there’s no documented -V option), so it should give an error, just like python -F does.

Users should use py -V:3.12, just like they currently do on Windows, or use python3.12, just like they currently do on Unix. We shouldn’t provide auto-install, users can manually install the versions they need (and speaking as a user, I’m perfectly happy with a tool that says “what gets downloaded and installed is under your control” rather than downloading things for me).

@steve.dower

If you’re taking this seriously (you don’t have to, but I am), work through your own proposed scenario and see whether you feel successful or whether you feel beaten up until you’ve done exactly what the tool wants you to do.

OK. Two scenarios.

First Scenario

New user, on a new machine. Personally, I’d go to the Store and look for Python to install it manually, but let’s say I somehow went to the command line and typed python. It opens the store. I’m a little confused, but whatever, I can now download “Python”, so I do. I’d hope the Store page explains that this will install the Python interpreter and a pymanage tool to manage my Python installations, but I’m cynical enough to expect that it just says “Woohoo! Python is awesome, have fun doing AI and data analysis with the best language ever!!!” Now, I go back to the command prompt and run python. I’d expect Python to start. I’d expect it to work exactly like the python command documented in the Python documentation, and in all the Reddit and StackOverflow articles I read rather than reading the documentation I’d hopefully know (either from the store page or from Reddit/StackOverflow - or maybe even by reading the Python docs!!!) that there’s a pymanage command which lets me install new runtimes, and do other cool stuff. But for most day to day usage, I’d expect things to “just work”.

As a naive user, I’d expect to be able to double click on .py or .pyw files, and be able to associate them with Python. I’d prefer it if the installer had done that for me, but I can live with having to do it myself. But if I have to do it myself, I’d want there to be only one pair of choices - “Associate with Python (Console)” or “Associate with Python (GUI)”. This probably means using the py launcher, as it was explicitly designed for this use case - which is why I want it installed alongside Python. I would not want to have to pick a Python version. In fact, I would probably not even have multiple Python versions. The only use I’d make of the manager would be to upgrade Python, and I’d probably wonder why the Store app didn’t auto-upgrade. But the first time I upgraded my Python and my existing scripts got broken, I’d be enlightened. At that point, I’d want to be able to use multiple versions, and I’d need to find out about shebangs, the launcher, maybe even virtual environments. Now I’m not a new user, though, and while my experience starts to degrade rapidly from here, that is not the problem we’re trying to solve in this PEP.

Second Scenario

I’m a relatively experienced Python user on Windows. I’ve been using Python for a number of versions - maybe I have multiple versions on my PC. I am familiar with virtual environments. I generally use the official distribution, not the Store one. Maybe I use the launcher, maybe I add python to my PATH. I’m grumpy and cynical, because it’s all annoyingly hard, but I’ve got something that works. Now the new version of Python comes out, and it uses the new launcher. Maybe I’ve heard that you can now get “proper” Python from the Store, or maybe I got the MSIX from python.org. Either way, I run the installer. I want it to just install, not to start Python for me. But I do expect it to install the latest version of Python.

I run python. I expect to see the new version of Python start. If I typically use py, I’m trying out whether python now works. It does. Unless I disabled the python AppAlias, in which case I probably forgot I’d done that and wonder why I’m getting “Command not found”. But I try py and it works. Great. Now I try pymanager. Also great - I can uninstall all those old Python versions, and reinstall them from here, managing everything from one place. At this point, I might break everything, because I forgot that all my venvs are tied to the existing installations. But maybe I’m lucky and remember that. And after all, “recreating venvs is easy” isn’t it? Sigh. Maybe I’ll keep the old versions as they are for now. (A note here - what happens if I have Python 3.12 installed and then install a second copy via pymanager? Will the two be separate in the registry?) I try some venvs, both old ones and new ones I create for testing. Running Python using py or python works just fine, as expected.

But overall, everything is good. It’s not clear to me (writing this scenario) whether the py command is now running the old py installed with previous Python versions, or a new py installed with the manager, but I (the user in the scenario) don’t actually care, as long as it keeps working (and will continue to do so when I uninstall the older Python versions).

On the other hand, if I added Python to my PATH, I expect to need to change my PATH, but I’m not sure what to change it to, because the installer didn’t talk about PATH and didn’t offer an “Add to PATH” option. Maybe I just need to remove old Python versions from PATH? Should I have done that before installing? Anyway, python is still running the old Python version. That’s annoying. I remove Python from PATH by manually hacking it - it’s annoying that you can add Python to PATH when installing, but there’s no supported way to remove it later (short of uninstalling). But I’m sort of used to that - I had to do it every time I upgraded^[2]. OK, now python runs the new version. Cool.

Most of the rest goes the same as the py user, although the benefits of switching to managing older versions of Python via pymanager are more compelling here - the pythonX.Y aliases and easier switching of the default version being the two clear wins.

(Apologies if I’ve made “add Python to PATH” seem like a terrible experience - I haven’t used it myself in years, and maybe it’s not as bad as I remember it being. The point is that I think it would be a rougher ride than using the launcher, but not in any way that would be unfamiliar to someone who used that approach).

---

1. This relates more to why we’re proposing msix/pymanager as the official python.org installer, I’m assuming we have no choice over msix for the store app ↩︎

2. or if I didn’t, my PATH is a bit of a mess, and I have a tidying up exercise to do… ↩︎

[steve.dower]

@pf_moore

In trying to understand why you framed the question the way you did, it occurred to me that maybe MSIX installers are like wheels, and can only unpack files and set up shortcuts, with no capability to do “post-install actions” like running the newly installed app. If that’s the case, then the problem is that you’ve completely missed the fact that most participants in this discussion have no idea what the constraints are on the new MSIX installer. And in that case, I think you need (both here and in the PEP) to more clearly explain the constraints we’re working under, and why we have to accept them

Thanks, this is a key point, and I’ve certainly not been clear about our abilities in this area.

First, the reason we want MSIX is because it’s the future and MSI is the past. It won’t ever be removed, I’m sure, but I’ve reported bugs in Windows Installer that have been wontfix’d. As far as I can tell, most other standard installer technologies have all the same issues/risks as MSI (in terms of things not working reliably or being corrupt), while MSIX has far fewer (and I’ll call them out below).

The biggest constraint on MSIX is indeed, that it is essentially like a wheel and cannot do arbitrary post-install actions. Moreso, it’s like a wheel that’s installed into a protected directory and symlinked into each environment (user account, for MSIX), so whatever is installed will be per-machine paths with special APIs required to locate the per-user (modifiable) areas. This is why we can’t simply drop the “regular” python.exe into a Store app, and it has extra functionality to locate the correct paths.

All app executables must be part of this package if they’re going to be associated with global aliases, Start menu shortcuts, or file associations (or any of the other MSIX-supported metadata, which is fairly extensive, though not quite at the “arbitrary command” level). And only users are able to modify these after install - most notably, the “Manage app execution aliases” has no programmatic equivalent (and my request to add one was rejected).

This is unfortunately also one of the “issues” with MSIX, in that user modifications to global aliases take priority over whatever an app might request. So once a user has said “I don’t want python.exe alias”, they won’t get one.^[1]

So changing the actual alias is not viable. Making it point to something that knows what to do is viable, but unless we put that all into python.c under a big #ifdef, it won’t be “regular” Python. I don’t see the point in putting it there, since it would mean we can’t release the manager independently of the runtime, and that’s one of the biggest benefits here.

“Installing PyManager also installs an initial Python interpreter” isn’t really an option, or rather, it’s one that would be deemed unacceptable. Hypothetically, let’s say PyManager 1.0 included Python 3.13. Because of the MSIX install process, this 3.13 wouldn’t get a PEP 514 registration^[2] and would be completely unmodifiable (which pip today would detect and infer --user, so not a big deal, but this wasn’t the case back when we first moved to the Store). Okay, now we release 3.13.1, but users can’t get it until we release PyManager 1.1, since it’s part of that app and so can’t be updated independently. Now we release 3.14, and what do we put into PyManager? If we update 3.13 to 3.14, all users will get the newer runtime and potentially break. If we don’t update, in ten years time we’re still shipping 3.13 by default.^[3]

My actual proposal is that the latest released runtime is bundled into the PyManager app, so that it can be extracted on first use if nothing else is found. With the uncertainty around automatic installs, this may not be viable (we’d have to come up with an obvious way to say “just install something, I don’t care, I just want to run Python”), but assuming it went through, it would mean that the first run doesn’t require any network^[4], and that we can later update what’s bundled without affecting what the user has on their machine. So the latest PyManager would be pretty close to the latest CPython on a clean machine, and be safe to upgrade in-place for existing users.

@pf_moore

First Scenario

Good news, the current design (and prototype) will give you exactly this scenario! The only bit I suspect you’ll disagree with me on that is “I’d expect it to work exactly like the python command documented in the Python documentation”^[5], but it does. For any command that would work with the current python command, it will behave exactly the same with two caveats:

• if you haven’t got an actual runtime yet, you’ll see extra output and it’ll take longer on the first run - this is a technical need to allow us to do updates over time, as I discussed above
• if you are trying to run a file named list/install/etc. (not list.py or .\list or any variation) then you’ll get different behaviour - this is a UX preference, to allow the use of a single command for related tasks, rather than having to discover alternates

Other than those two, everything that works today will work exactly the same under my current proposal.

@pf_moore

Second Scenario

This has a lot of aspects where we don’t really have a choice but to say that the user opted out of our supported path. If they’ve modified PATH by any means other than checking the box in the traditional installer, or modified app aliases, then we can’t guarantee that they won’t have to modify them again by hand to get certain behaviour. It’s just impossible to do anything more than this.

@pf_moore

what happens if I have Python 3.12 installed and then install a second copy via pymanager?

The two are separate on disk, and depending on how the original was installed they might be separate in the registry. We use the same entries (for compatibility), and PyManager avoids stomping on existing entries (for compatibility). If the original was installed for all users, the keys will be in a different location from what PyManager uses and you’ll get both.

@pf_moore

It’s not clear to me (writing this scenario) whether the py command is now running the old py installed with previous Python versions, or a new py installed with the manager, but I (the user in the scenario) don’t actually care, as long as it keeps working (and will continue to do so when I uninstall the older Python versions).

Assuming supported paths, it’ll be running the old one, because the old py is always added to PATH ahead of MSIX app aliases. So you’ll still get the old sort behaviour (3.14.0a2 is preferred over 3.13.1; free-threaded is preferred over non-free-threaded), the old registry merging behaviour (a per-user install takes precedence over a per-machine install with the same tag), and won’t get updates for it once you stop installing the traditional installs (we can’t update a different app’s files from an MSIX, which I hope is a nuance that was assumed ).

@pf_moore

On the other hand, if I added Python to my PATH, I expect to need to change my PATH, but I’m not sure what to change it to, because the installer didn’t talk about PATH and didn’t offer an “Add to PATH” option. Maybe I just need to remove old Python versions from PATH? Should I have done that before installing? Anyway, python is still running the old Python version. That’s annoying. I remove Python from PATH by manually hacking it - it’s annoying that you can add Python to PATH when installing, but there’s no supported way to remove it later (short of uninstalling). But I’m sort of used to that - I had to do it every time I upgraded

To be honest, I consider this trauma, and I’m sorry you (and everyone else) ever had to learn to live with it. Persistent PATH modification in the presence of side-by-side installs is an absolute nightmare. It’s the reason I disabled the option a decade ago and formalised registry registration instead. It’s also the reason why python3.exe has never been added to the traditional installer, because we really don’t need yet another randomly ordered versioned executable causing confusion.

The right approach is for us to have a single directory containing all the commands, so that users only add one item to their PATH and we ensure that the correct aliases are created. That’s in the current PyManager design, and it’s how we can make python3.exe work reliably (there’s a dynamic alias as part of the app, in case users run python3 before modifying PATH, but there’s a static alias created for the latest stable 3.x runtime installed in the additional directory).

And yes, in this case, you’d uninstall all your existing runtimes, or remove them from PATH. It ought to be possible to Modify the existing install and disable that option without removing the whole runtime, but depending on what Windows has decided to show today the modify option isn’t always obvious. Rerunning the traditional installer will get you to it, though.

---

The main points that I’d like to emphasise though is that your scenario 1 would work with the current proposal, and scenario 2 is very likely impossible for us to resolve from upstream and it requires (more) intervention by the user.

---

1. I managed to get a special feature for the redirector stub to allow a new Python install to “steal” the alias, but normally once one app has the alias, the next app will be disabled. e.g. install Python 3.12 from the Store and then install 3.13 - see which one gets python.exe and python3.exe. ↩︎

2. We tried for a while to use a private MSIX extension to write registry keys, but it doesn’t work well with side-by-side installs and non-MSIX installs. ↩︎

3. Incidentally, this is the same argument I used to convince the team at Microsoft to not just put a full Python install in the OS by default. That was the original plan, and if it had gone through then we’d still be getting 3.6 (which at least had f-strings, so not a complete loss)… ↩︎

4. We probably want to check for a newer version, so perhaps we grab the index. But if no internet is available then we can go straight to the bundled runtime if it matches what the user requested. ↩︎

5. I’d update the documentation, FWIW, so in that sense it would work exactly like it. But I know that’s not what you mean ↩︎

[pf_moore (Paul Moore)]

@steve.dower

We balance compatibility on our supported scenarios, but as soon as packaging/distribution/installation gets involved, we have hundreds of technically unsupported scenarios.

I think that’s a bit of a hardline view. I’ve seen you helping people with install issues enough to know that you don’t actually work like that. And this is open source - “supported” is a bit of a vague concept anyway. What matters more to me is what users do we care about. And in my view, not breaking people who’ve made a good-faith effort to work with the approach we’ve promoted up to now is a big part of that. Because I’m focusing on people making a good-faith effort, I think we can assume they will be sympathetic to changes that offer long-term benefits, but I don’t think we should push that sympathy too far.

@steve.dower

Of the items you list, you’ve missed all the little changes that if we made them on their own would constitute a breaking change (e.g. adding python3.exe, or fixing service account installs, or changing the order of py.exe’s results, or adding installs to PATH by default, or making free-threaded binaries install properly). But I didn’t list them in the PEP either, because they’re rationalisations, not reasons.

I think those are all important justifications. I don’t see them as rationalisations - it’s perfectly fair to say “making this change will allow us to include a large number of quality of life improvements that we haven’t been able to justify individually”, and then listing those examples you gave (and any others that you think of).

@steve.dower

Yes a manager is great, but anyone can do it.

In some ways “having a manager” (specifically, an “official” manager) is just another example of a quality of life improvement. It’s a bit bigger to implement than adding python3.12.exe, but for the user I’d argue it does just feel like part of a suite of improvements.

@steve.dower

I don’t particularly care to do anything much with the py launcher… The question really is, given PyManager, should/why should users also use py?

I don’t really care whether it’s simply copied, or reimplemented, or even just made available as pymanage run. What I do care about is that the functionality it enables (as described in the PEPs that define the launcher’s features) isn’t dropped. And “go and find an old version of the launcher, and by the way I’m sorry but there’s no launcher-only install hosted on python.org” counts as “dropping it” in my book.

Maybe we made a mistake bundling it with the Python interpreter all those years ago. But it’s our mistake, and we should own it. Even if all that means is splitting off the launcher into its own repo, releasing a standalone installer for it, and then just letting it tick over as an independent project (still owned by the core devs) that needs almost no maintenance.

In practice, I think that file associations and shebang support (to allow selecting the interpreter version) are the key features that we shouldn’t just abandon^[1]. Keeping the launcher is just the simplest way of retaining them.

@steve.dower

But the more serious point is that the lack of focus already exists. You keep referring to pip’s documentation, but go check out the range of other options already used in other projects

I know. My point about pip’s docs is that py was the best semi-official answer we could come up with (at least in part because it’s backed by a PEP, but also because of the mess around adding python to the PATH). PEP 394 sets a precedent for the core devs defining a policy on the command name to use via a PEP. I can understand if you don’t want to spearhead something like that, but I don’t think it’s unreasonable if this PEP adds to its motivation section that a goal of the change is to reduce the confusion by making python, plus the versioned names pythonX and pythonX.Y, the canonical way of running the Python interpreter on all platforms.

I just think that in doing so, we should acknowledge that it’s a change, and make sure we document how we intend to provide help in making the transition to those people who (in good faith) followed the previous recommendation to use py.

@steve.dower

Going back to all those user surveys, you’ll see that most users expected it all to be under a single command

I think everyone has a tendency to read those surveys in a way that suits their own preference. I personally view them as mostly about packaging and packaging tools. So “one command to do all packaging stuff”. I’m pretty sure that if you asked users “should we remove the python command and make running the interpreter into a run option on a workflow manager command?” they’d say “no”. On the other hand, tools like uv, conda, hatch, PDM and Poetry show that people like the “one manager command” approach, but that they can’t agree on which manager. And discussions around the indygreg standalone builds make it obvious that people would like interpreter runtimes to come from the core devs, but they like the “installation manager” approach. So having pymanage and python fit my reading of the various surveys and debates…

@steve.dower

But it’s not a technical point - it’s a policy point.

Fair point. And as a policy point, it’s probably something the SC should decide on.

Personally, I take a more nuanced view than “just run Py_Main”, but I agree with the principle that “python should run the interpreter”. In my view, the python command should behave as documented in the language documentation, and should behave consistently across all platforms. Subcommands are a departure from current conventions, rather than a violation of policy, but having python -3.12 be documented to find and invoke a Python 3.12 installation, on all platforms wouldn’t be a violation of policy in my view.

Theroretically, nor would installing a runtime on demand, but that’s technically far more challenging to achieve cross-platform, and could be hard to get core developer agreement on. It would also be difficult to get right in a way that supported distros - I don’t imagine conda would like it if you could run a Conda build of Python 3.13 as python -3.12 and get a non-conda Python 3.12 installed… Because of these complexities, I personally think keeping installation in a separate manager is a better reflection of the broad policy that “python runs the interpreter”.

---

1. FWIW, I don’t personally make much use of either of those two features, so this isn’t about me advocating for my use cases, it’s about what I think is right for Windows users in general ↩︎

[steve.dower]

@pf_moore

having python -3.12 be documented to find and invoke a Python 3.12 installation, on all platforms wouldn’t be a violation of policy in my view

@pf_moore

I don’t imagine conda would like it if you could run a Conda build of Python 3.13 as python -3.12 and get a non-conda Python 3.12 installed…

One thought I had on this, which I’ll just drop here so that it’s written down (I don’t really want to focus on it unless someone thinks it’s a great idea) is to make the -3* and -V:* arguments supported by the main command, but valid behaviour is to error out if they don’t match what’s being launched. That way, “imitation” commands like PyManager/py can use them for switching/selecting, but for the core runtime they’re valid options that just filter. Running a command like python3.12 -3.13 would be nonsense, of course (and you’d get an error), but if the command name is from the user then it could be convenient for filtering.

(Also, an active conda environment would override python with their own executable, since they put it higher in PATH than PyManager would be.)

[kapinga (Jon Harding)]

In general, Paul’s scenarios align more with my intuition, but I also recognize that I’m, personally, much closer to his Scenario 2 than Scenario 1. (In full disclosure: I’ve never internalized using the py command, and, much worse, have python installs managed by conda, the official python installer, uv, and perhaps even a couple from pyenv-win. Suffice to say that I’m very excited by the idea of an official installation manager on Windows.)

As a user, what I’d love to see in the long term is a well thought out, unified set of commands that apply to all the major platforms for user-provided python scripts (i.e., Windows, OSX, and Linux at minimum). Someone above brought up rust and cargo, which I think is a reasonable analogy (although there are also clear differences). As I understand it, that design has three fundamental parts: rustc (the compiler), cargo (the environment manager), and rustup (the version manager/installer/upgrade tool). If we analogize this to Python, one can imagine a similar trio of python (the interpreter), py (the environment manager), and pymanage (the version installer/upgrade tool).

However, a key part of what makes the rust trio of commands work is that they work everywhere. As a Windows user, I don’t have to worry (too much) if a rust article/blog/document was written by a Windows user or a Unix one; the guidance around how to “do stuff” is the same.

That’s not the case in Python currently: Windows users are instructed to run Python via the py command, while the rest of the world uses python or python3 (and I don’t expect random blog posts to show two versions of every command snippet). Similarly, there’s complaints about the lack of an official Python installer on Linux, which, if I understand correctly, largely centers around the (entirely reasonable) desire to not break the distro’s system installed python or scripts.

While I don’t expect this release to solve this issue in one go, it would be a sorely missed opportunity if this proposal is not thought through with a long-term, cross-platform goal in mind. The easiest way to keep the clean separation is to have separate commands that are (mostly) used to perform the separate activities listed above (as others have noted, imbuing python on with special install powers is likely a non-starter on Linux).

Reading through Steve’s recent description of the technical constraints is helpful, and I get the “first use” challenge. If the Windows implementation of the python command needs a special ability to install an interpreter to handle the first use scenario, that’s fair enough - but this ability should ideally be disabled as soon as a functional interpreter is found or, as a poor second, strongly discouraged in the documentation.

I’m probably too naive, but I can imagine an implementation that has the python, py, and pymanage aliases all point to the same executable, and then have that executable determine its behavior based on the name it was called by and whether or not it was able to find an installed interpreter. However, it should be a goal to make sure that, outside of the first use, python behave the same as it does when called from within a venv or on non-Windows systems.

[pf_moore (Paul Moore)]

@steve.dower

First, the reason we want MSIX is because it’s the future and MSI is the past.

IMO, this is a bad argument. Microsoft has a very long track record of abandoning technologies in favour of “the new hotness” - I’m pretty sure there was an installer before MSI, then MSI, now MSIX; we had DDE, then COM, then .NET; there was ODBC, then ADO; the list goes on and on. At some point, MSIX will be the past and there will be a new “flavour of the month”.

“We want MSIX because our policy is to use the latest installer technology on the platforms we support” is a better argument. But along with it goes the understanding that we shouldn’t assume that future technologies will support what MSIX does (or have the same limits as MSIX).

So the fact that MSIX makes it hard to have installing the launcher also install the runtime is a limitation we need to work around, rather than something we should base our user experience on.

@steve.dower

All app executables must be part of this package if they’re going to be associated with global aliases, Start menu shortcuts, or file associations (or any of the other MSIX-supported metadata, which is fairly extensive, though not quite at the “arbitrary command” level).

OK. That’s not something I was aware of. So when the user runs python, or pythonX.Y, or any other app alias, it cannot run the interpreter directly. In that case, I think the app alias needs to run a launcher - in effect, the py launcher.

That puts Microsoft in technical violation of the trademark policy, but we can mitigate that by ensuring that we only ever install an app alias which behaves identically to the interpreter. And that’s exactly what the py launcher was intended to do.

@steve.dower

if you are trying to run a file named list/install/etc. (not list.py or .\list or any variation) then you’ll get different behaviour - this is a UX preference, to allow the use of a single command for related tasks, rather than having to discover alternates

This is the preference I disagree with, and is the one other people view as “adding manager functionality to the interpreter”. I see the inconsistency of having python have extra functionality in some places but not in others (on different platforms, in a venv) as being a downside, rather than a convenience. And I’m not convinced by the discoverability argument, as the existence of the pymanage command will quickly become common knowledge, whereas “python install works, in most cases” is more nuanced, and therefore harder to internalise.

@steve.dower

This has a lot of aspects where we don’t really have a choice but to say that the user opted out of our supported path.

Well, yes, everything other than “using the py launcher” is to some extent off the supported path. But it’s still a user trying in good faith to make the most of a messy situation that we created. And to be fair, I didn’t suggest anywhere that the user had manually changed PATH. So all of the examples I gave were of someone working with what our installers did for them.

Maybe you’re OK with just saying “tough, deal with it” to users who are faced with re-learning a bunch of stuff when they thought they were doing things the way we wanted them to. I’m not. I’m OK with explaining why they need to change, and helping them as much as we can in that process - I’m not advocating we never change anything. But I don’t think we should be viewing these users as having got themselves into this situation, or somehow not worthy of our support.

@steve.dower

If the original was installed for all users, the keys will be in a different location from what PyManager uses and you’ll get both.

My intention was that the original was installed with the python.org installer, taking everything as default. So it would be a user install, so I guess the PyManager version wouldn’t show up in the registry.

@steve.dower

Assuming supported paths, it’ll be running the old one, because the old py is always added to PATH ahead of MSIX app aliases. So you’ll still get the old sort behaviour…

OK. That seems fair. And uninstalling the old py will effectively switch to the new one. All of the new features you mention sound like reasonable improvements to expect in a new version, so “uninstall the old one to get the upgrade” seems like a reasonable thing to say. I’d still want it to be documented that this is how you switch to the new launcher, though - it’s definitely not something I’d expect people to discover on their own.

@steve.dower

And yes, in this case, you’d uninstall all your existing runtimes, or remove them from PATH. It ought to be possible to Modify the existing install and disable that option without removing the whole runtime, but depending on what Windows has decided to show today the modify option isn’t always obvious. Rerunning the traditional installer will get you to it, though.

OK. Is there any reason this can’t be documented as the recommended upgrade path (along with a note about how to handle persistent venvs like those created by pipx)? Because that’s really all that I’m asking - that we provide documentation on how to transition, and we do what we can to make the transition as simple as possible (but no simpler).

@steve.dower

The main points that I’d like to emphasise though is that your scenario 1 would work with the current proposal, and scenario 2 is very likely impossible for us to resolve from upstream and it requires (more) intervention by the user.

I got the impression that handling scenario 2 for the py user was possible. What did you consider wouldn’t work in that one? I think we should make that work, as it’s our current documented approach. Not necessarily without any manual steps, and maybe with the resulting workflow no longer being the “recommended” one (we support py -3.12 while recommending python3.12 for example). But if we don’t provide a transition path for people using our current recommended approach, we’ll be open to valid criticisms that “Python has broken everything yet again”, which I’d like to avoid

While I’m in favour of making the experience as easy as possible for new users, I don’t think it’s right to ignore existing users. That’s the “introductory offer” fallacy that makes people think that no-one values loyalty any more.

[DavidCEllis (David Ellis)]

@steve.dower

after installing PyManager, should python install and launch Python on first run? Or should it give an(other) error pointing to pymanager and/or activating a venv?

I think it should install and launch, I think pointing to pymanager when someone is just trying to get Python for the first time would be a downgrade from the current behaviour.

---

@steve.dower

should running pymanager install and launch Python on first run? Or should it give an(other) error telling users what all the commands are?

Thinking about it this morning I actually think the most useful thing pymanager could do when launched with no arguments would be to give the user a basic UI with launch/run script/install/uninstall options. That way it could potentially also be something useful if run from the start menu.

Something along these lines?

Source as a PEP-723 script^[1].

Sorry if this is taking things too far away from your main goal here.

---

I think I’ve already made any arguments I have on whether the commands should be added to python.exe so I won’t comment further on that.

---

@pf_moore

@steve.dower

All app executables must be part of this package if they’re going to be associated with global aliases, Start menu shortcuts, or file associations (or any of the other MSIX-supported metadata, which is fairly extensive, though not quite at the “arbitrary command” level).

OK. That’s not something I was aware of. So when the user runs python, or pythonX.Y, or any other app alias, it cannot run the interpreter directly. In that case, I think the app alias needs to run a launcher - in effect, the py launcher.

That puts Microsoft in technical violation of the trademark policy, but we can mitigate that by ensuring that we only ever install an app alias which behaves identically to the interpreter. And that’s exactly what the py launcher was intended to do.

I’m actually quite used to this being the behaviour of python already. If you look at the file pyenv uses in place of python it’s essentially running pyenv exec python <args> and pyenv itself will figure out which version of python it should actually launch and pass everything through. The only real downside I’ve experienced is that the pyenv implementation of this can be fairly slow^[2].

---

1. Caveat: I’ve not used Textual before, so if I’m doing things incorrectly (and I almost certainly am) that’s not unexpected. ↩︎

2. On linux, querying all of the pyenv shims actually makes uv python list take an extra 2 seconds for me - which is almost all of the time the command takes. ↩︎

[steve.dower]

@pf_moore

I got the impression that handling scenario 2 for the py user was possible. What did you consider wouldn’t work in that one? I think we should make that work, as it’s our current documented approach. Not necessarily without any manual steps

(Emphasis mine). I’ve run out of time this week to respond in detail, but here’s a point where I think we’ve been talking past each other.

Every time I’ve been saying “not supported”, I mean “without the user taking manual steps”. Everything is fixable with the user taking manual steps, but discovering and telling them those steps in the general case is necessarily out of our scope.

We can sometimes detect cases where they need to take manual steps and report it (e.g. the new aliases directory missing from PATH), but we can’t do anything others (e.g. an old py launcher higher on PATH). It’s those latter cases, where the only thing we can do to help is write a big “if this, try that” document and hope that people find it and figure out which case applies to them.

If such a document meets your idea of “support”, then we can support it. What we can’t support is automatically detecting or fixing most of these situations.

[pf_moore (Paul Moore)]

@steve.dower

If such a document meets your idea of “support”, then we can support it. What we can’t support is automatically detecting or fixing most of these situations.

Ah, yes we have been talking past each other. Yes, I consider such a document to be sufficient.

I’d prefer the “manual steps” to be one-off actions rather than changes to workflow (hence why I want the py command to be present).

Longer term spelling changes (like moving from py -3.12 to python3.12) are acceptable. But people take time to learn new commands, so we should give them that time. They should just be about changing the command used, the behaviour should be the same.

If we want to remove any functionality (I don’t think we do, so this is mostly for completeness) then that should be the subject of its own PEP.

[pf_moore (Paul Moore)]

@steve.dower

(Also, an active conda environment would override python with their own executable, since they put it higher in PATH than PyManager would be.)

My point was in the case where the Python core mandates that python -3.12 must install and run a Python 3.12 interpreter. Then, even conda builds of Python must (by policy) and would (because they use the core sources) install Python 3.12. Unless conda patches their Python builds to get runtimes from the conda archives, that would mean that a conda python -3.12 would implement the default behaviour of installing Python 3.12 from python.org.

It’s a minor point, only intended to support the (probably obvious anyway) claim that making it a language guarantee that python -3.12 will install and run Python 3.12, is unlikely to ever happen. Sorry for making a big deal out of something that doesn’t warrant it…

[pf_moore (Paul Moore)]

@steve.dower

You keep referring to pip’s documentation, but go check out the range of other options already used in other projects

I will note by the way that you never answered my question about the pip docs. I would genuinely like to know what you recommend we put in the pip documentation should this proposal be accepted - and I’d like that same recommendation to be in both the “How to teach this” section of the PEP, and the transition document you suggested.

[BrenBarn (Brendan Barnwell)]

@kapinga

As a user, what I’d love to see in the long term is a well thought out, unified set of commands that apply to all the major platforms for user-provided python scripts (i.e., Windows, OSX, and Linux at minimum).

@kapinga

While I don’t expect this release to solve this issue in one go, it would be a sorely missed opportunity if this proposal is not thought through with a long-term, cross-platform goal in mind.

I basically agree with this and it’s at the core of my reaction to this proposal. I’d go a bit further, though, and say that a proposal like this which explicitly adds new behavior on one platform may actually make things worse from the perspective of moving toward that kind of unified set of tools. It can smooth the experience for Windows users who type python at the prompt, but it will likely increase the overall confusion about ways of installing and using Python.

@steve.dower

uv and conda are downstream of us (we are upstream), and they are non-authoritative for anyone’s system (unlike your OS provider). We don’t control them, and they aren’t responsible for coherence among all the software on their system.

Yes, but: a) by speaking with authority the Python core team (and/or PSF) can define what is authoritative; and b) the binaries that Python currently provides also don’t really provide coherence on existing systems, as evidenced by the constant confusion people encounter when using them (e.g., when multiple versions are installed).^[1]

@steve.dower

So the proposal is for upstream to stop making binary releases and let other people do them

It doesn’t even have to mean that we stop making binary releases. It just means we don’t have to present those releases as the only, best, or recommended way of installing Python. It means we can say “Most users will probably want to use uv” or “Many users may prefer conda” or just about anything but “Click here to download Python” without even mentioning that there are other ways to install and use Python and that many people use them specifically because they find they provide a smoother UX than the official installers.^[2] Python.org could still release binaries, but just frame them as a more “barebones” install path.

@steve.dower

and we wasah our hands and say “we hope these guys don’t charge you/collapse/mess up”.

I see where you’re coming from, but I’m looking at this from a user perspective. A lot of users already think that the existing Python.org installers are messed up! That’s why they’re using stuff like uv.

I think for a large swath of people, the theoretical risk of a redistributor collapsing is not really more worrisome than the very real risk of getting yourself into a tangle via use of the official installers. Even though the latter is much less damaging, it happens way more often.

Also, python.org already doesn’t provide binaries for Linux anyway.

@steve.dower

I’m not entirely against that - Linux has been working fine on a sources-only basis forever - but if you want to use compatibility as any sort of reason at all then this is a much more disruptive route.

I’m with you on the idea that breaking compatibility can be well worth it. I just think we should try for an overall more unified workflow across all platforms if we want to break compatibility.

@steve.dower

Additionally, uv is maintained by a for-profit company, and conda used to be, which apparently python.org are not allowed to endorse. We even had the link to our own Python app in the Windows Store taken down because it was apparently too strong an endorsement (which I opposed, but lost). These are legal arguments, to be had amongst the PSF’s officers/counsel, but the current precedent is “don’t recommend any businesses”, which makes it impossible for us to point at uv, even if it were technically responsible.

I’m not a lawyer but I would suggest that those decisions should be revisited. Certainly there are many nonprofit organizations that endorse, rate, or recommend products produced by for-profit companies (e.g., Consumer Reports, or various environmental organizations that give ratings for how eco-friendly your tuna is or whatever).

Aside from that, my understanding is that conda as an organization is now independent of the Anaconda company and is under the NumFocus umbrella, although perhaps someone with more direct knowledge could comment on that.

@steve.dower

Why isn’t “installing an app from the Store” a good enough activation command?

For all the same reasons that running everything in one global system-level environment isn’t good enough. Like I said, there could be some kind of auto-activated environment, but it’s an environment, not just a version of Python.

@steve.dower

So the questions on behaviour I’d like to see actual consensus on:

I’ll give my answers but (unsurprisingly) I think they diverge from the way some other people here are looking at things .

@steve.dower

• after installing PyManager, should python install and launch Python on first run? Or should it give an(other) error pointing to pymanager and/or activating a venv?
• should running pymanager install and launch Python on first run? Or should it give an(other) error telling users what all the commands are?

Typing python should never install Python. I’d be okay with pymanager installing some default Python version when it is installed. I don’t think running pymanager on its own should launch Python. It should either be pymanager run ..., or pymanager activate ... followed by python.

@steve.dower

• If we decide python points to the other command, then after installing a global runtime, should PyManager’s python launch that runtime? Or should it point to pymanager/venv still? (to encourage users to move to a single command unless they’ve explicitly activated)
• If we decide python launches global runtimes, should it support -V: and shebang processing for users to select? Or should we only allow those on pymanager?

This is where we diverge even more, because in my conception there is no such thing as global runtimes. I mean, people could still have them if they used a separate installer to install Python, but that would be discouraged because the point of a Pymanager would be to. . . manage your Pythons for you.

The key thing for me is that there should always be some kind of command or visual indication of which environment’s Python is being used. This could be typing an activation command, or doing pymanager run <env-name>, or maybe even some kind of autodetection based on being in a certain project directory. But “I opened the command prompt” is simply not enough information on its own to be able to run Python. My reasoning here is that the friction of this extra step of having to be aware of and specify an environment is minuscule compared to the pain that will eventually be caused by not being aware of environments.

@steve.dower

If we decide python can have version selection, should it also support auto-install? Or should we error in that case and require manually using pymanager?

python shouldn’t have version selection. I guess I sort of said this above, but I think version selection per se is not really what we want. What we want is environment selection, of which version is only a part. When I was using Windows as my main OS, I definitely appreciated the ability to select a version when the py launcher came out, but over time I ran into the same kinds of dependency problems that people run into on other platforms and which can only be solved by environments.

Now, I do think for some of the stuff above where I said “it shouldn’t do this at all”, there could be a message that says like “You need to choose which Python environment to use by typing pymanager run .... Type pymanager list to see which environments you have available”. So the user can be guided towards the right set of commands. But I think making the path towards the actual Python interpreter prompt too quick can actually wind up hurting users in the end, so it’s better to stick those extra steps in (maybe with some hand-holding) than try to skip over them.

I realize of course that most of what I’m saying is expanding the concept beyond what you’re trying to do. But like I said above, my perspective is basically that, although it can definitely be worth it to break workflow compatibility for the right new, better workflow, that “activation cost” can’t be met without incorporating the idea of environments.

---

1. On rereading, I guess maybe by “authoritative” you meant not in the sense of “this is Python official” but “this is installable in the way that is official on your platform”? If that’s the case though my second point still holds. ↩︎

2. Yes, some alternatives are mentioned on the rather hidden “alternative implementations” page, but my point is that at least some of those should be foregrounded rather than backgrounded. ↩︎