PEP 773: A Python Installation Manager for Windows

PEPs
41

I’ll let Steve confirm but I don’t believe so.

  1. MSIX installs can’t alter PATH at all - the “App Aliases” are all that they can add, and those are checked after PATH.
  2. The manager can’t even add something to PATH when it’s first run, because it’s a user program, and so it can only add to the user section of PATH, which is checked after the system path of PATH.

It’s all something that is easy enough to fix manually (undo any changes you made to PATH, maybe uninstall py if you have an “all users” install of the launcher), but there are way too many variables to do it automatically, without causing more breakage than you’re fixing…

1 Like
42

I like this mental model and naming very much:

  • python = runtime
  • pythonup = installer
  • py = “hybrid” and shortcut for both python and pythonup

Whether py is an alias of pythonup or whatever is an implementation detail and doesn’t matter to me.

I would like if py (once installed) is the canonical entry point for all use cases:

  • running a script in a venv or not
  • discover installed versions
  • running the installer
  • path manipulation (possible with running under admin rights?)
  • …

Nevertheless the documentation should inform that py is the shortcut for python and pythonup so that the user can recognize the established pattern analogue to the other languages.
And ideally a future Linux version of py can catch up and harmonize the platforms.

2 Likes
43

Paul’s correct here. Users should manage their PATH manually - it’s not well set up for programs to do it automatically.

The only technical correction to Paul’s comment is that the app aliases are already on PATH, just through a directory that’s managed by the OS and not something we have to get right ourselves.

I don’t like pythonup as a name, but the mental model is fine (and matches the current design), so I’ll see what I can do to make it clearer (short of never using py in examples).

44

I also (strongly) dislike the pythonup name. I’m perfectly fine with pymanage, and honestly I see no need to bikeshed it[1].

  1. not that I expect that to stop people doing so :slightly_smiling_face: ↩︎

4 Likes
45

python isn’t quite cross-platform, as according to PEP 394 (which could use an amendment IMO):

Distributors may choose to set the behavior of the python command as follows:

  • python2,
  • python3,
  • not provide python command, allow python to be configurable by an end user or a system administrator.

The py command enables easy and consistent access to different Python versions on Windows, as there are no pythonX.Y.exe executables on Windows, and using it for file associations makes them less prone to change when installing/upgrading Python, according to the PEP. Although I guess this is less relevant now than when side-by-side 2+3 installs were a thing.

1 Like
46

There will be with this proposal.

2 Likes
47

The “up” suffix means nothing to me. Is it short for “updater”?

At least pymanage has the advantage that it says what it does.

4 Likes
48

The “up” suffix means nothing to me. Is it short for “updater”?

I think it’s supposed to mean “I’m more accustomed to/would rather be using Rust, so please make Python’s tools look more like the Rust tools I’m familiar with/prefer.”

This too shall pass.

2 Likes
49

Maybe I have missed something amongst everything, but – why?

1 Like
50

I think this is a potentially exciting idea, so here is a condensed version of my notes from testing.

Testing on Windows 10, latest updates applied.

Before

C:.\temp_py> py -0p
Installed Pythons found by .\py.exe Launcher for Windows
 -3.13-64       C:Users\John\AppData\Local\Programs\Python\Python313\python.exe
 -3.12-64       C:\Programs\python312\python.exe
 -3.10-64       C:\Programs\python310\python.exe
 -2.7-64        C:\Programs\Python2\python.exe *

Downloaded python-manager-0.1a4.msix.

Doubled click on python-manager-0.1a4.msix, which I believed was the installer for the python manager. Instead it opened a command window and installed a new version of python 3.13

I think I should have been told
It was going to install python.
What version of python would be installed.
Where it would be put.
What the configuration was, as in the official download.
It should have detected 3.13.1 was already installed.

There should be a Cancel button which leaves the system unchanged.

On the plus side it was very quick.

Once installed then from a clean cmd prompt

C:\Windows\System32>py list
Managed By  Tag     Name                  Version  Alias          ←[0m
←[32mPythonCore  3.13.1  Python 3.13.1         3.13.1   python.exe, ...←[0m

←[36m* These runtimes may be run, but cannot be updated or uninstalled. *←[0m
PythonCore  3.13    Python 3.13 (64-bit)  3.13.1                  ←[0m
PythonCore  3.12    Python 3.12 (64-bit)  3.12.8                  ←[0m
PythonCore  3.10    Python 3.10 (64-bit)  3.10.4                  ←[0m
PythonCore  2.7     Python 2.7            2.7                     ←[0m

I was unable to get rid of ‘←[36m‘ etc.

2.7 and the original version of 3.13 could not be run despite the message ‘These runtimes may be run, but cannot …’.

Ignoring the fact that I already have and use multiple versions of python. How do I download 3.13 to c:\programs\python313?

Assuming traditional downloaders don’t exist how do I get a new version of 3.12 if I don’t want to download 3.13 as it would break something?

I found it very difficult to remove the manager from my system and it has left lots of stuff in the registry.

Using ‘app execution aliases’ to get round adding things to path was part of the reason. Running the manager seemed to add more app execution aliases every time.

In addition the new commands could only be accessed from cmd and powershell not from TCC, git bash and possibly other shell programs. That other shells may not recognise app execution aliases should be stated clearly upfront.

I am specificity interested in downloading to a target directory. I was not able to get this to work. Here is the log file generated when I tried to run the example given.


>> Writing logs to C:\Users\John\AppData\Local\Temp\python_install_20250125134151_12568.log
>> Python installation manager 0.1a4
Copyright (c) Python Software Foundation. All Rights Reserved.

BEGIN install_command.execute: ['3.13-arm64']
>> Searching for Python matching 3.13-arm64
Fetching: file:///C:/Program%20Files/WindowsApps/PythonSoftwareFoundation.PythonManager_0.1.0.164_x64__7qnp2ymwnadz2/bundled/index.json
Downloaded: <Index('file:///C:/Program%20Files/WindowsApps/PythonSoftwareFoundation.PythonManager_0.1.0.164_x64__7qnp2ymwnadz2/bundled/index.json', next='index-legacy.json', versions=[...174 entries])>
!! An error occurred. Please check any output above, or the log file, and try again.
>  Log file for this session: C:\Users\John\AppData\Local\Temp\python_install_20250125134151_12568.log
>  If you cannot resolve it yourself, please report the error with your log file at https://discuss.python.org/t/77900/ (during testing).
TRACEBACK:
Traceback (most recent call last):
  File "manage\install_command.py", line 528, in execute
  File "manage\install_command.py", line 405, in _install_one
TypeError: 'CompanyTag' object is not subscriptable

END install_command.execute
SILENCED ERROR
Traceback (most recent call last):
  File "manage\install_command.py", line 528, in execute
  File "manage\install_command.py", line 405, in _install_one
TypeError: 'CompanyTag' object is not subscriptable

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "manage\__init__.py", line 47, in main
  File "manage\commands.py", line 597, in execute
  File "manage\install_command.py", line 531, in execute
  File "manage\install_command.py", line 487, in _fatal_install_error
manage.exceptions.SilentError: 1
2 Likes
51

Notice that py is not a Windows only command. On Linux by default there’s only python3, but there are optional packages to add py. From my Lubuntu machine:

~/sources$ py
Command 'py' not found, but can be installed with:
sudo apt install pythonpy
52

Hmm, this got longer than I expected…

The MSIX is a no-go on Wine. It doesn’t know what to do with it. It’s builtin implementation of msiexec complains about something to do with pfx import/export. The usual workaround of 7z x extracting the MSIX and running the contents leads to an unhandled page fault. I guess that’s fine as long as the legacy installers stick around.

On normal Windows (well a VM and it’s on ARM64 but closer to the real thing than Wine), I got a bit lost over the pymanager command not existing but it was on me. I really hate the existing would you like to install python from the windows store python.exe shim so I normally remove C:\Users\bagpuss\AppData\Local\Microsoft\WindowsApps from PATH – guess I shot my own foot there!

I really like that switching architectures is treated as a first class feature. This is really important for ARM64 since you’re often forced to switch to AMD64 due to one of your dependencies not supporting anything else. What did surprise me though is that AMD64 is installed by default – I have to explicitly specify the native binary (pymanager install 3.12-arm64). That one sounds like a mistake to me?

Minor UX quibble but I’m not massively fond of the install [no argument] behaviour of doing an install.

$ PyManager.exe install
Python 3.13.1 is already installed.
Installed Python 3.13.1 as python.exe, python3.13.exe, python3.exe, pythonw3.13.exe, pythonw3.exe
This version will be launched by default when you run 'python'.

I’d rather it was equivalent to install -h and I had to type install latest or install --latest for it to actually do something. (There’s also the fairly minor defect of it telling me it’s installed 3.13.1 when it was a no-op!)

What’s the intention with multiple versions of the same minor version? Last I checked, the traditional installers would only let you have one of each minor version. It looks like that rule is staying?

C:\Users\bagpuss>pymanager install 3.11.4
Python 3.11.9 is already installed.

I’m seeing references to a pywmanager.exe (in the MSIX and under the Application Aliases menu) but I can’t see any mention of it in the PEP. What’s that one for? Is this a secret GUI equivalent of PyManager? :slightly_smiling_face:

One thing I can’t figure out is how to tell my firewall (in my case simplewall) which program to allow given that pymanager.exe isn’t a real executable – just some weird 0 byte alias thing. Currently, I’m just having to turn the whole firewall off to let pymanager run. Hopefully fixable with a sentence in the documentation saying what program does the actual download?

The no internet (or blocked by firewall) handling is a bit grim – a redirect to a log file followed by a seg fault. Could I request that we always write all errors to stderr? Being told to look at a log file that was put on a throwaway filesystem of some CI/CD machine and now no longer exists is a pet hate of mine.

I’d also echo what’s already been said about pymanager being the only command that should be allowed to do installation management. An application being its own package manager messes with my head. This one feels like it’s being forced fit into py’s CLI interface (even the --help looks discontinuous with it). And I also don’t welcome the next 5 or so years of people with the old py installed getting errors like:

C:\Users\bagpuss> py install
C:\Users\bagpuss\AppData\Local\Python\pythoncore-3.13\python.exe: can't open file 'C:\\Users\\bagpuss\\install': [Errno 2] No such file or directory

Overall though, I’m liking this. It’s not quite pyenv but it’s close! The finding of $python/Scripts/$commands issue is unfortunately still there but I can’t think of a way of ever solving that one (best I can think of is another alias (e.g. python-exec-3.12 pytest)).

One thing I would like (which I think others might not) is to be able to set the current default Python version to something other than the latest. That would solve both the problem of py switching to Python pre-releases if available and the scenario where a user upgrades their Python to a shiny new 3.x.0 version only to discover that their dependencies haven’t caught up yet. Maybe better discussed/addressed separately though?

4 Likes
53

Fantastic notes, thank you! I have a few clarifying questions and comments, but for the most part I accept the feedback.

I don’t understand what you mean by “configuration” here?

Yes, it should have. I’m not sure what happened there, as your later output clearly showed that it detected your other installs. Perhaps it’s related to not being able to launch the existing installs?

What terminal were you using? These are standard ANSI colour codes, so if your terminal doesn’t support them, I guess I need to detect it better. (Setting PYTHON_COLORS=0 will turn them off, just like for the runtime.)

Do you have any more info on why this happened? If you set PYMANAGER_DEBUG before py -V:2.7 it should display a lot more output about what’s happening.

The idea is that you shouldn’t care about the final location, but there are two options.

To move all your installs to a different directory, create a %AppData%\Python\PyManager.json configuration file containing {"install_dir": "C:\\programs"}.

Or to do one (unregistered) install use py install --target C:\programs\python313.[1]

py install 3.12. Having 3.13 present shouldn’t break anything, but if you don’t want it then py uninstall 3.13 and it shouldn’t come back if you’ve got another install there.

It left all your installs behind, but nothing else. So you can remove the manager and keep using Python. py uninstall --purge will clean up everything (unfortunately, we can’t run this automatically on uninstall, but I’m not sure that’s a good idea anyway).

It shouldn’t have. The manager adds its aliases to its own directory, and they are real executables. They should also work fine from Git Bash and other shells based on Cygwin (which apparently still has the bug where they refuse to run app execution aliases…).

  1. Which appears to have a bug right now, as you pointed out. Thanks for the log! ↩︎

54

Yes, I work with the maintainer of that project. Do note that it’s not an official CPython project, while the Windows one is, and so from an upstream POV we are perfectly entitled to dictate how the Windows one works while being able to say almost nothing to the independent ones.

1 Like
55

Not a problem, I appreciate the info!

I’m sure they’ll catch up one day, but until then there will be an MSI version as well. It’ll be pretty bare bones compared to what people may be used it (it’s meant for administrative scenarios), but Wine should be fine with it. (We also don’t officially support Wine at all, so while making stuff work there is okay, it won’t factor into any UX decisions.)

I’ve got a note somewhere to add platform detection, but haven’t implemented it yet :wink: As it stands, the ARM64 builds are still non-default anyway, and we officially recommend the AMD64 builds for regular users (i.e. people who are just trying to use libraries, not to make sure they work on ARM64).

I also thought about supporting latest as a meta-tag for this, so that’s two votes (a bonus with a metatag is that py install 3.10 3.12 latest would work and install all three versions).

Same as today, you get the latest. Using --target will get you the precise version though, and you can --force to downgrade if you need to.

I think there’s one mention of “and also the windowed variants”. But yes, it’s the version where pywmanager exec -V:3.13 will launch the pythonw.exe from 3.13 and avoid creating a console window. It also seems helpful if you’re trying to run it silently, but I’m not 100% convinced on that yet. It’s free to have it (since we need pythonw.exe and pyw.exe), so it seems harmless to keep it.

Yeah, that’s bad. Sounds like a bug - do you have that log file? You can pass -v or -vv when installing, or set PYMANAGER_VERBOSE or PYMANAGER_DEBUG to get extra output, but the switch here is really between “point to log file” vs “fill the screen with traceback” and there’s not yet a lot of middle ground. As we figure out which errors people run into, we can handle those with more helpful messages, but given the vast majority of users will (a) not hit fatal errors and (b) won’t know what to do anyway, pointing at a log file (as we do today) seems a fine next step.

If you set PYTHON_MANAGER_DEFAULT to the tag you want (e.g. the bit that gets passed to py -V:<tag>), or create %AppData%\Python\PyManager.json containing {"default_tag": "TAG"} then you should get this.

Thanks for the detailed feedback!

56

By my reading of the feedback, these seem to be the current ideas to consider for the PEP (as opposed to being bugs in the implementation):

  • prompt before installing on first use?
  • make py install show help instead of installing default version
  • add unsupported platform/terminal support as a reason for users to use the MSI
  • clarify uninstall behaviour (intentionally leaves Python installs)
  • consider pymanage instead of pymanager (since a few people have typo’d it)
2 Likes
57

I wonder - would it be worth adding a pymanger command to register an install after the fact[1]? And maybe also an explicit py install --target C:\programs\python313 --register? IMO it’s right that --target shouldn’t register by default, but being able to deliberately register an installation could be useful in some cases (and it’s tricky enough to do manually that having a command for it would be good).

  1. And if I’m already introducing scope creep, why not unregister as well? :slightly_smiling_face: ↩︎

58

There’s also some good feedback for the docs. There are a number of examples where people couldn’t do something, and you pointed out the solution (a number of which weren’t obvious - I’ve been following the discussion from the start, and some points made me go “hey, that’s cool, I hadn’t realised that was possible” :slightly_smiling_face:). A FAQ or “Recipes” section in the docs based on what people are asking here might be a useful addition.

2 Likes
59

The install instructions are encoded in the feed, not in the runtime, and the registration process writes them into the install. So an unregistered install doesn’t have the things that registration would add, such as what registry keys to write and shortcuts to generate.

Putting this information into a normal install would require standardising it so that non-core distributions could play, and I’m not real keen to do that. The closest I’ll go is to merge info from the package into the info from the feed, as this will dramatically help reduce the size of the feed.

But inspecting random files on disk and generating Start menu shortcuts and registry keys that claim to authoritatively represent those random files is definitely out of scope :wink:

60

Alternative is to add an additional alias for pymanger that prints "Blessed are the cheesemakers"

6 Likes