To those wanting only a pymanager (say) command, how would you envision first time use on a clean Windows 11 install?
One would type python and the store page for “Python Install Manager” (PIM) opens. Click install and close the window. Type python again and the store page opens again stating it is installed and can opened from the Start menu. Ok, close window, open the Start menu, and click where? If you are lucky, it is in the “Recommended” group, otherwise click “All >” and scroll until you reach the entry for PIM, or a more advanced user may know to just type python after opening the Start menu to search. Click PIM, it installs the default (current) version of Python and then launches the REPL.
OR
One would type python and the store page for “Python Install Manager” (PIM) opens. Click install and close the window. Type python again, it installs the default (current) version of Python and then launches the REPL.
Option 2 is what currently happens (substituting regular Python for the PIM, of course). This has been the default behavior for at least the last seven years!
First time use aside, I can see how the commands themselves could conflict with existing usage of python.exe install executing a script named install in the current directory (albeit very unlikely). To that end, prefixing the commands with -- (making them executable options, --install and the like) eliminates that problem as --opt is current invalid.
I like the Store Python for its simplicity. The multiple version launching is also identical to Linux, making me time in WSL painless, python3.13 launches Python 3.13 in both environments.
You type python, it opens the store page for the installation manager and installs the manager and launches, installing the latest release of Python as the current demo does, it also installs pymanager.exe as the test build currently does.
Typing python again launches the version of python controlled by the manager. If you wish to change this you need to use the pymanager command instead of python. You do not need to use pymanager to launch Python, only to install/uninstall versions or change which version the python command launches by default.
This way the management commands won’t clash with anything python chooses to add in the future or cause help/--help confusion. As pymanager is not included in a venv, the pymanager commands also won’t be shadowed by an active venv.
Your python3.13 aliases and such would be handled by the global shortcuts directory added by this new manager or accessed through the py command.
Unfortunately, launching the installation manager from the Store page is not possible, at least without a big change to how it is presented (GUI app). Notice how there is no “Launch” button on the Python app Store’s page (at least on latest Win11).
P.S. Therefore, a secondary step would still be required and a degraded user experience.
P.P.S. The MSIX install behaves differently from a Store install.
I’m not sure what the secondary step you’re talking about is?
I see there’s no launch button from the store, but maybe I wasn’t quite clear, the user experience would essentially be what you say for option 2. You enter python, install, enter python again and it installs and runs the latest release. Exiting and relaunching python would relaunch the release it had already downloaded.
The main difference here compared to the current Windows store is that there would be one “Python Manager” application for all releases of Python, rather than having an app for each release, simplifying the management of the Windows store application.
The main contention is that the install/list/uninstall/help subcommands shouldn’t belong to the main python.exe executable. Those should belong to a separate manager that is installed alongside (and used by) the main executable. Adding them to the main python executable will be more confusing as they won’t always be available (venv, other operating system, separate Python install that has shadowed the Windows store python) and there exist similar commands (list/--list, help/--help) that will now do different things on the same executable.
Python intallation manager 0.1a2
Copyright (c) 2001-2024 Python Software Foundation. All Rights Reserved.
Subcommands:
help Show help for Python installation manager commands
install Download new Python runtimes
list Shows all installed Python runtimes
uninstall Remove runtimes from your machine
Global options:
-v, --verbose Increased output (log_level=20)
-vv Further increased output (log_level=10)
-q, --quiet Less output (log_level=30)
-qq Even less output (log_level=40)
-y, --yes Always confirm prompts (confirm=False)
--config=PATH Override configuration with JSON file
python --help
usage: C:\Users\ducks\AppData\Local\Packages\PythonSoftwareFoundation.PythonManager_7qnp2ymwnadz2\LocalCache\pkgs\pythoncore-3.13\python.exe [option] ... [-c cmd | -m mod | file | -] [arg] ...
Options (and corresponding environment variables):
-b : issue warnings about converting bytes/bytearray to str and comparing
bytes/bytearray with str or bytes with int. (-bb: issue errors)
-B : don't write .pyc files on import; also PYTHONDONTWRITEBYTECODE=x
-c cmd : program passed in as string (terminates option list)
-d : turn on parser debugging output (for experts only, only works on
debug builds); also PYTHONDEBUG=x
-E : ignore PYTHON* environment variables (such as PYTHONPATH)
-h : print this help message and exit (also -? or --help)
-i : inspect interactively after running script; forces a prompt even
if stdin does not appear to be a terminal; also PYTHONINSPECT=x
-I : isolate Python from the user's environment (implies -E and -s)
-m mod : run library module as a script (terminates option list)
-O : remove assert and __debug__-dependent statements; add .opt-1 before
.pyc extension; also PYTHONOPTIMIZE=x
-OO : do -O changes and also discard docstrings; add .opt-2 before
.pyc extension
-P : don't prepend a potentially unsafe path to sys.path; also
PYTHONSAFEPATH
-q : don't print version and copyright messages on interactive startup
-s : don't add user site directory to sys.path; also PYTHONNOUSERSITE=x
-S : don't imply 'import site' on initialization
-u : force the stdout and stderr streams to be unbuffered;
this option has no effect on stdin; also PYTHONUNBUFFERED=x
-v : verbose (trace import statements); also PYTHONVERBOSE=x
can be supplied multiple times to increase verbosity
-V : print the Python version number and exit (also --version)
when given twice, print more information about the build
-W arg : warning control; arg is action:message:category:module:lineno
also PYTHONWARNINGS=arg
-x : skip first line of source, allowing use of non-Unix forms of #!cmd
-X opt : set implementation-specific option
--check-hash-based-pycs always|default|never:
control how Python invalidates hash-based .pyc files
--help-env: print help about Python environment variables and exit
--help-xoptions: print help about implementation-specific -X options and exit
--help-all: print complete help information and exit
Arguments:
file : program read from script file
- : program read from stdin (default; interactive mode if a tty)
arg ...: arguments passed to program in sys.argv[1:]
py list
Publisher Tag Name Version Alias
CPython 3.13.1 Python 3.13.1 3.13.1 python3.13.exe, python3.exe, pythonw3.13.exe, pythonw3.exe
py --list
-V:3.13 * Python 3.13.1
Incidentally a py liist typo currently gets an internal error triggered by the FileNotFoundError.
This one I think is a showstopper for the py alias. I’d rather the new installer didn’t provide a py alias than have it fail to follow PEP 486. At the very least, we would probably need a new PEP to withdraw PEP 486. (As the author of PEP 486, I’m not willing to withdraw it without a PEP…)
In all honesty, I’m strongly against the new installation manager creating an alias for py. People rely on the behaviour of py (which is defined in a series of PEPs) and we shouldn’t change that. It’ll be disruptive if we suddenly have users who don’t have py available, but IMO it will be more disruptive if py exists but doesn’t work as it currently does. And at least if py isn’t available, people can download the standalone version[1].
I’m a core Python user myself. I’ve never liked (Ana)conda, and have only used it occasionally, mostly for just long enough to remind myself of how much I dislike it… So I’m very biased here. Having said that, this proposal (by design) does nothing to help people who want an out-of-the-box “Python plus a GUI environment[2] plus a bunch of scientific packages”. If having 3rd party packages preinstalled is a requirement, you’re always going to need some form of “distribution”. That could be Anaconda, but it could be as simple as “Run python -m pip install -r our_package_bundle.txt”. The latter may be more than your target audience is willing to handle, though.
The new proposal does remove the “the store Python isn’t a normal Python install” issues, I believe. It doesn’t alter the fact that python script.py will run the MS-provided alias by default - that’s something Microsoft did (which I’m not particularly happy about myself), not something the core devs have any control over. It’s arguable that the new proposal gives a better experience for that alias, although personally I’d be happy with just expecting people to manually install Python from the store.
@steve.dower - this might be something to consider. If people have to disable the app alias to allow their own Python install to use the python command name, then there probably will be a significant number of users trying to install Python from the app store while having the alias disabled. Doing the install and then having python not launch the interpreter they’ve just installed is a bad user experience, that we should try to avoid.
Surely it’s Microsoft that is already in violation of that trademark as the preinstalled python alias isn’t the interpreter, but instead opens the Store?
If this could be made to work, I wouldn’t be against it. My instinct is, though, that it wouldn’t be as easy as you think - I’m sure Steve would have done that if it were straightforward. And having tested the new installer a little, I have to say that the extra subcommands available under the python command aren’t as jarring in practice as they seem right now.
That is presumably “just” a bug. Anything that isn’t a known command should be passed to the Python interpreter unaltered.
We might need to make that available - PEP 397 says that a standalone version will be available, but we don’t seem to have kept to that promise ↩︎
It doesn’t - it launches Python The nuance appears if you haven’t actually installed Python yet, in which case it’ll download and install and then run it, but the alternative to this is “Command not found”, which is incredibly unhelpful.
Are you okay with python doing an install on demand? If so, and if you’re also okay with printing additional output to help the user learn about the separate command, then it might be workable. If you’re not even okay with python doing anything other than error when Python isn’t installed, then we’re just going to have to butt heads, because I really don’t want to inflict that on our users.
FWIW, there’s actually no way to make the global “python” command “really” Python. The Store package already isn’t “really” Python (though it’s an executable built from the CPython repo, so I think most people would consider it “really” Python). What you’re asking for is a not-really Python executable that gives you errors instead of functionality.
You also need to tell me whether python -V:<version> is okay, and whether python <script.py> should check for shebangs or inline metadata. Those are part of the manager, not “really” Python, but any proposal to make it only be “really” Python is going to take those away. (Also justify that removal in terms of making it a better user experience.)
I would argue this is very close, and since it’s coming through this discussion forum (and hopefully will be “owned” by the core team), it shouldn’t run afoul of trademark laws.
Yeah, it really does leave a bit to be desired. We don’t control any of that UI, unfortunately. Having the “Launch” button do what it currently does is about the best we can do - or we could make it open to a doc page (which would also mean the main app is one that just opens the doc page, which may have other implications), but I think that’s more harmful to users with some idea than it is helpful to users with little/no idea.
Yeah, I need to align those names at some point It should only show up in the Start menu once, and I expect most people will look there before they go searching Program Files directly.
Most useful feedback so far! I do like the idea of pymanager to show help by default, though I also like that when you click “Launch” after installing that it runs Python by default, so I’ll have to figure out a way to make both work.
I’m hesitant to add extra output to any normal Python invocation - I heard bug reports due to extra output in Microsoft’s built-in redirector due to extra output, even when no arguments were being passed (people were piping scripts to stdin, which looks like no arguments).
This was a complex and obscure bug in other people’s installers and in Windows. The main culprit (.NET Core) has fixed their installer, and I’m not aware of anyone else causing this issue.
On a bug-free machine, the preinstalled Python alias is lowest priority, and definitely lower than the python.org installer’s usual PATH modification.
Not at all. It may even help - Anaconda could theoretically provide their own distribution index and python install --source https://anaconda.example/index.json ... would grab it and register it. But that’s up to them, not us.
If Anaconda register their install using PEP 514, and it’s the only install, PyManager will run it when you just type python. This should be the same behaviour as the current py launcher, in case you ever tested that (I’m not sure which Anaconda installers use PEP 514 though).
No, it provides a better alternative to the Store Pythons for apps that want to share secret files with other apps. Typically Windows apps are not meant to do this - they should be putting shared files in user-visible locations such as Documents rather than TEMP - and Store apps manage state based on this assumption. But the Pythons installed by PyManager aren’t running as Store apps, so they will have full freedom to scatter files all over the system (user permissions permitting).
I considered it, and if consensus forms around that then I won’t be too upset. But I’ve heard some pretty strong arguments from people that you shouldn’t use options to do things that subcommands are meant for - what happens if you specify --list --install --uninstall? (I think they’re being a bit precious about Unix command line norms, but I do prefer the feel of subcommands here.)
The challenge here is the “How to teach this” bit.
Thanks. Probably trying/failing to look for a shebang.
Well, this whole discussion is becoming a PEP, so don’t rule that out But I’d forgotten we’d actually specified that whole behaviour. I’ll take a look at it and see whether it flows nicely.
My hesitation is that I don’t like environment variables that the user didn’t set messing with behaviour like this. Though that’s kind of the whole point of a venv.
Does it make any difference if the py alias can be turned off? Or if it’s shadowed by any existing py launcher install (this is already true)?
As I suggested above, you could put a whole distro into your own feed, provided it can be installed by extracting a ZIP file and doing the other supported registrations (currently PEP 514 and Start menu items).
“By default” meaning “instead of giving a vague error”. If you’ve installed Python through any supported path, it should replace Microsoft’s alias. If you’re still getting that message, you’ve either got a misconfigured machine, or you haven’t installed Python.
Note: I think that parsing inline metadata is going to be a bad user experience. The critical part of script metadata is the dependency declarations, and if python <script.py> doesn’t respect those (and it won’t, unless we want to make the python command into an environment manager), then this will confuse and frustrate people.
Not really, IMO. My main concern is very similar to @malemburg’s, in a way. We have provided the py command on WIndows for many years now, and in fact we’ve documented its behaviour in PEPs, as well as in the core documentation. People have come to rely on that behaviour, and are likely doing so in ways we may not even be aware of. Changing the behaviour will be a compatibility break, and given that we’ve promoted use of py over use of python on Windows, quite likely a significant one.
It’s probably going to be bad enough if we see an increase in users who don’t have py installed at all. But if we have people with a py command that doesn’t work like tools expect it to, we’ll be dumping a whole lot of breakage on people who are least able to handle it (the users of those tools, rather than the maintainers).
Some examples of things that might use py. I don’t know for certain as it’s a long time since I checked, but they are certainly plausible.
Using py -X.Y to find the interpreter given a version number. Or using py -0p or py --list-paths to list what’s installed. You can use the registry and PEP 514, but who’s going to dig around in the registry when the launcher will do it for you?
Scripts and documentation use py to mean “run with the current interpreter”. Pip recommends py -m pip install ... to install packages on Windows.
To be honest, I think this is a really nice feature of the new launcher, and one that’s not been advertised nearly enough. I can see it being marginalised due to lack of awareness, or ending up as some sort of “corporate distribution” feature simply because no-one understands its applicability.
Maybe we should have some tools available to take a standard Python install, add some preinstalled packages, and bundle the result as a new feed? It would be great, for example, to have a “Jupyter-enabled Python” feed available, that the Jupyter project could publish and maintain with minimal effort. Or a “Scientific Python” feed. While that might not cover all the features @oscarbenjamin is looking for, maybe it would be a good middle ground.
That was a misunderstanding on my part based on what @oscarbenjamin said. I’m happy to know it’s now no longer the case. Although I still think the behaviour on a clean machine, where python script.py (I believe) pops up the store rather than giving a proper error, is bad. But that’s Microsoft’s fault, not yours.
An analogue of this is that on a fresh MacOS install you can open a terminal and type git and then a GUI pops up and offers to install the MacOS Command Line Tools. After install the tools are available and git runs Git as you would expect.
The real command to install command line tools is xcode-select --install but before git is installed there is a shim that opens the installer GUI instead. After install you can also configure which version you want and other things using the xcode-select command.
The fact that xcode-select is the real command for managing the installations but git initially pops up the GUI does not seem to cause any significant confusion. It was quite helpful for me the first time I tried to set up a MacOS machine. These are my nice easy instructions for students to install git on MacOS: “try running git and install it if needed”. They don’t start out needing multiple versions of anything so they don’t need to know about xcode-select yet.
I would consider it like this:
Users should install pymanager from the store and then use it to install and manage Pythons.
If they don’t have pymanager or any Pythons then running python in a terminal pops up the pymanager entry in the store.
The thing that is different as compared to xcode-select is that you don’t have pymanager until after it is installed from the store so probably “run python” would end up being the standard instruction for how to pop up the pymanager installer but as @pf_moore points out that is not great because the shim python.exe might get shadowed.
Perhaps the command from the launch button could check if the aliases exist? I assume if you’ve got it from trying python in a command line that the command must exist, not so sure what you can do about someone searching for and installing directly through the store interface if they’ve disabled the alias though.
Maybe, not me though .
Huh, didn’t know that would look like no arguments. That does make things a little trickier. Ideally you’d only see such a message if the REPL is also launching. Although the user is going to be getting extra messages if the manager is installing Python for the first time anyway so I’m not sure it’s completely avoidable.
I think it’s easier after you get past the ‘pymanager is the command/tool you want’ stage. There’s less to trip you up, such as similar but different commands and subcommands working/not working depending on environment. But you do have to get past that first stage.
However they only need that extra detail if/when they need to change which release of Python is installed, so it’s not necessarily something they’re going to need immediately? Still, it would be nice if there was a way to provide a related message on launch.
If you pass any arguments, it just prints an error and fails with the same exit code as a command not found would in cmd.exe. I think it was also updated to do the same if the standard streams are not attached to a console, so it should really only be the python case that opens the store. Any scripted/automated use will get roughly similar behaviour as with a missing command (with the exception of using where or searching PATH).
If python is replaced by something else then that something else ought to be Python, and so you don’t need the manager.
The XCode case is more akin to typing python and having the Visual Studio installer pop up, which I’m sure would be popular enough with some Microsoft leaderspeople, but really not where I’d like to see things go. As long as we can make the python redirector point to a PSF-owned resource, I’d like to keep it there.
But yes, not having pymanager until you’ve typed python to find and install it is indeed the usability issue I worry about.
Sure, and at that point you can stop using python for anything other than regular Python commands. Nobody is forcing you to type python install when you could type pymanager install, and nothing about the behaviour of python <anything else> is going to interfere any more than having a regular Python install on PATH.
Fine by me. I hadn’t implemented it yet because it seemed harder than it looked
Honestly, I would expect the majority of users would open the Store from the start menu, and look for “Python” and install it from that. The python.exe shim that launches the store is cute, and it helps publicise Python on Windows, but I don’t see it as the main way of getting the store Python installed on Windows.
Maybe I’m wrong, though. Microsoft have huge resources for UX research, and I doubt they added that shim without a good idea of its benefits.
Of course, if you’re OK with doing something that’s “harder than it looked” then by all means implement dependency handling as well - I’d love to have that out of the box
Heh, don’t give too much credit. It’s run by humans, just like anywhere else.
That said, because it directs to a PSF store page, we have the numbers on how many people search vs. use the stub. In short, 75% of installs come from the stub.
To be clear, it’s not about forcing me to type python install it’s about preventing people from typing python install and wondering why they get an error message because they have a venv activated. If the recommendation is going to end up being to use pymanager install every time because (assuming it’s installed) that alias won’t end up being shadowed[1] then why have the python install command in the first place?
Unless someone else releases their own pymanager.exe ↩︎
There is an important difference: it doesn’t install XCode but rather the Command Line Tools. XCode is a large many GB install with many things that someone learning Python doesn’t need much like Visual Studio. The Command Line Tools is a much smaller install that provides at least the “basic” things that a developer on Unix expects to be there so it is somewhat closer to e.g. apt-get build-essential in Debian-ish systems.
This is not completely off topic: part of the reason that I prefer to suggest typing git in the terminal is because it definitely pops up the installer for the current version of the Command Line Tools whereas any mention of “XCode” results in many students going off and downloading the wrong thing. It is hard to beat that for its “simple instruction that doesn’t get outdated leads to many people installing exactly the right thing effect”.
Hm, maybe. I guess expectations of the start menu change over time. I still expect it to list the installed programs and run them rather than giving me “web search results for Teams”.
And then python install 3.15 doesn’t work. The situation with instructions for how to install Python and how to run Python/pip etc is already so confused. Now we will have different python commands that may or may not be installation managers depending on some history of what other things were installed. There will be a never ending string of threads in the help category with the title “python install doesn’t work” and then people trying to help but not being sure if python has been shadowed.
If the command to run is pymanager install 3.15 then it is clear whether or not pymanager is not installed (command not found). The solution is also clear: download pymanager from the store.
IMO it doesn’t matter how we handle this, there’s going to be a lot of complaints from people whose muscle memory will need retraining. I think the PEP is going to need to do a lot of work in the “How to teach this” section. Just saying “it’s a mess at the moment, this is less of a mess” won’t cut it - for any individual, it isn’t a mess, we have all come up with our own conventions and workflows, and breaking that will be disruptive. For new users, it will be fine either way - they have no preconcieved notions, so they will learn whatever we give them (although they may find some things clumsy, and be annoyed at us old folk saying “yeah, but it was far worse in my day” )
For me, I use py almost exclusively to run the interpreter, and I expect it to work whether I’m in a venv or not - to the extent that I don’t really track whether I’m in a venv. With the new installer, assuming I still have access to py, I can imagine viewing python very much as the “manager” command. As such, I’ll definitely be surprised and frustrated that python install 3.13 fails to work when I have a venv activated.
I could train myself to use python instead of py and pymanager for the manager features. But for that to work, I’d need python to have features like python -3.10, shebang support, and “list versions”[1]. So python is still the manager in some senses, and I’m bound to get caught out sometimes by python install not working in a venv.
There really is no perfect answer here. And because of that, I think it’s probably the right call to (at some point) accept that there will be annoying edge cases, and concentrate on how we handle the transition. My big fear is that we just implement this, and expect to “take the hit” on the UX side. Not least because we don’t actually have much in the way of resources to handle user education and publicity
Let me ask a very specific question. In the pip documentation, we made a deliberate decision to make py -m pip ... the canonical form for running pip on Windows. The documentation recommends that form exclusively. When pip gives error messages (for example, the “you should update pip” message), it uses the Python interpreter path instead of py[2]. If this new launcher gets accepted, what should pip do to ensure that users don’t get confused over the right way to run pip? Remember that we need to support Python versions back to 3.8 (regardless of whether they were installed by the new launcher or the old installer) and we know that recommending python -m pip didn’t work for older Python versions.
This will be a genuine cost of the new launcher, and I think it’s important that the PEP addresses it properly, and doesn’t handwave it away. (It will hit a lot of other projects similarly, not just pip).
Honestly, at this point I think the only reason we don’t have our own xkcd strip is because Randall doesn’t use Windows…
I’d take ages to stop typing python -0p, though ↩︎
And we get complaints that we don’t quote that value properly, even though there’s no shell-agnostic way of doing so… ↩︎
The nuance appears if you haven’t actually installed Python yet, in which case it’ll download and install and then run it, but the alternative to this is “Command not found”, which is incredibly unhelpful.
It should only show up in the Start menu once, and I expect most people will look there before they go searching Program Files directly.
.
