Ah yeah, there are going to be a set of entries initially. It won’t grow over time, though.
Yeah, I acknowledged it in a footnote. It’s still the intended way to install once-off to a specific directory. I’ve fixed it now - next test release should be fine.
I’m afraid there’s no way to resolve the balance between “sensible default location” and “personalised default location” without making the second case require configuration. The default case for those who just want to run Python is to not worry about where it’s installed - everyone else needs to configure their location. --target is the way if you don’t want Start menu items or updates/etc.
Saw this on re-reading. The other reason to not do this is because I don’t actually have (or want) a central register of what’s installed - the file system should speak for itself. That way, there’s nothing to get out of sync.
So there’s a single install directory and searching it looks for *\__install__.json in that directory to collect the installs. Moving any out of that directory means we can’t do that search, and now we have multiple things to keep in sync, and need to add more detect/repair functionality for some database somewhere. So basically, the complexity goes way up just for this feature.
A viable middle ground is to add the PEP 514 and Start menu registrations, but leave it all unmanaged. So you can’t remove or update it, and the only way to clean that stuff up is manually. I don’t think that’s a good place to leave users, plus it complicates the install model even further.
I might be convinced some changes here are worth it if people can justify the need for multiple simultaneous user-managed installs into different locations, but I genuinely can’t come up with a reason for that other than personal preference (which doesn’t win over reliability for those without a preference here, sorry).
Ah. I was only thinking of “registration” in terms of PEP 514 (I wasn’t even thinking of the start menu, as I never use that for Python).
I have no need for it myself, so don’t change anything on my behalf. I’d only see the need if we intended to promote pymanager install --target XXX as a way to install your “system Python” to a location of your choice. Like you, though, I think the %AppData%\Python\PyManager.json approach should be enough.
What are people’s thoughts on updating the current launcher to detect these subcommands and provide a more useful error message? Probably directing people to the doc page, which will eventually contain instructions on migrating from the old py.exe to the new.
It would be most useful if you did it 10 years ago. Otherwise though we would not have this problem if the recommended command was pymanager install rather than py install.
Throughout this thread and the previous ones the proposals have included changes to python.exe or py.exe that are not necessary for the pymanager.exe part. I would be totally on board with these changes if they were being proposed as part of a cross-platform reunification of the commands but if this just making yet another kind of platform inconsistency then I think it is better to leave py.exe as it is. Conceptually it is much cleaner that pymanager manages the Pythons and py just runs them. If the installers can sort out PATH properly then the advantage of being able to type py install 3.12 rather than pymanager install 3.12 seems miniscule to me whereas the confusion about what py does and the fact that people have old and new versions does not.
Okay, now travel 10 more years in the future and ask whether you’d be wishing for three commands instead of two
Churn costs something, for sure, but I believe the cost in this case is worth the lack of regrets in the future (which is also why I initially wanted all the commands on python itself, though the cost of that churn is clearly way too high).
That’s why I want a cross-platform reunification of the commands. Your two commands instead of three does not simplify anything if it is only on Windows. I want to be somewhere in 10 years time where I can walk into a room with 100 students and say “this is how you install Python plus packages and then open a Jupyter notebook. Go do that and then run this code”.
I was in this exact situation on Friday last week and I honestly just skipped over the installation part because it is easier to let students try and then come around and help anyone who is having problems than it is to give actual instructions that people can use.
I’m absolutely in favour of cross-platform consistency. However, on Unix (and in particular Linux) that involves negotiating with multiple distribution maintainers, all of whom appear (to me) to be pretty strongly against packaging any sort of “installation manager” commands. (It’s been enough of a problem to get pip available in Linux distros in a way that everyone’s happy with, I don’t imagine getting a Python manager will be easier…)
I’d much rather we made progress where we can (on Windows) than block every platform on getting agreement from Linux distros.
Outside of the core, there’s always Brett’s py for Unix, which could gain the same capabilities as PEP 773’s py command. You still have the problem of getting py on your Linux box, but that’s no different from getting anything that’s not part of the distribution on a Linux box…
I’m not trying to block anything. My point is that I don’t think that py install 3.12 is any better than pymanager install 3.12 and the latter avoids any confusion about what the commands do since current installs of py.exe don’t have these commands and also the CLI is unambiguous.
The cross-platform point is that to me it would seem worthwhile to go through a process of churn and confusion if the end result is a big improvement in the general story for installing Python. If we are not going to do that cross-platform improvement then I would like to avoid making any decisions that would limit future improvements.
Putting this another way: does the proposed interface for py.exe seem like something that we might end up having on other platforms? If the answer is no then this is explicitly designing an interface that will never be cross-platform.
I think it does a similar thing that tools like uv, Hatch, and PDM do that manage your Python installation as well as run an interpreter (although in the Python Launcher’s case it’s explicitly an interpreter while uv runs Python code while hiding the interpreter a bit more).
That’s actually Debian -- Details of package pythonpy in sid which isn’t quite the same thing. What you’re after is the Python Launcher for Unix (which is normally at python-launcher.app, but CloudFlare isn’t happy ATM, so you can get to the docs at https://python-launcher.pages.dev until I switch to a different web host).
This was essentially my point too, which is what prompted my (short and easily overlooked) question:
I just haven’t understood what the actual advantages of using py are, and even less what is gained by preferentially recommending py in the official documentation. I’m happy to be convinced, it’s just non-obvious to me.
Well, “all of whom” is definitely not true as on Fedora I can install rustup from the repos and on openSUSE I have both rustup and juliaup available to me.
IMHO python currenlty has at least 5 different modes
pip installs modules
venv manages virtual environments
python is the interpreter
py runs versions of python
management is a human who has to browse and click.
The first four work well but the latter is a mess and pymanager goes a long way to fixing it.
but this is a command line environment, and making things multi-function just makes the commands more complex. I dont think pymanager needs to do everything everybody wants to do. It should do what MOST people want well.
I would vote strongly against mergeing everything into one. Personally I would be quiet happy never to use registered versions.
I discovered a feature last night. I removed the managed 3.13 and typed py. Instead of running one of my legacy instalations as I wanted it reinstalled 3.13 again.
As for python on linux, the apparnt dificulty of having multiple private versions makes me wamt to keep running Windows 10 for many more years. But this is another discussion to be had.
Given that the version is known, the location is known. If the user supples a tag then. It should be possible to generate HKEY_LOCAL_MACHINE\Software\Python\PythonCore<Tag>.
pymanager install 3.15.9 --target --tag 10.1
What anyone would do with it is up to them.I wonder if py -10.1 would work if the registry entry existed?
After hacking the registry
H:.\PythonTools> py -0p
Installed Pythons found by c:\programs\Apps\py.exe Launcher for Windows
-10.33-64 C:\Programs\python313\python.exe
-3.12-64 C:\Programs\python312\python.exe
-3.10-64 C:\Programs\python310\python.exe
-2.7-64 C:\Programs\Python2\python.exe *
H:.\PythonTools> py -10
Python 3.13.1 (tags/v3.13.1:0671451, Dec 3 2024, 19:06:28) [MSC v.1942 64 bit (AMD64)] on win32
Type “help”, “copyright”, “credits” or “license” for more information.
H:.\PythonTools> scite py10.py
H:.\PythonTools> type py10.py
#!python10.33
import sys
print(‘python 10.33’)
print(sys.version)
H:.\PythonTools> py py10.py
python 10.33
3.13.1 (tags/v3.13.1:0671451, Dec 3 2024, 19:06:28) [MSC v.1942 64 bit (AMD64)]
I’m absolutely in favour of cross-platform consistency. However, on Unix (and in particular Linux) that involves negotiating with multiple distribution maintainers, all of whom appear (to me) to be pretty strongly against packaging any sort of “installation manager” commands. (It’s been enough of a problem to get pip available in Linux distros in a way that everyone’s happy with, I don’t imagine getting a Python manager will be easier…)
I’d much rather we made progress where we can (on Windows) than block every platform on getting agreement from Linux distros.
I think there’s an incorrect mental model held by most people who spend a majority of their time dealing with a single operating system. “Linux” (and even “Unix” when they don’t realize that also means MacOS) seems like an operating system. And then they want to “port” things to Linux/Unix as if it’s just one platform (sure that’s sort of the goal of the POSIX standard but it barely covers the tip
of an iceberg).
The Linux kernel and GNU toolchain are pieces you can use to create an operating system, but they’re not themselves an operating system. Every Linux distro and Unix derivative (which also sometimes share some pieces despite having a separate lineage) is essentially its own distinct operating system with its own maintainers, tools, workflows and conventions. Some might have components in common or be derived from each other, but rarely in consistent ways that allow you to reason about their respective similarities to the next.
Over the decades I’ve seen very well-funded companies try to make their own cross-platform “Linux installer” for their software, but I’ve never seen it work out. If they’re lucky it might work on the three most popular distros (they end up making three different installers with a common frontend), but even that is a lot to keep working and updated as those platforms evolve release to release. Users who think they want a one-click distro-agnostic installer for some piece of software eventually come to realize that when they do that for 10 different pieces of software each of which chose a slightly different installer implementation, they’d ultimately prefer to use software directly packaged by/for their distribution so they get consistency and simplicity.
All of this to say people who expect to have one set of steps for consistently installing a piece of software (including whatever tools are needed to do that) across every conceivable platform are going to have to get used to disappointment.
I really don’t know what you’re asking me to do. “Add more commands because it isn’t worth adding less commands” isn’t a strong argument, and as others have quite eloquently pointed out, we can’t make Linux distros do anything. The best hope we’ve got is Brett’s py launcher, which already has the py name and is entirely capable of following suit (but is independent of core, so you won’t see a PEP describing it).
In terms of launching Python, the advantage is that it can calculate the most recent version of Python to launch, whereas relying on the traditional installer’s PATH modifications will either launch the most recently installed or updated version, or the originally installed, or the latest installed version, but it won’t do any ordering based on the Python version itself. So you can very easily end up with python launching 3.12 instead of 3.13. py can guarantee which version you are going to launch.[1]
In terms of managing Python, the advantage is that you only need to learn about one command. This would be an official command, not a third-party one, and so the distinction isn’t like e.g. python vs pip (which also commonly confuses people who don’t realise there are multiple commands). Since we can have one command for the job, I believe users will prefer that (once they’ve learned about it… obviously people with prior assumptions will - hopefully only briefly - get mad about their presumptions being invalidated, but that’s what this public discussion period is about).
The things it does are:
install Python
show which Pythons are installed
uninstall Python
launch Python
Which of these is excessive?
This “feature” has been fixed It won’t install a runtime if you’ve got an old one already there (I’m a little worried this will confuse people who expect to get the latest and greatest Python out of the box, but it’s already confused more people who want older versions, so I’ll cede to that for now and see where it goes).
Sure does! Though the short option has to start with 3 - but py -V:10.1 works just fine. You can even replace the PythonCore bit with your own name and then py -V:MyCompany/10.1 or py -V:MyCo/ will launch from that key. This is all defined in PEP 514, and supported by the existing py launcher (to varying extents… the support is better in PyManager, because it’s easier to do parsing and comparison in Python code than in C).
Great post! Nothing to add, just +1 to everything you said.
If I had a time machine, I’d have called it python and added python3.13.exe etc… But the py name pre-dates me by a number of years, as do the name collision issues, so the best I can do is fix things for the future. ↩︎
Thanks for expanding on this and I recognise your points. I’m still not personally convinced that it wouldn’t be better to soft deprecate py and only retain it for compatibility, but I suppose that’s for the decision makers to weigh up the advantages you’ve described against the legitimate disadvantages that others have put forward.
I think the sentiment about separation of concerns is still valid, but it’s actually not incompatible with a preference to promote py.exe (see my concrete suggestion below). The “installer” aspect of the PEP seems in reality to be entirely fulfilled by pymanager.exe, itself installed by an MSIX, which is why py.exe seems superfluous to many. Based on how you talk about it, py.exe looks to be intended more as a multi-purpose program/command for handling Python in the vein of uv, hatch, pdm etc. – and this is obviously a nice thing to have too. But the functions of the two are very much conflated and duplicated at the moment, and they are treated as mostly interchangeable, with mostly the same subcommands, but with occasional subtle differences.
You could solve the concerns by streamlining the roles of py.exe and pymanager.exe and pitching things differently within the PEP (and the future documentation). pymanager in my view ought to have orthogonal responsibilities to the other core components – python (or any interpreter by any name), pip, venv being the key ones. It definitely shouldn’t be in the business of launching an interpreter or executing scripts. And if py is what is being promoted, why would people even try to run a script with pymanager?
I think I am coming from a different perspective to the user you are responding to, but from the list above:
I would think 1, 2, and 3 are an appropriate remit. Providing other programs with the path to an interpreter on request is reasonable, however, and key to the following.
The expansion/rejuvenation of py.exe can then be introduced as a second aspect of the PEP (despite the narrower responsibilities, my understanding is that they still need to be introduced at the same time for technical reasons). Exactly as you did above, it can be pitched as a workflow tool that allows you to work with Python using a single command instead of both python and pymanager. It definitely doesn’t need to be overly ambitious and try and do everything that things like poetry or uv do, but in the same way as those tools often make use of things with single responsibilities like venv, flit or hatchling, the uv resolver, twine, and so on, it would be a higher level tool that would primarily just direct lower level components (to begin with, just pymanager and interpreters) to actually do things.
Separating concerns like this would also allow third-party tools to also use pymanager themselves to handle interpreter management rather than having to write their own logic for it (even if it never spread cross-platform).
And then rather than trying to have the command line behaviour of py and pymanager mirror each other, the likely confusion due to the subtle differences (e.g. shebang processing) can be avoided entirely; the two will be understood to be intended for different purposes, will be expected to have different interfaces, and can be written to behave how their respective responsibilities dictate.
Would this not essentially achieve your vision while also addressing concerns about overlapping functionality?
But at some point [1] that recommendation to use AMD64 on ARM64 will be dropped and pymanager will have to either make the breaking change of suddenly flipping the default to the native architecture or be a forever increasingly weird anomaly. If we’re happy to make that breaking change in the future then fine. I just don’t want to be in a world 5+ years from now when ARM64 is normalised and people are asking why pymanager install installs the wrong architecture and an older version of you is answering “Uh yeah, it made sense back in the days of pymanager 0.1.a4 but afterwards we didn’t switch to native binaries for backwards compatibility reasons”.
Sorry, I will get back to you on this one. (My VM bricked so I’ll need to make a new one before I can pick up where I left off.) Do you want to take debugging conversations elsewhere? These implementation issues don’t have any bearing on the PEP and this conversation is already getting hard to follow.
probably not so far away now that GitHub Actions’s ARM support is happening ↩︎
Honestly, you and Paul are the only two people I’ve ever heard give any indication that you understand why py even exists. There are differences but they’re not particularly significant or even always good:
It saves the installer from having to touch environment variables but that’s not really a problem for users.
It has different handling of multiple Python versions but not necessarily more correct handling. If a user installs a newer version to switch permanently then py’s behaviour is right. If a user installs it just to try it then it’s wrong. Last come, first served isn’t great either but at least with python.exe, you can just swap some rows in Windows’s environment variable manager to set your preferred version.
The Windows specific aspect makes writing cross platform scripts/CI workflows/dev guides a mess. Yes, a shrinking pool of old Linux distributions still make you to append a 3 (or run ln -s /usr/bin/python3 /usr/bin/python) but that is nothing compared to the asymmetry that py induces and even less compared to the version managing py being proposed.
Writing py -3.12 is kind of nice but then python3.12 is nicer and has a pip3.12 equivalent (I really struggle to explain the concept of python -m to anyone who hasn’t packaged before).
All in all, I wish py would be recommended only as a tool for people who want to juggle many Python versions. And if three commands is too many then I’d be pushing to at least not overload (and preferably stop recommending) what to me has always been the officially recommended distraction.
It won’t install a runtime if you’ve got an old one already there (I’m a little worried this will confuse people who expect to get the latest and greatest Python out of the box, but it’s already confused more people who want older versions, so I’ll cede to that for now and see where it goes).