Also not when pip no longer has to be installed into every venv. That’s the slow part - creating the environment is nearly instantaneous.
3 Likes
This part can be improved if finding __pypackages__ is built into CPython, if it is possible.
1 Like
As of the next version, pip will have a --python option that lets you manage a different environment. So you can have a private copy of pip and do pip --python .venv install xxx.
Also, pip will hopefully be available as a zipapp, so your private copy of pip doesn’t even need to be installed - .venv/Scripts/python /path/to/pip.pyz install xxx will work too.
1 Like
Without looking it up, --python seems like a weird option name to choose a venv. I would think it would be used to choose a Python executable or version. Or do you mean that --python could be used to point to the python inside a .venv?
It points to an executable. As a shortcut, if you give it the name of a venv, it will use the Python executable inside the venv. Sorry, I described it confusingly, in an attempt to show how it would typically be used in this situation.
1 Like
From reading recent comments, it seems to me that the main problem we have is there are two distinct use cases that both want to use the shortest possible command invocation form:
- People running
pythondirectly want it to automatically pick up the environment since difficult to teach newcomers otherwise (any additional flag creates friction). - People using
pythonto run a script (say in a shebang) want it to not pick up an environment, since doing so creates security implications to all existing scripts regarding cwd (similar to DLL injection).
And I would venture to say this is probably a sign that we really should have two commands (both shipped in the base Python distribution), instead of trying to argue which one python should default to, or even intelligently guess based on context (which I suspect is not possible to do perfectly and would create more secutiry concerns). Those are just very different scenarios and should have different entry points, with people being instructed to use one or the other depending on the use case.
Just a thought. I wonder if py would be a possible choice? Given where the community is at, it’s probably better to keep python as-is, and add auto environment discovery to py.
5 Likes
py is the standard way to invoke Python on Windows. In fact, python often isn’t on sys.path, so py is the only option. So I don’t think changing py is any more practical than changing python. But adding a new command could be an option. Or maybe just a command line flag? Which, if linked to an environment variable, could be set by the user as a preference[1].
However, the sys.path side of this feels almost exactly like the debate over whether the CWD (or script directory) should be part of sys.path by default. New users like the idea, experienced users find that it’s a source of errors. There’s endless debate but no real consensus. Python 3.11 added the -P option to disable adding the CWD to sys.path. It seems to me that we’re heading for a situation where __pypackages__ causes similar controversy and confusion.
Although then it would be something else we’d need to check when debugging user issues… ↩︎
2 Likes
The Launcher for Unix already automatically picks up .venv. As for supporting more possibilities, there’s a proposed solution at Support a way for other tools to assist in environment/interpreter discovery · brettcannon/python-launcher · Discussion #168 · GitHub .
I don’t know if that quite holds as much as it used to thanks to the Microsoft Store install of Python.
I think it depends on how it’s changed, but it would probably be easier than you might think. Since the Launcher is Windows-only on one side and an entirely 3rd-party install on Unix on the other, it isn’t nearly as constrained as you might expect (based on how both Steve and I have played with that CLI for that command 
).
I would rather not go that route; there’s already enough command cruft that we install (e.g. pydoc). If people really want to go the route of more flexible commands, your best is probably making py be a separate install and then expanding its functionality to be flexible enough to do what people are suggesting.
4 Likes
I’m completely happy with extra options/flexibility (in either py or python). The debate isn’t (as far as I can tell) about that. It’s about wanting to change the default behaviour - and that’s a backward incompatibility, which I feel the proposal should handle as such.
I’ll also say it isn’t just about picking up an existing environment, it’s also about creating the environment to begin with.
I like this direction. I prefer to keep python (or python3) cleaner with py as a nice launcher that can have some more user friendly defaults and configuration.
I opened Provide pluggable/extensible subcommand support · brettcannon/python-launcher · Discussion #222 · GitHub to talk about supporting sub-commands.
Do you expect the Windows launcher and this one to merge at any point? If not, aren’t we risking yet more confusion for newcomers?
Don’t get me wrong - I actually really like the idea (and I’d love to see it in the Windows launcher), but it seems a bit incongruous in the context of a discussion that’s aimed at finding a way to reduce the confusion for beginners…
Steve and I discussed that this week. It’s a possibility.
Depends on how py on Windows gets distributed long-term (i.e. does it stay in the installer or become something you get from the Microsoft Store or as a separate download?). It also depends on how hard we would push such a workflow on to people, as well as whether the wider community would even buy into such a workflow.
1 Like
Any solution to a technical problem using the words “just” or “simply” is inherently likely to be far more complex than the author of the solution anticipates.
Touché! In fact, that’s exactly why I wrote the third part of my blog posts on packaging.
Alas, I also believe that the Python packaging is too complex, and too complex especially when compared to other languages’ setups. So please pardon that the simplicity of the __pypackages__ proposal combined with past positive experiences with Node.js’ similar setup caused injudicious exuberance.
In short, I respect the skepticism, but if not __pypackages__, then what else will take its place?
I’m all for simplicity, but the length of this thread suggests that whatever solution is chosen will seem counter-intuitive to a significant number of people. For the record I think that PEP 582 is a worthwhile proposal that has merit. It offers a relatively easy way for utilities to ensure their requirements, at the cost of a certain amount of additional discipline on the part of the packager.
My only regret is that four years’ discussion hasn’t resolved the issues in time to make it into the fourth edition of the Nutshell - unless the SC pronounce really soon, which seems unlikely.
I’d like to understand what is the issue or what drawbacks exist??
- Is only the name suggestion? If so suggest a better one if we allow a config/argument to change it so the final user can choose the favourite name for it.
- I don’t think I follow the UX dictatorship on pip mentioned earlier. If your project is a wrapper/extension of PIP it will have to follow the interface of PIP.
- There’s no need to compare with other existing environments. Other environments are a solution for other problems or use cases, more complex potentially. it’s like comparing logging with print, I think this is the same scenario.
Have you read the thread? There’s a lot of detail on the potential issues.
Yes, it looks like @brettcannon address/cover them very well in his replies.