This thread also: Standalone app deployment story
And Iâd like to repeat my opinion that pip (and the dependency management paradigm it serves) is fundamentally the wrong approach to deploy standalone applications like flake8, Black, etc. I am ambivalent on whether pypackages should support console scripts, but if it does, it should be because of other reasons, not to bolt standalone application distribution back onto it.
@uranusjr This is about use cases again :-). If your use case is âI want to install a standalone application that happens to be written in pythonâ, then I agree that pip/pypackages/etc. are probably not the right approach. If your use case is âI want my project workflow tooling to let me install specific versions of applications into isolated, project-specific environments, and coordinate those applications with project collaboratorsâ, then pipâs approach is fine, and people might reasonably expect pypackages to address that use case.
Pip has nothing to do with the approach here - this is virtualenvâs influence.
The only thing required to make pypackages work âthe sameâ here is to do a mkdir and pip install into that.
Not that Iâd recommend doing that either, just as Iâd hesitate to say âmanually create a venv for each of your toolsâ (pipx is much closer to the best way here), but it doesnât help to let invalid comparisons go unanswered.
I agree. But then Iâd argue entry points cause more confusion than benefits in this case; python -m works much better since it eliminates a layer of indirection (PATH). Iâve certainly seen too many people complaining venvs not activating correctly, but ending up finding they have their bashrc misconfigured. pypackages would definitely help this, but console scripts donât.
I worried next problem comes up: They may use wrong Python.
Sadly, itâs very common to having multiple Python instances are installed. For example, system Python, Homebrew, pyenv, and conda.
When activating virtual env, the python used to create the venv is used automatically.
How people can use correct Python easily with PEP 582?
I happen to be thinking about exactly the same topic today 
This also goes back to Nathanielâs fantastic elephant; these topics are somewhat separate, but in the end they need to work well together so we can provide a seamless experience.
It seems to me that we can learn things from recent works on Windows. I just posted in Ideas on a related topic: Provide a PEP 514-like registry for macOS
SoâŚ
mkdir foo
py -m pip install -t foo flake8
foo\scripts\flake8.exe
Youâd expect that to work, and for the exe wrapper in foo\scripts to run the system Python (because pip hard-codes sys.executable in the wrapper shebang), somehow with the foo directory on sys.path so that the imports needed for flake8 would work?
Thereâs quite a lot of pieces here that donât work like that right now, and Iâm not 100% sure what you expect to change and how. Also, thereâs no pypackages involved here.
I may well be missing something - quite possibly flake8 isnât the sort of example youâre thinking of (I picked it because, to me, itâs an obvious example of âproject workflow toolingâ that @njs referred to). This discussion seems to me to be getting bogged down in generalities, and weâd be better talking about very specific and precise examples, so that people know whatâs being discussedâŚ
Like I said, neither approach is the right one for flake8 (I like using black as the example because it autocompletes on my phone 
). âpipx run blackâ is far better, and if pipx is looking at a local config file to choose the exact version of black for the current project then I think that fits the need nearly perfectly.
My point is that the difference between venv and pypackages is in how they locate packages (env variables and absolute symlinks vs. relative to script and fully relocatable) and pip doesnât actually do anything different between them (once it supports the latter, at least, which as you point out it currently does not, but thatâs been a regular point throughout this proposal - the proposal is about CPython and not pip, by request of Donald, so we have to assume that pip will figure out how to work with changes to CPython).
You should have had -t foo/__pypackages__ and âsomehowâ had a script wrapper that does âcd foo; python -m flake8 (magically in the original CWD but without changing sys.path back)â. This is certainly a piece that isnât figured out yet, but it is not part of the change to CPython as such, and so itâs not in the PEP.
Adding standard script wrappers to CPython would take another PEP, but Iâm not totally against it. Then I could go fix problems like embedding absolute paths with invalid quoting
These two comments are confusing to me, because they seem to contradict each other. Are you saying that __pypackages__ is or is not intended as a way to manage which version of flake8 is used for your project?
I completely understand the idea of having a script put its runtime dependencies in __pypackages__. Thatâs basically just a simpler and more flexible way of having a zipapp, and seems entirely reasonable to me.
But when the conversation moved on to âproject workflow toolingâ, it seemed to divert to discussing development-time dependencies, and I lost track of whoâs saying that __pypackages__ is intended to handle that case, and whoâs saying itâs not. Hence my plea for specific examples.
There are way too many ways of making âapplicationsâ, âtoolsâ, or âcommand line utilitiesâ (call them what you want) in Python already. The __pypackages__ proposal adds another one, but doesnât really explain which of the existing approaches itâs intended to replace. Obligatory XKCD.
My point is that you may give any script its own dependencies in its own folder. So if you always run it as python C:\flake8\flake8.py then you can put its dependencies in C:\flake8\__pypackages__ and itâll just pick them up.
Having tooling so that you donât have to type the full path is an extra task that is outside the scope of the PEP.
OK, so this is nothing to do with console scripts. Good, I think I follow now.
From talking to Kushal, and from reading the âmotivationâ section of the PEP, my understanding is that development-time dependencies are the only motivation for __pypackages__. In particular, development dependencies for absolute beginners, where you canât reasonably teach them about virtualenv until sometime after youâve taught them about if and print.
So IIUC, the project workflow tooling case is 100% what the PEP authors are trying to solve.
OK, so Iâm very confused. I guess I need to re-read the PEP and try to understand it betterâŚ
More likely we need to rewrite the PEP better 
Itâs very focused on mechanics right now and only tells half the story because only half the story exists in the tool (CPython) that the proposal applies to.
Reading with the context of âif CPython would infer this search path from the startup file, what could other tools do to make userâs lives easierâ will yield the best results.
Thanks. With that suggestion (and a determination not to read anything into the proposal thatâs not there1) the PEP makes perfect sense to me.
The whole thing about having development tools in the __pypackages__ directory is, as you say, part of âthe other half of the storyâ, in the sense that itâs perfectly possible to install a tool like black into __pypackages__, but how that tool then becomes available as something the user can invoke from the command line depends on how other tools handle this. python -m black would just work, as that only involves the core interpreter, but if people want a shell black command, thatâs when things like console entry points, and how they get built, becomes relevant. One problem I foresee with python -m black is that its meaning will change if you change directory. Thatâs a pain, as it means you need to cd back to the project root every time. So having better tool support would be useful (although even with tool support, Iâm still a bit concerned that we might have just made python -m into something that we would rather not recommendâŚ)
Thereâs also a fairly bad (but hard to prohibit) security concern that the PEP doesnât address. Imagine a directory containing a hidden __pypackages__ folder containing a malicious pip package. We pretty strongly recommend python -m pip as the canonical way to invoke pip, so thereâs a strong possibility that this would give an attacker a way to trick users into invoking malicious code. OK, it requires the attacker to have access to a directory that the user will run commands from, but even so it seems risky (consider putting something like this in tmp on a Unix box - at least Unix doesnât have hidden directories like Windows doesâŚ)
So IMO, the __pypackages__ proposal seems like a nice simple and understandable mechanism (subject to the security question I mentioned above). But to be actually useful will involve a whole other exercise in updating tooling and project workflows to take advantage of that mechanism. And that other exercise is likely to be by far the hardest part of the processâŚ
1⌠and a fanatical devotion to the Pope
This, and the rest of the example, is a very good point. I donât think it came up in any of the discussions.
Itâs not necessarily fatal to the proposal, as a âpip.pyâ in the current directory will have the same effect today, but perhaps we ought to think about -m some more? I like it, but its ability to search a lot of places to find code to run is a bit of a risk at times.
There are fundamental differences between .py files and .exe files (as I know you know). As a simple example, subprocess.run(['pip', 'install', 'foo']) will fail to find your pip.py. As another example, VS Code doesnât recognise .py files as OS commands (at least it didnât for things like pylint when I last triedâŚ) This is probably fixable at the tool level, but that was sort of my point - it needs tool support, and it doesnât fix -m directly.
IMO, -m is an incredibly useful and effective feature. The problem is not so much with -m as it is with the whole question of âwhat is a Python applicationâ. In practical terms, there are various ways of deploying Python code:
-m is perfect for (2), as are console scripts. But they get used for (3) as well, because thereâs no particularly good solution for (3). And of course __pypackages__ is ideal for (1) (supplementing things like zipapps).
My point about python -m pip or python -m black wasnât so much that -m is bad, as that pip and black should probably be distributed as type (3) applications, but the lack of a good lightweight solution for that makes it impractical. This brings us back to this thread, of courseâŚ
I believe what Steve meant is that pip.py in cwd changes the behaviour of python -m pip, just like pypackages in cwd does. pip.exe should always do the right thing in this situation.
Iâm confused again, but Iâm going to put it down to âthis is a tooling question, and Iâll wait till someone proposes concrete changes for the tooling side before worrying about itâ 