I’m on the fence about PEP 582, but I think that the existence of some magic directory toggling the behavior on or off is spooky action at a distance that will just confuse people. We should either decide that this is the path forward as the default and commit to it, or we should not do it at all.
I don’t think the .git comparison is a particularly good one, because .git doesn’t have a behavior other than to error if you run the command outside of of a directory tree that doesn’t have a .git directory, so while the presence of a .git directory signifies this is a git repository, it doesn’t change the behavior of git from one non error state to another.
Compare this to PEP 582, where the end result would be people accidentally installing things into their global environment when they didn’t mean to, causing more confusion about where exactly they’re going to install to or not.
I’m on the fence about PEP 582 because as a casual, only sometimes user of node.js and npm, I find node_modules behavior incredibly frustrating, and I don’t think I’ve worked on a project in Node where I wasn’t wishing for something akin to virtualenv by the time I was done. Obviously a lot of people use and like that particular feature of the node ecosystem, so it’s not clear to me whether I’m just used to my workflow or whether there’s some underlying problem with the node_modules approach.
I do know that one place this causes a lot of frustration for me personally is using within a Docker container that you’re bind mounting a host path to. Because the node_modules, or in this case __pypackages__ lives in the same place the code does, it makes it difficult and annoying not to share the same set of installed packages in the host and inside the container [1]. This can work sort of ok in an ecosystem like Node where a lot of projects may never have anything but a pure javascript dependency tree, but it stops working the moment you have any sort of compiled artifact in your build, which as it turns out, most projects in Python end up having at least one unless they’re very small or take great care not to.
Now a observant person would notice that this is essentially a problem of trying to re-use the __pypackages__ across multiple operating systems and/or computers, which is obviously a nonsense thing to do, but by colocating the installed environment with the project, you’re more or less making the behavior of re-using __pypackages__ between OSs/computers the default behavior anytime you have a single directory project being shared among multiple, which can happen for a number of reasons such as:
- Docker containers bind mounting (or even just copying the files in if
__pyproject__already exists on the host). - WSL2 mounts on Windows
- Bind mounting file systems into virtual environments.
- Dual booting computers with a shared data drive
- Any tool like a NAS, Dropbox, etc that shares a directory between multiple computers.
If we were to go forward with something like PEP 582, I would want to at least see the PEP be updated to acknowledge these problems, and explicitly declare that it thinks the improvements to the new user workflow is worth the cost of breaking all these other workflows (which may also impact new users of course) or come up with some solution to that particular problem.
See Warehouse for instance as an example of trying to avoid getting
node_modulesfrom getting mixed up between the host and the container. ↩︎