Would it be appropriate/acceptable to put sys.path inserts such as PYTHONPATH and/or
'' when that is added at a lower priority than the standard library, instead of always top priority?
This is related to Python flag/envvar not to put current directory to sys.path (but don't ignore PYTHONPATH) (and several of the historical links in this comment) in that it involves the behavior of PYTHONPATH and/or current directory (when that happens) being added to the front of sys.path, but I haven’t seen this specific idea discussed.
Overriding Python’s standard library is harder than it used to be (long ago, in the before-pip times, eggs were regularly added to the front of sys.path instead of at site-packages priority), and I think that’s great! So “installed” packages are now no longer added to the front of sys.path, but PYTHONPATH / “script dir” imports are still added to the front, which means they have priority above the standard library, unlike other third-party installs. That makes it easy for a file called
code.py or a directory called
code (or any other stdlib module) to cause real problems, as has been discussed numerous times over the years.
In IPython, we mitigate this issue in our own “script path” implementation by inserting the CWD immediately ahead of site-packages instead of ahead of the standard library, and this has helped us a lot! Would a similar change to the default behavior be appropriate? Is there a strong intentional reason why overriding the standard library is preferred by default? I’ve only ever used it myself to deliberately break things.
Such a change only protects the standard library, not anything in site-packages, but that seems reasonable to me because I think users are more likely to be aware that they have and need an importable
numpy than a
code, for example.
Another (much bigger!) related option to protect the standard library would be to add a
stdlib namespace to allow
from stdlib import re so that there’s only a single possible name for conflicts. That would be a years-long migration, but I think it would be nice if stdlib modules could be clearly and explicitly imported from the stdlib instead of sharing a single namespace with third-party packages.