It took me embarrassingly long to get a rough idea of what was happening here. So I thought I’d post it to see if I missed something obvious and if there could improvements be made.
Usually, at some point when starting to learn Python, something like this happens:
$ python3 -VV
Python 3.9.2 (default, Feb 28 2021, 17:03:44)
[GCC 10.2.1 20210110]
$ ls -a # empty folder
. ..
$ cat <<EOF >gettext.py
> print("I am a local gettext.py")
> EOF
$ python3 -c 'import argparse'
I am a local gettext.py
$
(Depending on your Python version, you might also get an ImportError immediately. I am on Debian 11 “bullseye” with Python 3.9.)
Explanation: The user has an (unrelated) gettext.py in their folder, without knowing that there’s an stdlib module of that name. The user wants to use argparse which imports gettext which leads to errors, because the local gettext.py hides stdlib’s gettext.
I’m aware of this and wanted to deliberately take advantage of the behavior to edit a local copy of a stdlib module to quickly try out some changes. However, I noticed the exact opposite behavior:
$ rm gettext.py # clean-up
$ python3 -m venv venv # create venv for later
$ cat <<EOF >reprlib.py
> print("I am a local reprlib.py")
> EOF
$ python3 -c 'import reprlib'
$ # notice no output from line above
$ python3 -c 'import reprlib; print(reprlib.__file__)'
/usr/lib/python3.9/reprlib.py
$
To my surprise, the local reprlib.py has been ignored.
Also, interestingly, the virtual environment behaves differently, despite being the same version:
$ ./venv/bin/python3 -VV
Python 3.9.2 (default, Feb 28 2021, 17:03:44)
[GCC 10.2.1 20210110]
$ ./venv/bin/python3 -c 'import reprlib'
I am a local reprlib.py
$
With a current build, the issue disappears for reprlib when executed directly:
$ # build from source, steps skipped
$ ./python -VV
Python 3.12.0a0 (heads/main:ffcc7cd57f, May 11 2022, 16:39:40) [GCC 10.2.1 20210110]
$ ./python -c 'import reprlib'
I am a local reprlib.py
$
But it is still there for the interactive shell:
$ ./python
Python 3.12.0a0 (heads/main:ffcc7cd57f, May 11 2022, 16:39:40) [GCC 10.2.1 20210110] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import reprlib
>>>
Furthermore, for other stdlib modules like io, the issue persists for both command line and interactive shell:
$ cat <<EOF >io.py
> print("I am a local io.py")
> EOF
$ ./python -c 'import io'
$ ./python
Python 3.12.0a0 (heads/main:ffcc7cd57f, May 11 2022, 16:39:40) [GCC 10.2.1 20210110] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import io
>>>
I assume all of this has something to do with some modules being imported - every time, by default - when starting the interpreter. Which specific modules this concerns or at which time the import happens, seems to depend on Python version, virtual environment and interactive shell.
The inconsistent behavior can probably be explained by looking at the internals, but this made it really confusing for me to find out what’s even going on. While I’m aware that it’s generally good practice to just avoid re-using names of stdlib modules, I thought that my use-case warrants to do so. In any case, it’s unfortunate that the whole “wrong import” situation results in a, more or less, silent error.
Is all of this missing from the docs or at least not easy to find or - most likely - am I just really bad at searching for it? Both for my issue as well as the more frequent “hiding an stdlib module with a local module”? Is there a conclusive list of names to avoid? If my assumption from above is correct: which modules are automatically being imported at start-up? I guess python -c 'import sys; print(sorted(sys.modules))' might be misleading because I manually import sys?