Unexpected imports with readline.py and rlcompleter.py and others files (library injection) + pip module

Spoiler: I’ve emailed security@python.org and they told me that it’s not a security issue so I feel free to create a public discussion here.

Vulnerable packages: PythonV3 (latest) and PythonV2 (latest)

Example1:

  1. mkdir /tmp/test
  2. cd /tmp/test
  3. echo ‘print(“hacked”)’ > readlines.py
  4. python (version 3, no arguments)
  5. you will see result of step №3.

Example2:

  1. mkdir /tmp/test
  2. cd /tmp/test
  3. echo ‘print(“hacked”)’ > rlcompleter.py
  4. python (version 3, no arguments)
  5. you will see result of step №3.

Example3:

  1. mkdir /tmp/test
  2. cd /tmp/test
  3. echo ‘print(“hacked”)’ > runpy.py
  4. python -m pip (version 3)
  5. you will see result of step №3.

Proof-of-concepts:
0day

(i am a new user, so i can’t add more than 1 embedded image and not more than 2 links), you can check another PoC here:

https://i.ibb.co/s3qvFvS/photo-2021-10-13-17-45-16.jpg

Description:

This allows hacker to execute arbitrary python code if another users
runs python command in certain directory.

So, for example, USER1 creates runpy.py with malicious code inside /tmp/ folder. USER2 switched current directory to /tmp/ and runs “python3 -m pip install test_package_name” and this command triggers malicious code of first user.

Full list of files:

# trying /tmp/py3/runpy.py
# trying /tmp/py3/readline.cpython-39-x86_64-linux-gnu.so
# trying /tmp/py3/readline.abi3.so
# trying /tmp/py3/readline.so
# trying /tmp/py3/readline.py
# trying /tmp/py3/readline.pyc
# trying /tmp/py3/rlcompleter.cpython-39-x86_64-linux-gnu.so
# trying /tmp/py3/rlcompleter.abi3.so
# trying /tmp/py3/rlcompleter.so
# trying /tmp/py3/rlcompleter.py

Security@python.org answered (short fragment):

The issue that you are talking about is described as a feature, not a
vulnerability.
. . .
No longer adding the current directory to sys.path has been discussed
multiple times, but so far, it remains the default behavior.

But I think that it’s not a problem with sys.path. It’s problem with python core which,
for some reason, started to input its’ core files from current directory, not python installed path.

Also about “feature, not bug”: there is no information in documentation about that python interpreter will run readline.py, rlcompleter.py and other files from current directory by default. So, if “feature” is not documented and can cause security problems - its a security issue. In this case, this feature leads to library injection vulnerability.

And last one my reply from email about isolation mode:

Yes, may be it can fixed by isolation mode, but without isolated
mode it looks like an unpredictable behavior which leads to a security
issue inside python core, which for some reason starts to load these
files from current directory.

Full email history:

https://drive.google.com/file/d/1i5YkOkxwpI2edxQ3yY_69K3F72iNWr2R/view

I will follow this topic, so feel to ask more information about this “feature” :smiley:

It is a sys.path issue from the perspective of that’s where the Python interpreter specifies the search order for modules, so it defines where this functionality occurs or not.

https://docs.python.org/3/library/sys.html#sys.path

As initialized upon program startup, the first item of this list, path[0] , is the directory containing the script that was used to invoke the Python interpreter. If the script directory is not available (e.g. if the interpreter is invoked interactively or if the script is read from standard input), path[0] is the empty string, which directs Python to search modules in the current directory first.

As for those specific modules, that’s because you’re launching the REPL. Launch your own module and what gets imported will be different (not only because you will trigger different imports but it even changes between Python releases).

As the security team said, this is all by design and known. If you don’t trust the files on your file system when executing Python files then that’s the first security issue to address as Python’s security model does not encompass untrusted files on your file system.

System utilities should also use isolated mode -I.

Yes, it solves problem. But it’s a “bad” bug fix, because solves problem only if script can be run in isolated mode. To fix this problem correctly, it’s needed to fix python core code.

P.S. One same example: there is a Remote Code Execution inside someone software - hacker can call os.system() function. And developers of this software advise users to turn off os.system function, although they can fix it in 2 lines of code : )

UPD: there are more information about bug inside next comment.

Let’s check file /usr/lib/python3.9/site.py:

. . .
    def register_readline():
        import atexit
        try:
            import readline
            import rlcompleter
        except ImportError:
            return
. . .

So, logically, if file /usr/lib/python3.9/site.py calls import rlcompleter, it must import file from current directory → /usr/lib/python3.9/rlcompleter.py.

But! With some code bugs it starts to import file in priority from current user directory, not /usr/lib/python3.9/, which must be in priority at sys.path for /usr/lib/python3.9/site.py.

P.S. I didn’t make a full source code analysis, but this looks weird for users, who are not expect that command python3 without arguments will start to run files from current directory (and same with python -m pip)

UPDATE:

I was wrong, there is information in documentation about sys.path: priority to the directory containing the input script, but anyway its not obvious for users that python3 will run these files from local directory by default, so think it’s better to fix (or write about it at documentation, but this is a bad way).

FIX
It can be fixed with adding full path of import script or temporary editing sys.path inside site.py file.

/usr/lib/python3.9/site.py:

. . .
def register_readline():
        import atexit # it is not affected because was imported earlier
        print(__file__)
        p_dir = os.path.dirname(__file__)
        sys.path = [p_dir] + sys.path
        try:
            import readline
            import rlcompleter
        except ImportError:
            del sys.path[0]
            return
        del sys.path[0]
. . .

P.S. will later add fix for pip

Please understand that changing these semantics will break people who rely on it. And these semantics have been around for decades, so there’s a reason why they have not changed even when people have brought it up previously.

This question isn’t about semantics and how instruction import works. It’s about an unexpected behavior of Python Core while using python in interactive mode.

Noone fixed it before because, as i think, they didn’t recognize this bug (didn’t find any bugtrack thread/github issue/forum thread). For users, functionality of including rlcompleter from current directory is undocumented and hidden so 99.9% users even does not know about it.

Can you, please, give an example when my fix of file /usr/lib/python3.9/site.py will break anyone’s code? As i tested now - it influenced only at python interactive mode so you can still use it as a library.

P.S. I’m not asking to change whole python semantics. I’m asking about some fixes only in site.py file to make Python Core better with less bugs/security issues.

At least every core developer knows about this, so it’s definitely known.

You’re deleting sys.path[0] which is the current directory, so if I purposefully wanted to override a module in the stdlib you just prevented me from doing that.

I appreciate the concern, but as I am not about to push for the change I will be unsubscribing from this topic.

And noone of core developers created any single topic about this unexpected behavior during python interactive mode (about importing rlcompleter.py & readline.py from current directory)? If there are any topics, I didn’t find.

I don’t delete current directory path, only appending path /usr/lib/python3.9/ at the start of sys.path and deleting it after importing rlcompleter & readline, so current directory will still be at sys.path[0]. Please, recheck it again.

. . .
p_dir = os.path.dirname(__file__)
sys.path = [p_dir] + sys.path
. . .
del sys.path[0]