How subprocess run() works?

Running python sys.executable in pycharm shows it uses virtual environment interpreter

(venv) PS C:\Users\SJ\Desktop\Programs\Python\PyTest> python
Python 3.11.4 (tags/v3.11.4:d2340ef, Jun  7 2023, 05:45:37) [MSC v.1934 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys                                                        
>>> sys.executable
'C:\\Users\\Bob\\Desktop\\Programs\\Python\\PyTest\\venv\\Scripts\\python.exe'
>>>                   

But why does subprocess in pycharm use system interpreter?

from subprocess import run
result = run(["python", "-c", "import sys; print(sys.executable)"], capture_output=True, text=True)
print(result.stdout)

It prints out location of system interpreter.
C:\Users\Bob\AppData\Local\Programs\Python\Python311\python.exe

I thought subprocess run() simply types the command in the pycharm terminal and runs it?

I am using PyCharm and I don’t get that result.

Maybe it was how you had set up PyCharm during installation?

Consider using sys.executable.

No. It finds the executable (i.e. the first element in the list) and then uses OS specific C level functions to start the process with the given list of strings as arguments. How it finds the executable is essentially by looping through the PATH variable, which is similar to what powershell does, but not the same. Specifically, venv adds python as an extra alias that isn’t anywhere on path and wont be found by subprocress. Just use run([sys.exectuable, ....

subprocess.run() spawns a new process, which on Windows is implemented by calling the system API function CreateProcessW(). Currently, it lets the system API resolve the command line, and the search path that’s used by the system always begins with the application directory of the current process, and may include the current directory, and finally the directories in the PATH environment variable. Thus searching for “python” will find “python.exe” in the application directory. This will not honor an active virtual environment because venv on Windows defaults to using a launcher instead of a symlink, so the application directory is that of the system Python, which in your case is “%LocalAppData%\Programs\Python\Python311”. To spawn a process in the virtual environment, execute sys.executable instead.

CreateProcessW() is not finding “python.exe” in a PATH directory in this case. It’s a bit more complicated than that.

Yep, fair, I used it as a short hand.

I need to stress that subprocess.run() itself does not search for “python” on Windows, unlike how it’s implemented on POSIX. It just lets the system API find “python”, which is where the peculiar behavior with the application directory comes into play. If, for example, the virtual environment gets created with py -m venv --symlinks venv – which uses a symlink for “python.exe” instead of a launcher – then subprocess.run('python') will instead find and execute “python.exe” in the virtual environment.

1 Like