I am trying to do a debug build of Python-3.11.1 on NonStop systems(custom OS and somewhat similar to linux), I see that whenever I invoke the interpreter, sys.path does not have the current directory by default. This goes against the default behavior of windows and linux. I have not configured any thing related to path or set PYTHONSAFEPATH environment variable. I have also not used any commands while invoking the interpreter like -P etc. Where can I start looking in CPython source code to debug this issue? what might be causing this issue? any pointers would be helpful.
This miiiight have been better as a continuation of the previous thread, given that it’s still looking at sys.path issues and it’s unclear whether something has suddenly inverted or not. Anyhow.
The default sys.path does NOT include the current directory on any platform. What it usually includes is the script directory.
You’ll sometimes see this shown as an empty string, in which case the script directory does happen to be the current directory, but in general, it’s the script dir that you have at the head of sys.path.
If safe-path mode (-P) or isolated mode (-I) isn’t enabled, then the script directory is prepended to sys.path if running a script, else an empty string is prepended to sys.path. The latter is the case when running the interactive shell (REPL) or a -c command. When sys.path is searched, an empty string gets evaluated to the current working directory, which can be changed via os.chdir(). It’s similar when running a -m module without safe-path mode or isolated mode, except in this case the fully-qualified initial working directory is prepended to sys.path, not an empty string.
I have a .py file named ham which has the following code,
raise KeyboardInterrupt
when I start an interactive session in this same directory using ./python(my 3.11.1 build) and then run the following command,
>>> import ham
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'ham'
>>>
however when I try to do the same with a installed version of python(I have 3.6.15 as installed version currently) by first invoking terminal using python3 then then the following command
>>> import ham
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/mohammed/python/Python-3.11.1/build/ham.py", line 1, in <module>
raise KeyboardInterrupt
KeyboardInterrupt
>>>
Which seems to work fine. My question being what could have caused this change in behavior?
also I do not see my current directory in sys.path either specified as empty string or as absolute path.
Even when running an interactive shell I do not see an empty string being present in sys.path. I invoked interactive session using ./python and following is the output for sys.path.
The fact that you invoked via “./python” and sys.path starts with “/home/mohammed/python/Python-3.11.1” is peculiar. A fully qualified directory is getting set instead of an empty string. Change the initial working directory and run Python via “path/to/python” instead of running it from the build directory. I want to find out whether that directory is always prepended as either the fully qualified build directory or the fully qualified initial working directory.
have added ham.py file in the directory /home/mz and the output of sys.path is as follows when I invoke the python interpreter using /home/mohammed/python/Python-3.11.1/build/python (i.e path to binary)
Was the working directory “/home” or “/home/mz” when you ran Python? It’s all the more peculiar if you ran Python from “/home/mz”, but “/home” was prepended to sys.path. Do you get a similar result when running Python with various random directories as the initial working directory? Try running with -E to ignore all PYTHON* environment variables.
My working directory was /home/mz, I see a pattern here, it’s prepending the current directory’s parent to sys.path. I will try with -E and place ham in some random directories as well, will get back on this shortly. Where exactly is the code for adding paths to sys.path on startup located in CPython source?
The implementation starts in pymain_run_python() in “Modules/main.c”:
The value of config->safe_path is true if either safe-path mode or isolated mode is enabled.
Most of the implementation is in _PyPathConfig_ComputeSysPath0() in “Python/pathconfig.c”:
The value of argv->items[0] should be one of the following: an empty string (i.e. run the REPL), “-c”, “-m”, or a script path. For a script or an empty argument string, have_script_arg will be true. For “-m”, have_module_arg will be true. For “-c”, both have_script_arg and have_module_arg will be false. The end result is that the value of the path0_p out parameter gets set to an empty string object when running the REPL or a “-c” command; or the fully-qualified path of the current working directory when running a module; or the resolved script directory when running a script.
Thank you Eryk, regarding your previous question, yes I am getting the same results on trying this out with various random directories, only the parent of current directory gets prepended to sys.path. tried running with -E option, but that didn’t make a difference in my case, I got the same sys.path as without -E and again only the parent of current directory got prepended. I will step into the functions you mentioned, will get back on this, till then any comments and suggestions are welcome.
I found the issue, it is related to some additional defines and behaviour of api’s slightly different than linux. Thank you Eryk, you pointing out the parts of program where sys.path initialization takes place was great help and I was able to debug this issue. I have added necessary changes to make it work similar to linux.