PYTHONPATH when the path contains a colon ( : )

Python Help
1

This one is tricky (and probably funny). At first I was going to suggest this in “ideas” as “lets change the way PYTHONPATH works” but decided to first seek help/rubberduck get a reality check, do some therapy sessions, etc here…

The problem is the following, if i want to add to PYTHONPATH a path that contains a colon (for instance /tmp/foo:bar) I cant. I’ll explain in code:

➜  ~ cd $(mktemp -d)
➜  tmp.81bfPh4UH3 mkdir foo:bar f00
➜  tmp.81bfPh4UH3 touch foo:bar/me.py f00/me.py
➜  tmp.81bfPh4UH3 chmod a+x */me.py
# using f00 as part of the path works
➜  tmp.81bfPh4UH3 PYTHONPATH=f00 python -c 'import me'

# using foo:bar does not
➜  tmp.81bfPh4UH3 PYTHONPATH=foo:bar python -c 'import me'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
    import me
ModuleNotFoundError: No module named 'me'

now to be fair for PATH its impossible to add a : in the PATH itself for instance

# can "execute me.py"
➜  tmp.81bfPh4UH3 PATH=f00 me.py

# can't execute it
➜  tmp.81bfPh4UH3 PATH=foo:bar me.py
zsh: me.py: command not found...

So there is a reason why PYTHONPATH would fail as according to the docs of PYTHONPATH pythonpath uses os.pathsep as the way to split this dirs, And i don’t think there is a good way of fixing this on a way that backward compatible, I also dont know if this is something that can or should be fix (I also don’t know if this is an issue in windows)

where this can bite is:

  1. when you have a directory name “foo” (or “bar”) and another named “foo:bar” , adding “foo:bar” to pythonpath would fail
  2. when you have dirs and projects that users name them, i need to make sure to assert that : is not a valid name

Th problem is that we cant really allow the escaping of pathsep, as this has never been the behaivure, i would not be surprised that people rely on \: . at some point i though that another colon would be a good escape :: but that would break even more scripts that ely on empty path to ignored.

So assuming that I didn’t miss the solution and this is not a solved problem, and if I had to come with ideas on how to solve it, I would propose either pass a flag to python that can be used to change the PYTHONPATH sep, something like -X PYTHONPATHSEP=; . and probably a new env variable to control this also.

2

The fix is for you not to use special characters in your folder names.

3

Colons are used as a separator in the PATH environment variable (Windows uses semicolons for the same purpose), so having one in a directory or file name is a bad idea.