PEP 394: Clarify Bullet for #!/usr/bin/env python

PEPs
1

Could the following bullet in PEP 394 please be clarified?:

For scripts that are only expected to be run in an activated virtual environment, shebang lines can be written as #!/usr/bin/env python, as this instructs the script to respect the active virtual environment.

I believe it would help “newbies” like me better understand what’s going on here if there was a little more, like this:

The shebang #!/usr/bin/env python should be used in scripts where it is desired to have a single shebang with the greatest degree of cross-platform compatibility, both for multiple versions of Python and multiple operating systems.

  • env is a system binary in /usr/bin that searches $PATH for strings containing the provided argument and returns the first instance it finds. In the above syntax, env will search for the first instance of python in $PATH and return it.

  • This shebang option works well both when a script is to be run from within an activated virtual environment or if it is uncertain whether the python interpreter exists in /bin , /usr/bin , /usr/local/bin, or another custom path.

  • Since python may be an alias for python2 or python3 (typically, it is an alias for python2), the developer may choose to be specific and use the shebang #!/usr/bin/env python3 where desired.

Here is my rationale:

  1. env in /usr/bin is not just for virtual environments (i.e. python virutal environments, which the bullet seems to suggest at first read). env is available by default in most Linux distros, but technically not part of the POSIX standard. For Windows users, it is easy to conflate env with Python virtual environments as opposed to the env tool in Linux. I discovered I was making this mistake after seeing that python is separated from /usr/bin/env with a space, indicating it was an argument passed to env per the #! interpreter [optional-arg] shebang syntax.

  2. The following Migration bullet in the PEP recommends being explicit wherever possible using python3 as opposed to python where intended:

It is strongly encouraged that distribution-specific packages use python3 (or python2) rather than python, even in code that is not intended to operate on other distributions. This will reduce problems if the distribution later decides to change the version of the Python interpreter that the python command invokes, or if a sysadmin installs a custom python command with a different major version than the distribution default.

  1. The fact that the PEP states the shebang usage should be used for activated virtual environments has appeared to be a source of confusion for most readers of the PEP. I believe this is rooted in confusion over the env tool and python virtual environments. Specifically, as there is no guarantee the end user will have an activated virtual environment even if they are encouraged to do so, specifying #!/usr/bin/env python to new readers seems odd. Without clarification, especially for Windows users and the uninitiated, /usr/bin/env seems like a folder that gets created when virtual environments are activated. For example, please see the following Stack Overflow posts regarding this topic:

I believe such a clarification would also benefit PEP 397 as the Python Launcher admits “virtual” shebangs, so it will provide further impetus for Windows users to add #!/usr/bin/env python3 to their scripts to ensure the greatest degree of cross-platform compatability between Linux, MacOS, and Windows.

2

For #!/usr/bin/env python, the py launcher searches the application directory, current directory [1], system directories, and PATH for “python.exe”. If it’s not found, the launcher uses an active 2.x or 3.x virtual environment, per the VIRTUAL_ENV environment variable. If there’s no active environment, it falls back on running the preferred installed version.

Currently the launcher does not, however, support searching for “pythonX[.Y].exe”. For something like #!/usr/bin/env python3, it runs the preferred version of 3.x that’s installed. It does not search for “python3.exe”, and it does not use an active virtual environment. I think it could at least do the latter by checking the version or version_info in the environment’s “pyvenv.cfg” to ensure that it’s a compatible version.

[1] Including the current directory before PATH is insecure. This could be averted by explicitly passing the value of PATH to SearchPathW instead of using the system’s default search. If one really wants the current directory included, include a “.” entry in PATH.

3

@eryksun
This is good information! I think this clarification should be added to PEP 394 and/or PEP 397 as well (wherever deemed appropriate, unless it’s already there).

For PEP 394 (geared towards Linux users), I’m primarily interested in clarifying that env is a Linux system binary that searches $PATH for the provided argument. That, and the fact that #!/usr/bin/env works regardless of whether one is in an activated virtual environment, I think would be a huge win. Such a sidenote clarification would give a great deal of clarity to readers, as would your information.

4

@eryksun I’ve posted this question to the python-ideas mailing list as well. I’m hoping to get some feedback from the larger community on this. Thank you again for your reply!