Sys.argv command line parsing (documentation)

As I understand it, on Windows python.exe gets a command line string, which it parses into command line arguments.

Since sys.argv documentation doesn’t define a parsing method, I guess that argv parsing is both undefined and implementation dependent. But,
(1) I could be wrong: perhaps there is a python definition that hasn’t been documented.
(2) If it is undefined and implementation dependent, it would be nice if the documentation said so.
(3) If it is undefined and implementation dependent, cpython may use a library function: it would be nice if the documentation said so.

As it turns out, the Windows shell apparently does parse the command line when “starting” a script (script.py), using a quoted space delimited strategy, and then constructs a command line for python.exe. That is not at all a feature of the python library, but is perhaps worth mentioning, if only for the fact that it’s different to linux, and seems to have been missed by most if not all python distributions, which all are configured to drop the arguments when “starting” a script on Windows.

sys.argv is passed from the os itself as part of the initial execution. In other words: the splitting is done before python gets it and is OS-specific.

The args are given all the way up as a char** in C to main() (the program’s entry point). The char** is essentially an array of pre-split strings.

Or, in case of Windows, as a wchar_t**.

On Windows the the command is passed in as a single wchar_t string. The Microsoft C Runtime has code that parsing that string into the argc, argv that is passed to the C level main() function.

In other words the parsing is done my python on Windows.

I think this is a reference to how that parsing is done: `main` function and command-line arguments (C++) | Microsoft Learn

1 Like

Barry is correct. We don’t currently write our own command line parser for splitting arguments, though it would be technically possible (but nobody really wants to do it). The C runtime splits them for us.

What does this mean? I don’t understand what you’re referring to. Can you provide a more detailed example of how you would start a script with arguments and then Windows drops them?

On windows, I create a new script like this:

copy con tmp.py
import sys
print(sys.argv)
[CTRL][Z]

Then I run it like this:

tmp.py one two three

What result do you see? On all my installations, (at present, conda, python.org, activestate), I only see [“fullpath\tmp.py”]

This isn’t exactly “windows” dropping the arguments, this is the python distributions, which are installed in such a way as to drop the arguments.

This is different to what happens on linux/debian (I’ve left the shebang comment out of the script shown). Which is where I came in: it took me a while to realize that I didn’t have a script problem, or a python problem.

What do you get if instead of calling tmp.py directly, you run py tmp.py one two three? I think if the file association is not set up just right in Windows, the arguments will not be passed.

1 Like

Yes, “running python” is not the same as “starting a script” on Windows. I don’t have a deep understanding of unix, but it seems that on unix ‘running a script’ is the same as ‘running python with a script’. At least for the purposes of argv handling.

I went down this hole because I had a disk crash, and need to move my users to a different machine. While fixing up the expected problems, I naively expected argv to just work the same.

While reconfiguring Windows to run my scripts as my users expected, I had to think about the way python.exe parses the command line. It wasn’t a big part of my problem. It wasn’t even a medium part of my problem. It was just something I went through as I was trying to identify the problem, and considering how to configure Windows to provide the parameters in the way that python.exe understands.

It works for me. I have included the assoc and ftype config that is in the registry for running python. These settings are setup when python is installed. Compare what you have with what I have.
I’m running on Windows 11 and have lots of python.org kits installed for development reasons. py.exe is defaults to using 3.13.

C:\Users\barry\tmpdir>type a.py
import sys
print(sys.argv)

C:\Users\barry\tmpdir>a.py a
['C:\\Users\\barry\\tmpdir\\a.py', 'a']

C:\Users\barry\tmpdir>assoc .py
.py=Python.File

C:\Users\barry\tmpdir>ftype Python.File
Python.File="C:\WINDOWS\py.exe" "%L" %*

C:\Users\barry\tmpdir>py -0
 -V:3.13t         Python 3.13 (64-bit, freethreaded)
 -V:3.13 *        Python 3.13 (64-bit)
 -V:3.13-32       Python 3.13 (32-bit)
 -V:3.12          Python 3.12 (64-bit)
 -V:3.12-32       Python 3.12 (32-bit)
 -V:3.11          Python 3.11 (64-bit)
 -V:3.11-32       Python 3.11 (32-bit)
 -V:3.10          Python 3.10 (64-bit)
 -V:3.10-32       Python 3.10 (32-bit)
 -V:3.9           Python 3.9 (64-bit)
 -V:3.9-32        Python 3.9 (32-bit)
 -V:3.8           Python 3.8 (64-bit)
 -V:3.8-32        Python 3.8 (32-bit)
 -V:3.7           Python 3.7 (64-bit)
 -V:3.7-32        Python 3.7 (32-bit)
 -V:3.6           Python 3.6 (64-bit)
 -V:3.6-32        Python 3.6 (32-bit)
 -V:3.5           Python 3.5
 -V:3.5-32        Python 3.5-32
 -V:3.4           Python 3.4
 -V:3.4-32        Python 3.4-32
 -V:2.7           Python 2.7
 -V:2.7-32        Python 2.7-32
2 Likes

assoc and ftype are the keywords I had long forgotten and not looked up, and what I intended to reference above. Thank you Barry :slight_smile:

1 Like

This is probably what you want to look at instead:

in the settings for “Apps → Default apps”

If I remember correctly, the settings from assoc/ftypes are no longer primary, and can be overridden in “interesting” ways, so the advice may help, but may not. If @eryksun is listening, I’m sure he can explain.

1 Like

The file associations created by installing recent versions of the Python launcher (since 3.5, unless we fixed it a version or two later) include the %* specification needed to make sure that the arguments get passed along. These are the only associations created - we don’t associate directly with a particular version, because then when you update an older version it would change which one runs (plus there’s no shebang support).

We also install a shell extension so that we can control the arguments passed when you drag and drop other files on top of a *.py file. Again, this is only with the launcher.

If you’ve created the association manually, perhaps because you didn’t include the launcher, then it may not include these and it won’t behave properly. But the ones we install should look exactly like what Barry posted.

1 Like

I think that the legacy commands assoc/ftype will report the current config accurately as seen by the current user.
But if you use them to change config its problematic, for example change only applies to the current user.

That is good to see. PBKAC: I’m working with outdated versions.

Comment: I’m not sure that “%L” is correct. That may be a typo? (%1)

This is only a tangent to my original suggestion: that the documentation include something about how argv is parsed on Windows.

On my production machines (which range from ‘old’ to ‘very old’), software/classes is secondary to Explorer/FileExts. I don’t think they will report correctly. On Win10/11, setting may cause problems even for a single user, since you can’t use them to generate hashes. Doing an association without the program name hash may cause problems further down the line.

That is what was output by the command.
I think the %L is correct, should be documents by microsoft, but do not have ref to hand.

Ok, it’s just odd to see a Win95 compatibility option there.