Adding created python scripts to path and using them wherever

Python Help
1

Hello, I am running Windows 11 and python 3.12. I am getting into python personal projects more and want to make some helper scripts that I can call from anywhere.

However I do not understand how to get my script to run as a python script when I call for it.

For context, the script is called path_test.py and as of right now it is meant to simply print the path of the current working directory it is called in. The path to this script is:
C:\Users\ian\my_scripts\py_scripts\path_test.py

Things I have tried based on looking through various stack******* threads:

  • Creating a PYTHONPATH with *C:\Users\ian\my_scripts\py_scripts\ * in my USER environment variables
  • Adding *C:\Users\ian\my_scripts\py_scripts\ * to PATH in my SYSTEM variables

path_test.py contents:

import os

cwd_path = os.getcwd()
print(type(cwd_path))
print(cwd_path)

env_path = os.environ['PYTHONPATH']
print(env_path)

This is what I see from the various test commands I ran in GitBash when i tried to run the script from my Documents folder:

image
image1261Ă—877 61.2 KB

From what I have read, the script is not being associated with the interpreter for python. I used the python.org installer when I installed python, and my python folders are properly in PATH. PATHEXT also has .PY in it yet path_test is not recognized.

One thing I did notice is when I put:
#! python
at the top of the path_test.py it will actually work.

Why is this the case? Is there a way to get the command lines to work without this line? I have read that the shebang line wont necessarily work in all cases depending on the OS or how the user has installed python.

2

Nothing that’s going on here has anything to do with Python. It’s really about understanding a) DOS and b) Git Bash. And yes, you really do need to understand both, because Git Bash is a rather flimsy attempt at creating a vaguely Linux-like environment on top of DOS while also allowing you to do some DOS-like things. It mainly exists so that Windows users can use Git (in particular the msys build) without massive headaches, and access a few common utilities that are handy to have available while using Git.

Since you apparently have a fairly new setup, you might prefer to try working with Windows Subsystem for Linux (WSL). However, I would recommend that you first make sure that you can do what you want directly in DOS.

This tells Python where to look (additional places to look besides the defaults) when it tries to import a module. It has nothing to do with running your code as a script.

This tells DOS (indirectly through Git Bash) where to look for programs (which could include the Python interpreter itself). It has nothing to do with running your code as a script. Of course, Git Bash allows you to treat the script as its own executable. But, while there are advantages to doing that, setting PATH is still probably not what you actually want.

If you try python path_test.py from the Documents folder, Python will look in the Documents folder, and nowhere else, for the script - because that is your current working directory. It’s the same as if your script’s code tried to open a file by specifying only a name without any path. With this command, Python is the program, and path_test.py is an argument passed to that program, which the program is free to interpret as it wishes. (As it happens, Python will interpret it as the path to a script to run.)

The simplest way to solve the problem is to navigate to the folder that contains the script first, using the terminal’s cd command. Then python path_test.py will work normally.

Otherwise, you will need to specify a valid path to path_test.py. This can be either absolute or relative.

Because you are using Git Bash. It implements the Linux idiom of “executable text files” (but, judging from the screenshot, does not respect PATHEXT while doing so).

This is also why some of your attempts give you “no such file or directory” errors where there are missing backslashes in the reported path. In this environment, forward slash is the expected path separator and backslashes require escaping.

This is also why, when you don’t have that shebang line and try path_test.py, you get the errors that you do. The tool is implementing the “executable text file” idea for you, but when there is no shebang, an executable text file is a shell script in whichever flavour of shell is in use (here, that would of course be Bash - note that this has nothing to do with Windows batch scripts).

2 Likes
3

Thank you for responding so quickly. I very much appreciate the knowledge you brought.

I understand this, invoking python path_test.py in its own directory does work as intended, however this was not my concern. What I want to achieve is the ability to invoke path_test.py anywhere I want (that is why i was showing what happens when I try to call it from another folder, ex. Documents).

So if PYTHONPATH is where it is looking to import modules, my script is not a module, so I can see now how that does not belong. Python is already in my system PATH, and if that is the default search spot should I just get rid of PYTHONPATH?

You also mentioned the run environment and the back/forward slashes. I have re-tested $ python C:/Users/ian/my_scripts/py_scripts/path_test.py in Documents, and with the forward slashes it works properly outputting the correct path to Documents.

I have not looked into DOS and have very limited experience with Git Bash. The CLI’s i have currently on my machine are:

  • Command prompt
  • Terminal
  • Powershell
  • Git Bash

From the brief searches I have done the closest thing I would have is Command prompt which is related to MS-DOS. Do you have any recommendations on which of these I should be favouring for testing my scripts?

I am new to these concepts. Is there a way to set up my environment that enables the recognition and execution of my created scripts as a python programs without the use of the shebang?

I would like to be able to make my own folders so I can create small helper scripts that can be called from wherever I want to invoke them.

4

DOS has not been a part of windows for about 20 years ago…

People sometimes call the CMD shell DOS, but its is wrong to do so.
CMD command runs Windows .exe files.

1 Like
5

Small update, if I use command prompt:
image
just calling path_test.py does work.

I should look into seeing if I can get Git Bash to respect PATHEXT as you mentioned here.

6

If you only set it to try to solve the current problem, sure.

Some people find it useful for configuring their Python environment e.g. when developing multiple projects at the same time that depend on each other. However, if you plan to distribute the code anyway, it will be easier to just do the packaging work anyway and install in a virtual environment - since you’ll want that setup for testing anyway.

“Command prompt” in the Windows UI should mean a plain DOS command line. I’ve never heard of Windows explicitly offering an option anywhere labelled “Terminal”; rather, “terminal” is a generic term that people use for this kind of CLI window. Except for Mac users, whose terminal actually is called Terminal.

Not that I’ve ever heard of, and it sounds like a bad idea.

Windows and Linux have two completely different approaches here.

The Linux approach, following the UNIX tradition, is that every file on your computer (including actual compiled programs) is just a file, and files can have an “executable” permission (which you can set using chmod). If something is executable, then it can be the the actual program name (the first word) in a shell command (a command you type at a Bash etc. prompt), and the operating system decides how to run it based on the first few bytes of the file. Compiled code will typically be stored in the ELF format, recognizable by that signature. Shebang lines work the same way: the operating system notices the bytes (not characters; it must use an ASCII-transparent encoding, so e.g. UTF-16 will not work) #! at the start of the file and then interprets the rest of that line to determine what “interpreter” to use. The default (and a given Linux system might be quite reliant on this default) is to treat it as a shell script.

(Of course, Python is bytecode-compiled in much the same way as Java or C#; but because that bytecode runs on a virtual machine and is not machine code, the operating system still thinks of this as an “interpreter” setup.)

Note carefully that this system means the Python file gets to choose the interpreter. It can specify an exact path to a specific Python executable, or it can ask /usr/bin/env to find one (using the same logic that the shell does when you write python my_file.py etc.).

The Windows approach is that only files containing compiled code, in an appropriate format (typically labelled as .exe) are “executables”. Other files are necessarily “opened by” an actual executable - including scripts that get run by interpreters. So, instead of making the .py file itself “executable”, you can configure the system to “associate” those files with a specific Python executable that will open them when you double-click the file. The association is based on the filename extension (Linux in general doesn’t really care about these), not the initial bytes of the file.

But this means that the OS configuration is in charge of exactly which installation of Python gets used - not the file itself, like how it works on Linux. To ameliorate this, the Windows distribution of Python includes a py executable which looks at shebang lines and interprets them according to its own logic, and otherwise falls back on a default Python installation. So, if you have multiple Python installations on your system, you can (at least in theory - I haven’t used Windows in years) associate the .py extension with the py executable, double-click the file, and still have it run with the correct Python installation.

As for making path_test.py work from a command line (or even just path_test) I’m afraid I can’t advise. If you found something that works, great. But I do know that Python packaging includes the ability to create “entry points” for the installed code that work like commands at the command line - and (at least with Pip) when you install on Windows, this creates actual .exe wrapper files (on Linux it creates some simple Python scripts that include a shebang, do some sys.argv manipulation and then imports your code). I assume there’s a good reason for that extra complexity.

I thought it was compatibility layers all the way down? Did they seriously re-implement all of that dos-like logic, including all the weird quirks of the original design like the magic filenames (NUL, CON etc.)?

7

Not true. You can setup to execute scripts aromatically as well.
For example on Windows .exe, .bat and .cmd files are executable by default. When python is installed .py files are added as executable files.

But if the .py files that the user wishes to run are not on the PATH they will not be found, as is the case with .exe as well.

You can CD to the folder that has rge .py file then run it.
Or you can use the full path to the file.

Personally I create a bin folder for my user and copy scripts into it so that I can run them conveniently. I added %userprofile%\bin to my PATH.

1 Like
8

Oh, has this changed over the years? I distinctly remember having difficulties in the past and having to wrangle with ftype and assoc.

9

I’ll double check when I get a moment, Windows is not my main OS.
ftype and assoc still work, but are not the modern way I understand.