Pip installing off PATH: should I do something?

(Quick note: apologies if this should have been posted in Packaging, but since it’s a cry for help, I thought this space better suited. I’ll delete and repost over there if needed.)

My question:

After updating to access pip, I installed Numpy, tabulate, and prettytable. The install confirmation for Numpy and tabulate was accompanied by a warning stating the package location and that the location was not on PATH. I did not receive this warning for prettytable though.

Are there any issues, or foreseeable future issues, if a script is installed not on PATH? I’ve been learning Python for a few months through university (ie I’m a noob). I’m totally ignorant about where these modules should be stored to be considered on PATH, or what problems, if any, may arise if they remain in their current location.

exact warning in cmd.exe:

WARNING: The script tabulate.exe is installed in 
'C:\Users\atomi\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\
LocalCache\local-packages\Python310\Scripts' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, 
use --no-warn-script-location.

Do I need to update the module’s location or add the directory to PATH? (another topic I’ll need to research)

Thanks in advance. While waiting, I should probably research the differences between modules and packages, and how they relate to the term ‘scripts’.

After updating to access pip, I installed Numpy, tabulate, and
prettytable. The install confirmation for Numpy and tabulate was
accompanied by a warning stating the package location and that the
location was not on PATH. I did not receive this warning for
prettytable though.

Are there any issues, or foreseeable future issues, if a script is installed not on PATH? I’ve been learning Python for a few months through university (ie I’m a noob). I’m totally ignorant about where these modules should be stored to be considered on PATH, or what problems, if any, may arise if they remain in their current location.

exact warning in cmd.exe:

WARNING: The script tabulate.exe is installed in
'C:\Users\atomi\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\
LocalCache\local-packages\Python310\Scripts' which is not on PATH.
 Consider adding this directory to PATH or, if you prefer to suppress this warning,
use --no-warn-script-location.

Do I need to update the module’s location or add the directory to PATH? (another topic I’ll need to research)

No, you don’t need to do anything. unless you want to use the script as
a command.

Thanks in advance. While waiting, I should probably research the
differences between modules and packages, and how they relate to the
term ‘scripts’.

A module is a single .py file. A package is a directory/folder with a
collection of files within it; it is common to break a module into a
package when it gets cumbersome.

WRT PyPI, everything there is called a package, even if it is a single
.py file. In that context, “packaging” is a the process of preparing a
module or (ahem) package for publication and distribution.

“scripts”: When you package something up, part of the metadata can say
that certain functions within the package can be presented as command
line tools. These are scripts.

For example, my cs.app.ydl package contains:

 ydl = cs.app.ydl:main

in its metadata, saying that the package installation should create an
executable script named “ydl” (or in your case on Windows “ydl.exe”)
which calls the main() function in the package cs.app.ydl as a
command.

The error message you have is saying that the tabulate package
provides a tabulate.exe command, and that the place in which is was
installed is not in your PATH environment setting, which governs where
to look for “commands”.

Typically, there is some kind of “activate” process people use enable
use of a particular Python environment, and one of its side effects is
to place the environment’s scripts directory in PATH so that the
scripts which got installed in that environment are available as
commands.

You can just add that directory to your PATH to achieve the same effect
because the installed script should invoke Python in a way which
utilises the environment’s Python install.

As a specific example, I’m on UNIX (a MacBook).

My usual Python environment is a virtualenv in ~/var/venv/3 - a
virtual env is an environment derived from some specific Python install,
but separate enough that package installs live in the environment.

Now, my own $PATH (the UNIX parallel to Windows %PATH) includes
~/var/venv/3/bin, which is where packages install their scripts on my
system. So I have a tabulate command:

 [~]fleet2*> py3 -m pip install tabulate
 Requirement already satisfied: tabulate in ./var/venv/3.10.6_1-homebrew/lib/python3.10/site-packages (0.8.10)
 [~]fleet2*> which tabulate
 /Users/cameron/var/venv/3/bin/tabulate
 [~]fleet2*> ls -ld ~/var/venv/3/bin/tabulate
 -rwxrwxr-x  1 cameron  cameron  246  4 Oct 10:55 /Users/cameron/var/venv/3/bin/tabulate

Here’s the bit about your warning message:

You do not need that directory in your PATH unless you want it.

Here is the tabulate script on my machine; it was created by pip
when it installed the package:

 [~]fleet2*> cat /Users/cameron/var/venv/3/bin/tabulate
 #!/Users/cameron/var/venv/3.10.6_1-homebrew/bin/python3
 # -*- coding: utf-8 -*-
 import re
 import sys
 from tabulate import _main
 if __name__ == '__main__':
     sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])
     sys.exit(_main())

An important feature is the shebang (#!) line at the top, which
specifies the python3 executable which will be used to run this
script.

Because of that, I could (a) not have the venv bin folder in my
$PATH and (b) just invoke the tabulate script explicitly instead:

 [~]fleet2*> /Users/cameron/var/venv/3/bin/tabulate --help
 Usage: tabulate [options] [FILE ...]
 .......

and it would still work correctly courtesy of that shebang line. So you
do not need that folder in your PATH inless you want the convenience
of invoking tabulate as plain command:

 [~]fleet2*> tabulate --help
 Usage: tabulate [options] [FILE ...]
 .........

I do. You may or may not.

On Windows, the Python launcher also understands shebang lines and the
tabulate.exe file on your Windows machine will probably be quite
similar to the one on mine but with a different shebang path.

Cheers,
Cameron Simpson cs@cskk.id.au

1 Like

Thank you Cameron. A lot of information to unpack. It’s like climbing a hill to find a mountain blocking the view. But, according to Aristotle, “…there is no learning without pain.”

I am experiencing problems with import < module >. I just installed colorama and import colorama works fine. import tabulate, import numpy, form prettytable import PrettyTable, (including import prettytable / import PrettyTable) all raise ModuleNotFoundError. Yet all four were installed to the directory provided in original post.

Let me ask this: would the location my .py files are stored to and opened from impact anything? I created directories for university assignments, which is where I’ve been storing to/opening from.

Thanks again. I’m still going through your wonderfully detailed reply.

I’m wondering how you’ve been doing the install of these modules.

I noticed (just now) that the path in your install warning:

 WARNING: The script tabulate.exe is installed in 'C:\Users\atomi\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\Scripts' which is not on PATH.

which refers to the path C:\Users\atomi\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\Scripts
which does not look entirely like a normal install path. In particular,
the PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0 looks like
scratch/temporary directory name. That may also speak to why the
Scripts folder is not in PATH.

1: Exactly what command did you use to do the pip install of numpy etc?

2: Exactly what command did you use to run the Python where you were
trying import numpy etc?

3: Exactly what is in your %PATH variable?

The usual advice for running pip is to invoke it like this:

 py -m pip install numpy

(to use numpy as an example package). In the above, py is the Python
launcher. You should be using the same command (eg py) to invoke pip
as you used to get the Python environment where you’re trying the
import numpy command. By using the same command you ensure that the
package is installed to the same Python environment which you’re using
to test things.

Can you post a complete transcript of (a) a pip install command,
including the command and all messages and (b) invoking your Python
environment and trying import numpy.

Please copy/paste these into your reply as text between triple
backticks, eg:

 ```
 paste transcript here
 ```

Cheers,
Cameron Simpson cs@cskk.id.au

This is what I used:
C:\Users\atomi>pip install tabulate

Requirement already satisfied: tabulate in c:\users\atomi\appdata\local\packages\pythonsoftwarefoundation.python.3.10_qbz5n2kfra8p0\localcache\local-packages\python310\site-packages (0.9.0)

Sorry for the delay.

Edit: Entered py -m pip install numpy for this result:

Collecting numpy
  Using cached numpy-1.23.5-cp310-cp310-win_amd64.whl (14.6 MB)
Installing collected packages: numpy
  WARNING: The script f2py.exe is installed in 'C:\Users\atomi\AppData\Local\Programs\Python\Python310\Scripts' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed numpy-1.23.5

So… progress?

Just guessing: is not it a path used by Python when it is installed using MS Store or winget?

I downloaded Python 3.10 from python.org a few months ago. But pip never worked for me. Yesterday, I ran python from within cmd.exe which pulled up the Microsoft store’s Python version 3.10.8 download option and I downloaded it.

After that, I was able to use pip. Using pip install as the command, I installed prettytable, tabulate, numpy, and colorama. However, I received cmd warnings for each that they were installed off PATH.

In my editor, I tried import numpy, import tabulate, from prettytable import PrettyTable, and import colorama. For all of them (except colorama, strangely) a ModuleNotFound error was raised.

When I run pip list the following is returned:

C:\Users\atomi>pip list
Package     Version
----------- -------
colorama    0.4.6
numpy       1.23.5
pip         22.3.1
prettytable 3.5.0
tabulate    0.9.0
wcwidth     0.2.5

They are all installed to the same location:

c:\users\atomi\appdata\local\packages\pythonsoftwarefoundation.python.3.10_qbz5n2kfra8p0\localcache\local-packages\python310\site-packages

I reran py -m pip install numpy which installed numpy to:

C:\Users\atomi\AppData\Local\Programs\Python\Python310\Scripts

though it’s not on PATH either (according to cmd).

I’ve been looking up how to change environment variables to fix PATH issues. But I’m wondering if it would be better to uninstall everything and reinstall it.

This is what I used:
C:\Users\atomi>pip install tabulate

Requirement already satisfied: tabulate in c:\users\atomi\appdata\local\packages\pythonsoftwarefoundation.python.3.10_qbz5n2kfra8p0\localcache\local-packages\python310\site-packages (0.9.0)

Ok, this still looks a bit bogus based on the install path. I think
you’re finding the pip command from the wrong place (which is why the
py -m pip version is preferred - always correct for py).

Sorry for the delay.

No worries. I’m in GMT+11.

Edit: Entered py -m pip install numpy for this result:

Collecting numpy
 Using cached numpy-1.23.5-cp310-cp310-win_amd64.whl (14.6 MB)
Installing collected packages: numpy
 WARNING: The script f2py.exe is installed in 'C:\Users\atomi\AppData\Local\Programs\Python\Python310\Scripts' which is not on PATH.
 Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed numpy-1.23.5

So… progress?

That install path looks far more reasonable.

So if you start an interactive Python prompt by typing py, does
import numpy now work?

Example (again, on a Mac, so the command’s a little different from
yours:

 [~]fleet2*> py3
 Python 3.10.6 (main, Aug 11 2022, 13:47:18) [Clang 12.0.0 (clang-1200.0.32.29)] on darwin
 Type "help", "copyright", "credits" or "license" for more information.
 >>> import numpy
 >>>

For clarity, the bits I typed above were py3 to start my Python, and
import numpy to try the import.

Cheers,
Cameron Simpson cs@cskk.id.au

I downloaded Python 3.10 from python.org a few months ago. But
pip never worked for me.

That’s unfortunate. Probably better to uninstall this version if that’s
easy. I’m not a Windows person, so all my advice here is a bit…
abstract.

Yesterday, I ran python from within cmd.exe which pulled up the
Microsoft store’s Python version 3.10.8 download option and I
downloaded it.

Generally a better route - you get the “official” version MS distribute.
The guys involved in that are active in the Python community, BTW, so it
should be pretty current and good.

After that, I was able to use pip. Using pip install as the command, I installed prettytable, tabulate, numpy, and colorama. However, I received cmd warnings for each that they were installed off PATH.

There may be some information here:

though i can’t see something specfic to that warning.

In my editor,

BTW, what’s your editor/IDE?

I tried import numpy, import tabulate, from prettytable import PrettyTable, and import colorama. For all of them
(except colorama, strangely) a ModuleNotFound error was raised.

I’d guess you’ve installed colorama using pip -m install colorama.

When I run pip list the following is returned:

C:\Users\atomi>pip list
Package     Version
----------- -------
colorama    0.4.6
numpy       1.23.5
pip         22.3.1
prettytable 3.5.0
tabulate    0.9.0
wcwidth     0.2.5

Is the listing for py -m pip list any different.

Again, this is on the premise that pip at your command prompt is a
different Python install from that invoked by py. Therefore
different sets of installed packages.

They are all installed to the same location:

c:\users\atomi\appdata\local\packages\pythonsoftwarefoundation.python.3.10_qbz5n2kfra8p0\localcache\local-packages\python310\site-packages

Right. The… sus looking one.

I reran py -m pip install numpy which installed numpy to:

C:\Users\atomi\AppData\Local\Programs\Python\Python310\Scripts

Right. Saner looking. Clearly your pip command is installing to a
different Pythin install area than the area used by py. That will be
why the “plain” pip installs are not available.

though it’s not on PATH either (according to cmd).

Hmm. A Windows person will need to chime in here. But again, unless
you’re invoking these scripts as commands, this doesn’t matter.
import colorama from inside Python code etc will all work fine.

I’ve been looking up how to change environment variables to fix PATH
issues. But I’m wondering if it would be better to uninstall everything
and reinstall it.

I think at the least you need to uninstall the version you got from
python.org. The clean thing is:

  • uninstall both
  • make sure there’s no pip command any more and no working py command
  • install the Python from the Windows Store
  • see if py -m pip and pip then install to the same area
  • report back!

Cheers,
Cameron Simpson cs@cskk.id.au

1 Like

Will do, give me a few hours and I’ll have results to share. Thanks again!

I uninstalled both Pythons and VSCode. Afterwards, pip was still functional but py -m pip was not. The uninstaller left behind iPython, .venv, and vscode folders.

I reinstalled Python 3.11 from MS but it recreated the same sus directories. I wonder if perhaps it’s local to my machine?

Either way, I edited environment variables, adding to Path the install location and script directories, then installed numpy via py -m pip install and did not receive a warning regarding the location being not on PATH. I launched IDLE, import numpy, ran a quick print("Hello World") and it did not raise a ModuleNotFound exception!

I’m hoping this resolves the issue? I didn’t try running any code specific to numpy, but if it was gonna throw an error, it would immediately I assume.

(Edited to fix the code blocks which I couldn’t enter properly on my phone.)

Short follow-up to Cameron’s post at (Pip installing off PATH: should I do something? - #8 by cameron). I try to remember to use pip -V prior to installing stuff using the pip command (vs python -m pip …) just to confirm what pip module I’m using as that location suggests where stuff ought to be placed as a result of pip install. Because every so often, I’m catching myself using the wrong virtual env. I try to trust myself but I know I still ought to verify.

1 Like