Python doesn't find local module although it is installed and its location exists in sys.path

Python Help
1

I installed a local module using the pyproject.toml approach by following the great tutorial by RealPython: Packaging Your Python Code With pyproject.toml | Complete Code Conversation - YouTube

Although the module seems to be locally installed, I am unable to invoke it from outside the project directory. I have spent about two hours trying out various things, reading forums, watching videos, etc. to understand my mistake, but I have not found anything.

Can someone help me figure out the next steps I could take to figure this out?

My pyproject.toml looks like this:

[build-system]
requires = ["setuptools", "setuptools-scm"]
build-backend = "setuptools.build_meta"

[project]
name ="incometax"
version = "0.0.1"
dependencies = [
    "pytest",
    "prospector[with_everything]",
    "black",
    "pandas",
]

Here’s my sys.path: (Note the last entry matches the location in pip show below)

C:\
(venv_incometax) λ python
Python 3.11.1 (tags/v3.11.1:a7a450f, Dec  6 2022, 19:58:39) [MSC v.1934 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path
['', 'C:\\WORK\\python', 'C:\\Users\\aravi\\AppData\\Local\\Programs\\Python\\Python311\\python311.zip', 'C:\\Users\\aravi\\AppData\\Local\\Programs\\Python\\Python311\\DLLs', 'C:\\Users\\aravi\\AppData\\Local\\Programs\\Python\\Python311\\Lib', 'C:\\Users\\aravi\\AppData\\Local\\Programs\\Python\\Python311', 'C:\\WORK\\dragondive\\heavens-arena\\venv_incometax', 'C:\\WORK\\dragondive\\heavens-arena\\venv_incometax\\Lib\\site-packages']
>>> exit()

Here’s the pip show of my local module:

C:\
(venv_incometax) λ pip show incometax
Name: incometax
Version: 0.0.1
Summary:
Home-page:
Author:
Author-email:
License:
Location: C:\WORK\dragondive\heavens-arena\venv_incometax\Lib\site-packages
Requires: black, pandas, prospector, pytest
Required-by:

However, python doesn’t find the module:

C:\
(venv_incometax) λ python -m incometax
C:\WORK\dragondive\heavens-arena\venv_incometax\Scripts\python.exe: No module named incometax
2

What commands did you use to build the package?

What commands did you use to install the package?

Does venv_incometax\Lib\site-packages actually contain a package called incometax? If so, does that package contain the modules you would expect?

3

I created a python venv, then from within that venv, I called python -m pip install . (in the directory where pyproject.toml exists) to install the package. I am able to invoke the module properly from within that directory.

In the site-packages, there’s a directory named incometax-0.0.1.dist-info/, with the below files. It doesn’t look much different from the other modules in that directory (although I’m not sure if I’m looking at the right things):

direct_url.json
INSTALLER
METADATA
RECORD
REQUESTED
top_level.txt
WHEEL
4

That’s because your working directory contains a directory called incometax. In a situation like this

ls
incometax  pyproject.toml

python will happily import the incometax directory if invoked in this location:

Python 3.8.10 (default, Mar 13 2023, 10:26:41) 
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import incometax
>>>

That’s just metadata, the actual package would just be called incometax. If that’s missing, that explains why you can’t import it.

Did you get any warnings when installing the package? What version of pip are you using?

1 Like
5

That’s just metadata, the actual package would just be called incometax . If that’s missing, that explains why you can’t import it.

Ah I see, this must be it. It’s still not entirely clear to me though why both pip freeze and pip show show that package. I will investigate this further and update here when I find out what went wrong.

Did you get any warnings when installing the package?

No. Now I also did a clean build from scratch, but there were no warnings. I followed the RealPython tutorial, paying attention to their outputs and comparing them with mine, but I didn’t find anything unusual.

What version of pip are you using?

pip 22.3.1 (python 3.11)

6

Weird. FWIW, I tested this with a minimal package in a fresh venv, and it worked:

mkdir testproj
cd testproj
mkdir incometax
echo "print('Tax')" > incometax/incometax.py
nano pyproject.toml  # Identical to your pyproject.toml
python -m venv .venv
.venv/bin/python -m pip install .
[output redacted]
cd ..
testproj/.venv/bin/python
>>> from incometax import incometax
'Tax'

I don’t know why it’s not working for you.

7

Thank you so much for pointing me in the right direction, @abessman. I figured out (again) that I am an idiot. :smiley:

I had the following lines further down in my pyproject.toml:

[tool.setuptools.packages.find]
where = [
    "incometax/src"
]

After your comment above, I read the documents carefully again, and found this:

Automatic discovery will only be enabled if you don’t provide any configuration for packages and py_modules . If at least one of them is explicitly set, automatic discovery will not take place.

Ironically, those lines are blockquoted and marked as important, but I had ignored them due to banner blindness.

I replaced those lines in my pyproject.toml with the following, and now it works properly:

[tool.setuptools.packages]
find = {}

I’m using this project for self-learning the pyproject.toml based approach as distutils is planned for removal in python 3.12. This mistake was a useful learning for me.

1 Like