Two similar imports one is failing

I am getting following when I try import, any idea???

[root@localhost PyTorchStepByStep]# python3 -c "import data_generation.square_sequences"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'data_generation.square_sequences'

I made similar boilerplate code dir1/dir1a.py and it works:

python3 -c "import dir1.dir1a"
[root@localhost PyTorchStepByStep]# cat dir1/dir1a.py


def dir1a():
    print("dir1a:")

[root@localhost PyTorchStepByStep]# git remote -v
origin  https://github.com/dvgodoy/PyTorchStepByStep.git (fetch)
origin  https://github.com/dvgodoy/PyTorchStepByStep.git (push)

(By the way, please don’t use the root account directly like this - you should have a user-level account, and sudo only for commands that need it. Otherwise, you increase the potential damage that any malicious software could do to your computer.)

Well, what exactly are you expecting to happen instead? And why - what steps did you take to try to make that happen?

Similar to what?

When you try the non-working example, are you trying to import your own code? Or a library?

there is a folder called data_generation and in which square_sequences.py module defined but it is not importing those are modules I am leverating from github repository I was studying.
I created small similar example which is working.

In order to provide any useful help here, we would need to know the full file structure under your working directory. By chance I noticed you happened to include an otherwise unrelated command output that indicates you’re in a local copy of a a cloned GitHub repository, dvgodoy/PyTorchStepByStep, I cloned that repo, cd’ed to the root directory (as you appear to be) and ran python -c "import data_generation.square_sequences". However, I cannot reproduce the error you’re getting on either Python 3.7 or Python 3.11; the module imports successfully.

Therefore, it seems there is something different in your working copy of the repo that is causing the issue—the most obvious being possibly having deleted or moved the module or its parent directory—or else something different with your environment/configuration. Please ensure you are in the root of the repository, confirm your specific python version (share the output of python --version) and try running with the -E and -s flags, that will ignore local environment variables and site configuration: python3 -Esc "import data_generation.square_sequences".

2 Likes

What version of Python are you using? Please provide that in every new post you make. It helps us that much more.

Yes, there seems environment issue. It is running from docker container with centos 9 and import fails. I exit docker ( and create another copy outside docker in normal bare-metal environment) and and import is successfully. When comparing sys.path on inside and outside docker, I see syspath inside docker does not have current directory entires “.” whereas outside docker it has:

outside docker:

python -c “import sys ; print(sys.path)”
[‘’, ‘/usr/lib64/python39.zip’, ‘/usr/lib64/python3.9’, ‘/usr/lib64/python3.9/lib-dynload’, ‘/home/amd/.local/lib/python3.9/site-packages’, ‘/usr/local/lib/python3.9/site-packages’, ‘/usr/lib64/python3.9/site-packages’, ‘/usr/lib/python3.9/site-packages’]

inside docker:
[‘/usr/lib64/python39.zip’, ‘/usr/lib64/python3.9’, ‘/usr/lib64/python3.9/lib-dynload’, ‘/usr/local/lib64/python3.9/site-packages’, ‘/usr/local/lib/python3.9/site-packages’, ‘/root/gg/git/benchmark’, ‘/usr/lib64/python3.9/site-packages’, ‘/usr/lib/python3.9/site-packages’]

But that still did not explain because, my small example should have failed if sys.path does not contain current dir: “”. I added anyways but problem essentially remains the same.

python --version
Python 3.9.18

Makes no sense why would using -Esc parameter would fail numpy import whereas -c would fail the other one??

[root@localhost PyTorchStepByStep]# python3 -Esc “import data_generation.square_sequences”
Traceback (most recent call last):
File “”, line 1, in
File “/root/gg/git/codelab/gpu/ml/tf/tf-from-scratch/3/PyTorchStepByStep/data_generation/square_sequences.py”, line 1, in
import numpy as np
ModuleNotFoundError: No module named ‘numpy’
[root@localhost PyTorchStepByStep]# python3 -c “import data_generation.square_sequences”
Traceback (most recent call last):
File “”, line 1, in
ModuleNotFoundError: No module named ‘data_generation.square_sequences’

y
Ok, i got it:

E: ignore PYTHON env.
s: ignore user env.
WIth that numpy fails.
[amd@localhost git] python --help | grep "\-E" -E : ignore PYTHON* environment variables (such as PYTHONPATH) -I : isolate Python from the user's environment (implies -E and -s) [amd@localhost git] python --help | grep “-s”
-s : don’t add user site directory to sys.path; also PYTHONNOUSERSITE
[amd@localhost git]$ python --help | grep “-c”
usage: python [option] … [-c cmd | -m mod | file | -] [arg] …
-c cmd : program passed in as string (terminates option list)

I tried using -v switch output is huge, too verbose with thousands of lines. It is hard to discern anything. I see -v verboseness can be controller by PYTHONVERBOSE where 0 is default (with no debug output). But as soon as I set to 1, it is same as using -v huge output.
I put snippet below, I will try to see how i can attach whole log:

1979-# code object from ‘/usr/local/lib64/python3.9/site-packages/scipy/stats/pycache/kde.cpython-39.pyc’
1980-import ‘scipy.stats.kde’ # <_frozen_importlib_external.SourceFileLoader object at 0x7f7b75151df0>
1981-# /usr/local/lib64/python3.9/site-packages/scipy/stats/pycache/morestats.cpython-39.pyc matches /usr/local/lib64/python3.9/site-packages/scipy/stats/morestats.py
1982-# code object from ‘/usr/local/lib64/python3.9/site-packages/scipy/stats/pycache/morestats.cpython-39.pyc’
1983-import ‘scipy.stats.morestats’ # <_frozen_importlib_external.SourceFileLoader object at 0x7f7b75151640>
1984-# /usr/local/lib64/python3.9/site-packages/scipy/stats/pycache/mstats_basic.cpython-39.pyc matches /usr/local/lib64/python3.9/site-packages/scipy/stats/mstats_basic.py
1985-# code object from ‘/usr/local/lib64/python3.9/site-packages/scipy/stats/pycache/mstats_basic.cpython-39.pyc’
1986-import ‘scipy.stats.mstats_basic’ # <_frozen_importlib_external.SourceFileLoader object at 0x7f7b751606d0>
1987-# /usr/local/lib64/python3.9/site-packages/scipy/stats/pycache/mstats_extras.cpython-39.pyc matches /usr/local/lib64/python3.9/site-packages/scipy/stats/mstats_extras.py
1988-# code object from ‘/usr/local/lib64/python3.9/site-packages/scipy/stats/pycache/mstats_extras.cpython-39.pyc’
1989-import ‘scipy.stats.mstats_extras’ # <_frozen_importlib_external.SourceFileLoader object at 0x7f7b751602e0>
1990-# /usr/local/lib64/python3.9/site-packages/scipy/stats/pycache/mvn.cpython-39.pyc matches /usr/local/lib64/python3.9/site-packages/scipy/stats/mvn.py
1991-# code object from ‘/usr/local/lib64/python3.9/site-packages/scipy/stats/pycache/mvn.cpython-39.pyc’
1992-import ‘scipy.stats.mvn’ # <_frozen_importlib_external.SourceFileLoader object at 0x7f7b751600d0>
1993-# /usr/local/lib64/python3.9/site-packages/scipy/stats/pycache/statlib.cpython-39.pyc matches /usr/local/lib64/python3.9/site-packages/scipy/stats/statlib.py
1994-# code object from ‘/usr/local/lib64/python3.9/site-packages/scipy/stats/pycache/statlib.cpython-39.pyc’
1995-import ‘scipy.stats.statlib’ # <_frozen_importlib_external.SourceFileLoader object at 0x7f7b75160880>
1996-# /usr/local/lib64/python3.9/site-packages/scipy/stats/pycache/stats.cpython-39.pyc matches /usr/local/lib64/python3.9/site-packages/scipy/stats/stats.py
1997-# code object from ‘/usr/local/lib64/python3.9/site-packages/scipy/stats/pycache/stats.cpython-39.pyc’
1998-import ‘scipy.stats.stats’ # <_frozen_importlib_external.SourceFileLoader object at 0x7f7b75160400>
1999-import ‘scipy.stats’ # <_frozen_importlib_external.SourceFileLoader object at 0x7f7c17bb51c0>
2000-import ‘data_generation.dgen’ # <_frozen_importlib_external.SourceFileLoader object at 0x7f7cb8ffe7f0>
2001-# /usr/local/lib/python3.9/site-packages/data_generation/pycache/papers_dataset.cpython-39.pyc matches /usr/local/lib/python3.9/site-packages/data_generation/papers_dataset.py
2002-# code object from ‘/usr/local/lib/python3.9/site-packages/data_generation/pycache/papers_dataset.cpython-39.pyc’
2003-import ‘data_generation.papers_dataset’ # <_frozen_importlib_external.SourceFileLoader object at 0x7f7b75160910>
2004-import ‘data_generation’ # <_frozen_importlib_external.SourceFileLoader object at 0x7f7cb8ffe400>
2005-Traceback (most recent call last):
2006- File “”, line 1, in
2007- File “”, line 1007, in _find_and_load
2008- File “”, line 984, in _find_and_load_unlocked
2009:ModuleNotFoundError: No module named ‘data_generation.square_sequences’

OK, i got some breakthrough:!!!
I recall to import package you have to create init.py in that folder and then it works!
I am giong to move on now, however:
It still mystery while it explains why inside container it did not work by fixing by creating init.py in data_generation folder but it does not explain why when I exit container, i am able to do same import with no init.py present!??