Package Relative Imports - double dot doc example not working

Hi there, I’ve been redirected here by our former BDFL :smile: (see Issue 43005: Package Relative Imports - double dot doc example not working - Python tracker) here is my problem :

I’ve been trying to do a double dot import without success, I’ve tried going back to the official documentation in order to figure it out however even the example shown does not work.

Official example : 5. The import system — Python 3.9.1 documentation

The following instructions mimic pretty well the doc example :

mkdir package
cd package
touch __init__.py
mkdir subpackage1
touch subpackage1/__init__.py
mkdir subpackage2
touch subpackage2/__init__.py
echo "eggs = 42" > subpackage2/moduleZ.py
echo "from ..subpackage2.moduleZ import eggs" > subpackage1/moduleX.py
python subpackage1/moduleX.py
cd subpackage1
python moduleX.py

cd ..
echo "from subpackage1 import moduleX" > moduleA.py
python moduleA.py

However I get 3 times the following error :
from …subpackage2.moduleZ import eggs
ImportError: attempted relative import beyond top-level package

Even though the docs say “In […] subpackage1/moduleX.py […] the following are valid relative imports: […] from …subpackage2.moduleZ import eggs”.

I have no idea how to fix the doc as I would love to do “from …subpackage2.moduleZ import eggs” from “subpackage1/moduleX.py” since that’s the structure I’m trying to set up for a Python project of mine but obviously it does not seem to work.

My system uses Python 3.9 and I’ve used Docker to try this scenario on Python 3.8, 3.5 and even 3.2: it does not work either. I’ve also tried on another machine and asked a friend to try, no luck.

I’m hoping someone here can explain how double dot imports work and help fix the documentation accordingly.

The error message does not say that your use of relative import is wrong:

It says what you try to import is not in your current package. When you execute touch subpackage1/__init__.py you make subpackage1 into a package. You cd into the directory, which means that for import you work from subpackage1, not your original package. So
..subpackage2.moduleZ says go up one directory, which is no longer in the package you are working in, although it would be in package.

If I’m right, here’s how your package hierarchy looks like with the additional instructions inside the scripts:

package
     |----- __init__.py
     |----- moduleA.py: from subpackage1 import moduleX
     |----- subpackage1 (dir)
                  |----- __init__.py
                  |----- moduleX.py: from ..subpackage2.moduleZ import eggs
     |----- subpackage2 (dir)
                  |----- __init__.py
                  |----- moduleZ.py: 'eggs' == 42

python moduleA.py

I believe your issue has to do with how you’re running moduleA.py. Try running it as a package after moving to the parent directory of package/:

cd ..
# so current directory (pwd) ia the root of package/
python -m package.moduleA # .py is omitted

To elaborate Menno Hölscher’s answer, Python reads the __name__ attribute for each module to know its relative location, or more specifically, whether the module is part of a parent package or not. When a module is executed directly, like you did, python moduleA.py, the __name__ is set to __main__ which is to tell Python to execute moduleA.py as a top-level script. So Python doesn’t know moduleA.py is part of package, and relative imports found inside fail.

If moduleA.py has to be run directly with python ./package/moduleA.py, you can define __package__ attribute to the package name that is relatively imported. See PEP 366 for details.

python -m: 1. Command line and environment — Python 3.9.1 documentation

First of all : thanks for looking at my issue, I appreciate it. Now regarding the error message, I now realize I probably got different ones : my main problem being that I can’t seem to find any way to make it work.

I’m obviously missing some understanding of this whole situation so sorry if this sounds stupid but is your suggestion that I remove subpackage1/__init__.py ? If I do that then try the following for example :

cd package/subackage1
python
>>> from ..subpackage2.moduleZ import eggs
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: attempted relative import with no known parent package

I’ve tried this many different ways always with some import error or another.

Can we agree that detailed examples are good in documentation and that they should always work ? In 5. The import system — Python 3.9.1 documentation the example isn’t detailed so I can’t just copy paste it : how would you go about recreating the doc example ? I think a working example would really help me understand the system.

Thank you as well for looking at this. However package -m package.moduleA from the parent folder does not work either unfortunately : I get the following error :

    from subpackage1 import moduleX
ModuleNotFoundError: No module named 'subpackage1'

I have never user __package__ and don’t quite understand the whole PEP 366. If you were use how would it be exactly ? Add __package__ = "package" to package/__init__.py ? It would really help me to have an example in the official documentation that I could simply copy-paste and play around with.

Having a problem with relative imports I had first look at StackOverflow and quickly found a link to the official documentation so I figured I’d find my answer there which how I then found myself here looking to make the doc example work.

I’ve gone back to read in more details:
StackOverflow : Relative imports for the billionth time
StackOverflow : How to do relative imports in Python?

Looks like you were on the right track @culdesac, if I try from above the package folder the following command: python -m package.subpackage1.moduleX.

This doesn’t quite work out for me (sorry my real problem was actually getting FastAPI, SQLAlchemy and Alembic to work together in a simple, minimal project structure) however now that I have a working example I can probably figure something out.

Looking at the StackOverflow answers it seems I’m not the only one finding the current Python relative import system to be pretty “limited” : however this has long been the case and seems unlikely to change in the near futur.

Nevertheless, looking to be productive if I wanted to suggest a change in the official documentation like for example "You can run this example with python -m package.subpackage1.moduleX", how would I do so ?

First: you are not stupid, I also made a mistake in my answer. It is different from what I remembered: I never use relative imports.

Which points me at the first solution: don’t use relative imports.

My file structure is:

python
|
- test1
  |
  - sub1
  |
  - sub2

I have created a file fromimp1.py in sub1 that contains

from sub2 import toimp

print(“Imported!. In sub1”)

and in sub2 file toimp.py that contains

print(“From sub2”)

Lets go:

(venv) mennoh@Vecht12-2:~/python/test1> python
Python 3.6.12 (default, Dec 02 2020, 09:44:23) [GCC] on linux
Type “help”, “copyright”, “credits” or “license” for more information.

import sub1, sub2
from sub1 import fromimp1
From sub2
Imported!. In sub1

This does what you were after, so if you use this, no worries about relative imports. This is my favourite solution, so I could not resist mentioning it.

With relative imports I have changed fromimp1 to:

from …sub2 import toimp

print(“Imported!. In sub1”)

In the repl we go like here:

(venv) mennoh@Vecht12-2:~/python> python

Python 3.6.12 (default, Dec 02 2020, 09:44:23) [GCC] on linux
Type “help”, “copyright”, “credits” or “license” for more information.

from test1 import sub1, sub2
from test1.sub1 import fromimp1
From sub2
Imported!. In sub1

Now it works. Why?

The documentation refers to “the top package”. The top package is not referring to the file system, but to how you load it. I have started python from the directory above test1 so I have to write from test1 import sub1, sub2 to load test1 as the top package. In your actions package is loaded as __main__, not as a package so you cannot pass through it to reach sub2. I also did that in my previous answer, so that is not going to work.

And no, I did not want you to remove __init__.py, it is immaterial to this if sub1 is a package or a module.

I’m glad to see you’ve made progress and might have figured it out by now.

Python documentation isn’t known for its beginner friendliness, and like yourself, many have pointed out the confusion in relative imports in comments of your links. If you want to bring this issue to the attention of the core maintainers, you can start a new thread in Ideas category, there’s a dedicated documentation tag. If you’re looking to open an issue in Python issue tracker, and eventually make a PR, there’s a documentation contribution guide.