Adding contents to top-level native namespace

Python Help
1

I’m building several packages which are developed in separate repos, but all share the same native namespace, top.

import top.package1
import top.package2
import top.package3

I’d like to add a docstring, and a __main__.py to top to help direct users to all of the sub packages, common usage, etc.

Is there any way to do this?

2

So you are using the identifier top as a namespace package, right?

Attention: The original suggestion at the end of this post is misleading.

I am leaving it here collapsed for reference. It will work only if the real package is not before any namespace package in the Python path. When the real package (containing __init__.py) is found then the search is finished. The remaining namespace packages would be ignored.

Click to show the original misleading suggestion.

You will need to create the real import package top (containing __init__.py) so you have to decide in which repo you will put it. If one of your packages can be considered as a main one and will be always installed before the others, put it there. If not, create a new one.

The directory structure would be:

pyproject.toml        # definitions for the distribution package
src/                  # It is best practice to have the code directory structure here
    top/
        __init__.py   # This file makes the top package a real import package.
        __main__.py

You will put the docstring inside __init__.py.

2 Likes
3

If I read PEP 420 correctly, there can not be an __init__.py file in top.

Namespace packages cannot contain an __init__.py.

I am not sure about a __main__.py file, or any other kind of file in top, it is not clear to me. My interpretation is that top can not contain any file, only sub-directories (sub-packages).

4

Yes that is right but that is exactly what we need to do - create the real top package in one of the repositories. In the other ones it will still exist as a namespace package though.

Namespace packages can be used to “overlay” the real package.

Correcting edit: This is not the intended use-case for namespace packages. It is not recommended because the “overlay” would work only in specific cases.

5

I would need to dive into this topic again to be certain, I do not recall any of the details anymore. Anyway, as far as I can recall it is not a good idea to have an initializer in top. I think it is this post that helped me understand the whole topic better at the time.

6

Okay it seems like this is either…

  1. Not supported, and this “overlay” behavior is undefined and could change at any time, or…
  2. The “overlay” behavior is defined, but this use case isn’t encouraged.

I can live with the second option! I do think this is an important distinction. Does anyone think either option above describes the implementation of Python today?

7

Some more reading:

Note especially this comment in the 1st thread from the author of PEP 420 (read the whole thread for context of course):

My understanding is that it will most likely work (at least for now), but it is not the way it was intended to be used and may break today already under some specific conditions (installations spread over different sys.path entries for example).

2 Likes
8

I am sorry for the misinformation. I was convinced that one of the main use-cases of namespace packages was to overlay an existing regular package. I think I read this in some tutorial. Now I am disappointed :frowning:

I will add a warning to my wrong suggestion.

Thank you for the references. I did not test it yet but now I understand it that my suggestion would work only if the Python path would not contain any directory with a namespace package after the directory with the regular package. It would be foolish to count on that.

It is a pity that the official documentation contains only a short note about namespace packages in the glossary. The PEP is too long for normal Python users to function as a documentation.

3 Likes