Can't import editable packages

Python Help
1

Hi,

Note: I rewrote this question as the problem seems to be broader than I initially thought. My old post is kept below.

I can’t import an editable package.

I did the following:

  1. venv .venv and activate it
  2. git clone https://github.com/rm-hull/luma.lcd.git
  3. python -m pip install -e ./luma.lcd
  4. Create a test.py file in the root dir and try to import luma.lcd

luma.lcd cannot be imported but its (non-editable) dependency luma.core can.

sys.path returns:

['', '/usr/lib/python311.zip', '/usr/lib/python3.11', '/usr/lib/python3.11/lib-dynload', '/home/mathieu/Kodi_rpi5/lcd_display/luma2/.venv/lib/python3.11/site-packages', '__editable__.luma_lcd-2.11.0.finder.__path_hook__']

Old post:
I’m trying to set up a virtual environment where I can contribute to two packages from separate Github repositories where one depends on the other. I’ve cloned the child package and installed it in editable mode (pip install -e). The parent package is in the virtual environment’s site-packages. I would like to install it in editable mode as well from the Github repository, and have it recognized as a dependency of the child package.

The project is luma.lcd and its dependency luma.core

Thanks for your help.

2

How did you set up the virtual environment?

The way I’d do this is would be:

  1. Install the higher-level package first, as an editable package, and include its dependencies in the venv
  2. Install any additional dependencies of the child package
  3. Install the child package in editable mode, but use --no-deps to avoid overwriting dependencies
3

I created the virtual environment with the command:

python -m venv .venv

I did what you suggested, the packages and their dependencies are installed, but I can’t import them in a python script in the virtual environment (it cannot resolve the import).

I suspect it has something to do with the following namespace having conflicting definitions ?

The directory structure is:

.
├── luma.core
│   ├── luma
│   │   └── core
│   │       └── (…)
│   └──luma.core.egg-info
├── luma.lcd
│   ├── luma
│   │   └── lcd
│   │       └── (…)
│   └── luma.lcd.egg-info
├── test.py
└── .venv
    └── lib
        └── python3.11
            └── site-packages
                ├── luma
                ├── luma_core-2.5.2.dist-info
                ├── luma_lcd-2.11.0.dist-info
                ├── distutils-precedence.pth
                ├── __editable___luma_core_2_5_2_finder.py
                ├── editable.luma_core-2.5.2.pth
                ├── editable___luma_lcd_2_11_0_finder.py
                └──__editable.luma_lcd-2.11.0.pth

The files editable___luma_*_finder.py contain the strings:

MAPPING: dict[str, str] = {'luma': '/home/user/luma/luma.core/luma'}
NAMESPACES: dict[str, list[str]] = {'luma': ['/home/user/luma/luma.core/luma']}

and:

MAPPING: dict[str, str] = {'luma': '/home/user/luma/luma.lcd/luma'}
NAMESPACES: dict[str, list[str]] = {'luma': ['/home/user/luma/luma.lcd/luma']}
4

Hm that might be the issue, I’m not conversant with how those files actually work.

It might be worth opening an issue on the repo in question to see how they recommend setting it up.

1 Like
6

Even if I install one of the packages as editable and the other as a regular dependency, the editable package cannot be imported. So this may not have anything to do with the project’s structure but something could be wrong with my venv.

I started from scratch in a new virtual environment:

  1. venv .venv and activate it
  2. git clone https://github.com/rm-hull/luma.lcd.git
  3. python -m pip install -e ./luma.lcd
  4. Create a test.py file in the root dir and try to import luma.lcd

luma.lcd cannot be imported but its (non-editable) dependency luma.core can.

sys.path returns:

['', '/usr/lib/python311.zip', '/usr/lib/python3.11', '/usr/lib/python3.11/lib-dynload', '/home/mathieu/Kodi_rpi5/lcd_display/luma2/.venv/lib/python3.11/site-packages', '__editable__.luma_lcd-2.11.0.finder.__path_hook__']
7

I found this issue that led me to reinstall the package using an older version of setuptools that doesn’t create the file __editable__.luma_lcd-2.11.0.finder but relies on some older mechanics. The import now works in VS Code. Still not sure what’s wrong with the newer setuptools.

8

It turns out, the code runs fine. VS Code’s pylance extension shows a hint under the import statement saying the module cannot be resolved, and I trusted it, but running the code produces no error. I’m sorry for wasting everyone’s time.

1 Like