How to make pip install -e . work for non-standard layout

Packaging
1

I have a project that looks like

project/--setup.py
          +-README.md
          |
          +---src/pkg/__init__.py

If I run pip install -e . in project it all appears to work. However, I cannot import pkg. I assume that’s because the pkg-egg.link file points at project and I have the pkg dir in src not directly parallel to setup.py. Is there a way to control where the egg-info is placed and where the link file points?

My setup.py looks like

from setuptools import setup, find_packages
def main():
    setup(
        name="pkg",
        version='0.0.0',
        license="BSD license (see license.txt for details)",
        description="A dummy thing",
        long_description="""A dummy thing for testing only""",

        author="Modified gluons",
        author_email="nobody@nowhere.org",
        url="https://python.org/",
        packages = find_packages("src"),
        package_dir = {'pkg': "src/pkg"},
        package_data = {'pkg':[
                                '00README.txt',
                                ],
                        },
        )

if __name__=='__main__':
    main()

If I put a pth file in site-packages pointing at project/src then everything appears to work.

2

I recommend:

setup(
    # ...
    packages=find_packages(where='src'),
    package_dir={'': 'src'},
    # ...
)

You might want to cleanly uninstall the project first. Then do the modification and finally install again (as editable).

3

Thanks, that seems to work for me.

4

You can skip all this extra configuration by getting rid of setup.py entirely and just switching to pyproject.toml exclusively. It “just works” then.

2 Likes
5

I’ve been using python for long time and my real setup.py has a lot more than just the build instructions. I find it strange that python prefers to offload its packaging to third party packages.
Perhaps the lack of a clear simple standard way to do distribution has led to the multiple ways of doing stuff.

Looking in my array of python builds I find only 3.12.0a1 has tomlib. So I suppose setup.py will have to endure for a while longer.

Luckily I am getting too old to worry much about this. Explicitly installing the required build tools is very easy and obvious; I will stick with that.

1 Like
6

Can you show me?

You don’t have 3.11 installed then? tomllib — Parse TOML files — Python 3.11.7 documentation
Older Pythons will have the packaging tools use tomli, transparently from the user’s POV.

7

mea culpa I do see

LOCAL/3.12.0a1/lib/python3.12/tomllib
LOCAL/3.11.0/lib/python3.11/tomllib

but I’m building/testing 3.7-3.11 and checking 3.12 for issues. I’ll be away before 3.10 is EOL

8 
$ ls venv/lib/python3.7/site-packages/pip/_vendor/
[...]
tomli

So pip brings its own TOML parser along anyway, just in case it’s
needed.

9

https://packaging.python.org/en/latest/tutorials/packaging-projects/