How to make pip install -e . work for non-standard layout

I have a project that looks like

project/--setup.py
          +-README.md
          |
          +---src/pkg/__init__.py

If I run pip install -e . in project it all appears to work. However, I cannot import pkg. I assume that’s because the pkg-egg.link file points at project and I have the pkg dir in src not directly parallel to setup.py. Is there a way to control where the egg-info is placed and where the link file points?

My setup.py looks like

from setuptools import setup, find_packages
def main():
    setup(
        name="pkg",
        version='0.0.0',
        license="BSD license (see license.txt for details)",
        description="A dummy thing",
        long_description="""A dummy thing for testing only""",

        author="Modified gluons",
        author_email="nobody@nowhere.org",
        url="https://python.org/",
        packages = find_packages("src"),
        package_dir = {'pkg': "src/pkg"},
        package_data = {'pkg':[
                                '00README.txt',
                                ],
                        },
        )

if __name__=='__main__':
    main()

If I put a pth file in site-packages pointing at project/src then everything appears to work.

I recommend:

setup(
    # ...
    packages=find_packages(where='src'),
    package_dir={'': 'src'},
    # ...
)

You might want to cleanly uninstall the project first. Then do the modification and finally install again (as editable).

Thanks, that seems to work for me.

You can skip all this extra configuration by getting rid of setup.py entirely and just switching to pyproject.toml exclusively. It “just works” then.

1 Like

I’ve been using python for long time and my real setup.py has a lot more than just the build instructions. I find it strange that python prefers to offload its packaging to third party packages.
Perhaps the lack of a clear simple standard way to do distribution has led to the multiple ways of doing stuff.

Looking in my array of python builds I find only 3.12.0a1 has tomlib. So I suppose setup.py will have to endure for a while longer.

Luckily I am getting too old to worry much about this. Explicitly installing the required build tools is very easy and obvious; I will stick with that.

1 Like

Can you show me?

You don’t have 3.11 installed then? tomllib — Parse TOML files — Python 3.11.0 documentation
Older Pythons will have the packaging tools use tomli, transparently from the user’s POV.

mea culpa I do see

LOCAL/3.12.0a1/lib/python3.12/tomllib
LOCAL/3.11.0/lib/python3.11/tomllib

but I’m building/testing 3.7-3.11 and checking 3.12 for issues. I’ll be away before 3.10 is EOL

$ ls venv/lib/python3.7/site-packages/pip/_vendor/
[...]
tomli

So pip brings its own TOML parser along anyway, just in case it’s
needed.

https://packaging.python.org/en/latest/tutorials/packaging-projects/