Struggling to avoid namespace packages

Python Help
1

In trying to simplify a repo which contained three nonoverlapping packages, I find myself stuck with something called a namespace package. It has a __file__ attribute of None:

>>> csvprogs
<module 'csvprogs' (namespace) from ['/Users/skip/miniconda3/envs/python312/lib/python3.12/site-packages/csvprogs']>
>>> csvprogs.__file__
>>>

but it has a subpackage named csvprogs which does have a useful __file__ attribute:

>>> import csvprogs.csvprogs
>>> csvprogs.csvprogs.__file__
'/Users/skip/miniconda3/envs/python312/lib/python3.12/site-packages/csvprogs/csvprogs/__init__.py'

I don’t want that subpackage csvprogs. I’ve got no packages declaration in pyproject.toml. python -m build seems to be figuring all this out on its own. I’m using setuptools.build_meta as the build backend.

I can’t provide a reliable directory structure at this point either. In trying to reorganize my directory tree, I seem to have git mv’d my way to a f*cked up directory tree as well. If the top level directory of my Git repo is ~/src/csvprogs, what subdirectory should contain the scripts/modules?

Any help would be appreciated.

2

Well this is probably what’s going wrong…what is the directory structure? setuptools will try to guess what you want but you might need to tell it where your code is.

3

That’s the problem. Since I last built and installed my stuff, I tried moving things around and messed that up. I can give you the last directory structure from which I installed the package. This is the wheel built by python -m build with some of the Python files elided.

% unzip -t dist/csvprogs-1.1.0-py3-none-any.whl 
Archive:  dist/csvprogs-1.1.0-py3-none-any.whl
    testing: csvprogs/csvprogs/__init__.py   OK
    testing: csvprogs/csvprogs/args.py   OK
    testing: csvprogs/csvprogs/atr.py   OK
    testing: csvprogs/csvprogs/bars.py   OK
    testing: csvprogs/csvprogs/csv2csv.py   OK
    testing: csvprogs/csvprogs/csv2json.py   OK
    ...
    testing: csvprogs/csvprogs/square.py   OK
    testing: csvprogs/csvprogs/take.py   OK
    testing: csvprogs/csvprogs/xform.py   OK
    testing: csvprogs/csvprogs/xls2csv.py   OK
    testing: csvprogs/tests/test_csv2csv.py   OK
    testing: csvprogs/tests/test_csvfill.py   OK
    testing: csvprogs/tests/test_ewma.py   OK
    testing: csvprogs/tests/test_square.py   OK
    testing: csvprogs/tests/test_xform.py   OK
    testing: csvprogs-1.1.0.dist-info/LICENSE   OK
    testing: csvprogs-1.1.0.dist-info/METADATA   OK
    testing: csvprogs-1.1.0.dist-info/WHEEL   OK
    testing: csvprogs-1.1.0.dist-info/entry_points.txt   OK
    testing: csvprogs-1.1.0.dist-info/top_level.txt   OK
    testing: csvprogs-1.1.0.dist-info/RECORD   OK

I think I figured it out. The extra level of csvprogs which remained from the older structure was the culprit. It used to be

~/src/csvprogs
    csvprogs
    data_filters
    mpl

with corresponding package bits in each of the three subdirectories. I git mv’d the contents of data_filters and mpl into csvprogs but failed to remove the extra directory layer. After building and installing, I now get what I expected:

>>> import csvprogs
>>> dir(csvprogs)
('[args]', '[atr]', '[bars]', '[csv2csv]', '[csv2json]', '[csv2xls]', '[csvcat]', '[csvcollapse]', '[csvfill]', '[csvmerge]', '[csvplot]', '[csvsort]', '[dsplit]', '[ewma]', '[extractcsv]', '[filter]', '[html2csv]', '[hull]', '[interp]', '[keltner]', '[mean]', '[mvavg]', '[regress]', '[roundtime]', '[sharpe]', '[shuffle]', '[sigavg]', '[spline]', '[square]', '[take]', '[xform]', '[xls2csv]', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__path__', '__spec__')
>>> import csvprogs.csvsort
>>> dir(csvprogs)
('[args]', '[atr]', '[bars]', '[csv2csv]', '[csv2json]', '[csv2xls]', '[csvcat]', '[csvcollapse]', '[csvfill]', '[csvmerge]', '[csvplot]', '[dsplit]', '[ewma]', '[extractcsv]', '[filter]', '[html2csv]', '[hull]', '[interp]', '[keltner]', '[mean]', '[mvavg]', '[regress]', '[roundtime]', '[sharpe]', '[shuffle]', '[sigavg]', '[spline]', '[square]', '[take]', '[xform]', '[xls2csv]', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__path__', '__spec__', 'csvsort')
>>> csvprogs.__file__
'/Users/skip/miniconda3/envs/python312/lib/python3.12/site-packages/csvprogs/__init__.py'

(The bracketed entries represent modules which have yet to be imported – a personal dir() tweak.)

2 Likes
4

Yeah I was asking to see what the structure of the problematic installation was. As you discovered: if setuptools sees something that looks like a namespace package it will assume that’s what you wanted to do.

For a standard package it wants to see

- src
  - csvprogs
    - __init__.py

The additional layer (without an __init__.py at that level) suggested you were creating a namespace package.