How do I package "everything" in the directory containing `pyproject.toml`

Python Help
1

Hi all,

After spending considerable time trying to locate relevant information in both PEPs (517, 518, 621 off the top of my head) and “Python Packaging Guide”, I still find myself bewildered by simultaneously too much information and not enough of exactly what I seem to be after.

I have a non-Python software repository with a tests directory we decided should contain Pytest-discoverable test modules to test said software, writing both unit- and black-box tests. Since the software itself has nothing to do with Python, there’s obviously no pyproject.toml governing it, however since tests is a Pytest project, and requires dependencies to be satisfied (at least pytest), I thought it’d be fitting for it to have its own pyproject.toml. All well and good, but the tests are just comprised of various module files directly in the tests directory, like test_foo.py and test_bar.py. I imagine the directory may contain package directories, longer-term, but in any case I can’t seem to find a way to have python -m build or pip install -e . (with tests as CWD, obviously) to just package the module files into the wheel and the sdist files, with the following pyproject.toml that I have cooked up just for the build to succeed (without the setuptools-specific sections, Pip and python -m build complain my project layout isn’t standard enough for it to proceed with assumptions):

[project]
name = "foo-tests"
dependencies = [
        "pytest"
]
dynamic = [ "version" ]
requires-python = ">=3.11"

[tool.setuptools]
py-modules = [ "test_*" ]

[tool.setuptools.packages]
find = {}

The building does only work if "tests_*" isn’t a globbing pattern – that is apparently unsupported as py-modules items must be literals, or at least are only ever interpereted as module names proper, not pattern(s) thereof.

I have a feeling I am trying to do something that is well supported by Python packaging tools, but going wrong about it.

What am I doing wrong, and how can I package all of the modules that are discoverable in tests, automatically? If I can package assets like README.md etc, that would be nice too.

I have build wheels/sdist before, but it’s been too long and the time I did it was a more traditional package layout (src-based).

This time I can’t have one – like I said, this isn’t Python software outside of tests, and I see no reason to complicate content of tests any further either – if I can get everything to package with current layout, that’d be ideal.

2

What gets included in a wheel is build backend specific, not part of the packaging spec. setuptools has so many options, and controlling what to include in the build is a fundamental requirement, I’m sure there’re config options for it. There are plug-ins to use what’s checked into version control (setuptools-scm).

2 Likes
3

Perhaps let’s take a step back first, and tell us why do you want to package them in the first place. The approach you seem to be attempting would result in installing a bunch of raw test_* files straight into site-packages directory, which doesn’t sound particularly useful.

The common approach for non-Python packages that use pytest as a test runner, is to provide a plain list of dependencies in the format supported by pip install -r <filename>, and possibly wrap that in your build system.

For example, you could have a tests/requirements.txt file such as:

pytest >= 7.2
some-other-test-dependency

and e.g. Makefile targets such as:

test:
    python -m venv tests/venv
    tests/venv/bin/pip install -r tests/requirements.txt
    tests/venv/bin/pytest tests

(Note that these are just quick, dirty examples.)

4

You use setuptools, so MANIFEST.in still works as it always did.