How to write unit tests for setup.py itself?

Packaging
1

How does one write unit tests for verifying a setup.py itself is correct?

I’ve been looking for examples to study but I haven’t uncovered any so far. It’s hard to figure out what the right search terms might be. Pytest examples preferred as that’s the testing tool used in an upstream project but I’d happy with anything at this point.

2

You should seek to move as much dynamic code outside of setup.py as possible; ideally, all of it (but perhaps a trivial stub).

Then, the real test is simply if

python -bb -X dev -W error -m build

to build your package, and

echo dist/*.whl | xargs -I % python -bb -X dev -W error -m pip install --upgrade %'

to install it (in a clean venv, of course) works without error, and the unit tests for your package itself all pass when running on the installed copy (a src directory tox/nox, or other tools are helpful here) on the various platforms and Python versions you support.

There isn’t great offline validation currently for package metadata, though

pip check
twine check --strict dist/*

will cover a few very basics (environment consistency and your package readme’s syntax, respectively). It is hoped to improve the latter in the future (or other tools) to cover more metadata.

3

Thanks for the suggestions. I’ll work through them and see what I can unpack.

It’s the “install in a clean venv and verify works” part I’m working out at the moment. I can do it manually but am coming up short on how to automate and verify. My process looks like:

python setup.py check
python setup.py bdist_wheel
python -m venv test-wheel
cd test-wheel
source ./bin/activate
pip install ../dist/*.whl
# run console script-1
# run console script-2
# ...
exit
rm -rf build dist *.egg-info test-wheel
4

FYI…

So, you should replace python setup.py check with twine check dist/* (or better, twine check --strict dist/*, especially in CI and python setup.py bdist_wheel with python setup.py bdist_wheel with python -m build, or better, python -bb -X dev -W error -m build.

I’m not quite sure why you’re doing this, but it does ensure that you don’t pick up the local copy when testing.

I’m also not quite sure what

is all about, but here is where you would run pytest ../path/to/tests (or whatever your test runner is) to unit and/or integration test the installed package.

5

yep the cd is about helping to make sure we don’t pick up stray files.
the # run script bits are place holders for what will be tests but are right now just “run and look at screen to see if seems ok”.
exit should have been deactivate

Thanks for Ganssle article; marked for study.

6

Hi @maphew, I think the best way of testing if setup.py works is to run your test suite against an installed version of the package (instead of relying on PYTHONPATH or an editable install).

(I understand that this is not an unit test, its more of an integration test, so not exactly what you are looking for).

Tools like tox and nox may help you with that (if you go for tox, make sure to configure isolated_build=true[1]).

These articles might also help:

  1. I also recommend not setting usedevelop, otherwise your package will not be installed like it would in production. ↩︎

2 Likes