Generating python files before sdist build

Python Help
1

I try to improve an old project (mercurial / mercurial-devel · GitLab) with a pyproject.toml and a setup.py using setuptools.

I’d like to know what are the good practices for generation of files at different stages.

  • Before the sdist build (actually before setuptools uses MANIFEST.in because this is for dynamic generation of this file)
  • During sdist build (translation, __version__.py, hgext/__index__.py)
  • During wheel build (mercurial/__modulepolicy__.py)

Currently, these file generations are called through setuptools commands/subcommands in setup.py and also with Makefile targets.

I’d like to know if this is still the best methods when using setuptools through PEP 517.

I’d like them to work fine in different cases:

  • pip install .
  • pip install --editable .
  • python -m build (produce sdist and wheel from the sdist)

I’m a bit lost with the association between setuptools commands/subcommands (build_py, install, etc.) and the PEP 517 functions (defined for setuptools in setuptools/setuptools/build_meta.py at main · pypa/setuptools · GitHub).

I’m a bit lost where the generated files have to be generated. For example, currently, mercurial/__modulepolicy__.py (produced during build_py) is produced in the wheel but not for the editable install.

2

A long while ago, I was wondering the same thing, I investigated, and came up with this: https://sinoroc.gitlab.io/kb/python/setuptools.html#commands-dependencies. Now, many years later, I am quite sure it is at least incomplete, and there might even be some errors. Especially since setuptools itself has changed since then.

1 Like
3

The whole subcommands business always was a bit of a mess. You could add print(sys.argv) to setup.py to see what subcommands are being used by different frontend actions.

Having looked only very briefly at the mercurial repo it looks as if the build is sufficiently complicated that I would recommend using something like meson rather than setuptools. Any migration of a complex build system is a lot of work but I think that using a more general purpose build system will pay off in future.