I maintain many Python-based FreeBSD ports.
It would help a lot if it would be possible to extract build, runtime and test dependencies from any Python project automatically.
Some projects have requirements.txt, some other projects list dependencies in setup.py, yet some other projects list dependencies in pyproject.toml
Is there a way to extract these dependency lists automatically, with a script?
If by “project” you mean something like a GitHub repository, then no, not without building first. pyproject.toml is a step in that direction, but it can’t function in that capacity as long as there are other supported ways to do it.
requirements.txt files really describe an environment, not dependencies. The format is not really just package names - it’s a list of lines with command-line arguments to Pip (except the install argument). For example, it allows for specifying package index URLs, wheel/sdist preference, platform etc. Even if you can reliably parse the package name out of each line, the application may be expecting things to be installed in a specific way, not just to be available at a specific version. In short - it’s for applications, not libraries.
setup.py still contains arbitrary code that runs at build time, and can thus manipulate the dependencies in arbitrary ways. pyproject.toml can assert the dependencies, but is also perfectly allowed to describe them as “dynamic” (i.e., must be determined by building the wheel).
On the other hand, determining dependencies from a built wheel or sdist is trivial: they contain metadata files that were created by the packaging process, taking the execution of the setup code into account. You’re looking for Requires-Dist lines in a “core metadata” (plain text) file. In an sdist tarball, this is PKG-INFO in the root; in a wheel, this is METADATA in the {package name}.dist_info folder. You should use whichever corresponds to the installation method you intend to use; while they are normally in sync for a given package, this is currently not guaranteed.