Currently, the PEP 517 can specify a
wheel_directory, and I believe it is guaranteed to be executed from the repository root (though I think
pip first copies your directory into a temporary directory so the “repository root” is ephemeral anyway), but there is no standard way for front-ends to pass a non-ephemeral location other than the repo root to the backends.
One problem this causes is that
setuptools will generate a bunch of build detritus directly into your repo root like
lib/, which is not a great practice. I think this also makes it so that it is difficult for
pip to cleanly implement any sort of incremental builds, because the only available location for the build detritus is the repo root.
There is open issue in
setuptools to allow moving the locations of these folders, but I am worried that this will end up with
tox or other front-ends writing setuptools-specific code to pass these options in order to avoid polluting the local development environment. I think we can solve both this problem and the “allow incremental builds” problem by adding a new “persistent cache directory” which is created by the front-end and passed to the back-end. The idea would be that the front-end should create a directory where the backend can store expensive-to-create objects that should persist between builds. It is up to the front-end when this cache is cleared.
I think there are two options:
We modify the hooks in the PEP 517 build interface to add an optional
cache_directoryparameter, like so:
def build_wheel(wheel_directory, config_settings=None, metadata_directory=None, cache_directory=None): ...
For backwards compatibility, I think backends that support this feature would maybe have to specify something like
backend.__SUPPORTS_CACHE_DIRECTORY___ = True.
We add a top-level configuration function that can be called prior to calling a build hook that would configure this or other global options, possibly like this:
def set_backend_configuration(*, cache_directory=None, **kwargs): ...
In this case frontends would just check for the existence of
set_backend_configurationand pass options if and only if it exists. This version also has some built-in forwards compatibility by taking arbitrary keyword arguments that will be ignored if unsupported (possibly with a warning). We could also add a
get_backend_configuration_optionsfunction so that frontends could have different behavior based on what the backend supports.
We don’t have to specify what backends should interpret a missing
cache_directory as, though I imagine
setuptools would default to the repository root. I’m thinking that this will allow
tox to specify a cache manager in some per-env directory under
pip could easily grow a flag like
--incremental-builds that turns off the isolated build behavior of copying the repository into a temporary directory.
I am OK with writing a new PEP since this may be beyond the scope of what we want included in PEP 517 itself, but I’d like to hear thoughts and criticisms before moving to the “draft a PEP” stage.