Taking advantage that the PEP is still provisional I propose to remove this guarantee from it:
Frontends should call each hook in a fresh subprocess, so that backends are free to change process global state (such as environment variables or the working directory). A Python library will be provided which frontends can use to easily call hooks this way.
I feel this guarantee offers little benefit at a very costly price. The benefit only affects build backend maintainers, who no longer have to care about the global state. However, the drawback is effecting the entire python user base, because now all frontend-backend interactions need to pay the full interpreter startup/teardown (plus imports) price every time. On a normal tox run for example tox needs to do the following 3 calls:
- get-requires-for-build-sdist
- prepare-metadata-for-build-wheel
- build-wheel
This overhead (on my high-spec MacBook Pro) is around 50ms, but things get much worse on a Windows machine where starting subprocesses is even more expensive. And this is an overhead that must be paid on every tox invocation. I think it would be beneficial to pay it just once, rather than 3 times.
Now from the POV of the build backend, this seems not that expensive to handle. You can wrap every PEP-517 hook with a os.getcwd
/os.chdir
and os.environ
mangling. It’s a one-time cheap cost, but most backends don’t mangle the global state, either way, most of the time. I think at the very least we should allow build backends to opt-out of the need for the fresh subprocess on every call.
PS. I did do a POC of an interface that does not require fresh subprocess calls, https://github.com/gaborbernat/tox/blob/p-impr/src/tox/util/pep517/backend.py#L43-L74, and excluding some rare edge cases it actually works fairly well.
I think as far as guarantees go it would suffice to enforce that the PEP-517 invocation part (the part that invokes the backend method in the subprocess) will not alter the global state.