I’m fine with calling it build-backend-bootstrap-path or something like that, to make the intended usage clear. But I don’t see a conflict between “this is where the in-tree backend is located” and it being a list. For the vendored dependencies case, the backend is itself spread across multiple directories.
Once we go down the sys path option, I don’t think we can stop from people using it to add their own source path for non build backend boostraping. So I would say as much as to be upfront and explicit about it as prepend_sys_path, and document it that the main intended usage is for self bootstrapping back-ends.
But we cannot limit its usage for this only, for example it will be hard to argue that
prepend_sys_path = ["."]
is more declarative and straight forward than sys.path.insert(0, ".") in the setup.py file.
OK. I’m not sufficiently motivated to argue in favour of the idea of “build backend location” any more, the consensus is clearly leaning towards just having a facility to add directories to sys.path. So subject to a final review of this thread, I’ll be putting together a PR for PEP 517 to add the following information:
- Projects MUST NOT include cycles in their build requirements and front ends MAY report an error if a cycle is detected.
- Projects (particularly backends) MUST NOT assume that build requirements are available in wheel form - they MUST be prepared to handle the case where everything has to be built from source. For backends, this likely means being very careful about your dependencies (as otherwise one of your dependencies could suddenly change to depend on you).
- We’re adding a new
python-pathkey to[build-system]that is a list of locations which will be added tosys.pathbefore the backend is loaded. - The
python-pathentries will be treated as relative to the source root, and MUST all resolve to locations within the source tree. - If
python-pathis specified, the build backend MUST be loaded from a directory inpython-path. This restriction is explicitly to limit the use of this key to handling in-tree backends.
Some outstanding questions:
- Do we add the
sys.pathentries at the start ofsys.path? I think that’s been the assumption, but we need to be explicit. - Do we insist on the check in (5) above? It’s the one place where we do make this feature explicitly for in tree hooks and self hosting only. I think that’s important, given some of the concerns expressed here around making the feature too open to misuse, but the people making those arguments have been relatively quiet recently, so I’d prefer it if they explicitly make their views known (I can only proxy their concerns up to a certain point
) I will assume that in the absence of anyone arguing otherwise, we keep the restriction, so removing it will need someone to champion that change.
I don’t have a strong view on either of these, but I think they are important questions - someone with a good feel for security issues needs to review the proposal to confirm we’re not creating any new risks here (yes, I know, if you’re running a build from sources off the web, all bets are off, but we should at least ensure we’re not creating any new or easy to overlook vectors for exploits).
Last call for arguments! 
Assuming the package to be installed and the build backend are well-behaving, this should be inconsequential AFAICT, but it could be a good idea to make no mention of this in the spec (only guaranteeing it to be added), add to the end instead, and discuss if edge cases arise.
I think this is a good restriction to have, but how can it be reliably enforced? Does the front end check for the __file__ attribute of the loaded module?
I’m -1 on leaving it unspecified, as it’ll only result in people relying on implementation-defined semantics. If we don’t want to make a choice for now, let’s at least be specific - “projects MUST NOT reply on the path elements being added with any particular priority”.
That’s how I’d imagine doing it (and I’d probably note that in the PEP itself, so we have consistency).
1 Like
A possible variant to be even more specific: it must be loaded from the first specified directory.
That seems like the obvious way. A possible alternative would be to use importlib to resolve the module before loading it. I think importlib.find_loader() could do what we need, but it might be tricky to make it compatible with Python 2.
+1 for always inserting at the start of sys.path, and requiring that the build hooks be loaded from there does make me feel less concerned about abuse, if only because it makes the intent clear and people read PEPs to determine intent. Whether anyone ever enforces it or not is secondary.
We can write that, but in practice it will just mean that projects end up relying on whatever pip does. So I think we should pick one now.
I think either prepend or append could work. I’m leaning towards prepend, because (1) it’s more consistent with $PYTHONPATH, (2) the general theme here is that the source tree gets to decide how it is built, and putting the source tree’s code first feels consistent with that.
Regarding __file__ checks: this sounds like the sort of thing that could have tricky side effects. (What does __file__ get set to if you load from a zip file?) Before we dive into that complexity, can someone articulate what, specifically, we’re worried about happening if we don’t enforce this? (I’m fine with describing the intent in the spec, that’s just being honest. But adding extra runtime checks has costs in terms of complexity, so I want to see the costs and benefits laid out next to each other.)
To get a better understanding, what would a front end do if it didn’t report an error – doesn’t it have to result in an error? (Edit: actually, I remember now that I had asked this question earlier in the thread and someone had replied, so no need to respond to this.)
I think it might as well make clear in what order the sys.path entries will appear among themselves if multiple paths are provided. In other words, will the order within sys.path match as it is listed, or in reverse (e.g. by calling sys.path.insert(0) in sequence on each element)?
I think somewhere up-thread, Donald agreed that the version with path manipulation and the version with a magic known-location module are equivalent in terms of capabilities, and the main difference is in the UX.
The problem with this UX is that it is too broadly scoped for the problem. The problem to solve is “how do you write a self-bootstrapping backend”, but the solution is to add an option that, as a side-effect, pretty seriously changes the behavior of existing backends (including allowing things like code injection). It’s also not trivial for backends to avoid those side-effects, because this is a front-end configuration option.
Practically speaking, what will happen if the sys.path version goes in is that setuptools.build_meta will start parsing pyproject.toml and throwing an error if it detects that the sys.path option is specified and the project being built is not setuptools itself. This will have the unfortunate side effect of banning self-bootstrapping backends that are wrappers around setuptools.
Using any of the “known location in tree” options, it is trivial to get the behavior you want. Assuming we went with an option that says “I supply a build-backend in the known magic location”, you could do this:
[build-system]
requires = []
build-backend-bootstrapped = True
And then your __build_backend__.py would look like this:
import os
import sys
sys.path.insert(0, os.dirname(__file__))
from backend import backend
__backend__ = backend
Similarly, as I’ve mentioned before in this thread, if you don’t like the fact that you would be required to implement a specific named-method (or use a DSL or whatever), with a “known backend location” method, it is possible to implement a meta-backend that has basically any semantics you want (e.g. intreehooks) without running into scoping problems in the PEP.
There’s a clear asymmetry here. Both mechanisms have exactly the same power to solve the self-bootstrapping problem, but the "insert things into sys.path" mechanism has side-effects that are difficult for backends to undo, and as a result of that the "insert into sys.path" mechanism will end up giving a worse user experience to the class of end user who wants to write a simple PEP 517 backend for their own setuptools-based project for whatever reason.
The version of the "add stuff to sys.path" proposal that requires that the build backend be in the path added is probably acceptable on these grounds, though the implementation is much more complicated for frontends to get right (and if it’s anything less than a MUST, setuptools will need to enforce it on the backend).
If we’re adding bootstraping capabilities, I’m still in favor of the simplest thing possible to implement to enable this pattern: a known-location magic-module that can be used to do all configuration. It’s not a particularly uncommon pattern in Python (e.g. __init__.py, __main__.py) and as far as I can tell has no practical downsides.
I mean… it’s Python right? All of those things are already perfectly possible if I add a package that has a .pth file in it. If someone wants to do something sufficiently gross then they’re going to do it, no matter what we do here. Python is the language of consenting adults, if someone wants to do some gross code injection… then more power to them?
I also don’t think that is the problem we’re solving. It is one of the problems, but it’s a more generic solution because, upon further examination, there were related problems where it would be useful to have this capability that weren’t directly related to self bootstrapping backends. More generic problems calls for a more generic solution, instead of two specific solutions.
Honestly, that sounds pretty un-pythonic to me, but I’m not a setuptools contributor.
Can you elaborate, I must have missed something somewhere, as it stands now even the title of this thread suggests that self bootstrapping backends are the problem to solve.
1 Like
See earlier comments:
Which part of that is the “new problem”? People already have the ability to use one-off build code, that’s what setuptools does, and presumably any other backend that provides imperative builds. It’s not only supported, it’s the default thing.
In any case, I still don’t see any argument here for why the scope needs to be broadened. You yourself have admitted that the “known location” and “add to path” solutions are equivalently powerful, but the “add locations to path” solution only gets you a side effect, which is that you, as a user, can change how the paths are configured for a backend you are merely using. It’s simultaneously creating the ability for the frontend to find a custom backend (the problem we want to solve) and forcibly adding a configuration option for the backend that the backend doesn’t even know about (not a desired outcome of any problem anyone has described here).
Given that we have two equivalently powerful solutions, I think we should go with the one that has no side effects and allows you to build things like intreehooks for people who want to opt-in to the other solution.
People shouldn’t have to use setuptools to get one off build code, they should be able to do that wholly within PEP 517. It’s only the default thing if you’re using setuptools.
shrug, I likewise don’t see the argument for why we should artificially limit the scope. All I’ve seen so far are nebulous claims that “bad things” will happen, with no actual scenarios to back that up and why those same bad things won’t happen in every other possible iteration we’re discussing here. Having the same capabilities, but with a worse UX because nebulous unspecified reasons seems entirely silly to me.
1 Like
We seem to go round and round on this point. There’s no “broadening of scope”, in the sense that we’ve all agreed that all of the proposed solutions can be implemented in terms of the others. I get that you’re uncomfortable that the discussion is moving in a direction that has the PEP remaining neutral on the question of what the feature is “for” (although the restriction that the backend must be loaded from the first added path entry remains to make it abundantly clear that it’s only for building backends) but the “consenting adults” principle is a pretty strong design principle in Python, and I think it’s something we should stick to here.
I hope we can all do better than throwing out ultimatum-style statements like that.
Well, I said that I proposed making it a MUST, and no-one objected. How much more assurance do you need before you’re willing to drop the threats? Setuptools can do what they like, there’s nothing anyone else can do to change that, but we’re trying really hard to build something that’s acceptable to setuptools (more specifically, to you - I haven’t heard much from any of the other setuptools maintainers in support of your position) and it would be nice if we could compromise a little here, or at least assume a level of good faith.
I suggest we let tempers cool down a little. I’d rather we could all agree to try to make the final proposal work, but if we can’t, then ultimately we’re going to have to either pick a solution based on majority preference, or walk away with no solution. And I don’t think any of us have the stomach to leave this looming over us 
2 Likes
I’m sorry if it came off as a threat. At this point it’s incredibly hard to gauge where everyone is coming from, we’ve had too many arguments and it’s not at all clear who has taken what points to heart.
I was intending to highlight that a solution where someone can simply add things to the backend’s PATH is not, for the purposes of implementing a backend (not just setuptools), functionally equivalent to one where you specify the location of the backend. Early on in this thread, @ncoghlan suggested that backends could simply choose to remove whatever values are added to the sys.path before executing whatever they want to execute. What I was trying to suggest is that if we, as setuptools, do not want to offer the option to arbitrarily manipulate the path as part of the package configuration, the only option we have is this very hacky mechanism that will itself bring great limitations on how people can use setuptools. Backends, including setuptools, will have to choose whether they want to try and hack around the problem or not.
On the other hand, a “known backend” solution allows everyone to have exactly the semantics they want because it enables both meta-backends and allows individual backends to choose their semantics.
To be clear, what I meant was that the proposed mechanism to solve the problem is broader than the scope of the problem. I don’t see anyone arguing that allowing users to manipulate a third party backend’s sys.path is a benefit - there are just people who think it’s a downside and people who don’t care. I’m saying there’s a solution that should satisfy people who think it’s a downside and people who don’t care.
I think they are functionally equivalent though, as already called out, you can trivially emulate the ability to add an arbitrary number of items to sys.path and then use setuptools backend directly by just making a __build_backend__.py that modifies sys.path and then imports the setuptools build backend.
Is the assumption here that there is a large contingent of people who want to add some arbitrary thing to sys.path before setuptools, but they don’t want to do that bad enough to just stick a 3 line file in their sdist?
From my POV it seems that the two solutions are practically the same as far as setuptools POV, before the 517 API is called, some items got added to sys.path. I can’t figure out a way that it matters for setuptools whether a frontend set the path from a declarative file or if it was set imperatively by an in tree backend before shimming setuptools-- in either case before setuptools is even imported those items are on sys.path. Why is one OK and the other not?
Buried in the thread somewhere, the point was made that there’s no easy way to technically implement “specifying the location of the backend” except by adding something to sys.path. I proposed adding the location just for the import of the backend, but people pointed out that removing things from sys.path was error-prone and likely to confuse people (backend authors in particular) so that was a non-starter. It might be possible to do something robust with importlib, but we can’t assume that because we have to support Python 2 
I’d prefer to avoid mandating fragile behaviours in the PEP, so I consider that as essentially a done deal.
OK, I would strongly recommend that you don’t, as it’s entirely likely that users caught out by such restrictions would consider this a setuptools bug, and I’d probably agree with them. It’s unlikely that we can find a way of writing the language in the PEP to allow backends to implement such hacks without opening up loopholes you could drive a truck through. The best we could do is to leave the exact behaviour unspecified, and we’ve already seen how that works out (see the setuptools “legacy” backend debate)…
But I’m open to editorial comments on the PEP PR which clarify and/or constrain this. I’m just not happy with trying to specify the semantics in such a way that frontends cannot implement them by adding the location(s) to sys.path.
Well, yes, if you insist that the only problem we should solve with this iteration is “make it possible for backends to self-host”. I mentioned this in a couple of the drafts of my post that I threw away, but the general trend has been that we should also look at making it easier for projects to host in-tree hooks. So the current solution is intended to also address that problem. Again, sorry if you missed that. But I consider solving two problems with a single PEP update to be a good thing, so again, I’m not keen on deliberately ignoring the chance to simplify in-tree hooks.
But it’s only the problem space that has broadened. The solutions all remain capable of addressing both problems - just with different convenience and intuitiveness trade-offs.
Hmm, you’ve not been looking very hard - @njs has been arguing for that pretty much from the start 
But even so, “manipulating a third party backend’s sys.path” is not the goal here. You seem to keep missing it, but the whole point of restricting the feature to require that the backend is loaded from one of the added sys.path entries is to ensure that this feature can’t be used to affect “third party” backends, but only ones you supply yourself. In particular, this feature cannot be used to specify the setuptools backend, and modify sys.path for it.
Modifying sys.path is the implementation technique for the proposal, it’s not the goal of it.
Hopefully, this is clearer - thanks for restating your views, it was worthwhile as I think we understand your concerns better now. And with luck, the above has addressed them (or at least explained why the general view differs from your preference…)
Surely the value of modifying a backend’s sys.path is only for modifying the search path used by your own arbitrary code anyway, right? What would you do with Flit other than locating it? And once located, it’s not going to pull submodules out of somewhere else.
Isn’t this just a “Python code as configuration” problem (aka setup.py)? People want to import their module and helpers because they can (and yeah, I do it all the time too), but I don’t get why that’s an inherently worse thing than having yet another Python-code-as-configuration file to do it for them?
Now, if some backend became a library for that new config file to make declaring custom imperative backends easy, I’d be interested But that’s just moving the arbitrary code around and I don’t think anyone is signing up for that job. And we already have setuptools handling it fine indirectly via the package and then the setup.py.