An “editable” wheel uses the wheel format not for distribution but as ephemeral communication between the build system and the front end. This avoids having the build backend install anything directly, but it is probably not useful to cache or distribute these wheels.
And in that scenario, we don’t have to care about the tags at all.
TBH I’m inclined to update the text to say that tools, and in particular build frontends, MUST NOT expose that wheel to end users.
And, if and when use cases appear for exposing it to users, then we can worry about the actual wheel name, or metadata in it. But for now there are no such use cases that I know of, so nothing obliges us to address that now. The evolution path to that hypothetical future is easy: “build front ends MUST NOT expose the wheel” to which we add in that future “unless it meets whatever criteria we come up with in that future spec update”.
The only reservation I have about this (and otherwise, I think you’re right and we can just ignore most of the issues by making the spec say that tools MUST NOT expose the wheel) is that the filename is part of the wheel spec, and so the backend has to choose a valid wheel name because the frontend is allowed to reject an invalid name as “not a wheel” (or to do things like abort if the version in the wheel filename doesn’t match the version in the metadata - which is something pip does, and I hadn’t thought of before but kills the “use a local version” idea). That should be fairly obvious, but it might be worth stating explicitly¹.
But yes, saying “tools must not expose the wheel returned from build_wheel_for_editable” pretty much solves all of the concerns here. (I’m assuming that we’re all in agreement that we’re not trying to make it impossible for people to do unsupported things here, we’re just trying to be explicit about what’s valid usage and what’s not).
¹ I sort of hesitated over saying this, because it’ll probably start the whole “but how do people recognise an editable wheel” debate again, but maybe the easiest thing for a backend to do is to use the same wheel name as it would for build_wheel. After all, if the wheel is never going to be user visible, why not?
I have updated the text in the linked pull request:
clarified that Frontends must not expose the wheel obtained from build_wheel_for_editable to end users. The wheel must be discarded after installation and must not be cached nor distributed.;
rejected the editable local version identifier idea;
made it more explicit that the wheel file name must be spec compliant.
This is covered by the PEP, as Requires-Dist is mentioned as the only metadata that can differ from what prepare_metadata_for_build_wheel generates.
It’s been quiet for a a few days, so it looks like we have settled the naming issue ?
The other main issue raised is the scheme argument. I personally can see value to opening the door to relative paths. I also see the potential for misuse, which we could duck by saying explicitcly that backends must not use this information to write directly in the locations provided by the scheme argument.
Are there other opinions or advises regarding this topic?
Nothing new here, but I remain of the opinion that we should omit scheme. It can be added in a later iteration if needed (if we say that backends must ignore extra arguments passed to this hook, we can do that in a backwards-compatible way).
If you can put an argument for supporting this in the PEP, then that’s fine. My objection is to adding it because of vague “it might be useful” arguments. Without good examples, it’s impossible to give good guidance to backends or frontends on how to use the parameter (look at the config argument to the existing PEP 517 hooks for an example of how this plays out…)
I have to say, I really dislike this whole approach because it’s once again asking build backends to do the installation in all but name. This is problematic for a number of reasons, but the biggest one is that I don’t think that “how this gets installed” should be something to configure at the per-project level — it is much more something that should live in the domain of the user doing the installation.
For example, we already know that some users like me would strongly prefer editable installs to not just pick up “everything in the directory” and expose it to Python, since that means your editable install may have a different set of modules than the actually installed version, whereas others see this as a feature, because they can add new files without a separate installation step. If the build front-end were given a “virtual wheel” with a list of files to install and given the leeway to install as desired, two people working on the same library could use two different front-ends or a command line flag to decide to do the editable installation differently for their different use cases. The way it’s written, it’s entirely dependent on your build backend. Even if build backends supported both options, the information would have to go into a build configuration file, it couldn’t be a command line switch or something of that nature.
“We should put light switches in this house, so that people can choose when the lights are on or off.”
“You get to choose lights on or off when you build your own house.”
The point is that different people, or the same people at different times, working on the same project, may want to make individual choices about this based on their current preferred workflow. Making it a project-wide choice is bad UI.
No, it is not ideal. There is a trade-off to be made that depends on individual workflows, not workflows that need to be done on a project-wide scale.
The question of strict vs. not strict editable installs is just one of many potential switches someone could theoretically expose for various reasons. The problem to be solved here is that the user is asking the build tool to install the project in such a way that updating the files on disk will also automatically be reflected when the module is imported, without further installation steps. The build backend gets its configuration from the project (and choices made about the build backend are scoped to the entire project), whereas the build frontend gets its configuration, and choices made about the build frontend, are scoped to the individual user. The interesting questions about how something gets installed all vary by platform, file system, user workflow and individual user choices, not by project.
That is why I think having the build backends do the “installation” is a bad idea.
But the backend is not doing the installation. The front end is. The backend is just making available the files on the disk for the frontend not? That seems a different thing.
Build front-ends already are used to do the wheel installation. Build backends create a standalone artifact in a standardized format and know nothing about how it will be installed. Build frontends then take those artifacts and install them. So “where does this live” and “how does this get installed” and all of those details are already configurable from the front-end.
To the extent that there are configuration options where what gets included in the wheel should be configurable at compile time (and thus up to users), that is a major failing of the current packaging setup, and one that we constantly see people complaining about (“How do I have optional dependencies?” “How do I build wheels with compile-time flags?”).
Using the PEP 660 approach locks us in to “the install scheme configuration is all managed per project”, whereas using an approach like the one that we proposed and agreed upon at the 2019 PyCon US PyPA summit would allow the install scheme to vary by tool (or within a tool).
Practically speaking, in PEP 660 the backend is doing the interesting / meaningful part of the installation. It has to choose how the files are exposed to the system (.pth file, the editables hack, etc), within some constraints offered by the PEP.
The backend is doing the interpreter mangling to make the editable logic work, however it does not handle any addition/removal of the files from within the python prefix. From this POV practically only the frontend is installing files. The backend only alters the interpreter startup logic (via the pth file installed by the frontend), so the way I see it does no installation.