I was initially against using wheel but it turns out that the build backend knows how to make them. Mine puts extra files in the build directory that don’t make into the archive so it would be extra work to produce a clean directory.
The ‘editable’ wheel will be tiny. It includes the metadata and a reference to top level modules. Most incremental builds will not change the ‘editable’ wheel.
I’m not sure how much performance benefit we can have when reading a few kilobytes of file. Considering this wheel is likely to be just a few small shim files that expose the actual source files the difference probably is negligible.
Build backends already have a local cache folder they can use to support incremental builds. The only benefit you would be saving is the performance hit for zipping and then unzipping those few small shim files. The drawback would be though that you’d need an installer that can understand directories as an editable install target (this time copy it please, don’t use the build wheel/sdist and install we do with them today), which would complicate the installer code.
I think in this case using the wheel makes the logic easier for installers, and the price for it is a very small performance hit of zipping those few shim files that need to be installed (just to spell it out: an editable install will not install the package itself - which could be big - it only needs to install a few python+pth files that are loaded at interpreter startup and makes the actual source tree discoverable within the interpreter - IMHO the easiest is to just inject some importer hooks as @pf_moore editables project does).
Do we need metadata_directory here? Has anyone written a build backend that uses that parameter? I expect that for the editable wheel the metadata is likely to be slightly different especially if you count RECORD
metadata_directory is only meaningful if the frontend calls prepare_metadata_for_wheel first (and use this argument to pass the result back into the backend). pip has always used this two-step method to build wheels. The main benefit to this is that it can avoid building the entire wheel when only metadata is needed.
It would be nice if build_wheel_for_editable would have the same mechanism, since it makes it easier to implement in pip 
But the performance benefit would be minimal here, since editables are always specified by direct URL or local path, and a wheel is almost always needed in the end.
pip used to call setup.py egg_info first to install dependencies before building and installing the package. Now it can call prepare_metadata. The backend probably generates metadata twice which is fine because it’s very fast to write a few text files.
https://www.python.org/dev/peps/pep-0517/#build-wheel does have an optional metadata directory argument, so the backend can check if the content of that is up to date and not generate it again.
Do any existing backends use that parameter? It seems like it would take more time to check than to rebuild the metadata.
At least Setuptools does. And the backend does not need to check—PEP 517 mandates the frontend must pass in a directory identical the backend had previously built with prepare_metadata_for_build_wheel, so it should be faster. And a backend is not allowed to ignore the parameter either, if it implements prepare_metadata_for_build_wheel.
Not really, for example, setuptools can calculate the hash of the setup.py and setup.cfg and if that stays constant do not regenerate the metadata. And that means it can avoid evaluating runtime code in setup.py, not?
Here’s setup tools build wheel not passing metadata directory down to the builder: https://github.com/pypa/setuptools/blob/b6bbe236ed0689f50b5148f1172510b975687e62/setuptools/build_meta.py#L214
I don’t mind the directory parameter. Seems like as long as we can convince Paul that “scheme” is okay we can proceed to add this to pep517, or the packaging documentation or (hopefully not) a new pep. @takluyver @njs
I don’t think you need to convince me. I was only speaking on a personal level, so you can ignore me if you like 
On a procedural note, and speaking with my packaging interop BDFL-delegate hat on, I think this is more than we should just do as an edit to PEP 517. I think this deserves a new PEP defining editable mode properly. So the next step is for someone to put together such a PEP, start a discussion on it and (hopefully fairly quickly) confirm consensus. I’m happy to sponsor the PEP if you need a sponsor.
I rebased my pip proof of concept and made a quick poc in flit.
@dholth since there does not seem to be any serious objection to your proposal above and @pfmoore is kindly proposing to sponsor, I’m considering to convert your text into a PEP. Do you agree ?
I’m inclined to drop the scheme argument in a first version for the sake of simplicity in the specification and implementation, knowing it could still be added later as an evolution.
Sounds like a plan 
Thanks Stéphane. I’ve been thinking it was time to complete this and I’m very glad you’re willing to do the pep… if it really can’t be just a packaging standard. Hopefully this thread and the github discussion get us most of the way there and I’ll be happy to help edit.
Good plan 
It’s not realistic to expect that it will be added later as an evolution. Otherwise the plan is okay.
One thing that came up in this issue is that the implementation technique in editables doesn’t support exposing single Python modules as editable on sys.path, only directories.
Personally, I think that’s fine, and backends should be permitted to only expose packages, and not single-file modules when creating an editable-mode wheel. But I do think the PEP should be explicit about this - and should probably require backends to at least give a warning if they don’t produce an editable install that exposes the same things on sys.path that a normal install would.
It’s worth noting that setuptools’ setup.py develop doesn’t handle single-file modules properly either - it installs them, but it does so by exposing the whole directory they are in on sys.path (to demonstrate this, try doing import setup, which runs setup.py).
I think the PEP has no right to tell backends what they must do, but it should describe the history of editable installs starting with PYTHONPATH, the mechanism of creating a special wheel, and the current ‘gold standard’ of stub modules that expose no more and no less than what would be installed.