Discuss PEP 662: Editable installs via virtual wheels

[bernatgabor (Bernát Gábor)]

@dholth

Some backends, especially setuptools, do not know which files will be added to the wheel before actually doing it.

This is only true for now. @pganssle did try to fix this within setuptools, and definitely doable just needs someone to do the lifting for it. In the meantime setuptools can do whatever it does today, just expose the project root as is; which mean setuptools can maintain their status quo and other backends can advance along.

@dholth

Since there won’t be a direct link between the wheel and the editable mapping, where’s the advantage?

There’s no wheel in this proposal, can you clarify what you’re referring to; and why would it need to be a link between them?

@dholth

What if a flat directory of Python files is mapped to an installed tree

The build backend can do this transformation during their package build today, so why would this not make sense?

@dholth

What if the same file is mapped several times under different import paths?

The backend can do this today for a wheel during the package build, so why would this be a problem in the case of editable installs?

@dholth

How would I include a folder, exclude a single submodule, and allow anything else in my source directory, as permitted by the editables library?

In the same way, the backend today generates a custom importer that does this custom and dynamic logic the frontend can use a similar technic to achieve this. This PEP does not disallow any of PEP-660.
The backend is still free to generate an import hook as a source file and forward that to the frontend to expose within the target interpreter. However, the backend now doesn’t have only that option, it can also expose the source files directly by passing the list of files explicitly.

@dholth

The best feature of the alternative is that it breaks people’s expectations of an editable install by being very strict?

I did not follow what you were referring here to, so if you can clarify? Thanks! (but if the backend /frontend wishes this PEP also allows them to be very strict)

@dholth

In PEP 660 the complexity is all in the backend. Instead of having to debug interactions between a complicated “generate the mapping” in the backend and another complicate “use the mapping” in the frontend.

This is a trade-off. Sure you have most of the complexity in the backend, but now the backend is extra complicated because it effectively must use import hooks and nothing else to achieve the editable install effect. With this PEP for simple use cases, the frontend can just use symlinks, that are much simpler to debug and understand. And the backends response is also fairly simple, it’s just a .dist-info folder and a mapping of mapping.

Note PEP-660 can hack in symlink support as pointed out:

@sbidoul

As a side note I do have a requirement to use symlinks in a custom backend and I could implement it with PEP 660 by generating the symlinks on interpreter startup (via an executable .pth that is put in the wheel by the backend). It is not particularly elegant, but I just wanted to note that it is possible today.

However, this feels to me so hacky that I’d put it into the category of works by chance rather then by design and choice. With PEP-660 how would the user choose in between symlink mode and register new importer mode?

@dholth

It gives control to the package author, who should be the primary user of the editable feature.

The issue is that this is not a choice for the package author to make, but rather something that’s dependent on the machine the developer is running. That might not support symlinks so the user need to be able to request via the frontend the import mode, or if their platform does support symlinks for performance and simplicity reasons might wish to explicitly request a symlink mode (assuming their frontend supports both). The argument of this PEP is that how the editable installation is achieved should be in the hand of the user via the frontend, not the build backends exclusive right.