PEP 660: Editable installs for PEP-517 style build backends

In what way is it restrictive ? In terms of capabilities I have yet to see something that one can do and not the other.

What prevents a new version of setuptools that implements PEP 660 to generate the built extension in another directory and expose that one ?

You can define a C extension just fine with enscons’ SConstruct, no setup.py required. Maybe pymsbuild does it too. Even the distutils 2 attempt from a decade ago put extensions in setup.cfg. setup.py does no magic it just knows a few compiler flags.

C extensions should work fine with .pth or import hooks same as .py

The truest Python old-timers still set PYTHONPATH but the package would not show up in pip list which sometimes matters.

2 Likes

backend specific, no plan to standardize until we have more experience

How is it different for editables than a regular pep 517 build ?

Again I’m not sure how it is different than regular PEP 517.

I guess we’re having a disgreement on symlink functionality. You say one can do via inserting a startup code into the interpreter that performs the symlink operation. I don’t think this is a smart approach we should publicize. In essence the backend is creating files into the target interpreter the frontend is not aware. The frontend will not be able to uninstall these because it doesn’t know they exist. Or the backend needs to generate pre-emptively entries into the RECORDS that are not there, but can be there. Not sure if the RECORDS file within wheels supports this, or if frontend would not choke on it either at install or uninstall time.

In theory, it can. However, feels like there’s an easier solution by defining an interface where the frontend can explicitly create the symlink, and not mandate the need of a meta folder to achieve this functionality.

I don’t see any mention on this topic on PEP 517 either. So I’d like to ask for clarifications on the question here.

The existing hooks in PEP 517 do not pass in any target interpreter specific information in PEP 517. PEP 660 for some reason does pass in the scheme argument that seems to tie it to a specific interpreter and nothing else. So while you can take a universal wheel and reuse it across multiple interpreters, the defined interface seems to discourage this. The PEP should be clear if it does not plan to offer similar guarantees PEP 517 does.

I don’t think we should take the stance that reorganize your project or its configuration to make it editable friendly. I believe desiging a system that can solve the problem without this would be prefered.

The scheme argument is still subject to discussion and I’m more and more convinced we should remove it from PEP 660.

1 Like

I know this is only part of whats going on but once this PEP is completed our plan to kill setuptools is complete.

You don’t have to switch to enscons. But once you can do “develop” mode without setup.py then setuptools has no preferred place in packaging.

I’m not sure how this is relevant at all here. I’m by no means a setuptools maintainers. And both the virtual wheels and the PEP-660 gets you there independent of what alternative backend you want to go with.

Just avoid saying setup.py is required for extensions or that setuptools must implement all packaging standards or those standards cannot exist.

Why don’t we return to the other thread to discuss how virtual wheels implement ‘expose a whole path’ versus ‘expose a package’ versus ‘expose a list of .py files’

1 Like

I’ve been mildly interested in adding symlink support to wheel. What if we added a permissions column to RECORD. If it contains a lowercase l, interpret the content of that file as a symlink. Optionally set the standard zip file symlink attribute.

Then this PEP could request symlinks.

Yes that would open up additional possibilities for PEP 660 but that should treated separately, and not as a prerequisite for PEP 660 adoption, because it is useful without symlinks.

1 Like

I’m drilling down on the objections


1 Like

Regarding headers and data files, PEP 660 puts them explicitly out of scope in the Terminology and goals section, by saying we focus on making python files editable. If supporting editable headers and data files become a key criteria for adopting an editable PEP, then it’s symlinks or :man_shrugging:, I suppose ?

Regarding scripts, well, I’m personally fine to say they can’t be editable and recommend project authors to migrate to console scripts if they want that. Editable scripts does not sound like a strong differentiator, especially if the only known way to implement that is symlinks.

1 Like

You run your script in the checkout no problem and it can import your Python.

1 Like

I disagree. Sure it’s useful but I’d say not complete without it. In a world where PEP-660 gets accepted and symlink PEP rejected, we’d be worse off than a competing proposal that would add support for symlinks. So if we go down this lane I’d like the two PEPs to be accepted/rejected together.

In mind editable installs are about Python packages, not python files. So in this sense I guess the two PEPs currently have different goals. Similar to how PEP 517 covers packaging for editable packages, and not pure Python projects this pep should also cover packages not files.

I don’t think symlinks would work here, but proxy scripts generated would, so I see no reason why not go down that path.

1 Like

It’s not about setup.py file per se, but yes backends should be able to handle their existing use cases in the new standardized way, so we’re not progressing some while regressing in other parts.

Where did the symlink requirement come from? It seems odd to block on support for platform-dependent functionality.

Maybe, as a Windows user, I am seeing things differently?

2 Likes

Yes, I hope that we can decide that this implemented PEP is fine as is and without symlinks, making it easier to provide a feature that many people like. The “strictest possible simulation of an install without actually copying the files” idea is different, could we call it ‘preflight’ mode?

The problem would be adding new scripts, but that could be solved by adding a communication back channel to trigger the frontend to generate them.

Windows 10 from the last few years does support symlinks, you should try to use it helps a lot efficiency/performance in virtualenv for example :smiley:

I think based on discussion above we can conclude that we can’t agree and we’ll have to let the packaging BDFL or the steering council pick a solution. Many people do want editable installations, but both PEP provide this functionality so there’s no this reasoning can’t be used by either option.

We do still have to work out exactly what the other proposal would do before it can be claimed to be equivalent.