PEP 660: Editable installs for PEP-517 style build backends

Hi all, I spent an hour adding support for PEP 660 in Implement PEP 660 with editables by frostming · Pull Request #36 · pdm-project/pdm-pep517 · GitHub, which leverages importing redirection with editables.

5 Likes

I’m collecting links to PEP 660 prototypes to add in the PEP. I currently know about and have links to prototypes in pip, flit, enscons, pdm, and setuptools.

@ofek I think you mentioned you had one in hatch but I can’t readily find it. If there is something you want linked in the PEP, just let me know.

If I missed any other, let me know too.

Can we re-title this PEP to use the term “pyproject.toml based builds” instead of referencing PEP 517 directly?

It would be good to start using a non-PEP-based name for the interface defined in PEP 517. Previously discussed at:

Fine with me. Done in the open PR.

1 Like

While we’re changing the title can we change to something that covers the angle it takes? Because PEP-662 is also Editable installs for pyproject.toml based builds.

IMHO a better name would be Editable installs via build backend import hooks would be more appropriate and differentiate it better from the other proposal: Editable installs via virtual wheels.

-1 on implying that this PEP requires the use of import hooks (it doesn’t).

How about “PEP 660: Editable installs for pyproject.toml based projects (wheel based)” and “PEP 662: Editable installs for pyproject.toml based projects (virtual wheels)” for the two PEPs?

2 Likes

A virtual wheel is also a wheel, so it perhaps more appropriate to say that’s backend driven, or something around that.

I might have missed here something, what other options does a backend have? Likely the PEP should mention it, because currently it only references editable package that AFAIK uses always import hooks.

I suggest:

  • PEP 660: Backend-driven editable installs for pyproject.toml-based projects
  • PEP 662: Frontend-driven editable installs for pyproject.toml-based projects

Although PEP 662’s scope is quite broad and “frontend-driven” doesn’t quite capture it…

Have a look at the “What to put in the wheel” section of PEP 660. It mentions 2 other methods which do not rely on import hooks but rather PYTHONPATH update via a non executable .pth.

A .pth file, exactly like setuptools does at the moment. Or self-replacing modules, as the original version of the editables project used. Anything that can go in a wheel.

1 Like

I haven’t released it yet but it’s on PyPI: hatchling · PyPI

The hatchling backend is complete, I’m almost done the hatch CLI

2 Likes

Here is the PR to add links to prototypes in the PEP: PEP 660: add links to prototypes by sbidoul · Pull Request #1998 · python/peps · GitHub

1 Like

A new version of the PEP has been published.

Changes compared to the initial version include:

  • removal of the scheme argument
  • an approach to use symbolic links in “What to put in the wheel”
  • PEP 662 in rejected ideas, and why (in a nutshell, frontend interoperability)
  • links to available prototypes
  • new title
  • various minor clarifications

To the best of my knowledge, all feedback has been taken into account. Of course, do not hesitate to let me know if something is missing.

@pfmoore if nothing else pops up, it is ready for pronouncement.

Thanks. I would like to avoid having to pronounce on one of the two PEPs before the other is ready, so would you be OK if I delayed pronouncement for a little while?

@bernatgabor said here that PEP 662 is about 3 weeks away from being ready (he said “a month” about 6 days ago). That would mean I’d be looking at making a decision around the weekend of 17/18 July. @sbidoul is that OK with you?

@bernatgabor if you are going to struggle to make that deadline, I suggest you finalise your PEP without one or both of the POCs. Lack of POCs won’t harm your PEP’s chances as much as not being able to get consensus on an agreed design (I’d like to see at least an indication that @pganssle is OK with the final form of PEP 662, if it’s to be a serious contender for approval, as Paul was the main other person expressing strong reservations about PEP 660).

Sure, of course.

I’m not happy with the title of this PEP yet:

  • Do we need the for pyproject.toml-based projects part in the title? Feels redundant considering is the only type of project supported by PyPA.
  • PEP 662 is also wheel-based: virtual wheel-based. So the wheel-based part feels to me adds no value.
  • Does not mention the core trait of the PEP, that the backend is driving the editable aspect.

I’ll see when and how I can finish PEP-662, it’s a WIP.

I’m fine with the title. I don’t think hair-splitting over the titles is the key blocker here :slightly_smiling_face:

Let me know if you’re not going to make 17/18 July. Worst case, I’ll make a decision on PEP 660 in isolation. I don’t consider it essential for PEP 662 to be completed, there’s been just as long for a “virtual wheel” proposal to be developed and finalised as PEP 660 has had, the fact that no-one stepped up to write such a PEP until PEP 660 was almost complete is a point against the proposal, not a justification for more delays.

Eh, this sounds like dismissing my concerns about the topic (and considering you’re packaging BDFL and the one pronouncing on these PEPs, I’m worried that pronouncement becomes authoritative). I’ve enumerated three separate concrete problems, so I’d appreciate addressing why those are not valid.

I personally don’t care much about the title, as long as it remains accurate. To answer each concern:

I’m running out of ideas here. Do you have an alternative proposal, that also addresses Praduyn’s concern about using a friendlier name than “PEP 517” ?

Can we agree that PEP 660 is more wheelish than PEP 662, since it uses a spec-compliant wheel as interface ?

I’m fine to mention the backend in the title. However, backend-driven might imply that the backend does the installation, which is not true, as the backend remains responsible for a build step only.

Conversely, I’d find it misleading to characterize PEP 662 as frontend-driven only, as it requires close (closer actually) cooperation and understanding between frontend and backend.

So all in all, I prefer the current titles to, e.g. this proposal.