PEP 660: Editable installs for PEP-517 style build backends

PEP-662 already incorporated this feedback and is waiting for PEP-662: Add prepare_metadata_for_build_editable by gaborbernat · Pull Request #2003 · python/peps · GitHub to be merged. Otherwise ready.

Apologies for the last minute change. Here is the PR with the name shortening and addition of prepare_metadata_for_build_editable (spec text copied from PEP 517).

@dholth prepare_metadata_for_build_editable does not harm as both frontends and backends are free to ignore it, so I hope this is not too controversial.

I think it is “a foolish consistency” but I am also able to never implement it in my own code.

@pf_moore since

  • @dholth is okay(ish) with prepare_metadata_for_build_editable
  • the controversial scheme argument has been removed
  • no other argument has been raised since several weeks now (except PEP 662 of course)
  • the renaming does not materially change the PEP, and that naming was used in PEP 662 since its inception without objection

I think we can consider that relevant feedback has been handled and it is indeed ready.

3 Likes

After reviewing the two PEPs, I have formally accepted PEP 660. Please see this posting for the full details.

2 Likes

Thank you Paul and all who contributed to the discussion and built prototypes. And special thanks to @bernatgabor for driving the thorough steelmanning of the virtual wheel idea.

I’ll work on getting PEP 660 implemented in pip 21.3 (October).

5 Likes

A quick update: support for PEP 660 and PEP 610 editables has been merged in pip and will be available in the 21.3 release in October.

Feedback from backend authors willing to test with the pip main branch is welcome !

6 Likes

PEP 660 has now been implemented in pip, flit, enscons, hatchling, pdm, poetry (merged by not released), and probably other frontends and backends.

Can we move it to Final status like PEP 517 ?

IMHO until setuptools (which IMHO is 70% of packages) implemented it and released it with a few months of test time we should not. PEP-517 has been provisional for around 4 years, so I don’t see why we need to rush this.

Can we move it to Final status like PEP 517 ?

I’m inclined to say yes.

Given that pip implements a custom workaround using setuptools’ legacy behaviour, it makes very little difference to users whether setuptools implements PEP 660. The only real impact is that pip can’t remove the legacy mechanism until setuptools supports PEP 660.

The provisional status was never dependent on setuptools implementing the feature, but merely on having sufficient real-world experience to be comfortable that there are no unexpected issues. It looks like we have 6 projects with implementations, which seems like plenty to me.

I’d say that’s a bad take on this. You’ll only find out what gaps the standard has once it’s widely used. And until setuptools tries to use it (to make perhaps a more correct and smarter version of the current primitive way) I don’t think this standard is widely used enough to be considered good enough to be marked as final.

IMHO enough is not a few backends that are overall not that often used adopts it. But enough should be when majority of the projects using it adopt it (e.g. 80% of projects). Now I can see this by either setuptools implementing it or people moving away from using setuptools in time. In my mind this is not contingent on setuptools implementing it, but then people should stop using it :blush: Most projects that currently implement the standard don’t provide a generic build framework, as setuptools does, but instead only a subset so they don’t necessarily expose the current standards potential issues (think e.g. flit is restricting itself currently to purely toml configuration driven and avoids having a build step).

1 Like

You’re entitled to your opinion, but I disagree. Not much we can do about that, you’ve made your opinions on PEP 660 pretty clear :slightly_frowning_face:

PEP 411 is probably the best authority we have here, and it says

Essentially, the decision will be made by the core developers on a per-case basis.

A PyPA vote is probably the nearest equivalent for packaging, so if we can’t reach consensus, I suggest someone puts it to a PyPA vote.

I have no problems with PEP-660, I just don’t consider we have enough field experience on it to consider it final. My litmus test on this is that most projects are using it, while in your case from what I understand is that a few projects implemented it. As explained above if those projects are not widely used (to make it most projects) I consider that insufficient.

Done at Mailman 3 Vote: +1 to Mark PEP-660 final or -1 to keep it provisional - PyPA-Committers - python.org

Do we have some notion of what difficulties setuptools might face or is this entirely speculative? setuptools currently creates a single-line pth file. I don’t see how anything done via PEP 660 would make that worse (or better). It simply changes the delivery method of the pth file.

I don’t think the scope of the PEP is to standardize pth files but to allow editable installs in general and allow new mechanisms to achieve editable install effect. We don’t know what difficulties it might face until we try, hence why IMHO keeping it provisional would be better here.

By accepting PEP 660 we accepted that what can be accomplished is limited by what can be packaged in a wheel. It seems rather pointless to be bringing this up again.

Agreed, setuptools can implement this pretty easily I believe. I think there was even a POC to that effect. My understanding is that the only blockers for setuptools are lack of resource, lack of urgency (setuptools is special cased in pip, so doesn’t need PEP 660) and other, higher priorities.

You’re reading a lot more into the intention than was ever stated in PEP 660. If you want to re-read my comments on the approval of PEP 660, I said there very clearly that I approved it because it standardised the existing functionality while allowing some scope for improvement. Allowing new mechanisms was never core for PEP 660.

Agreed. Whatever (percieved or real) flaws there were in the decision to accept PEP 660, I’d rather we moved on.

There’s an open PR from a new(?) contributor for this.

The setuptools maintainers don’t want to implement their PEP 660 support as a drop in replacement via a pth file, but instead want to implement it via a different mechanism to “clean up some of the major failings of the current editable install mode”. If you feel that they should implement this via a .pth file (something I agree with), feel free to pitch in on that discussion.