It has now been nearly 3 years since PEP 708 was provisionally accepted, and we still have no implementation in pip (or in uv, as far as I can see). And I’m not aware of any index other than PyPI that implements it, either. So according to the criteria I set, the PEP is nowhere near to being ready for final acceptance.
I don’t believe this is a reasonable situation to be in - essentially, the effect of PEP 708 has been to stall any other standardisation work that might have gone on to address dependency confusion attacks, and slow or derail any tool-specific work (“Why not just implement PEP 708” is an entirely reasonable objection to a tool-specific solution).
At this point, I’m strongly inclined to withdraw/reject PEP 708 because the conditions for its provisional acceptance have not been met. Tools are moving towards index priority mechanisms for addressing dependency confusion, and it seems likely that work to standardise such approaches would be a far more productive option than trying to revive PEP 708.
I’m not making this decision lightly, and I’m aware that PEP 708, if actually implemented, would supplement other approaches, rather than replacing them, but I don’t think it’s good for the ecosystem to have a standard (particularly a security-related standard) that is essentially being ignored by implementors. It doesn’t send a good message about how we view supply chain security.
I’d appreciate feedback on this proposal - I won’t be making a formal decision immediately, and I’d like to get some sense of the community view on the situation first. In particular, @sethmlarson and @dstufft I’d like to hear your thoughts.
{PyPI Admin hat on}
I did some analysis of “how this has been used thus far”.
None of what I’m sharing isn’t public information, you can scrape all the data and assert this yourself, but in the interest of time, I queried the database directly.
Since being added in 2024, most of the entries are misused, and a fraction actually point to another Index.
The vast majority of folks adding this point back to their source repo (GitHub, GitLab, Codeberg, etc), or a ReadTheDocs site, or even the Python docs themselves, not an alternate Index.
Some folks even use the field to point back at PyPI or TestPyPI - which also doesn’t make sense.
Most of the remaining unknowns appear to be along the lines of “advertising links” - things we’d expect to be in the Project-URLs metadata, not this field.
Of the 13 “correct” users of an index - 7 are from a single user, pointing at an alternate index, and most of the remainder are broken links that look like an index path. {PyPI Admin hat off}
I don’t think this feature panned out as intended. I don’t know if that was because of lack of adoption by installers, or needed some better UX / documentation, but it doesn’t feel like it did the right thing. I’m +1 for removing it.
I was the one who implemented PEP 708 in PyPI. I thought I had the time & backing to also implement it in pip & another index. I can see I had started planning / design docs beyond pip. Unfortunately, my time & backing ran out early.
I’m sorry to see that PEP 708 hasn’t gone the way it was intended. I think there’s a couple of things here that could be learnings for future dependency confusion mitigation approaches.
For starters, I think PEP 708 expected a fair bit of coordination between users, installers, and indexes. I think it was doable, but it needed a coordinated push to get going. Unfortunately, that didn’t happen, probably due to time and energy constraints. That’s no one’s fault.
I think any future approach needs to lay out how the coordination will get off the ground, or use an approach that requires less coordination.
Secondly, I think backward compatibility is a real challenge for any dependency confusion mitigation, because there are such ingrained conventions working against most of the potential improvements I’m aware of. I think it is possible, but again, there will need to be a community consensus and plan from the start.
I think there is a fair push these days for security improvements, so hopefully something can be designed, planned, and implemented.
Thanks @dstufft for the work on PEP 708, I found it quite reasonable to implement. I just didn’t end up with enough time to see it through.
Thanks for the work you did do, and for your thoughtful analysis. I agree with everything you say, and you provide some good lessons to learn for the future.
I’m hoping that index priority solutions (already implemented in uv, and planned in pip) will offer a better way forward, and with luck someone will find the time to work on standardising the approach. In the meantime, tool-specific options are a good start.
I agree standardizing index priority seems like a good path forward. Perhaps it would be good to revive PEP 766 and modify it to be about resolver behavior when interacting with indexes.
I was less involved with PEP 708 than others in this thread, but it does seem that implementation has stalled. Index priority looks like a nice direction to move, there are fewer parties involved in “implementation” for that standard to stick and be successful.
uv mitigates some of them with its sources mechanism (allowing for per-package selection of alternative repositories), but it’s a whole lot of fiddly configuration that each affected end user has to set up for themselves, without any potential for an ecosystem level “it just works” outcome.
Offering prioritisation is still a good feature, it’s just not a direct substitute for what PEP 708 was trying to do.
There’s definitely a chicken-and-egg problem with the roll out, though: without PyTorch or piwheels configuring the metadata, there’s little immediate practical benefit to it for anyone from adding the feature to clients, but without clients adding the feature, there’s no benefit to repositories adding the metadata.
So if we wanted to revisit the rollout rather than abandoning it, the next concrete steps would be:
ask the piwheels.org admins about what they would need in order to report “tracks” metadata for their repositories
ask the PyTorch alternative index admins what would be needed to report “tracks” metadata for their repositories, or the more authoritative “alternative locations” metadata (and maybe ask about upload time metadata as well, since they don’t publish that either)
Agreed. It’s far from a simple “let’s just standardise what tools are already doing” exercise. However, the problem with PEP 708 doesn’t seem to me to be a technical one, rather it’s a social one. No-one is interested in moving it forward, and I think that’s something we have to acknowledge.
But any future PEP to standardise index priority will need to address the points raised in PEP 708. And for me, the most important thing is to be very clear on what the goals of such a PEP are. So far, people have been broadly supportive of “index priority” as a mechanism, but a mechanism isn’t enough. Unless someone can clearly state what problems they solve - and how they solve them - it will be hard to put together a compelling PEP for standardising index priorities.
Those steps could have been done anytime in the past 3 years. Having them happen now in response to the threat of withdrawal doesn’t seem like a healthy way of assuring that PEP 708 will be supported long term.
There’s been basically no objections, so I’m going to leave it another week or so (for personal reasons) and then mark PEP 708 as withdrawn.
PEP 1 process note: authors withdraw, delegates/SC reject. So I suggest you (as delegate) reject it.
A PEP can also be “Rejected”. Perhaps after all is said and done it was not a good idea. It is still important to have a record of this fact. The “Withdrawn” status is similar - it means that the PEP author themselves has decided that the PEP is actually a bad idea, or has accepted that a competing proposal is a better alternative.
I’m fine marking it as Withdrawn as well. I still think it’s a good idea, but hard to argue against the fact that nobody else has bothered implementing it
I’ve been thinking this over, and while I think index priority is a good idea and should probably be considered in addition to PEP 708, I’m also considering if another option here is we could implement PEP 708 on pypi.nvidia.com. There still wouldn’t be an implementation in pip nor uv, but that might help break the chicken and egg problem we’re currently in.
As I said above, I don’t really like the implication that it’s only the threat of rejection that prompts people to action. No-one is going to bother with the user side of PEP 708 unless it’s (fairly) widely adopted, so my measure of success isn’t just about the numbers, it’s about community interest in the solution as a whole.
This feels very similar to the discussion about deprecating the HTML index format, but with the added issue that PEP 708 is pointless unless multiple indexes adopt it. I don’t want us to get bogged down in meta-issues like “how do we get people to pay attention to our standards?”[1] but equally I’d rather we focused our energy on things where we can actually make a difference.
That week has now more or less passed, and I have some time available, so I’m going to formally declare PEP 708 as Rejected (thanks to @hugovk for pointing out that I reject PEPs, it’s the authors that withdraw them).
Thanks to everyone who worked on the PEP, both on the the PEP itself and on the implementation work that did get done. I’m sorry that work ended up being discarded, but I do think that as a community we have learned something from the process.
