PEP 610: Recording the origin of distributions installed from direct URL references

@pradyunsg Agreed. The PEP looks pretty good to me, and I was mostly just waiting for the formal request for pronouncement. I normally like to give the PEP a final review myself before pronouncing, but as I’m not particularly familiar with the subject matter, I’d be perfectly happy with you being BDFL-delegate on this one.

@sbidoul once you’re ready, just confirm here that the PEP is good to be submitted for approval, and @pradyunsg can take it from there. And congratulations on a well-run PEP discussion - I know it took a while, but thanks for sticking with it.

2 Likes

The implementation in pip did raise only one minor issue for which the PEP text has been updated. It is also complete, except for two optional features which can be implemented in follow-ups: 1/ generating the hash for URLs pointing to archives 2/ resolved_revision_type/resolved_revision.

So yes, @pf_moore @pradyunsg, I do think it is ready to be submitted for approval.

1 Like

@pradyunsg soft ping.

1 Like

@pradyunsg If you’re happy to formally pronounce acceptance for this, then the next steps would be:

  • post the acceptance here
  • submit a PR to the PEPs repo that changes the PEP status, links to that post in the Resolution field, and updates the BDFL-Delegate field (if Paul didn’t already update that)

Whoops, I didn’t. Thanks for the reminder - @pradyunsg go ahead and include that change in the acceptance PR, as @ncoghlan suggests.

Thanks for the nudge @ncoghlan @pfmoore!


I’m happy to provisionally accept this PEP! We’ll mark it as “Final” once the rollout is done, to allow for changes (if any are needed) after the initial release of the implementation.

Congratulations @sbidoul @cjerdonek!

2 Likes

PEPs repo PR: https://github.com/python/peps/pull/1330

A big thank you to all involved!

Hi all,

PEP 610 has been in pip since 2 releases now, with no negative feedback so far, to the best of my knowledge.

The implementation is complete except for two optional parts:
1/ generating the hash for URLs pointing to archives which is blocked on a broader hash recording feature in pip
2/ resolved_revision_type/resolved_revision which is complex to implement in pip and for which pip has no direct use case

Therefore I would like to propose to move PEP 610 to the Final state.

I will then convert the text to a specification at https://packaging.python.org/specifications/direct-url. I plan to leave out the unimplemented features from the specification and add them later if/when there is implementation feedback.

Sounds good to me.

I’m slightly worried that the direct URL record isn’t used much except in pip freeze, where the output is quite forgiving. I kind of want to see something that uses the direct URL info in its structured form to prove the current format is useful without backward incompatible changes. Specifically I want to know whether the current format has enough to support direct URL upgrade strategy.

Not an unreasonable suggestion, but there’s no ETA on that change, and I think the normal practice is for features to be provisional for a set period of time, rather than based on how much it gets used and what for.

https://www.python.org/dev/peps/pep-0411/#criteria-for-graduation is the only formal definition I can find for the process of moving from provisional to final, and it’s fairly strict on what would cause a provisional PEP to not get moved to final.

If upgrading wasn’t part of the discussion when PEP 610 was approved, then it’s reasonable to say it’s not in scope for the move to final. If the feature needs changing to handle upgrades, then we can always do a follow-up PEP that modifies the design to cater for that feature.

It’s @pradyunsg’s call (as BDFL-delegate) but I’d be inclined to not wait on a direct URL upgrade implementation, unless someone can propose a specific deadline by which that’s expected to be implemented.

I’m not too worried with the upgrade scenario because the URL is preserved in full, except for the fragments for which only the fragments that pip cares about are preserved (namely subdirectory). There might be interoperability issues one day but that would be because the URL part of PEP 508 is not normative, not because of PEP 610 itself.

Thanks for the clarification. Would it be allowed to specify additional fields in direct_url.json if we eventually find that more context is needed? (e.g. URL redirect destination.) The spec won’t need to be changed in an incompatible way if we are allowed to add to the format, and my worry wouldn’t matter in that case.

Adding optional fields to a json structure is something that is generally considered backward compatible for consumers of the data structure, so that should provide a sufficient evolution path indeed.

BTW, @pradyunsg, talking about things to record in .dist-info :slight_smile:
Do you think we can move forward and bring PEP 610 to final state?
I might then take a moment during the holidays to transform it into a spec at https://packaging.python.org/specifications/.

Yup. Adding the spec there this year, would be great. I think we can mark this as final once the spec is in the packaging.Python.org pages. :slight_smile:

I updated the PR at Add reference to PEP 610 by sbidoul · Pull Request #690 · pypa/packaging.python.org · GitHub with the full specification extracted from PEP 610.

And I resolved my previously blocking review, so this should be live soon :slight_smile:

@pradyunsg, gentle nudge for the last step, as we are now in a situation where the spec is published but the PEP is still Provisional.

1 Like