PEP 833: Freezing the HTML simple repository API

[dstufft (Donald Stufft)]

@woodruffw

If it helps, I’m happy to add some qualifying language to the PEP to explain the weight of the SHOULD NOT and why it doesn’t preclude all future changes to the HTML representation.

Perhaps a FAQ or adding to the Rationale to say that this PEP is not intended to be an absolute prohibition to further PEPs targeting the HTML representation, and still allows future PEPs to make changes to it, but rather it means that a PEP that wants to change the HTML representation needs to make the case for why the change they want to make to the HTML representation is important enough add it even though we’ve “closed” it.

I’m personally thinking of it kind of like Python 2.7 was treated, it technically was no longer accepting new features, but a number of features did end up being backported to 2.7 because a case was made that those features were important enough to override the standard “no new features in a patch release” rule.

@notatallshaw

I’ll freely admit I am poorly educated on specification word lawyering, and my objection may be entirely faulty based on my own misunderstanding. I think everyone understands by point though and I don’t have anything else to add, so I’ll leave it up to PEP authors and approvers to decide if my point is worth considering.

SHOULD NOT X is allowing for the possibility of X, but it’s got an implication that X is expected to be rare or have caveats, issues, or other such things that may cause issues if you X.

Roughly speaking:

• MUST and MUST NOT - These are hard requirements of the specification, and if you don’t follow them then you’re “wrong”.
• SHOULD and SHOULD NOT - These are things that ideally would be MUST or MUST NOT, but the spec recognizes there may be cases where they are needed, so it allows it, but “only if you understand the consequences”. In the context of a PEP like 833, the consequences of adding something like upload-time is probably pretty minimal, so low risk (but for instance, it’s not unheard of in the land of RFCs for simplified tools to make assumptions that get invalidated if someone doesn’t follow a SHOULD or SHOULD NOT.
• MAY and MAY NOT - These are things that the spec fully allows you to both do or not do, and there typically is an implication that either choice is perfectly valid and equally reasonable. Compliant tooling is expected to handle these being followed or not followed.
• RECOMMENDED and NOT RECOMMENDED - These aren’t actually part of the normal RFC 2119 language, but they get used sometimes. Most of the time they’re basically the same as MAY and MAY NOT, but with an implication that one of the choices is better for some reason.

I do agree it’s kind of an awkward look if we end up accepting 833 and then shortly after we accept a hypothetical PEP that adds upload-time to the HTML, but 833 does allow for that.

Unfortunately the nature of OSS is that we the stuff that gets worked on is the stuff that someone feels motivated to spend their time on, so sometimes that means those awkward ordering of things are just going to happen. ^[1]

@pf_moore

I’m curious. Has anyone ever asked an index provider to support upload-time, and been told “no”?

I’m not sure how useful of a anecdotal data point it is, but the linked PR on uv where someone wanted to use Last-Modified to hack in support for something like upload-time, the author of that PR mentioned that they choose to switch to an entirely different vendor that allowed them to effectively implement --exclude-newer server side instead rather than try to interact with their original vendor to try and get them to add support for anything.

That was about implementing PEP 691/700, not about adding upload-time to HTML which isn’t quite what you asked, but the impression I’m getting from that thread is, at least that person’s experiences, is that their company was dreading trying to get their original vendor to do anything.

Which does then circle back to 833, and IMO, adds strength to it’s argument that the HTML format is effectively de facto frozen already, by nature of basically zero uptake in new features by anyone that isn’t PyPI.

I suspect there is a very real chance that, if we get a hypothetical PEP to add upload-time to HTML, that the effective out come is the same whether we accept it or not-- that PyPI would support it (but installers that understand it are fetching JSON from PyPI anyways) and nobody else (or practically nobody else).

The only reason I’m waffling at all and calling it a chance, and not a nearly certain outcome, is that specifically for upload-time itself, the effort to add that data (if you have it) is so trivial that maybe folks would have an easier time getting those vendors to do things.

---

1. Of course, it also sometimes means that good features are blocked by something nobody actually wants to work on too! ↩︎