Makes sense! I’m going to update the PEP (via the current open PR) to remove any prescriptions about metadata producers needing to/performing label normalization, and move the standard entirely to the index/consumption layer.
Thanks for pinging me! We can support these changes in the uv build backend.
Currently, the only place using Home-page and Download-URL is uv publish, which attaches those fields as formdata. I have implemented the normalization changes on top of the build backend branch:
The PEP already looks very complete so I have only some minor comments:
The mechanism for presenting this warning or notification is not specified, since it will vary by index. By way of example, an index may choose to present a warning in the HTTP response to an upload request, or send an email or other notification to the maintainer(s) of the project.
Given then current underspecifiedness of the upload api, I don’t think the http response part is feasible without PEP 694, which specifies its own mechanism for notices.
If a distribution’s metadata contains both sets of fields, the index MAY choose to reject the distribution outright. However, this is NOT RECOMMENDED until a future unspecified major metadata version formally removes support for Home-page and Download-URL.
If we want, we can ban build backends from emitting Home-page and Download-URL in the next minor version, i.e. if you want to write let’s say metadata version 2.4, you must not include those fields. Timing is difficult since there are no “METADATA releases” independent of individual PEPs and PEP 639 is also getting stabilized.
Metadata producers SHOULD emit the normalized form of a user specified label, but MAY choose to emit the un-normalized form so long as it adheres to the existing 32 character constraint.
I could omit it entirely, but this was intended to reflect the current behavior of Warehouse: the current file_upload handling returns a list of (newline-delimited) warnings within the body of the 200 OK response, which twine logs when present.
(I could make it clearer that this isn’t intended to be normative/prescriptive over an upload API, just an example of something that’s already doable with PyPI’s upload contract.)
I would indeed like that, although I already have at least two other PEPs queued up after this so I don’t want to commit to a personal timeline .
FWIW the latest changes actually remove the obligation that build backends perform any normalization at all; the PR mentioned above (#4010) changes it so that the consumer side only performs normalization as part of processing/presentation.
I changed that based on @pf_moore and @ncoghlan’s feedback about producer-side normalization potentially modifying the user’s intent too much. But if you feel strongly that the producer-side should also be normalizing here, then I’m happy to revisit it
Reading the PR that removed producer normalisation, I think one clarification that is still needed is that indexes are only performing normalisation for their own use when deciding how to render their web UI - the metadata APIs still publish the original metadata (since the index is acting as the metadata producer in that case).
Tangent: I wonder if the duplicate label issue is the reason PEP 566 didn’t try to transform the Project URL list into a JSON dict when converting the metadata.
@woodruffw - Before I give a decision, I’d like to ask one question. Please note, I am not asking for the PEP to be revised here, I want to confirm my understanding of the text as it stands, and how that will impact index implementations.
The PEP says
Any changes to the interpretation of metadata of version 1.2 or later that result in previously recognized URLs no longer being recognized SHOULD NOT be retroactively applied to previously uploaded packages.
So, if project Foo 1.0 contains the following metadata:
an index that currently displays the “old” homepage for Foo 1.0, should continue to display that when the index is updated to support PEP 753. But if Foo 2.0 is uploaded, with no metadata changes, Foo 2.0 should show the new homepage (while the display of Foo 1.0 remains unchanged).
Two questions:
Have I interpreted the text of the PEP correctly here?
Will PyPI handle this situation the way I have described?
I ask because maintaining a distinction between Foo 1.0 and Foo 2.0, which have identical metadata, seems like it’s an additional burden on index implementations that isn’t clearly called out.
As this is only a “SHOULD”, not a “MUST”, indexes do have the option to skip this complication. But I’d hope that the intention is for PyPI to implement the full spec, which is why I want to clarify now.
Yes and yes: the intent behind the text is was to express the de facto behavior of PyPI when it comes to metadata rendering, which is that changes to rendering only affect new uploads and are not backfilled to anything already present.
(At least IME this is not a burden for PyPI – doing the backfill would actually be more burdensome, since every presentation change would obligate a somewhat costly data migration )
Excellent. With that understanding, I’m happy to say that PEP 753 is accepted. Congratulations!
Thanks for your work on this PEP. While it’s a relatively small change in some ways, it directly affects user presentation of project data, and that’s something that always stirs up people’s opinions Your work in managing the discussion here is appreciated. We’ve added some much-needed consistency in how project URLs are presented to the user, and given project authors better direction in how to present their key locations to users.
Please can you update the PEP to mark it as Accepted and link back to this post as the resolution?
To explain why I was unclear - I was imagining a model where the URL information was generated “on the fly” from the project metadata, rather than being precomputed. An index server that generates information on the fly might find adhering to this requirement harder, but as it’s a SHOULD, it’s acceptable for them not to do so.