Supporting signed file URLs in the simple package repository API

Packaging
1

Currently, PEP 503 (simple package repository API) and PEP 658 (serve metadata file in said API) specify that related distribution files (the GPG signature and metadata files) must be served at the same URL path as the distribution, but with a suffix (.asc and .metadata respectively).

This is not possible with signed URLs, where the signature (usually included as part of the URL’s query string) is computed from the URL path (for example, see AWS S3 presigned URLs), even if the distribution file itself can be served like this.

<a
  href="https://my-pypi.s3.us-west-2.amazonaws.com/files/foo/foo-1.42.0-py3-none-any.whl?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIASPAM%2F20220509%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20220509T014753Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&X-Amz-Signature=abcd1234#sha256=abcd1234"
  data-python-requires="&gt;=3.7"
>
  foo-1.42.0-py3-none-any.whl
</a>

I propose adding a way to specify alternate paths for these related files in the package’s file-list page. For backwards compatibility, I think the best solution is to introduce new HTML anchor tag attributes (eg data-gpg-sig-url and data-dist-info-metadata-url) which are exclusive with the existing attributes (eg both data-gpg-sig and data-gpg-sig-url can’t both be specified). This allows existing consumers to ignore the unknown attributes. data-dist-info-metadata-url would allow specifying the hash as a URL fragment (the same as the distribution).

If PEP 691 (JSON simple repository API) is accepted before this proposal, I’ll propose similar distribution object keys.

An alternate solution which is compatible with the current spec is to have package file URLs hit the index server, which then dynamically redirects to a signed URL for all three files (distribution, GPG signature, metadata file), but this adds latency and server load.

If there’s interest in solving this problem, I’ll draft up a PEP.

PS: PEP 691 (JSON API) was the first time I head of PEP 658’s (metadata file serving) acceptance (and the second time I heard of the PEP at all). I don’t think there was enough communication of its acceptance, given how active I am in following the community. What do I have to subscribe to to see packaging PEP announcements?

1 Like
2
3

Unrelated aside, but these kinds of URLs can’t be cached either (see Ignore URL query parameters when caching · Issue #10075 · pypa/pip · GitHub and Cache responses on the "other side" of redirects with status code 303 · Issue #10694 · pypa/pip · GitHub), which may become an issue for you.

If you’re building something around this, you may want to proxy file requests rather than redirecting to an authenticated URL.

1 Like