Discouraging/deprecating PEP 427-style signatures?

Packaging
1

Hi all,

I’m opening this up for discussion. The title isn’t intended to be leading – I don’t know whether this is a good idea or not, and I figure others in the community are better placed/informed on the current adoption of PEP 427 signatures than I am :slightly_smiling_face:

TL;DR: Are PEP 427-style “wheel signatures” vestigial and, if so, should they be formally discouraged and/or deprecated?

Background

PEP 427 and its living equivalent both mention “signed wheel files”, which essentially consist of a .whl that contains a RECORD.jws or RECORD.p7s which in turn contain JSON Web Signatures or a detached S/MIME signature, respectively.

The PEP and living spec call these “attached signatures” because they reside within their respective wheels, rather than outside of them (like the “detached” PGP signatures of the time, or the contemporary PEP 740 attestations).

The problem

“Signed wheel files” are a prominent section with the living binary distribution format specification.

Because of that, users may end up believing that they’re widely supported or used by the ecosystem. More specifically, users may end up conflating them/their usage with PEP 740, leading to cases where users attempt to use PEP 740’s machinery to provide PEP 427-style signatures like here.

This kind of confusion doesn’t violate any security properties, but it’s suboptimal in terms of sending ordinary Python community members on a bit of a wild goose chase through wheels, PEP 427, PEP 740, etc., all to discover that “attached” signatures are (probably, see below?) vestigial.

Proposal

Given the opportunity for confusion and their (probably?) vestigial nature, I’d like to propose either discouraging or deprecating PEP 427-style signatures.

“Discouraging” would mean modifying the living PyPUG spec to note that adoption of PEP 427 was limited, and that most users should probably prefer a PEP 740-style attestation instead.

“Deprecating” would mean formally setting RECORD.{jws,p7s} on a path towards hard removal, i.e. eventually being rejected outright by PyPI and other indices for some future wheel revision.

I’m not sure if either (or both) of these is the right thing to do, per the open questions below :slightly_smiling_face:

Open questions

  1. Are attached wheel signatures in fact vestigial? I have only a partial vantage point which is PyPI, so it’s possible there are other contexts where PEP 427-style signatures are widely used. If this is the case then discouraging/deprecating does not make sense, although perhaps some disambiguating language between the living specs would help users (which I could own doing).
  2. Procedurally, what would the process for deprecating these signatures look like? Other parts of the ecosystem (see links below) appear to have removed or never completed support for these, so I’m not sure if a PEP is a good idea or not.

Other links/resources

3 Likes
2

The signatures built on RECORD are based on the assumption that the RECORD is checked, but the hashes in RECORD itself are currently not checked by pip and subsequently also not by uv, instead we’re checking the hash for the entire wheel file on the simple index page. imho, we should deprecate the signature files plus the hashes in RECORD, not we them to be removed, but to signal that they are unused and should not be relied on, instead as you wrote having signature on the index as with the attestations.

1 Like
3

(post deleted by author)

4

Given that there were no further comments, I’d like to move this proposal ahead. Specifically, I’d like to propose:

Motivations:

  • RECORD.jws and RECORD.p7s haven’t found mainstream usage, and aren’t check by any major tool I’m aware of. They are not supported in pip nor uv.
  • These signatures are presented as security features in the spec, when they in fact don’t provide any security (they aren’t checked).
  • The newer, supported security features, specifically hash checking for archives and attestations, use external hashes and signatures, which are presented in the index, instead of internal signatures.
  • Since these features build on top of RECORD checking (which pip and uv don’t do), this is a first step before being able to discuss further RECORD changes that align spec and tool implementations.

@pf_moore I don’t know what the process here is, for a PUG spec change that is not a clarification but smaller than a real PEP, is there any process description i missed? I checked with PEP 1 but it does not seem applicable, and I couldn’t find anything on PUG itself. Unless there’s more opinions on this more concrete proposal, should we do something like a two-week final comment period with the above proposal?

1 Like
5

The possibilities are:

  • Readability improvements that don’t affect software interoperability
  • Changes which can affect interoperability, which are agreed as a text-only change
  • Changes which can affect interoperability, which need a full PEP

The lack of feedback here also includes a lack of approval that this can be a text-only change. Personally, I think this probably does need a PEP, mainly because it’s removing agreed functionality, and I think that is more than we should be doing as a text-only change.

I don’t think a PEP needs to be complex, but I do think it should describe what we recommend anyone who uses these signatures[1] should transition to.

You’ll need a sponsor for a PEP - I’m not sure it’s acceptable for me to be a sponsor and PEP-delegate, so you should look for someone else. But if you are having an issue finding someone, I’m willing to be put down as sponsor, and I’ll check with the SC whether that’s OK.

  1. As William noted, we don’t have full visibility of who might be using this feature, so we shouldn’t assume no-one is - although I think it is fair to take the view that covering the removal in a PEP is sufficient notification for such people ↩︎