I used to work for the US Government, and I shipped cryptographic solutions in a bunch of places – when the US Government makes it hard on it’s own employees, we shouldn’t go out of our way to encourage their mistakes.
In any event, stdlib vs. not-stdlib isn’t relevant to almost any projects/agencies. If you’re concerned about the US Government’s ability to use crypto, FIPS 140 is the dominant concern.
Fortunately, with recent updates to FIPS and other NIST SPs, Ed25519 is now acceptable. Given it’s good design choices, slim API surface, performance, and ease of implementation, it’s the obvious choice for any signature scheme.
Paul Kehrer informed me that PyCA has a pure Python implementation for Ed25519 signature validation. This makes Ed25519 the obvious choice. I can add a C wrapper for OpenSSL’s Ed25519 to Python’s stdlib for maximum performance and compliance. pip can fall back to pure Python implementation for older Python versions.
I contemplated that old ed25519 library briefly. IIRC, the vulnerability warnings are related to side-channel attacks when signing allowing some exposure of the signing key. That’s not a natural concern in this context:
The signing is done repo-side, and that tooling really doesn’t profit from only-Python constraints. (i.e. you can use heavier libraries like NaCl for it). On the client side, there is no risk from the concerns motivating those warnings on signature verification – there’s no private key to expose. Even using the library on the server side, you could probably mitigate these attacks without too much effort.
That said, I opted not to go with that ed25519 implementation anyway, for other reasons – and that it is not maintained is certainly a consideration.
This is encouraging, and we’ve been waiting for this for a long time. I’m working on something similar for another group, and when all is said and done, I’ll provide a breakdown of the cost in time, but that’ll be far too late for this RFI.
I guess I would only say… If you go with a model that does not involve developer signing, like the minimum security model, try to leave room in the design (the metadata spec in particular, at least) for the ability to expand later. (TUF would do that for you, but you could do it otherwise, if you’re not going to use TUF as-is.)
ed25519 is sort of maintained – which is to say if there were bugs we are available to fix them. There just haven’t been any, nor any users. If pip were using it we could absolutely give it more love.
@trishankatdatadog There are some more voices who will be speaking up here before Sept. 18th, such as @EWDurbin and (I’m predicting) @woodruffw – it’s ok if we don’t have a consensus yet.
I’m still a pretty big fan of using TUF for PyPI and I’ve yet to see something that I feel like would be a better fit. I’m happy to look at other options and have folks attempt to persuade, but so far I’m still pretty sold on TUF.
I think the rough idea of PEP 458 -> PEP 480 is solid. I haven’t yet looked at in-toto to know if it’s something that would be generally useful for us or not.
The Pure Python ed25519 library is perfectly safe to use for signature verification, just not for signature creation. If I remember my history correct the entire reason it exists was when we started talking about this years ago the Pure python djb code was really slow so we took it and improved it to see if it was possible to get something fast “enough” for signature verification. It likely has no real users at this point but I have no doubts in PyCA’s ability to maintain it if we were to use it in pip.
In my mind I think I was planning on using RSA for the root keys because of hardware support for them, that was years ago though so perhaps ed25519 has good hardware support now for realtively cheap HSMs we can use for the root keys.
Hi @sumanah – I’m relaying some answers from the team here, including @woodruffw
Using TUF to implement PEP 458 makes sense. Given the limit of funding available, we think it might be too much to attempt to implement both PEP 458 and PEP 480, but we definitely support a plan of eventually implementing 480. Perhaps with the offers of pro-bono support in this thread, the more ambitious goal of both PEPs is doable.
We think Ed25519 is absolutely the right decision here, assuming you intend to implement TUF. Regarding the concerns about the Python version, one of the maintainers of PyCA (which contains the pure-Python implementation the Ed25519 signature scheme referenced in PEP 480) works with us and we’d happily update it if necessary for interoperability. Additionally, EdDSA is supported via YubiKey HSM and probably some other cheap HSMs, so there is no need for root signing keys to be RSA.
Edit:
One consideration for this project is how to minimize the burden on package authors. If a solution like PEP480 provides greater security guarantees but achieves a low opt-in rate by the package authors, then a solution like PEP 458 might achieve greater impact overall, despite being referred to as “The Minimum Security Model.”
I slightly disagree, a low opt-in rate is does not necessarily indicate a low impact. Getting a popular package such as urllib, or numpy signed has a larger benefit than getting something less well-known like bullet signed. I do agree however that getting buy-in from developers, particularly of larger/popular packages, is important for adoption of end-to-end signing.
@mike-myers-tob Happy to support the implementation of PEP 458 any way I can over at Datadog.
For both Mike and @LucidOne: we evaluated several strategies for buy-in from developers in Section 7 of the Diplomat paper. (Diplomat is just a codename for TUF.) For example, we found that:
If the most popular 1% of projects are signed by developers (406 projects), then 73% of users are protected. If the top 10% of projects sign their project, then 96% of users are protected. This shows that users overwhelmingly download only popular projects and so focusing on their protection is highly effective.
