PEP 458: Secure PyPI downloads with package signing

[mnm678 (Marina Moore)]

To prevent any confusion, we can remove the mention of Microsoft.

@lukpueh

I still think this statement is vastly overstating what the PEP addresses.

I agree that it can be refined. What about something along the lines of…?

“This PEP aims to protect users of PyPI from compromises of the integrity, consistency and freshness properties of PyPI packages, and enhances compromise resilience, by mitigating key risk and providing mechanisms to recover from a compromise of PyPI or its signing keys.”

This looks good. I’ll add it to the proposed text.

@lukpueh

It also protects against malicious CDNs/mirrors, which usually don’t have access to the signing keys. Furthermore, the PEP recommends to store some upper-level role keys (root, targets and bins) offline, which allows a seamless recovery from compromises of online keys (timestamp, snapshot, bin-n).

These are additional benefits that should be added to the intro.

@steve.dower

MITM attacks and client-side redirection attacks seem to be the primary vector being protected against. They should at least get a mention.

I agree, these attacks should be mentioned.

Here is a quick rewrite with those changes. I think we could mention the attacks from @SantiagoTorres somewhere as well.

Attacks on software repositories are common, even in organizations with very good security practices (Attacks on software repositories · theupdateframework/pip Wiki · GitHub). The resulting repository compromise allows an attacker to edit all files stored on the repository and sign these files using any keys stored on the repository (online keys). In many signing schemes (like TLS), this access allows the attacker to replace files on the repository and make it look like these files are coming from PyPI. Without a way to revoke and replace the trusted private key, it is very challenging to recover from a repository compromise. In addition to the dangers of repository compromise, software repositories are vulnerable to an attacker on the network (MITM) intercepting and changing files. These and other attacks on software repositories are detailed here. This PEP aims to protect users of PyPI from compromises of the integrity, consistency and freshness properties of PyPI packages, and enhances compromise resilience, by mitigating key risk and providing mechanisms to recover from a compromise of PyPI or its signing keys. In addition to protecting direct users of PyPI, this PEP aims to provide similar protection for users of PyPI mirrors.

To provide compromise resilient protection of PyPI, this PEP proposes the use of The Update Framework (TUF). TUF provides protection from a variety of attacks on software update systems, while also providing mechanisms to recover from a repository compromise. TUF has been used in production by a number of organizations including Cloudflare, Datadog, DigitalOcean, Docker, Flynn, IBM, Kolide, LEAP, RedHat, and VMware. More details about TUF are included later in this PEP and in the specification.

This PEP describes changes to the PyPI infrastructure that are needed to ensure that users get valid packages from PyPI. These changes should have minimal impact on other parts of the ecosystem. The PEP focuses on communication between PyPI and users, and so does not require any action by package developers. Developers will upload packages using the current process, and PyPI will automatically sign these packages. In order for the security mechanism to be effective, additional work will need to be done by PyPI consumers (like pip) to verify the signatures and metadata provided by PyPI. This verification can be transparent to users (unless it fails) and provides an automatic security mechanism. There is documentation for how to consume TUF metadata in the TUF repository. However, changes to PyPI consumers are not required, and can be done according to the timelines and priorities of individual projects.