Current/upcoming funding for PyPI/packaging security (roundup FYI)

@brettcannon just wrote “How do you verify that PyPI can be trusted?” and mentioned that we need funding to implement PyPI compromise detection and to make PyPI harder to crack. He pointed to PEP 458 (“Surviving a Compromise of PyPI”) and PEP 480 (“Surviving a Compromise of PyPI: The Maximum Security Model”), which are both Deferred pending funding.

The Packaging Working Group has been seeking funding for improvements to packaging security, and now’s a good time for a roundup – no response needed, just FYI:

  • Facebook has already funded some work on cryptographic package signing and malware detection that we aim to start later this year. This work may involve using The Update Framework, which PEP 458 recommends. PSF is working on the Request for Interest now - we’ll publish that and use the responses to refine budget and architectural choices (like: TUF or not?), then we’ll have a Request for Proposals, then PSF will decide what to accept, then PSF and contractors will start the work.
  • The new PyPI audit log we just launched is an aid to compromise detection. (Funded by an Open Tech Fund grant.)
  • The in-progress rewrite of pip’s dependency resolver will help make it possible for PyPI to better know packages’ dependencies and dependents and enforce various kinds of metadata compliance more strictly. Packaging WG is working on requesting funds from Mozilla via MOSS to help us finish this implementation, and has already submitted a grant proposal to the Chan Zuckerberg Initiative to support complementary pip work to assure a well-tested & well-designed rollout.

The last few grants have been focused on PyPI. That’s meant @EWDurbin has had to spend a lot of time on this stuff instead of or on top of other Director of Infrastructure tasks. So, for the next several months, we need to delay proposing new funded work on PyPI specifically. So I think we should concentrate on finding funding for other packaging codebases, like pip, manylinux, wheel, auditwheel, etc. And if we want money for something that includes substantial Warehouse/PyPI work then we should plan it so that work doesn’t start till at least mid-2020.

More details on fundable projects on the wiki.

4 Likes