Good news!
New York University – specifically Professor Justin Cappos – and I have successfully asked the US National Science Foundation for a grant to improve Python packaging security. The NSF is awarding NYU $800,000 over two years – from mid-2021 to mid-2023 – to further improve the pip dependency resolver and to integrate The Update Framework further into the packaging toolchain.
https://nsf.gov/awardsearch/showAward?AWD_ID=2054692&HistoricalAwards=false
What we’re planning to do
NYU researchers and developers will
- Further assess and improve pip’s dependency resolver, following up on the work done in 2020 and making ResolveLib more reusable by other tools in the packaging ecology
- Secure the PyPI-to-user pipeline by integrating TUF support for signed packages throughout PyPI’s clients (we’re targeting conda, pip, and bandersnatch initially)
And I’ll be part of this work, paid to work on this part-time, doing some outreach, coordination, project management, and similar.
This is a plan and subject to change.
What this means in the short term
More of the same, in a lot of ways. You already see Justin’s Secure Systems Lab doing stuff like PEP 458 and helping with the TUF rollout on PyPI. And: Once the funding from CZI and MOSS ran out I started spending way less time on packaging-related stuff, but I’ll return to higher levels of active attention and participation once the NSF-funded work kicks off. With NYU folks, I’ll be helping nudge PEP 480 along so we can come to resolution on it.
I will of course keep folks apprised of progress through this and similar public channels. Since this wasn’t funded through the Packaging Working Group of the PSF, I won’t be indexing updates on that wiki page but I’ll find some other public place to archive them.
Why NYU & NSF?
Justin and other folks in his lab have been pushing forward on Python packaging security for several years, including authoring PEP 458 way back in 2013. He’s used to applying for NSF grants and it was great to work together with him on this and learn the ropes a bit. NYU will of course collaborate with the PyPA, the PSF, and other stakeholders as we move forward.
The National Science Foundation is an arm of the United States government that funds advances in science and engineering – see their About page for more. This funding is for a “Transition to Practice” project in the NSF’s “Secure & Trustworthy Cyberspace” program.
The future is bright for the PSF to participate more directly as a co-proposer on these kinds of proposals in the future, and Justin is working to help with that!
Thanks
Thanks to the NSF for the award, to Justin for leading this proposal and NYU for supporting it, and to y’all for supporting improvements in Python packaging security!
Please reply with any questions.
Best,
Sumana
(some relevant threads: PEP 458: Secure PyPI downloads with package signing , Prerequisites & vetoes -- improving packaging security , RFC: improving pip security with package signing (PEP-458) , Current/upcoming funding for PyPI/packaging security (roundup FYI) )
