It would also make legacy eggs arbitrary code execution during pip install or import a no longer a problem in term of security
Just for inspiration/precedent, the Rust ecosystem has a similar system for packages on crates.io: Advisories › RustSec Advisory Database and GitHub - RustSec/cargo-audit: Audit Cargo.lock files for crates with security vulnerabilities acting as an auditing tool for dependencies.
Ruby also has a community maintained database of vulnerabilities: GitHub - rubysec/ruby-advisory-db: A database of vulnerable Ruby Gems
And there is a Go proposal/prototype to do the same: vulndb - Git at Google
Thanks Oliver! I like the general idea. I’ll try to find some time to read your proposal and give feed after I have dealt with 3.10 feature freeze frenzy.
There is already a commercial security database for Python packages at https://pyup.io/. They release their database once a month at GitHub - pyupio/safety-db: A curated database of insecure Python packages
@tiran depends if this is more for the project on GitHub or coming to a general conclusion. I read:
as the key question. If it’s more for the idea then I can move it back.
@brettcannon Yes that’s the key question from me
For more general packaging/warehouse discussion, I opened an issue at github/pypa/warehouse/issues/9407 (I’m not able to post the full link), or should there be another discussion topic on here for that?
Just a note, the
pypa orgs are managed by different people and have different criteria to join. If joining
psf does not end up making sense (I don’t personally have an opinion but don’t have a say anyway), it may make sense for you to join
pypa instead if you are inclined to. See also: PyPA Members, And How To Join — PyPA documentation
I love the idea of such a database! I hope either the PSF or the PyPI guys will love it too…
I talked to @EWDurbin about this offline and we both agreed that this probably would be more appropriate under the
pypa org instead.
Can a mod move this back to the Packaging category so we can give this a little more time for discussion with the members there?
Assuming this doesn’t get any significant opposition there, I’ll probably kick that off sometime next week.
Moved over to Packaging
Here’s a sample of what an advisory would look like: python-vuln-examples/PYSEC-2021-63.yaml at main · oliverchang/python-vuln-examples · GitHub
I’ll be starting the pypa-committers vote to create this as a new repo,
pypa/advisory-db on Monday!
cool! I in particular like " The goal is to have the
pip install (and an additional
pip audit ) command automatically report vulnerabilities out of the box." - that would be supercool
And flexible for some don’t care about it at some point in time too.
The advisory repo looks like it’s being kept up to date by osv-robot · GitHub which as far as I can tell looks like a Google-run bot. Is the source for that bot available somewhere? Will a non-Googler be able to continue to maintain this repo if Google chooses to stop sponsoring the work?
python friends. I help maintain the advisory-db for RustSec, and also contributed a smidge to the OSV spec. If there’s any stuff I can share around our experience running a vulnerability DB, please just ask!
@ehashman Great question. The source for the robot (and the rest of the OSV tooling) is here: https://github.com/google/osv (the robot specifically is here: https://github.com/google/osv/blob/master/docker/worker/worker.py)
The goal of the repo is that while it can be backfilled/bootstrapped by this robot, individuals can also submit advisories directly to it as well.
The project is in its infancy but Google is making an effort to ensure it is not entirely Google-driven, e.g. there is a proposal that helped define the vulnerability format that had many community contributors: https://tinyurl.com/vuln-json
Thanks for the positive feedback everybody!
This is now public at pypa/advisory-db on GitHub (I’m not allowed to post the full link here for some reason) and currently contains 550~ advisories dating back to 2019 (more will be backfilled soon).
If you would like to use the CVE database, Thoth’s resolver can automatically remove packages with vulnerabilities stated in the database - see the demo:
Currently, we are ingesting dependency data for other operating systems than Red Hat Enterprise Linux 8 and UBI 8 (with Python 3.8).
If you would like to consume recommendations for other operating systems, feel free to let us know to eventually prioritize data ingestion based on the user base.
And last but not least, thanks for the database!