Detecting Quarantined Packages

Packaging
1

We’ve been trying to find a way to detect quarantined packages, but we haven’t found any first-class APIs to make it easy.

Is there an easy way to detect quarantined packages?

We tried screen-scraping, e.g., https://pypi.org/project/crazy-thursday/. This works sometimes, but mostly gets blocked by Fastly client challenges.

We finally settled on the following, which appears to work, but isn’t ideal:

# For quarantined packages, the simple API returns a 200 response
curl -I https://pypi.org/simple/pandass/

# But, the JSON API returns a 404
curl -I https://pypi.org/pypi/pandass/json

This heuristic of a 200+404 response from respective calls to the simple and JSON APIs seems to correctly detect quarantined packages.

Is there a better way to do this?

Jon

2 Likes
2

Hi @jonyoder ! Curious why you’re looking for quarantined projects - what’s the value of knowing this, vs knowing that a project isn’t available for installation?

3

Hey @miketheman!

We’re recording snapshots of packages over time, and it’s very useful to know that a particular package (that may or may not have been available in the past) is unavailable due to being quarantined. For example, then you can ask questions like, “should all historical snapshots of this package be made unavailable or added to a vulnerability list?”.

I can provide more details if you’re interested.

Jon

4

It’s in the XML RCP API. e.g.

echo '<?xml version=\'1.0\' encoding=\'ASCII\'?><methodCall><methodName>changelog_since_serial</methodName><params><param><int>26724070</int></param></params></methodCall>' | curl -XPOST -T- -L -H 'Content-Type: text/xml' -H 'Accept-Encoding: gzip' https://pypi.org/pypi | gzip -d

That’s an incremental/changelog API where you have to replace the number 26724070 with the with the serial of the last event you’ve seen in order to get the next 50,000 events. It’s not clear to me if you want to monitor the whole of PyPI or just some specific packages.

5

@bwoodsend Oh interesting. We’re already using the XML RPC API to monitor changes, and it’s been working well. I guess we missed the fact that information about quarantined packages was introduced in that feed. I’ll check it out! Thanks.