This less than ideal outcome is indeed the result of needing to protect PyPI against automated/scripted access/scraping.
Over the last week we saw a dramatic increase in this kind of activity against the (relatively) expensive to serve project, release, and search endpoints. Worse, this activity was coming from over 1000 unique IP addresses with randomized user-agents.
This caused two major and one minor outage that coincided with the floods.
Availability of a search API is a sore spot for PyPI, there’s no doubt. We used to have the XMLRPC search API which was ultimately disabled due to having no way to communicate with the end-users who ware flooding it.
It seems that the same kinds of automation were eventually pointed at the HTML search page at https://pypi.org/search/ and led to the same kinds of abuse.
Protecting availability of PyPI for known/intended/supported use-cases is a priority for myself and the team, and in this case browser validation was the only tool we had to reach for to ensure it in cases like this. I’m grateful for it, but understand the impact it has on use-cases like Thonny’s.
I don’t think this work is “complete” and feedback like this is helpful in understanding use-cases. We flipped the switch on Friday to keep PyPI stable over the weekend, and will be looking at this and other feedback in the future to understand the impact and consider how we might support use-cases like this down the line.