…and the error says it’ll be deprecated in the future? That’s obviously bad, why is it just being turned off instead of fixed? The need to find packages who’s exact name you don’t know won’t go away, just because the ability to search for them has been taken away, lol.
Is there any discussion about this anywhere? Is this a problem I can help fix?
I find it extremely hard to believe that doing a full text search is an impossible task.
Furthermore, the error says to check https://status.python.org for more info… but pip and/or package searching is nowhere on the status page, and there’s no indication at all that anything is other than 100% operational. Which is also extremely bad.
Because if I’m trying to install something, and apt search doesn’t have any results, the next step is naturally to check pip search.
But that doesn’t work, which is annoying.
So I check https://python.org, but there’s no search. There is at least a link to pypy from python.org, but if you don’t know that pip is pulling from pypy, that link is not helpful.
Because PyPI (Warehouse) doesn’t expose it for 3rd party use. Which is almost certainly because the last time they did, it got abused to the point where it was causing service issues.
I’ve updated the issue on the pip tracker here to propose that we finally deprecate and remove pip search, so that we no longer mislead people like yourself who think that it might work… (Please understand “why not just fix it” is not a helpful comment, and will not be received positively - we have considered the options and there really is no viable way to just make pip search start working again).
I appreciate your frustration here, but honestly, you should just go to the PyPI website and use the search box there. It may not be ideal for your workflow, but it works, which is more than can be said for blaming the pip or PyPI maintainers…
Not a problem, it’s just that a lot of people over time have asked “why not just…” in one form or another, which gets a bit wearying. My comment was more in the form of a pre-emptive strike against anyone who reads this thread, then goes to the linked issue and starts complaining. That’s one reason I think we should probably just remove pip search at this point.
No need to mark the thread as solved, if we just leave it here it’ll be fine.
I realize I’m not adding anything helpful here, but it will absolutely be a surprise for newcomers with any kind of experience with other ecosystems that virtually all familiar “package managers” have some solution for searching via a command line interface, and Python does not. Consider apt/yum/dnf/zypper/pacman/(others-I-left-out) for Linux environments, choco search for Windows users interested in Chocolatey, plus gosearch, npm search, gem search, ppm search and…
Please see some viable solutions below from a cybersecurity engineer with decades of experience such as one or more of the following:
Add a limiter on the server side to only allow X searches (especially larger ones) within X minutes and then lock out that specific IP for whatever time period is required to keep servers healthy. This can be done quickly and with no client updating and server side solutions only such as apache modules and iptables firewall rules.
Longer Term Solutions:
Change the default behavior of pip to cache search results by default and only grab new package lists every 24 hours unless overridden
By default, limit searches to X rate unless a user has some other identifier (such as an api key / login) so abusive clients can be disabled
Local caching is the approach of well-known Linux distribution package managers (dnf, apt, pacman, etc.). Don’t know if there are reasons why that is less viable in the Python world - possibly a lesser acceptance of great globs of data being stored locally?
My Debian/Sid workstation has over 200MiB of LZ4-compressed package
metadata. The current package count in Sid is around 165K entries.
Depending on what you want to search on PyPI, at best its project
count is more than double that right now, and the count of
releases/files pip has to take into account is one or two orders of
Granted, the type and volume of metadata used by apt and pip are not
the same, but it should be fairly apparent that there is a
significant difference in package quantity and churn between a (very
large) Linux distro and PyPI.
Perhaps an opt-in mechanism would be more appealing - for the Debian world analogy, think apt-file, which is not default, and has to have the database populated by request, but then does provide for a number of very useful queries,
The local caching approach was my motivation for GitHub - domdfcoding/pypi_search: Metadata for a client-side search of PyPI, which is updated every 30 minutes with new releases’ names and summaries. The tool is modelled after apt-cache, which only searches (the debian equivalent of) that metadata. The total size is about 35MB, but since it’s a git repository you only have to download the delta each time you refresh your local version.
Of course it’s not MS-DOS DOSKEY blasting in from the 1980s. It’s “doskey.exe” for the Windows NT console, blasting in from the 1990s. Each application that’s attached to a console session has a command history buffer, plus a set of input aliases that match at the beginning of an entered line of text. Attached applications (e.g. “cmd.exe” or “python.exe”) have nothing to do with this. For example, when an application calls ReadConsoleW(), the console host has already applied any matching alias such as “pip”.
“doskey.exe” provides command-line access to the command history and input alias functions in the console API. The API for input aliases is documented: