We’re figuring out how best to install a mixture of public & internal Python packages. The simplest solutions all involve using either
--extra-index-url to tell pip where to find our internal packages, and letting it fetch the rest from PyPI. However, if someone finds the name of an internal package (which is hardly a secret), they could upload a malicious package of the same name on PyPI, and we may get that installed. This is not considered a bug in pip (issue #8606).
The best workaround at present appears to be running an index server (with devpi, pypi-server or simpleindex) which can proxy or redirect requests to PyPI, but where internal packages can shadow public ones, so once I have made an internal
foo package, no
foo from PyPI will be found. You pass this as
--index-url rather than
--extra-index-url (though the latter will appear to work ). If you use a lot of packages, though, there’s a risk you one day want a public package with the same name, because it’s all the same namespace. And the server is extra complexity compared to just putting internal packages in a directory or e.g. a Gitlab package registry.
Alternatively, you could namesquat your internal packages on PyPI so no-one else can upload them. But it’s easy to forget to do that when you make a new package, and no doubt some organisations are reluctant to publish even package names.
Would it make sense to reserve some prefix for internal packages, so the same names could not be claimed on PyPI? I can see a couple of possible variants of this idea:
- Find some currently legal prefix like
internal-which is not yet used on PyPI (so
internal-itself is out), and make PyPI refuse to accept packages named with that prefix.
- Expand the legal characters in package names to allow a new prefix like
internal:, without accepting it on PyPI.
I’m suggesting this as a rough and ready way to make the easy options more secure, not a perfect solution. It would treat PyPI specially, as the global default index, rather than trying to describe index preference or trust levels in general.