How to specify extra-index in a pyproject.toml for pip and pip-tools?

[CAM-Gerlach]

@omnipy

the most simple method I could think of is to give pip a explicit method of telling which index to use for a specific package. package name collisions then could be handled on a package registry level. myindex.com/package-a would be something different then pypi.org/package-a. And a dependency could use any of them by explicitly adding the index to use, rather then the current “round robin” behaviour of pip that just cycles all indexes until it finds a first match (hopeing it’s the package we want, and not a different of even a malicious one).

Just to note, PEP 708 seeks to solve this problem, with what seems to be a fairly similar approach at the package index level to what you propose:

PEP 708 – Extending the Repository API to Mitigate Dependency Confusion...

[peircej (Jon Peirce)]

I don’t understand the resistance to this feature, at least for the option of specifying the new index for a specific package as supported in requirements.txt. I understand that you might not want to specify an additional index for all packages being installed for reasons of conflicts but specifying a url for a specific package is very useful

If a package has dependency that is not on PyPI then pyproject.toml is the natural place to define these in a way that they get picked up regardless of which build/install tool is being used. This isn’t a pip config thing, or a PDM or Poetry thing, it’s a project thing.

Is it insecure to allow a custom package index? No more insecure than allowing a dependency to be installed from a git repo, which is already supported in pyproject.toml

[sinoroc]

@peircej

No more insecure than allowing a dependency to be installed from a git repo, which is already supported in pyproject.toml

Is it? Ah, yes, but then PyPI would reject the upload of such package, right?

[sergiu (Sergiu Bivol)]

What PyPI does is irrelevant. This is about building and installing packages, not about publishing them to PyPI.

[AmmoniumX (Diego Garcia)]

This seems like the most logical way to add such a feature.

Allow a project to list extra index urls on pyproject.toml, and follow the already implemented logic of --extra-index-url of first searching on the official pypi, then look on extra index urls.

If a valid package is then found on the given repository, then explicitly ask the user if they want to “trust” this url in the future, then add it to their user list of extra urls.

This makes perfect sense and is clear about downloading packages from external indexes.