Continuing the discussion from Structured, Exchangeable lock file format (requirements.txt 2.0?):
What if we invented a new meta-namespace, that included both? So e.g.
"pypi:gulp"would mean “what pypi calls gulp”, and
"conda:gulp"would mean “what conda calls gulp”, and now we can use both vocabularies at the same time without namespace collisions.
And we’ll also need a meta-meta-namespace to tell us what the
More seriously, the idea is almost exactly what Fedora (and probably other RPM-based distros) does now. Let me summarize to avoid re-inventing eggs.
Fedora uses use “virtual provides”. Those are typically used for “aliases”: for example,
python2-requests package (still) provides
python-requests for backwards compatibility, so an old package requiring
python-requests doesn’t break (yet).
But it’s not just for aliases. For example, names of installed files are included as virtual provides. If I’m missing the
http command, I can install if directly with
dnf install /usr/bin/http, and forget that it’s from the project httpie Or was it
python3-httpie in Fedora? (I don’t mean to pick on the author here; naming is hard.)
Virtual provides do complicate the system, though – requirements are “searched for” rather than just “looked up”.
For Python packages:
- A package like
python-requests(from the Fedora namespace) can specify that it provides
requestsfrom the PyPI namespace.
- Other packages can Require either name.
- A Provides generator automatically looks for
dist-infometadata when any package is built, and adds proper
python3distProvides to the package.
- And a requirement generator now also automatically puts in the Requirements. (This has been turned on by default about a year ago.)
It’s taken a while to gradually get this implemented (and shepherd package maintainers to use it), but it works quite nicely.
Obviously, it’s not cross-platform, but it’s cross-ecosystem: in the same way, you can for example require headers for a C/C++ library, say
pkgconfig(Qt5) – and if you build an extension with that, another dependency generator scans the extension’s symbols and adds a Require to the right version of
It still doesn’t make it trivial (=automatable) to build Fedora-quality RPMs from PyPI packages (which often lack tests and licences, have different convention for descriptions, etc.), but it’s the first few steps :)
Please steal ideas as appropriate. (Most likely, though, this is not appropriate for Pip now.)
Digression: Distribution Package is the PyPA term for the stuff you download from PyPI, but I avoided that term here since I also discuss packages from a Linux distribution. The terms “project” and “package” and “module” are similarly confusing. (Thankfully, “distro” is always an OS distribution (where OS is Operating System, not OpenStack.)) Hah! Namespaces! Let’s do more of those.