PEP 725: Specifying external dependencies in pyproject.toml

[rgommers (Ralf Gommers)]

@msarahan

When you talk about something becoming a known name, I think there’s two parts, and I’m not sure the PEP is explicit about this (or maybe I’m just dense)

You are completely right, and @tgamblin said something similar higher up:

@tgamblin

Who defines canonical names and canonical package structure for externals as described in this PEP, though? That should probably be defined a bit more specifically.

These questions are nontrivial. A few thoughts:

1. Yes a central list seems reasonable to add to this PEP.

2. PURLs do not have to be pre-registered. They are pretty unambiguous and while a registry may be useful (for pkg:generic in particular), I don’t see a need to forbid using any PURL that is valid (parses correctly). It’s similar to Python dependencies: it’s your responsibility to put in a name that makes sense, but nothing is stopping you from writing dependencies = ['nonsensical-package'] I think.

3. For virtual dependencies the situation is slightly different, because (a) there will be far fewer of them, and (b) an interface or language standard with multiple implementations may not be unambiguously versioned.

4. There should still not be a validation step for virtuals that, for example, would stop pip/build from building a wheel or an upload to PyPI.

5. The central list that @msarahan describes can exist standalone, before the name mapping mechanism exists. All it needs is PURL/virtual string, and a description field (free-form).

6. Allowing duplicate PURLs for the same package may make sense and be allowed. E.g., pkg:generic/openblas and pkg:github/openmathlib/openblas are both reasonable, and authors may have different preferences here.

   • In that case, the central list may need an “is alias to” field as well.

7. If the central list does come into existence, it should be a project/repo under the PyPA org I’d think. Something like pypa/pyproject-external?

8. Regarding @tgamblin’s comment on package structure and non-default build options, I think that is something that can’t reasonably be handled I think. If we get to well-defined metadata that gives you the exact names and versions of dependencies that’s a significant step forward. I cannot think of a way to say “give me version 1.2 of package X built with flags --with-libfoo --no-gpl-components”. That level of detail has to remain a docs-only thing I’d say.

If the central list seems reasonable to everyone, I can try putting that together, seeding it with the virtual:* and pkg:generic from the demo I posted higher up.

@msarahan

Is this too concrete for a PEP? If I’m on the right track, let me know and I can spend more time fleshing this out and coming up with diagrams.

That sounds great. I think that diagram should go in the name mapping PEP.