PEP 633: Dependency specification in pyproject.toml using an exploded TOML table

To clarify, does this mean the valid TOML representation here would be like this?

aiohttp = { git = "ssh://git@github.com/aio-libs/aiohttp.git" }

Because this is the only valid URL form, which is mandated by PEP 508.

What this means is that both representations are valid:

aiohttp = { git = "git@github.com:aio-libs/aiohttp.git" }
aiohttp = { git = "ssh://git@github.com/aio-libs/aiohttp.git" }

but they must both translate to the following PEP-508 representation:

aiohttp @ git+ssh://git@github.com/aio-libs/aiohttp.git

Thanks for the clarification.

If you don’t hear from anyone in the next 24 hours I will sponsor to help keep this moving forward (but I am staying neutral between both proposed PEPs).

You’re right, there were errors in the examples. A VCS repository path must be a URL as specified in PEP 440. The PEP has been updated to reflect this.

1 Like

Looks like no one came forward so I submitted PR review comments to https://github.com/python/peps/pull/1595 to make myself the sponsor of this PEP.

1 Like

Some things that came up in the review:

  • Both the empty string "" and the empty table {} are currently meant to signify no constraint, which fails the Zen as there’s multiple ways to do things. In addition, the empty array [] is disallowed (specifically because an array is really supposed to be used for multiple requirements. Currently even now the spec allows an array of 1 item…)
  • Should unknown keys cause an error, cause a warning or just be silently ignored? I could see not erroring meaning tools could have specific functionality based on non-standard keys, for better or worse.

Done.

I can appreciate the irony :slight_smile:, but the motivation, atleast for me, is to make use of the structure that TOML affords us to allow for better downstream tooling. So, it does not necessarily have to be what other communities name things.

I think this should be left to the tools to handle, however, needs to be explicitly noted that if tool A specifies key K, tool B has no obligation to honour it. Alternative, is to enforce this and allow for tool specific configuration to be handled, similar to dataclass’s metadata parameter.

One could argue that practicality beats purity. That said, the use of [] failing introduces ambiguity. I feel it might be good to simply allow [] so that strings, inlinetables and arrays are allowed. I know its going in the other direction, but seems more pragmatic. But ofcourse this is a subjective view.

The Zen of Python says there should be one obvious way, not there must only be one way. And the obvious way may not even seem obvious unless you’re Dutch. Don’t disallow representations for the sake of it, but only if it makes sense. Lay out things logically, and good design will follow.

7 Likes

You could expand it to “any false-y value” and then even false in TOML would work :wink:.

If you are going to allow both string and table values then supporting both can make sense. But right now, the PEP is written where the empty string support is a single line mentioning it and the rest of the whole PEP just uses the {} format. I would either justify having both a bit more and think about which one will be the most common, or choose one.

But this also leads to potentially more serious future-conflict if we decide to officially add a key; that’s way more serious than two tools conflicting.

If you want to allow for tool-specific keys you will probably need to namespace them somehow. Otherwise I would disallow non-standard keys.

Are you all ready to discuss https://www.python.org/dev/peps/pep-0633/ in general with the PEP 508-related solution?

1 Like

All of the PEP 508 examples in this PEP have unnecessary parentheses. For example:

aiohttp (>= 3.6.2, < 4.0.0)
aiohttp = { version = “>= 3.6.2, < 4.0.0” }

Should really be:

aiohttp >= 3.6.2, < 4.0.0
aiohttp = { version = “>= 3.6.2, < 4.0.0” }

The other PEP consistently omits the parentheses, and TBH I’ve never seen them used in the wild. (Honestly, it’s not enough to sway me to this proposal over the other, but it jumps out as making the comparisons look a little unfair.)

2 Likes

As far as I know, parentheses are accepted by PEP-508 and, in my opinion, they improve readability by explicitly separating the package name from the version constraint.

And it’s the format used by the PyPI JSON API (see https://pypi.org/pypi/ipython/json). I know this API is not standardized but it’s still there. It also is the format used and specified by the PEP 345 for the Requires-Dist metadata.

it jumps out as making the comparisons look a little unfair

How so?

Since I don’t think any package author writes their dependency specifications with parenthesis – and that’s what this PEP is for. :slight_smile:

1 Like

I don’t understand. This PEP is for a TOML representation of dependencies so package authors, if they use this PEP to do it, won’t have to bother knowing the PEP 508 notation.

And honestly, this PEP is for a project’s metadata representation, so the acceptable generated specification format should be the core metadata specification (https://packaging.python.org/specifications/core-metadata/) which uses the parentheses notation (https://packaging.python.org/specifications/core-metadata/#requires-dist-multiple-use).

So, to me, it seems that this PEP gives a generated PEP-508 format that will be accepted everywhere without modification.

Parentheses are allowed in PEP 508, but using them in Core Metadata specifically is discouraged:

A version specifier. Tools parsing the format should accept optional parentheses around this, but tools generating it should not use parentheses.

1 Like

They will still need to know PEP 508 form for pip command lines, for requirements files, for reading package metadata directly, etc. IMO, it’s unreasonable to assume that users will never need to know PEP 508 syntax (at least for the simpler cases up to maybe the level of pkg[ex1,ex2] >=1.0, <2.0).

1 Like

@EpicWink re-reading the PEP I think there is some ambiguity introduced due to the optional-dependencies table mandating for-extra. IMHO, the use of for-extra makes for a case against needing optional-dependencies as it keeps thigns under a single dependencies table. I would suggest that we recommend optional-dependencies table be avoided.

Regarding the for-extra option, I think we should also make a mention of what needs to happen if marker contain something like python_version >= 3.6 and extra == 'http' and the specification contains for-extra = "http". The simplest resolution would be for for extra to be simply appended as and extra == '<for-extra-value>', I reckon.

2 Likes

You can recommend that optional-dependencies be removed, but I wouldn’t suggest avoiding it when we are doing something new like this. Do whatever gives the best ergonomics and experience for the user and if that means tweaking PEP 621 then so be it since it’s not an accepted PEP yet either.

This is what already happens in the reference implementation. I’ll make an explicit note about it

It’s have recommended to keep the style consistent in the examples. To satisfy both their’s and your recommendation, one of them should be dropped. Earlier in this topic the empty string was specifically requested, so it sounds like the empty table should be disallowed.