PEP 633: Dependency specification in pyproject.toml using an exploded TOML table

[EpicWink (Laurie O)]

Would anyone like to sponsor this PEP submission?

[xafer (Xavier Fernandez)]

This PEP in its latest form seems to allow the use of url or VCS specific keys (git, hg, etc).
I don’t see the reasoning behind the split ?

Would this mean that both:

aiohttp = { git = "ssh://git@github.com/aio-libs/aiohttp.git" }

and

aiohttp = { url = "git+ssh://git@github.com/aio-libs/aiohttp.git" }

are equivalent ? Is it really worth the extra keys ?

Also, the git specific format aiohttp = { git = "git@github.com:aio-libs/aiohttp.git" } seems to be allowed as it is present in the VCS Dependency example section.
Is it intentional ? If it is, we might want to explicitly say so and clarify that VCS specific locations are allowed and that all the backends must be able to transform them in a proper URI to be stored in METADATA.

[sdispater (Sébastien Eustace)]

My understanding was that the url was only to be used for direct remote links to package distributions and it’s specified in the PEP:

the URL of the artifact to install and satisfy the requirement. Note that file:// is the prefix used for packages to be retrieved from the local filesystem.

So VCS urls are not permitted for the url key.

@xafer

Also, the git specific format aiohttp = { git = "git@github.com:aio-libs/aiohttp.git" } seems to be allowed as it is present in the VCS Dependency example section.
Is it intentional ?

I think so, yes. It’s a common format for git ssh urls so it makes sense to support them. However, I think there is an error (or typo) in the PEP since it would translate to aiohttp @ git+ssh://git@github.com/aio-libs/aiohttp.git@master and not aiohttp @ git+ssh://git@github.com:aio-libs/aiohttp.git@master which is invalid for PEP-508, I think.

[uranusjr (Tzu-ping Chung)]

@sdispater

I think so, yes. It’s a common format for git ssh urls so it makes sense to support them. However, I think there is an error (or typo) in the PEP since it would translate to aiohttp @ git+ssh://git@github.com/aio-libs/aiohttp.git@master and not aiohttp @ git+ssh://git@github.com:aio-libs/aiohttp.git@master which is invalid for PEP-508, I think.

To clarify, does this mean the valid TOML representation here would be like this?

aiohttp = { git = "ssh://git@github.com/aio-libs/aiohttp.git" }

Because this is the only valid URL form, which is mandated by PEP 508.

[sdispater (Sébastien Eustace)]

What this means is that both representations are valid:

aiohttp = { git = "git@github.com:aio-libs/aiohttp.git" }
aiohttp = { git = "ssh://git@github.com/aio-libs/aiohttp.git" }

but they must both translate to the following PEP-508 representation:

aiohttp @ git+ssh://git@github.com/aio-libs/aiohttp.git

[uranusjr (Tzu-ping Chung)]

Thanks for the clarification.

[brettcannon]

If you don’t hear from anyone in the next 24 hours I will sponsor to help keep this moving forward (but I am staying neutral between both proposed PEPs).

[EpicWink (Laurie O)]

@sdispater

However, I think there is an error (or typo) in the PEP

You’re right, there were errors in the examples. A VCS repository path must be a URL as specified in PEP 440. The PEP has been updated to reflect this.

[brettcannon]

Looks like no one came forward so I submitted PR review comments to https://github.com/python/peps/pull/1595 to make myself the sponsor of this PEP.

[EpicWink (Laurie O)]

Some things that came up in the review:

• Both the empty string "" and the empty table {} are currently meant to signify no constraint, which fails the Zen as there’s multiple ways to do things. In addition, the empty array [] is disallowed (specifically because an array is really supposed to be used for multiple requirements. Currently even now the spec allows an array of 1 item…)
• Should unknown keys cause an error, cause a warning or just be silently ignored? I could see not erroring meaning tools could have specific functionality based on non-standard keys, for better or worse.

[abn (Arun Babu Neelicattu)]

@EpicWink

@abn can you sign the Python CLA (or add your GitHub to your bugs.python.org account) please?

Done.

@brettcannon

Correct. That can change with the proper argument. But do note that naming was done based on how other communities name things (which is ironic as that’s an argument this group is making against PEP 508 ).

I can appreciate the irony , but the motivation, atleast for me, is to make use of the structure that TOML affords us to allow for better downstream tooling. So, it does not necessarily have to be what other communities name things.

@EpicWink

Should unknown keys cause an error, cause a warning or just be silently ignored? I could see not erroring meaning tools could have specific functionality based on non-standard keys, for better or worse.

I think this should be left to the tools to handle, however, needs to be explicitly noted that if tool A specifies key K, tool B has no obligation to honour it. Alternative, is to enforce this and allow for tool specific configuration to be handled, similar to dataclass’s metadata parameter.

@EpicWink

Both the empty string "" and the empty table {} are currently meant to signify no constraint, which fails the Zen as there’s multiple ways to do things. In addition, the empty array [] is disallowed (specifically because an array is really supposed to be used for multiple requirements. Currently even now the spec allows an array of 1 item…)

One could argue that practicality beats purity. That said, the use of [] failing introduces ambiguity. I feel it might be good to simply allow [] so that strings, inlinetables and arrays are allowed. I know its going in the other direction, but seems more pragmatic. But ofcourse this is a subjective view.

[uranusjr (Tzu-ping Chung)]

@EpicWink

Both the empty string "" and the empty table {} are currently meant to signify no constraint, which fails the Zen as there’s multiple ways to do things.

The Zen of Python says there should be one obvious way, not there must only be one way. And the obvious way may not even seem obvious unless you’re Dutch. Don’t disallow representations for the sake of it, but only if it makes sense. Lay out things logically, and good design will follow.

[brettcannon]

@EpicWink

Both the empty string "" and the empty table {} are currently meant to signify no constraint, which fails the Zen as there’s multiple ways to do things. In addition, the empty array [] is disallowed (specifically because an array is really supposed to be used for multiple requirements. Currently even now the spec allows an array of 1 item…)

You could expand it to “any false-y value” and then even false in TOML would work .

If you are going to allow both string and table values then supporting both can make sense. But right now, the PEP is written where the empty string support is a single line mentioning it and the rest of the whole PEP just uses the {} format. I would either justify having both a bit more and think about which one will be the most common, or choose one.

@abn

I think this should be left to the tools to handle, however, needs to be explicitly noted that if tool A specifies key K, tool B has no obligation to honour it. Alternative, is to enforce this and allow for tool specific configuration to be handled, similar to dataclass’s metadata parameter.

But this also leads to potentially more serious future-conflict if we decide to officially add a key; that’s way more serious than two tools conflicting.

If you want to allow for tool-specific keys you will probably need to namespace them somehow. Otherwise I would disallow non-standard keys.

[brettcannon]

Are you all ready to discuss https://www.python.org/dev/peps/pep-0633/ in general with the PEP 508-related solution?

[steve.dower]

All of the PEP 508 examples in this PEP have unnecessary parentheses. For example:

aiohttp (>= 3.6.2, < 4.0.0)
aiohttp = { version = “>= 3.6.2, < 4.0.0” }

Should really be:

aiohttp >= 3.6.2, < 4.0.0
aiohttp = { version = “>= 3.6.2, < 4.0.0” }

The other PEP consistently omits the parentheses, and TBH I’ve never seen them used in the wild. (Honestly, it’s not enough to sway me to this proposal over the other, but it jumps out as making the comparisons look a little unfair.)

[sdispater (Sébastien Eustace)]

As far as I know, parentheses are accepted by PEP-508 and, in my opinion, they improve readability by explicitly separating the package name from the version constraint.

And it’s the format used by the PyPI JSON API (see https://pypi.org/pypi/ipython/json). I know this API is not standardized but it’s still there. It also is the format used and specified by the PEP 345 for the Requires-Dist metadata.

it jumps out as making the comparisons look a little unfair

How so?

[pradyunsg (Pradyun Gedam)]

@sdispater

How so?

Since I don’t think any package author writes their dependency specifications with parenthesis – and that’s what this PEP is for.

[sdispater (Sébastien Eustace)]

I don’t understand. This PEP is for a TOML representation of dependencies so package authors, if they use this PEP to do it, won’t have to bother knowing the PEP 508 notation.

And honestly, this PEP is for a project’s metadata representation, so the acceptable generated specification format should be the core metadata specification (https://packaging.python.org/specifications/core-metadata/) which uses the parentheses notation (https://packaging.python.org/specifications/core-metadata/#requires-dist-multiple-use).

So, to me, it seems that this PEP gives a generated PEP-508 format that will be accepted everywhere without modification.

[uranusjr (Tzu-ping Chung)]

@sdispater

As far as I know, parentheses are accepted by PEP-508 and, in my opinion, they improve readability by explicitly separating the package name from the version constraint.

Parentheses are allowed in PEP 508, but using them in Core Metadata specifically is discouraged:

A version specifier. Tools parsing the format should accept optional parentheses around this, but tools generating it should not use parentheses.

[pf_moore (Paul Moore)]

@sdispater

This PEP is for a TOML representation of dependencies so package authors, if they use this PEP to do it, won’t have to bother knowing the PEP 508 notation.

They will still need to know PEP 508 form for pip command lines, for requirements files, for reading package metadata directly, etc. IMO, it’s unreasonable to assume that users will never need to know PEP 508 syntax (at least for the simpler cases up to maybe the level of pkg[ex1,ex2] >=1.0, <2.0).