PEP 621: how to specify dependencies?

[sdispater (Sébastien Eustace)]

It feels like a really disingenuous way of representing each PEP (one being still in discussion). The TOML representation purposefully mixes notations, where there is no need to, which makes it less readable.

I have yet to mention it of the TOML PEP discussion but we could have a shorthand syntax for version-only cases to avoid clutter. At this point the TOML representation would be (still provisional though):

[project.dependencies]
cached-property = ">= 1.2.0, < 2"
distro = ">= 1.2.0, < 2"
docker = { version = ">= 4.2.2, < 5", extras = ["ssh"] }
docopt = ">= 0.6.1, < 1"
jsonschema = ">= 2.5.1, < 4"
PyYAML = ">= 3.10, < 6"
python-dotenv = ">= 0.13.0, < 1"
requests = ">= 2.20.0, < 3"
texttable = ">= 0.9.0, < 2"
websocket-client = ">= 0.32.0, < 1"

# Conditional
"backports.shutil_get_terminal_size" = { version = "== 1.0.0", markers = "python_version < '3.3'" }
"backports.ssl_match_hostname" = { version = ">= 3.5, < 4", markers = "python_version < '3.5'"}
colorama = { version = ">= 0.4, < 1", markers = "sys_platform == 'win32'" }
enum34 = { version = ">= 1.0.4, < 2", markers = "python_version < '3.4'" }
ipaddress = { version = ">= 1.0.16, < 2", markers = "python_version < '3.3'" }
subprocess32 = { version = ">= 3.5.4, < 4", markers = "python_version < '3.2'" }

[ofek (Ofek Lev)]

I simply copy/pasted the section from each PEP…

[sdispater (Sébastien Eustace)]

Oh, my bad then

It seems there is still work to do to iron out the TOML PEP. Like I said I still need to catch up on it and give feedback to improve it.

Again I am sorry for that comment but I think we should hold off on making comparisons just yet while it’s still a work in progress.

[EpicWink (Laurie O)]

@sdispater

The TOML representation purposefully mixes notations

You’re right, I deliberately mixed notations to showcase the flexibility of TOML, when perhaps I should have presented the most readable option. Should have followed the KISS paradigm.

@sdispater

we could have a shorthand syntax for version-only cases

A quick example of something that nearly achieves this is dotted-key notation

cached-property.version = ">= 1.2.0, < 2"

There’s immediately a likely point of confusion for users there, however, for projects with dots in the name (which is really an issue in all situations in the exploded table form)

"backports.shutil_get_terminal_size".version = "== 1.0.0"

The only project I had come across other than backports which had dots was ruamel.yaml, so perhaps it’s not common enough to be an issue, and can be solved via documentation in the project’s installation instructions

Probably should have put this post in the PEP’s discussion topic

[pf_moore (Paul Moore)]

@EpicWink

perhaps I should have presented the most readable option.

Ultimately, the PEP needs to present the agreed-upon option. If there’s no agreed-upon option yet, then framing it as a PEP was probably misleading (certainly to people who are not actively pushing for a TOML format, who are basically waiting for the people who are to come up with a consensus). If it’s still just a starter for discussion, then yes, it probably is too early to be comparing the two options.

Brett’s proposal was that there should be “a draft proposal being actively discussed as a topic here”. I take that as meaning that the people in favour of the expanded approach should have reached at least rough consensus on what they want to be considered as their proposal. It sounds like there’s still some work to be done to get that rough consensus. I’d consider the best time to post a draft PEP to be once you have that consensus on what you want to say, personally…

At some point we really do need to limit the decision to just two proposals - PEP 508 and some well-defined “expanded” syntax. If we can’t get to that point, we’ll need to decide what to do - drop the whole thing, go with PEP 508 syntax because it’s the only concrete proposal we have, or let things drift on for even longer. I personally think the debating has gone on long enough, and we need to make a decision and be done with it.

[brettcannon]

@pf_moore

At some point we really do need to limit the decision to just two proposals - PEP 508 and some well-defined “expanded” syntax.

Yes, and the point of the deadline was to make sure there were people dedicated enough to see each idea through to a conclusion.

@pf_moore

If we can’t get to that point, we’ll need to decide what to do - drop the whole thing, go with PEP 508 syntax because it’s the only concrete proposal we have, or let things drift on for even longer. I personally think the debating has gone on long enough, and we need to make a decision and be done with it.

Yes, if those interested in an idea can’t come to a conclusion on what to include in a PEP proposal then I would say it lacks enough support to be considered. I think it looks like both camps are on there way, though. My assumption is we’re are going to be done debating this by the end of the month:

1. Reach consensus on design by each camp by Tuesday, Sep 8
2. Have PEPs submitted by Tuesday, Sep 15 (since a design would have been agreed upon this is just writing it down succinctly)
3. Debate until Thursday, Oct 1
4. Make a decision sometime shortly after that

[uranusjr (Tzu-ping Chung)]

@EpicWink

The only project I had come across other than backports which had dots was ruamel.yaml , so perhaps it’s not common enough to be an issue, and can be solved via documentation in the project’s installation instructions

I believe the most popular project (likely by a large margin) using a dot in its canonical name is coverage.py.

[EpicWink (Laurie O)]

@brettcannon

Reach consensus on design

I’ve listed some open-issues in the draft. In addition I’d like comment about the switch to the PEP 610 direct property. Those invested in the exploded table can head to the discussion topic

[pf_moore (Paul Moore)]

There are also zope.interface and flufl.enum. Namespacing using dots isn’t massively common, but it’s not unheard of either…

[merwok (Éric Araujo)]

@uranusjr

I believe the most popular project (likely by a large margin) using a dot in its canonical name is coverage.py .

That may be its «marketing» name (it appears in its readme and documentation, sometimes even capitalized), but the metadata name is coverage.

[EpicWink (Laurie O)]

The PEP-draft for the exploded-table dependencies format can be found at: https://github.com/EpicWink/peps/blob/5531d87a5507233ad32131c41eeb3ee04f602c05/pep-9999.rst

[ofek (Ofek Lev)]

Here is the PEP 508-based version: https://github.com/python/peps/blob/master/pep-0631.rst

edit: note there is no support for syntax highlighting in the main repo, see https://github.com/python/peps/pull/1577

[brettcannon]

If the PEP is at a stable enough place to go into the PEPs repo then please submit a PR and I can get it assigned a number and checked in. It can obviously keep getting tweaks if you are mostly happy with where things stand.

Thanks to @ofek, @EpicWink and everyone who has/is/will participate in this!

And just as a more visible clarification here from the exploded table topic: the whole optional-dependencies table bit currently in PEP 621 can be changed with good reasoning behind it if that proposed split is too rigid or non-ergonomic for people.

[pf_moore (Paul Moore)]

@brettcannon

It can obviously keep getting tweaks

As far as I’m concerned, any of the PEPs can be modified right up to the point where things get submitted for approval. When I’m asked to make a decision, I’ll simply review the PEPs as they are at that point.

One thing I feel I should mention - these PEPs are mandating a particular input format for build backends. So far, the focus has been on refining the two dependency proposals, which is exactly as it needed to be. But once that’s done, I’d hope we can move back to consensus building and co-operation, and not simply leave it as “we don’t agree, pick one”. We may not get full consensus, but everyone should at least understand both dependency PEPs, as they will be expected to implement whichever one is approved. If we’re going to end up in a situation where it looks like projects might ignore the standard because a decision didn’t go their way, there’s no point in having a standard at all

[ofek (Ofek Lev)]

@pf_moore

situation where it looks like projects might ignore the standard because a decision didn’t go their way, there’s no point in having a standard at all

I can’t speak for other backend maintainers, but Hatch will only support the PEP that is chosen. I don’t have a strong enough preference to reject a standard

[steve.dower]

I likely won’t adopt any of this entire PEP until a library (hopefully packaging) can do all the work for me. One of the pymsbuild “features” is minimal processing/validation of metadata, so I’m certainly not going to inject my own interpretation of converting to a METADATA file in there.

[brettcannon]

I’m very much assuming there will be some library that will take the dict from reading the TOML and provide a function to spit out the appropriate METADATA and such.

[ofek (Ofek Lev)]

@brettcannon

• Reach consensus on design by each camp by Tuesday, Sep 8
• Have PEPs submitted by Tuesday, Sep 15 (since a design would have been agreed upon this is just writing it down succinctly)
• Debate until Thursday, Oct 1
• Make a decision sometime shortly after that

I’m not certain how the debate is supposed to go, but I guess I’ll start it off by saying currently I prefer the PEP 508-based approach (link: https://github.com/python/peps/blob/master/pep-0631.rst)

[brettcannon]

@ofek I’m going to ping the relevant PEP threads to make sure they feel ready to have a focused discussion and then I will start a new thread (this one is way too long to build off of).

[brettcannon]

I have created How to specify dependencies: PEP 508 strings or a table in TOML? to discuss the two PEPs. I’m going to lock this topic now to make sure future discussions happen over there.