If you really feel strongly about this feature, you can always add it in Hatch, no?
[project]
...
dynamic = ["dependencies"]
[tool:hatch]
dependency_file = "requirements.in"
I don’t think we need to make this impossible, there are legitimate use cases (particularly in the short run), but I don’t believe it should be in the standard.
If that’s the case, I think we should at least mention that in future versions dependency may become a table or a list containing tables. Maybe this wording:
Why? Surely anything can be changed in a new version. It’s up to the people who write the PEP for that update to handle backward compatibility.
Everything is reserved for future use. The definition of anything not in the [tool] namespace is reserved, so having dependency be a table is an error currently, and will remain an error if PEP 631 says that dependency must be a string. If a new PEP says it can also be a table, then (and only then) will it be anything other than an error. Tools that support the PEP 621/631 version of pyproject.toml will still reject tables as values, which is perfectly acceptable - we can’t mandate that tools support every as-yet undefined standard we introduce.
That’s already the case, because PEP 518 reserves all namespaces other than [tool].
I suppose that’s fine, but I think the main issue is that I think these two formats are the ones we’re most likely to adopt as extensions in the future, and it would be good if tools written today could do something reasonable with them even in the future. I think we already know what that reasonable thing to do is, too: consider the field dynamic and throw it to the backend to parse it (which will fail if it’s not implemented correctly).
I think it’s reasonable to call these out as valid possible future forms as compared (especially the second one) to, say:
[project]
dependencies = 3
I’m also on board with getting rid of the first reserved format (table of key/vaue pairs) and just saying that the list can contain tables, but the meaning of such tables are undefined — backends must not use them without a change to the spec, parsers that don’t know what to do with them should mark the field as dynamic. That would give us the freedom to implement something like this in the future:
[project]
dependencies = [
{file="requirements.in"}
]
But also the freedom to do stuff like this:
[project]
dependencies = [
"a_thing",
{dependency="something[extra]", editable=True}
]
I don’t really want either of these things, but allowing for the possibility would make the thing more extensible without breaking the entire ecosystem. The “exploded table” format does have the advantage that it’s easy to add additional keys in future specs in such a way that wouldn’t require updates to all existing parsers. If we reserve this format in the original spec and specify that parsers should fall back to “dynamic” in the event that they encounter this format that they don’t understand, we’ll also have that extensibility benefit.
Another possibility would be to add explicit versioning to the spec from the outset, and say that parsers should fall back to dynamic if this format is encountered and the spec version is higher than the highest version they understand.
I don’t feel terribly strongly about this — I don’t think it adds a huge amount of complexity to the parsing code, but I also understand the YAGNI concerns.
I’ll duck out at this point. I don’t really care, personally, I’m just pointing out that if the PEP gets over-complex and adds stuff that wasn’t part of the original discussion, there’s more chance people will object (or just not participate, because everyone’s appetite for these discussions seems to have got pretty low).
Opened a PR: https://github.com/python/peps/pull/1578
Sorry, I didn’t review the PR, but it says:
Backends MUST silently ignore these until the specification is extended.
This isn’t right. Backends should raise an exception if it encounters an inline table, because it’s definitely an error condition if your backend is coded against a version of the spec that doesn’t support features you are using!
If we include this “reserved for backwards compatible updates” logic, it is aimed at parsers other than the backend attempting to build the project. I think that any project that is not the build backend attempting to parse a pyproject.toml must consider a dependency specification including inline tables as invalid, but it SHOULD (or MAY) fall back to treating the relevant field as “dynamic”.
If we do it this way, if everyone’s implemented as specified and we never upgrade the spec, the presence of an inline table would always raise an exception when someone attempts to get the dependencies, because even parsers who treat it as dynamic would then invoke prepare_metadata_for_build_wheel, which would raise an exception in the backend. If we extend the spec in the future, parsers who don’t understand the updated spec will fall back to the backend, which will give the relevant information.
I was hoping someone other than me and Paul Moore would weigh in on this, though, because I’m ambivalent on whether or not it should be included. I think it’s an elegant way for us to keep open an escape hatch that allows for a bit of flexibility to extend the spec in the future, but on the other hand since implementing the spec correctly and incorrectly (i.e. falling back to the backend if an inline table is present) would have nearly identical behavior and there will be no real-life examples that involve inline tables, it’s very likely that support for this particular extension mechanism would be patchy, making future extensions less backwards compatible than desired.
Are you all ready to have https://www.python.org/dev/peps/pep-0631/ discussed compared to the TOML-based solution?
I am. Anyone else who is also ready may add a 
to this comment to avoid posting.
Honestly, TOML is pretty-Python like on most fronts:
dependencies = """
a == 1.0.0
b == 2.0.0
"""
We can use the textwrap.dedent / line.strip() logic, allowing all the lines to be indented.
I just read both this PEP and the “expanded tables” one before commenting on the comparison thread. In doing so, I found the “tools should allow for the possibility that we may have tables in future” provision in the PEP to be a serious distraction - it doesn’t say why the provision is there, it suggests that people can’t rely on the PEP remaining stable, and (worst of all) in the context of the debate, the lack of an explanation left the feeling that it meant “we may change our minds and go for expanded tables later”.
@ofek I strongly suggest you remove that part of the PEP - it adds nothing but confusion.
Shouldn’t this be in one of the broader threads (probably the underlying one about PEP 621)?
My problem here is that “pretty python-like” is not exactly “python”. The close-but-not-exact similarity adds more confusion for me, not less. For example, escaping backslashes and raw strings differ, and I have to look up how to specify Windows paths in TOML every time. Not an issue for this usage, I know, but nevertheless, you may be underestimating how confusing “nearly the same, but not quite” syntax can be for non-experts…
Oh, I’m very well aware of this. I wasn’t around when the initial design decisions on TOML were made and changing those things in TOML is non-trivial to put it mildly. 
There’s AFAIK three-ish points of difference:
= instead of : and disallowing newlinesHowever, I don’t know what the point of bringing this up is – it’s fine if it’s a passing statement expressing your (valid) concern around this area. OTOH, if you said that as “something worth addressing” – there’s no way we’re going to be able to achieve 100%-Python-like-semantics with anything other than a plain Python file (cough setup.py cough) and it’s going to be non-trivial to backtrack the choice of TOML for pyproject.toml now.
In other words, IDK if there’s anything that can be done to change it.
It was a statement specific to this PEP, and Brett said that we should stick to discussion PEP-specific in the PEP-specific threads, so I responded to that comment here.
Ah, OK. I see, you were just replying to “How to write a multi-line string”. I misread your comment as relating to “get rid of quoting and commas” which linked back to @steve.dower’s comment that setup.cfg is “the only one with neither quotes, braces or commas”. My mistake.
Can I please add an explanation rather than remove it or is this a blocker?
You can do what you want, as you’re the PEP author. My suggestion is a personal opinion¹. I’ve already spoken out pretty strongly against the {file = "xxx"} idea that this is a remnant of, so you know that I don’t want to see that included, and I have no interest in it being added later. But it’s ultimately your PEP, not mine.
In a purely structural sense, having a syntax “reserved for future expansion” with no further comment, particularly when the proposal is in competition with a different proposal that uses that syntax, is IMO confusing and distracting. I would recommend you address that regardless, but whether you choose to explain more about the intended possible future extensions, or just drop the whole idea for now, is your choice (and one you should expect to have to justify to people reading the PEP).
¹ My personal opinion becomes more significant if & when I’m asked to pronounce on the proposals, I guess, but at the moment that’s all this is - a personal opinion.
Okay, I will remove the inline table allowance.
To avoid making 2 commits, what do we think about allowing for the fields to alternatively be multiline strings in addition to arrays? @steve.dower and I are in favor PEP 631 - Dependency specification in pyproject.toml based on PEP 508
In that case, the reference example could be:
[project]
dependencies = '''
cached-property >= 1.2.0, < 2
distro >= 1.5.0, < 2
docker[ssh] >= 4.2.2, < 5
dockerpty >= 0.4.1, < 1
docopt >= 0.6.1, < 1
jsonschema >= 2.5.1, < 4
PyYAML >= 3.10, < 6
python-dotenv >= 0.13.0, < 1
requests >= 2.20.0, < 3
texttable >= 0.9.0, < 2
websocket-client >= 0.32.0, < 1
# Conditional
backports.shutil_get_terminal_size == 1.0.0; python_version < "3.3"
backports.ssl_match_hostname >= 3.5, < 4; python_version < "3.5"
colorama >= 0.4, < 1; sys_platform == "win32"
enum34 >= 1.0.4, < 2; python_version < "3.4"
ipaddress >= 1.0.16, < 2; python_version < "3.3"
subprocess32 >= 3.5.4, < 4; python_version < "3.2"
'''
[project.optional-dependencies]
socks = [ 'PySocks >= 1.5.6, != 1.5.7, < 2' ] # example of array of strings
tests = '''
ddt >= 1.2.2, < 2
pytest < 6
mock >= 1.0.1, < 4; python_version < "3.4"
'''
-1.
It seems to me that this has not become a point of contention in the discussion between the two, so I don’t think it matters that much, but it’s pretty easy to explain — in future versions we may want to expand the field such that the PEP 508-based specification is a strict subset of the allowed values. If that ever happens, it will use an inline table, and we’d like that to be considered a backwards-compatible change.
This alleviates the concern that several people brought up, which is that the TOML-based syntax is extensible in a backwards-compatible way and the PEP 508-only mechanism isn’t. If no one cares about adding in a mechanism for backwards-compatible upgrades to the spec, then I suppose remove it, but otherwise I can draft a 1 paragraph explanation.
I think setuptools allows multiline strings in some places where a list is otherwise acceptable and I find it confusing and annoying (and I think it’s caused its share of bugs). I think we should pick a single way to do it. I’m sure I can write a shell one-liner that converts quoted lists to unquoted lists for the purpose of copy-pasting these things into a requirements.txt or something; though it also seems simple enough to write utility functions that translate one representation into the other. The TOML→PEP 508 list direction goes like this:
#! python
import toml
with open('pyproject.toml', 'rt') as f:
dependencies = f.get('project', {}).get('dependencies', [])
print("\n".join(dependencies))
The other direction is similarly simple.