It’s strictly a necessity. This is all a bit OT, but…
discussion of specifying backends
If my project has its metadata declared in setuptools `setup.cfg`, then the build backend must be setuptools. flit-core, hatchling, etc won't work.
If my package uses `[tool.hatch.build]` configuration, then only hatchling works.
As for specifying the version, the general consensus is to specify only a lower bound or not specify the version at all.
There’s actually a bit of an issue in that different build frontends may behave differently even on the same backend. So, for example, pip install . and flit build; pip install ./dist/*.tar.gz may give you different results. But in practice pip is the only frontend which matters for sdists, so it’s not a huge issue most of the time. It can show up when building from source.
This thread kicked off around versioning of the [project] table, and specifically with respect to a feature discussed in the thread about Dependency Groups. It was regarding how we could ever safely introduce new syntactic forms into [project.dependencies] and [project.optional-dependencies].
I’ll show the syntax here, which was suggested as a way of including a dependency group in [project.dependencies]:
[project]
dependencies = [{include-group = "foo"}]
For the purposes of this thread, and to re-center this a bit, let’s assume that we’re all agreed that we want such syntax.
I see four ways for such a new syntax to be added:
- the whole
pyproject.toml file is versioned, and this is introduced in v2
- the
[project] table is versioned, and this is introduced in v2
- the syntax is added with no explicit version number, and its presence implies that
[project] is on v2
- the syntax is added with no explicit version number, and its presence does not imply anything additional
I much prefer (4) to (3) here, since it’s not clear what “implies v2” would mean. I have no clear preference between (1) and (2) but they each seem worse than (4), at least at present, since their meanings are still unclear.
I think the choices at present for a new spec are (4) – add new syntax with an awareness of what the fallout may be – or don’t add it.
Not adding syntax of any new kind to [project] may be the right choice most of the time. But I doubt it will be the right choice all of the time. Sooner or later, there will be a pressing need to make changes, possibly very subtle ones, to that table.
In certain network protocols – I’m thinking of SSL/TLS – two parties, a server and a client, need to send information back and forth to detect what behaviors are and are not supported. It’s not just a protocol version (although TLS has that too). Often they’re sending feature negotiation messages, information about their supported behaviors (e.g. ciphers). I think our situation with pyproject.toml is really much more akin to this kind of feature negotiation.
Of course, computers are very good at this sort of thing and pyproject.toml is human-edited. I don’t want to write my files with
features = "dependency-groups,path-specifiers,external,..."
[project]
...
that sounds very unpleasant and easy to get wrong.
To @fungi’s point about version numbers being good for documentation, I agree. And I really like that there’s a clear use-case there for the version number, with a specific user-facing impact. But we don’t have a spec which evolves in discrete linear steps. Assume that [external] and [dependency-groups] are both accepted, around simultaneously. What version number should be assigned to communicate that a tools supports [external] but not [dependency-groups]? And vice-versa?
We can’t put a version number on this multi-dimensional thing, which is the pyproject.toml spec, and call it a day. Version numbering even the spec requires that changes are in some way serialized. Which might be a good thing, but then we’ll probably need to discuss which features are mandatory vs optional to support on different versions. The worst case scenario here is that we reproduce much of the current state, with a small core spec considered “mandatory” and the rest considered “optional”.
It’s not at all version negotiation that tool authors want. They want to be able to negotiate their supported features with the user. And specifically, it’s tool<->user communication! Not tool<->file! pyproject.toml is just in the middle.
I don’t know how that communication should be handled and mediated between tools and users. Perhaps if there were a single library which converted pyproject.toml contents into a list of feature flags, that would help the tool authors?
