I am also -1 on loading from files, but this kind of thing came up as an advantage of exploded TOML formats, that they can be easily expanded in a backwards-compatible way to cover things that might be appropriate in a pyproject.toml but wouldn’t necessarily make sense in PEP 508. The most reasonable one is “read this from a file” (which we’re effectively foreclosing forever at this point if we don’t include a provision for extensions). Another possibility would be some metadata about the dependency that is relevant to the backend, but either isn’t going to be translated into metadata or is going to be translated in a way that would be awkward to represent otherwise.
I could imagine a future where we want to flag a specific dependency as “recommended” rather than strict, or tag it as part of a non-extra grouping — for example, if we move to a world where it’s possible to specify different backends with their own groupings, you might have the default value be “flask + werkzeug” but if you choose the [django] backend it pulls in django + some Django plugins.
I don’t know that it’s amazingly likely that we’ll ever need this, but it addresses the “easily extensible” argument for exploded tables.
I also think there’s a reasonable case to be made for doing this only for dependencies, because:
- Dependencies are probably the most complicated thing but also one of the most important pieces of metadata. It is likely to be a place where people get up to a decent amount of chicanery and keeping flexibility available so that we can comfortably expand the spec if we find that people are chafing under the existing mechanisms and just using the
dynamicescape hatch all the time. - Dependencies are also the place where people are disproportionately likely to fall back to
dynamicanyway, so I expect many parsers will need to build in the fallback mechanism anyway.
Another possible option is that we can add in an explicit pyproject-version field somewhere, to indicate which version of the spec you are using. To opt in to new backwards-incompatible features, you would have to set the version flag to the minimum version that supports all the features you need, and tools that see a version higher than they understand would be expected to switch to the “if I see something I don’t understand, ask the backend” behavior. That would probably be annoyingly manual, but it could be made relatively painless by having backends check to see if the pyproject.toml has the wrong minimum version and raising an exception that says what the correct minimum version is.