I’m going to need to rework the proposal quite a bit; IMO this is a key issue we must address:
I’ve not really looked at metadata 2.2+, due to missing PyPI support, so completely missed this issue.
Especially if we break it up a bit further, maybe we could even suggest or require a backend when computing a non-Dynamic dynamic metadata field (okay, not sure I like the two concepts being slightly different but having the same name!) to pull directly from the SDist instead of running the plugin when doing a build from SDist? Don’t know if that’s a good idea or not, but I know third-party packagers would love not requiring the plugins be present - packagers that build from SDist like conda-forge could just skip adding the plugin to their build dependency lists in that case.
I think both of the key initial use cases, version and readme, are probably always not Dynamic. And we should really be avoiding Dynamic whenever possible. An upcoming use case, pinning a dependency at build-time (whatever version of NumPy you use when building must be equal or newer to be used after you build the wheel) would have to be Dynamic, as it occurs when you do the build. So I think we need the flexibility to specify this and the current API return value doesn’t allow that. I think this should be specified by the plugin. This might be as simple as adding an optional hook function dynamic_metadata_is_dynamic that is assumed false if not present. Not sure.
I think the “additive metadata” portion could possible be proposed sooner. We’ll need to redo the specification a bit (and update the example implementation in scikit-build-core; I think I could also set up an example implemention based on flit and/or a helper for pyproject-metadata.
This is exactly what we have in scikit-build-core (in main):
[project]
name = "mypackage"
dynamic = ["version"]
[tool.scikit-build]
metadata.version = "scikit_build_core.metadata.setuptools_scm"
It’s based on the original form of the proposal (and was contributed by someone who hadn’t worked with scikit-build-core before!), so it’s still using a simple string, and there are no inline config options, and the API is the original version. I’ve got a branch that adds those, but it’s somewhat on hold until we iron out what’s going to be in the proposal. I’lll probably require the tool.scikit-build.experimental=true flag to be set to use it, at least with non-included plugins. Once we have a stable proposal I could propose it for meson-python, at least.
Though, really, we don’t have to have a PEP for the API - this doesn’t change the pyproject.toml at all at this point, it’s only a standardization agreement between plugins and backends. So perhaps we could propose both things (the API and the pyproject.toml changes), and then backends and plugins could adopt it via custom config, and then accept the PEP if adoption works well. I know I’ll at least have a tool-specific way to enable plugins for a while, since a user might need to add a plugin but work with a tool that doesn’t support the updated specification (just like any other update, including the TOML 1.1 update that includes trailing commas and newlines in inline tables! Very excited for that one.) So far the tools I’ve checked support the change to an inline table, though. I’ll be sure to have a discussion of the possible implications when we work on that part! FYI, the inline a.b syntax isn’t supported by older pytoml based versions of Pip - there is always some backward incompatibilities to worry about, but they disappear over time.
Having a single interface would be much nicer, but this is a good starting point if it’s better to go in steps.
Also, I don’t expect this to be added to beginner tutorials, or to cause packages that have static metadata to switch. I expect it to unify the existing usage of dynamic metadata and simplify converting custom setup.py’s that do README processing, optional-dependency merging, and version computation - which covers almost every non-binary build custom setup.py I’ve had to maintain. I think it will also help with unifying backends, since this currently is one reason to select one backend over another - with this proposal, you could select your favorite dynamic metadata plugin and your favorite backend independently.
This discussion has been immensely helpful, thank you to all that have participated! Quick question: If I make a major revision and split it, should we continue here, or should I start a new topic (possibly two for each side of the split)? Now might be a good time to bike shed on the names and API details for the next draft, as well.