Partly dynamic project metadata (eg. adding dependency constraints based on the build environment)

[steve.dower]

@pf_moore

Hang on, we’re mixing things up again. In a wheel, there’s no need for Install-Constraint - the dependencies are always fully specified in a wheel. In a sdist, it’s allowed (see PEP 643 for the details) for the build backend to mark the field as dynamic, but include the value “as a hint”. So Install-Constraint is somewhat useful, but not necessary (and the definition of Dynamic in core metadata means that I don’t think it would actually help in this case). That only leaves pyproject.toml where it’s not already possible to record this data. I can’t say why the authors of PEP 621 chose not to allow dynamic-with-a-value, but I do know that for most use cases where I’d expect to be processing project metadata, I’d be looking at core metadata, not reading pyproject.toml.

Thanks, this is a very helpful clarifying wall of text But it has genuinely clarified things. In particular, I hadn’t realised that the rules for pyproject.toml were different from core metadata.

FWIW, the core metadata rules make sense to me, and do indeed function in the way I’d like (for the most part, though I now don’t trust my belief that unspecified fields are meant to error, rather than being used at your own risk). The pyproject.toml rules are overly strict, IMHO, but people do like getting to reject user’s code rather than deal with it…

My only real interest here is in not adding more pointless restrictions on build backends (and I do really like the idea of making wheel dependencies stricter than the sdist, since it would allow us conda-like behaviour in PyPI packages, solving a whole lot of compatibility issues quite conveniently). It doesn’t seem like new restrictions are likely though, so I’ll bow out.

[FFY00 (Filipe Laíns)]

@pf_moore

OK. But allowing build backends to alter the value given in dependencies without an explicit marker to give them permission will also break a lot of stuff, so I don’t think you can reasonably claim that’s an acceptable approach either.

Can you give me an example using Requires-Dist-Dynamic?

I must be missing something here. I think what you are saying is true for core metadata, but not for pyproject.toml.

[brettcannon]

@pf_moore

I can’t say why the authors of PEP 621 chose not to allow dynamic-with-a-value

If you mean e.g. defining project.dependencies and dynamic = ["Requires-Dist"], I don’t recall if that was suggested. Best I can think of is it waters down the static data as it’s accurate but incomplete. Since that’s currently not allowed I could see that working around the issue of breaking pre-existing pyproject.toml files.

[pf_moore (Paul Moore)]

@steve.dower

Thanks, this is a very helpful clarifying wall of text

Sorry for the wall of text. I’m way too concerned about adding confusion, so I end up adding qualifiers. And I always default to “too much text” anyway, as you well know Glad it helped anyway.

@steve.dower

FWIW, the core metadata rules make sense to me, and do indeed function in the way I’d like (for the most part, though I now don’t trust my belief that unspecified fields are meant to error, rather than being used at your own risk). The pyproject.toml rules are overly strict, IMHO, but people do like getting to reject user’s code rather than deal with it…

I agree on both counts.

To alleviate your fear,

Fields defined in the following specification should be considered valid, complete and not subject to change.

the “complete” in this says that unspecified fields should not be present.

@steve.dower

It doesn’t seem like new restrictions are likely though, so I’ll bow out.

I agree it would be useful to solve this issue, I think it’s something that would be genuinely useful to have. Whether it happens largely depends on whether we can get an acceptable proposal, though (hence my insistence on clarity and precision…)

@FFY00

Can you give me an example using Requires-Dist-Dynamic?

No, because I don’t know how Requires-Dist-Dynamic is meant to work. Can you state your new proposal (including whatever needs to change in pyproject.toml, and any associated core metadata changes needed to include Requires-Dist-Dynamic)? If you do that, I’ll try to restate my example in terms of that proposal.

@brettcannon

If you mean e.g. defining project.dependencies and dynamic = ["Requires-Dist"]

Well, dynamic = ["dependencies"], but otherwise yes. We’re all doing it now

I think that’s the most reasonable possibility we have for moving forward, but @FFY00 isn’t willing to go that way.