PEP 643: Metadata for Package Source Distributions

[pganssle (Paul Ganssle)]

@pf_moore

I’m completely in favour of anything that makes it easier for setuptools to push for static-by-default behaviour as fast as possible. My expectation was that setuptools wouldn’t do anything, but bdist_wheel would verify that the metadata which got generated for the wheel matched the sdist metadata, and report an error if it didn’t.

I don’t really make a distinction between “setuptools” and “bdist_wheel”, but either way what you are describing is worse than any of the alternatives I suggested, because it will only detect an issue if the sdist build environment doesn’t match the wheel build environment. That’s way too late in the process. It means end users (who can’t do anything about it) will start having pip install <x> fail on them because the people who can make the change before a release don’t build an sdist then test building wheels from it on every other platform — they build and install all tests from source, and build wheels and sdists directly from source independent of one another (and on the same platform anyway), before uploading.

The only things we can do are:

1. have bdist_wheel pull anything static from PKG-INFO first, ignoring any values that are generated by setup.py that might be in conflict with those in PKG-INFO
2. Detect whether a value is static or dynamic at sdist build time and raise an exception. We can know that anything in setup.cfg is static, but without parsing the AST we won’t know if anything passed to setup() is. We’ll have to assume anything passed is dynamic, which will likely break a lot of builds.

Neither of these is especially desirable from a user experience or adoption point of view. If we go with the exception mode, we’d need to delay the roll-out of support for this until we have an AST-parser/reader that covers a sufficiently significant quantity of the setup.py files out there that we won’t be breaking everyone’s builds on day one. And AST parsing is going to need upkeep, because AFAIK the AST is not stable.

On the other hand, if we allow basically all fields to be dynamic, we can roll out today, and everyone with setup.cfg files will opt in immediately. We can gradually add some AST parsing heuristics without worrying about being perfect, because the metadata upgrades will happen silently as our heuristic processing gets better. We also don’t have to worry quite so much about the AST parsing getting out-of-date, because we can fall back on “if we don’t understand the AST, warn and emit a bunch of dynamic fields”.

@pf_moore

I honestly don’t care much. @brettcannon asked for it to be an error and I didn’t see any reason against that choice. I am concerned that under your proposal, we’d effectively be saying that the metadata value in a sdist is tool-defined if it’s marked as Dynamic , but if it matters enough to you to want me to change it (and no-one else objects) then I’ll do so.

I don’t really understand what you mean by “the metadata value in a sdist is tool-defined if it’s marked as Dynamic”. My proposal is that:

1. In an sdist, a field with a value and Dynamic set should treat the value as a hint as to what the true value will be, but the canonical value (as it is in the current version of the PEP) requires a build to determine, because it is supplied by the backend dynamically.
2. In a wheel, Dynamic refers to the provenance of the metadata. If a field is marked Dynamic in a wheel, the associated value is canonical for the platform / context you are using (as it is today), but you know that it was the result of a calculation at build-time, and may not apply to other wheels built from the same source distribution.

No one is clamoring for this information, but we definitely have it at build time and as I mentioned in my comments I can think of some situations where it would be useful.

@pf_moore

One more significant argument in favour of Dynamic is that if we look forward to a situation when static data is the norm, and essentially most package data is static, do we want to really want to see long lists of Static declarations? Like the following:

My original objection to this was that this is not really presented for human consumption anyway, so it doesn’t matter that much what it looks like, but I’ve changed my mind and actually find this compelling. Even if it’s not a human-centric format, it will almost certainly be read by humans more often than people will write new libraries to parse it. So, +1 for Dynamic rather than Static.