# Nobody is following the metadata\_directory promise in PEP 517 **URL:** \ **Category:** Packaging\ **Created:** [February 1, 2021, 10:34am UTC](https://discuss.python.org/t/nobody-is-following-the-metadata-directory-promise-in-pep-517/6964 "2021-02-01T10:34:48Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![uranusjr](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/uranusjr/32/103_2.png) [@uranusjr](https://discuss.python.org/u/uranusjr)\ **Post date:** [February 1, 2021, 10:34am UTC](https://discuss.python.org/t/nobody-is-following-the-metadata-directory-promise-in-pep-517/6964/1 "2021-02-01T10:34:48Z")
While exploring the possibility to add an interface to `prepare_metadata_for_wheel` in pypa/build, it was raised that no (as in _none at all_) PEP 517 backends actually fulfills the promise made in PEP 517: > `build_wheel` > If the build frontend has previously called `prepare_metadata_for_build_wheel` and depends on the wheel resulting from this call to have metadata matching this earlier call, then it should provide the path to the created `.dist-info` directory as the `metadata_directory` argument. If this argument is provided, then `build_wheel` MUST produce a wheel with identical metadata. The directory passed in by the build frontend MUST be identical to the directory created by `prepare_metadata_for_build_wheel`, including any unrecognized files it created. But instead, all major PEP 517 backends choose to ignore `metadata_directory`, re-generate metadata from scratch in `build_wheel` instead, and make no effort whatsoever to verify the generated metadata is identical. The issue has been raised to setuptools, flit, and poetry. [Poetry devs did not respond to the issue](https://github.com/python-poetry/poetry/issues/1078) (opened in May 2019). [Setuptools actively generates non-identical metadata](https://github.com/pypa/setuptools/issues/2529). [Flit is the only project that’s responded](https://github.com/takluyver/flit/issues/389). But it does have a point to not make use of the argument—It is easier to always re-generate, and since flit relies solely on static metadata, it does not expect anything to go wrong unless in extreme advanced and niche usages (custom backend extending Flit’s PEP 517 interface), so always verifying the output feels wasting. So now we have a rule that nobody is following, and people using the interface (frontends and backend extensions) are left in a bad place. How can we improve the situation? ---
**Author:** ![steve.dower](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/steve.dower/32/56_2.png) [@steve.dower](https://discuss.python.org/u/steve.dower)\ **Post date:** [February 1, 2021, 10:45am UTC](https://discuss.python.org/t/nobody-is-following-the-metadata-directory-promise-in-pep-517/6964/2 "2021-02-01T10:45:27Z")
Popularise the minor backends instead? 😁 Seriously, it was one line. I don’t know what to do if they don’t want to type it. [pymsbuild/\_build.py at 2c31968d4576a388701f50e8117187afef767d37 · zooba/pymsbuild · GitHub](https://github.com/zooba/pymsbuild/blob/2c31968d4576a388701f50e8117187afef767d37/pymsbuild/_build.py#L322) ---
**Author:** ![bernatgabor](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/bernatgabor/32/3003_2.png) [@bernatgabor](https://discuss.python.org/u/bernatgabor)\ **Post date:** [February 1, 2021, 10:48am UTC](https://discuss.python.org/t/nobody-is-following-the-metadata-directory-promise-in-pep-517/6964/3 "2021-02-01T10:48:42Z")
I consider these backend bugs that should be fixed by the maintainers. The frontends are free to raise an error and refuse to handle such backends. ---
**Author:** ![pf\_moore](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/pf_moore/32/35_2.png) [@pf\_moore](https://discuss.python.org/u/pf_moore)\ **Post date:** [February 1, 2021, 11:02am UTC](https://discuss.python.org/t/nobody-is-following-the-metadata-directory-promise-in-pep-517/6964/4 "2021-02-01T11:02:40Z")
I agree with just calling these backend bugs. Frontends should work on the assumption that the backend follows the spec, and if that causes an issue, direct the user to yell at the backend. At a minimum, every backend should be able to _compare_ the two sets of metadata and fail if they differ, so it’s not like this is complicated to implement. (Flit argues that it’s an unnecessary cost, I’d say that it’s only unnecessary as long as no-one can tell you’re not doing it 😉). Edit: Note that I don’t have a problem with backend bugs not getting fixed promptly. It’s a fairly rare edge case, and all of these projects are volunteer based, so prioritising more significant issues is entirely reasonable. But that doesn’t mean it’s not a bug… ---
**Author:** ![uranusjr](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/uranusjr/32/103_2.png) [@uranusjr](https://discuss.python.org/u/uranusjr)\ **Post date:** [February 1, 2021, 12:46pm UTC](https://discuss.python.org/t/nobody-is-following-the-metadata-directory-promise-in-pep-517/6964/5 "2021-02-01T12:46:34Z")
So it sounds like pypa/build (as a frontend) should stick to the standard and look for backends implement the expected behaviour in the future. I’ll bring this back to the discussion there, thanks. ---
**Author:** ![pf\_moore](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/pf_moore/32/35_2.png) [@pf\_moore](https://discuss.python.org/u/pf_moore)\ **Post date:** [February 1, 2021, 12:52pm UTC](https://discuss.python.org/t/nobody-is-following-the-metadata-directory-promise-in-pep-517/6964/6 "2021-02-01T12:52:21Z")
Out of curiosity, what was the pypa/build issue that triggered the question? ---
**Author:** ![bernatgabor](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/bernatgabor/32/3003_2.png) [@bernatgabor](https://discuss.python.org/u/bernatgabor)\ **Post date:** [February 1, 2021, 1:04pm UTC](https://discuss.python.org/t/nobody-is-following-the-metadata-directory-promise-in-pep-517/6964/7 "2021-02-01T13:04:01Z")
> > > Am I missing something, or is prepare\_metadata\_for\_build\_wheel() not implemented? > It seems like it's something that isn't specified in PEP-517, but I was... ---
**Author:** ![MrMino](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/mrmino/32/3322_2.png) [@MrMino](https://discuss.python.org/u/MrMino)\ **Post date:** [February 1, 2021, 2:32pm UTC](https://discuss.python.org/t/nobody-is-following-the-metadata-directory-promise-in-pep-517/6964/8 "2021-02-01T14:32:05Z")
> Out of curiosity, what was the pypa/build issue that triggered the question? TL;DR: me not knowing whether the metadata hook is an optional call for the frontend or not, and not seeing it in the [build](https://github.com/pypa/build) API. Thread linked up above by @bernatgabor - most of my chatter not related to the issue at hand. Start from this comment instead: [Support for metadata hook · Issue #130 · pypa/build · GitHub](https://github.com/pypa/build/issues/130#issuecomment-769727536) ---
**Author:** ![njs](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/njs/32/204_2.png) [@njs](https://discuss.python.org/u/njs)\ **Post date:** [February 1, 2021, 3:14pm UTC](https://discuss.python.org/t/nobody-is-following-the-metadata-directory-promise-in-pep-517/6964/11 "2021-02-01T15:14:45Z")
It sounds like flit is following the spec? The spec doesn’t say the backend is required to actively check for equivalence, it just says that the output has to be equivalent, however that’s accomplished. Otoh the frontend would be entirely within its rights to check and raise an error if there’s a difference. And the backend is free to check too, if they want. Might be a good idea for setuptools too, since it’s so hard for setuptools to know what user code and plugins are doing. The reason backends get the `metadata_directory` argument is to maximize their flexibility for implementing this rule, by reusing the old metadata, comparing against it, etc. ---
**Author:** ![pf\_moore](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/pf_moore/32/35_2.png) [@pf\_moore](https://discuss.python.org/u/pf_moore)\ **Post date:** [February 1, 2021, 4:06pm UTC](https://discuss.python.org/t/nobody-is-following-the-metadata-directory-promise-in-pep-517/6964/12 "2021-02-01T16:06:20Z")
> [@njs](#): > > It sounds like flit is following the spec? The spec doesn’t say the backend is required to actively check for equivalence, it just says that the output has to be equivalent, however that’s accomplished. Agreed. “The way that the backend works means that there can’t be a difference” is an entirely valid approach. > [@njs](#): > > Otoh the frontend would be entirely within its rights to check and raise an error if there’s a difference. Yes, but the frontend is _also_ entirely within its rights to assume the backend follows the spec, and _not_ check. That’s basically the point of a constraint like this, it allows frontends to avoid checks that the backend did what they are required to do. Yes, the frontend can check, and if they do, and find a discrepancy, they can give a friendlier error - but the error is still “the backend has a bug and I can’t proceed, please file a bug with the backend”… ---
**Author:** ![dholth](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/dholth/32/952_2.png) [@dholth](https://discuss.python.org/u/dholth)\ **Post date:** [February 1, 2021, 7:42pm UTC](https://discuss.python.org/t/nobody-is-following-the-metadata-directory-promise-in-pep-517/6964/14 "2021-02-01T19:42:29Z")
Yes, the requirement is OK, but the solution is ‘generate the same metadata every time’ not ‘copy the passed in folder because the metadata might be different each time’. ---
**Author:** ![bernatgabor](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/bernatgabor/32/3003_2.png) [@bernatgabor](https://discuss.python.org/u/bernatgabor)\ **Post date:** [February 1, 2021, 9:26pm UTC](https://discuss.python.org/t/nobody-is-following-the-metadata-directory-promise-in-pep-517/6964/15 "2021-02-01T21:26:01Z")
Well, implementation detail. Some backend might be cheaper to compare and generate delta than regenerate it (e.g. where you need to compile some binary to find out the records 🤔). ---
**Author:** ![wim](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/wim/32/1705_2.png) [@wim](https://discuss.python.org/u/wim)\ **Post date:** [May 15, 2022, 7:49pm UTC](https://discuss.python.org/t/nobody-is-following-the-metadata-directory-promise-in-pep-517/6964/16 "2022-05-15T19:49:05Z")
Shirking the responsibility to look at `metadata_directory` also makes wrapping the hooks unreliable, unfortunately. More info here: [build meta wheel does not respect metadata\_directory per PEP-517 · Issue #1825 · pypa/setuptools · GitHub](https://github.com/pypa/setuptools/issues/1825#issuecomment-1127008877) ---
**Author:** ![ofek](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/ofek/32/1033_2.png) [@ofek](https://discuss.python.org/u/ofek)\ **Post date:** [May 17, 2022, 10:24pm UTC](https://discuss.python.org/t/nobody-is-following-the-metadata-directory-promise-in-pep-517/6964/17 "2022-05-17T22:24:44Z")
As a backend maintainer, my view is that the main issue is `prepare_metadata_for_build_wheel` itself. It serves no purpose IMO and is bound to lead to this issue, see [Feature request: hatchling should implement the prepare\_metadata\_for\_build\_wheel hook · Issue #128 · pypa/hatch · GitHub](https://github.com/pypa/hatch/issues/128) If dynamic things happen during builds metadata will change unless you literally build the wheel for `prepare_metadata_for_build_wheel`. ---
**Author:** ![pf\_moore](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/pf_moore/32/35_2.png) [@pf\_moore](https://discuss.python.org/u/pf_moore)\ **Post date:** [May 18, 2022, 9:49am UTC](https://discuss.python.org/t/nobody-is-following-the-metadata-directory-promise-in-pep-517/6964/18 "2022-05-18T09:49:01Z")
From what I recall, the point was that backends might be able to determine metadata quickly, but building the wheel may be costly. For example, setuptools where the metadata is accessible statically (from `setup.cfg`, maybe) but there’s a binary to build. If frontends can signal “we only need the metadata just now, and we may not ever need the wheel”, this could be a huge win. In practice, though, it’s not turned out to be as useful as we’d hoped (you could in fact argue that adding it was a pretty clear case of premature optimisation…) But the standard is absolutely clear that `prepare_metadata_for_build_wheel` is optional, so it’s absolutely acceptable to simply not implement it. ---
**Author:** ![bernatgabor](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/bernatgabor/32/3003_2.png) [@bernatgabor](https://discuss.python.org/u/bernatgabor)\ **Post date:** [May 18, 2022, 1:25pm UTC](https://discuss.python.org/t/nobody-is-following-the-metadata-directory-promise-in-pep-517/6964/19 "2022-05-18T13:25:21Z")
I don’t think it’s premature optimization, especially not for c-extensions where building a wheel is more expensive. tox 4 uses this feature a lot. ---
**Author:** ![ofek](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/ofek/32/1033_2.png) [@ofek](https://discuss.python.org/u/ofek)\ **Post date:** [May 18, 2022, 4:17pm UTC](https://discuss.python.org/t/nobody-is-following-the-metadata-directory-promise-in-pep-517/6964/20 "2022-05-18T16:17:03Z")
What does it use it for? The metadata will change in this case btw ---
**Author:** ![bernatgabor](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/bernatgabor/32/3003_2.png) [@bernatgabor](https://discuss.python.org/u/bernatgabor)\ **Post date:** [May 18, 2022, 5:19pm UTC](https://discuss.python.org/t/nobody-is-following-the-metadata-directory-promise-in-pep-517/6964/21 "2022-05-18T17:19:54Z")
To determine dependencies of the run environment. Mostly only interested in the `Requires-Dist`, which should really not change between `prepare_metadata_for_wheel` and `build_wheel`. ---
**Author:** ![layday](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/layday/32/2771_2.png) [@layday](https://discuss.python.org/u/layday)\ **Post date:** [May 19, 2022, 7:06am UTC](https://discuss.python.org/t/nobody-is-following-the-metadata-directory-promise-in-pep-517/6964/22 "2022-05-19T07:06:13Z")
It’s unclear what purpose passing the metadata directory produced by the `prepare_metadata` hook to `build_wheel` is intended to serve. Is it to allow modifying the metadata, or is it for the backend to be able to verify that the metadata haven’t changed in the intervening period? The few backends that support the directory argument seem to think it’s the former but several people here (myself included) are uneasy with backends producing wheels with potentially arbitrary metadata. What are the pros and cons of either approach and what are their practical applications? As for @wim’s specific use case (or perhaps more generally), there exists an additional complication, which is that frontends are not required to invoke `prepare_metadata` prior to building a wheel or pass its output to `build_wheel` even if they do. Therefore, you would not be able to isolate the metadata transformation to `prepare_metadata` unless you controlled the frontend. ---
**Author:** ![pf\_moore](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/pf_moore/32/35_2.png) [@pf\_moore](https://discuss.python.org/u/pf_moore)\ **Post date:** [May 19, 2022, 10:35am UTC](https://discuss.python.org/t/nobody-is-following-the-metadata-directory-promise-in-pep-517/6964/23 "2022-05-19T10:35:49Z")
> [@layday](#): > > It’s unclear what purpose passing the metadata directory produced by the `prepare_metadata` hook to `build_wheel` is intended to serve. It’s so that backends can avoid doing a bunch of work twice. Imagine the extreme case, where the backend implements `prepare_metadata` by building the wheel in the metadata directory and then unpacking just the `.dist-info` part. In that case, the backend can implement `build_wheel` by simply noting that a metadata directory was passed, and returning the already-built wheel stored in there. In reality, such a backend would be better just implementing `build_wheel` and not implementing `prepare_metadata` at all, but intermediate variants where the backend builds most of the wheel (maybe just not running C compilers on extension modules) are plausible. > [@layday](#): > > Is it to allow modifying the metadata Absolutely not, the PEP requires the metadata is the same from both calls. (Edit: But I hadn’t considered wrappers when I said that, see below for the situation wor wrappers), > [@layday](#): > > is it for the backend to be able to verify that the metadata haven’t changed in the intervening period No, it’s so the backend doesn’t have to recreate the metadata at all, it can just re-use it. But the backend _can_ do this if it wants (but by doing so, it would prohibit wrappers that change the data - which it may or may not want to do). > [@layday](#): > > As for @wim’s specific use case (or perhaps more generally), there exists an additional complication, which is that frontends are not required to invoke `prepare_metadata` prior to building a wheel or pass its output to `build_wheel` even if they do. Correct. `build_wheel` _must_ be prepared to do the whole job, if `prepare_metadata` doesn’t get called, or if the metadata directory isn’t passed (a frontend that calls `prepare_metadata` but doesn’t pass th directory on to `build_wheel` is not technically broken, but it’s pretty stupid as it’s deliberately blocking the backend from a potential optimisation). > [@layday](#): > > Therefore, you would not be able to isolate the metadata transformation to `prepare_metadata` unless you controlled the frontend. Absolutely. You cannot isolate _anything_ to `prepare_metadata`, because the front end isn’t required to call it. I don’t quite understand the `setuptools-ext` use case, because it seems as if `setuptools-ext` is assuming the front end is required to call `prepare_metadata` if it’s present, which isn’t true. So if that’s the case, I think the problem here is simply that there’s a bug in `setuptools-ext`. I will say that we didn’t consider backend wrappers when defining `prepare_metadata` - the approach of a wrapper adding its own “stuff” to the metadata directory and expecting a wrapped backend to preserve it is a reasonable one, and requiring the backend to copy what’s in `metadata_directory` into the final wheel seems like a fair way of achieving that. But it’s not mandated by PEP 517 and it would need an update to the spec to require it (and, as this thread demonstrates, it would need existing backends to change so we’d need a way to detect which version of the spec a given backend conforms to). On the other hand, preserving the metadata directory could be a private arrangement between `setuptools-ext` and `setuptools`. We don’t have a use case of a _backend agnostic_ wrapper needing this functionality yet, so forcing all backends to conform might be a little premature. [Next page](https://discuss.python.org/t/nobody-is-following-the-metadata-directory-promise-in-pep-517/6964.md?page=2)