Description field in package metadata is defined here as being either encoded in the “message body” part of the RFC 822 formatted file, or in the headers in “folded field” format, with CRLF replaced by CRLF plus 7 spaces and a pipe character.
In practice, wheels appear to use the “message body” format, whereas sdists created with setuptools use a non-standard “header with extra lines indented by 8 spaces but no pipe” format. Flit ignores the long description in the sdist metadata, but uses the message body for wheels. To further complicate the matter, Python’s
email.parser.Parser class doesn’t seem to do anything with the “7 spaces plus pipe” format (or for that matter, the 8-space indentation), so client tools have to manually fix the indentation.
The result is frankly a bit of a mess for client tools. And unfortunately, there’s not much we can do to retrospectively fix older sdists. (On the plus side, I suspect no-one actually cares much about sdist metadata, and a bit of malformed whitespace isn’t a disaster anyway).
In the interests of making it easier to write conformant clients, while not making a mountain out of a molehill, I suggest the following:
- The core spec states that the “description in the message body” form is canonical, and the header form is retained for backward compatibility only. All new tools should only write the body form.
- The “indented without a pipe” form of header is noted as being a permitted variant (simply because we can’t do much about the fact that it exists in the wild).
- Clients reading metadata must prefer the “body” form, and are allowed to ignore the indentation rules. I don’t like this particular provision, but nor do I like requiring clients to implement complex rules to ensure correct round tripping of a format that we are deprecating. If, on the other hand, there’s a way to get the stdlib
For all practical purposes, I suspect that most clients would simply read the record and ignore the indentation-preservation rules for the header form anyway, so we’d really just be codifying what’s actually happening.
What do people think? In particular, @jaraco how much effort would be involved in fixing setuptools’ sdist code? I suspect this might actually originate in distutils, so it might need new code to be written.