To parse HTTP headers, the
http.client module uses the
email.parser.Parser class, as headers are in the same format. The relevant function for this is
http.client to parse its headers, this function is also responsible for parsing headers on the server side.
_parse_header_lines doesn’t pass an email policy to
email.parser.Parser. The documentation mentions that a policy should always be provided, and that the default policy will change in the future, so it’s probably a good idea to change the code to use a specific policy.
The current default policy for
email.policy.compat32, which is a bit outdated but provides complete backward compatibility. Because the documentation says that this won’t be the case forever, I suggest that we change
_parse_header_lines to use
email.policy.default instead. While this is a breaking change, it would greatly extend support for the many extra RFCs that
email.policy.default already supports.
One of these RFCs in particular that this change would start supporting is RFC 2047, which includes receiving encoded-word header values and would resolve part of issue #105530.
PS: Sorry that this is an exact copy of the GitHub issue I opened (#105622). I probably should’ve posted this here first, as it is a feature request/idea and not really a bug. That issue should also be a feature request, but I can’t edit the tags on it.