HTTP response headers

Python Help
,
1

I need to use http.client and access the response headers.
Code like this seems to work:

import http.client
response: http.client.HTTPResponse = ...
for name in response.headers:
    value = response.headers[name]
    print(name, value)

Is this code wrong?

mypy complains:

error: "HTTPMessage" has no attribute "__iter__" (not iterable)

http.client.HTTPResponse.headers is documented:

Headers of the response in the form of an email.message.EmailMessage instance.
(That is a bit surprising. Why email? An example how to access the headers would be nice.)

typeshed annotates it as a HTTPMessage. (Is that the same as EmailMessage?)

This issue explains HTTPMessage should not be considered a Mapping. (Not sure if that is relevant.)

From the EmailMessage docs:

The following methods implement the mapping-like interface for accessing the message’s headers. Note that there are some semantic differences between these methods and a normal mapping (i.e. dictionary) interface. For example, in a dictionary there are no duplicate keys, but here there may be
duplicate message headers.

So I guess the code is at least not ideal, as it will miss duplicate keys, and I should use items() to get all headers?

import http.client
response: http.client.HTTPResponse = ...
for name, value in response.headers.items():
    print(name, value)

At least mypy likes it. Problem solved. :slight_smile:

1 Like
2

I would not expect that. The documentation says that it is not a Mapping. It depends on what __iter__() returns and how the returned object iterates.

Unfortunately I know nothing about the object from http.client.HTTPResponse.headers to say anything specific :frowning:

Anyway it looks like you discovered a bug in the typeshed regarding the supposedly missing __iter__ attribute.

1 Like
3

I tested it with duplicate keys now. It iterates over both keys, but the lookup by key does not magically find the right values of course.

from socketserver import TCPServer, StreamRequestHandler
from threading import Thread
from http.client import HTTPConnection

def main():
    server = TCPServer(("localhost", 0), RequestHandler)
    thread = Thread(target=server.serve_forever)
    thread.start()
    try:
        demo_request(server, "Normal request", "GET", "/")
    finally:
        server.shutdown()
        server.server_close()

def demo_request(server, desc, *pos, **kw):
    client = HTTPConnection(*server.server_address)
    try:
        client.request("GET", "/drop-connection")
        response = client.getresponse()
        print("for name in response.headers:")
        for name in response.headers:
            print("    Header", name, response.headers[name])
        print("for name, value in response.headers.items():")
        for name, value in response.headers.items():
            print("    Header", name, value)
        response.read()
    finally:
        client.close()

class RequestHandler(StreamRequestHandler):
    def handle(self):
        assert self.rfile.readline().startswith(b"GET ")
        while self.rfile.readline().rstrip(b"\r\n"):
            pass
        self.wfile.write(
            b"HTTP/1.1 200 Dropping connection\r\n"
            b"Content-Length: 0\r\n"
            b"MyHeader: Hello\r\n"
            b"MyHeader: World\r\n"
            b"\r\n"
        )

if __name__ == "__main__":
    main()

(Based on this demo I found on bpo.)

for name in response.headers:
    Header Content-Length 0
    Header MyHeader Hello
    Header MyHeader Hello
for name, value in response.headers.items():
    Header Content-Length 0
    Header MyHeader Hello
    Header MyHeader World

(The “Hello” is repeated and the “World” is missing.)

1 Like
4

Oh, I overlooked that iterating response.headers gives only header names! The documentation says that you should use the get_all() method to get all the values for a given header name.

I find this Mapping-like behaviour of an object with multiple possible keys confusing.

1 Like