Add an option to http for socket protocol selection

Currently, the http module can only use TCP sockets. But since 3.10 IPPROTO_MPTCP is supported. It would be nice to be able to select it when using the http lib.

  • For the http.client, a new optionnal parameter protocol for the HTTPConnection could be used to specify the protocol to be used by the socket.
  • For the http.server, the socketserver.TCPServer class can be update with an optional parameter mptcp boolean that can be used to enable the protocol.

Based on RFC8684 - Design Assumptions, we can replace IPPROTO_TCP with IPPROTO_MPTCP.

An HTTP user doesn’t have to know anything about the socket or TCP.

yes, that would be the easiest way to do it. But as it stands, MPTCP is not supported everywhere and thus fallback to tcp needs to be implemented as described on mptcp.dev.
In addition, giving the user the choice to use MPTCP (especially for clients) is important because there is no real benefit to using MPTCP sockets if their systems are not setup for mutltipath (see setup guide)
Regarding servers the default use of MPTCP is advised but fallback still need to be considered as well as the option to disable it if needed.

The fallback mechanism can also be implemented in Python. HTTP operates three layers above TCP in the OSI model and is not directly related to TCP. HTTP/3 runs on UDP.

In future versions of Python, support for HTTP/3 may be included, rendering the proposed parameter meaningless.

Hi @elis.byberi

(I’m working on MPTCP in the Linux kernel and Maxime recently pointed me to this discussion.)

From what I understood from Maxime, he wants to start a simple HTTP service in Python and use MPTCP without having to use workarounds to force MPTCP sockets to be created instead of TCP. The issue is that the http library doesn’t allow using this TCP extension.
I understand that an HTTP user doesn’t have to know anything about the socket or TCP, but MPTCP could be useful in some use-cases.

A server could enable MPTCP support by default by simply replacing IPPROTO_TCP by IPPROTO_MPTCP when creating the socket: if the client doesn’t request MPTCP, the kernel will return a TCP socket from the accept() system call. Without additional configurations, the server will allow the client to create multiple paths from the same interface. Would it be OK for Maxime to do such modification? Do you think there is no need to add a way to avoid creating MPTCP sockets “just in case”? (even if MPTCP can be disabled with a sysctl knob per netns)

For the client side, it might be more interesting to allow the lib user to control its usage: e.g. MPTCP could be explicitly blocked in some networks (paranoia, blocking any TCP extensions), maybe there is no need to. Without additional configurations, MPTCP will only use one path at a time (likely when the default route changes): only interesting in mobility use-cases. What could be done here?

Maybe it is easier to look at the server case first.

In future versions of Python, support for HTTP/3 may be included, rendering the proposed parameter meaningless.

I guess there will always be a need to support different versions, UDP is still being blocked in some networks, some services might not offer it (SSL configuration needed, etc.). I don’t think HTTP 1 and 2 will be dropped soon :slight_smile:

You can subclass HTTPConnection and override the connect method.

Also, I would suggest doing the same for other HTTP libraries and publishing a PyPi package.

OK, thank you! So the idea is more to create a new lib overriding some methods just to change the socket call, instead of extending the HTTP/socket libs, right? So MPTCP would be used only if explicitly asked, by adding a new lib (which is fair, but required modifying apps as well, even for the server side).

@mux99 do you plan to work on such lib?

1 Like

yes. At least, I will try.
I will continue to post in this discussion if i have more questions.