Multipath TCP encounters its first middlebox
Like many Internet protocols, Multipath TCP was designed by a working group. By involving several network engineers, one expects that they will think about all possible problems and that the final design will be stronger. However, there is also a risk that this design by committee becomes a series of compromises and that more and more options are added to the protocol. The best way to avoid this risk is to implement the protocol while it is being specified or base the key decisions of the design on an existing implementation. During the first year, the MPTCP working group mainly worked by emails and many ideas were exchanged.
Fortunately, two members of the working group started to write an implementation based on the first drafts. Sebastien Barre announced his first prototype implementation in the Linux kernel on November 18th, 2009. Costin Raicu replied on November 24th, 2009 with a user space implementation that also extended the Linux TCP stack.
Sebastien Barre ‘s implementation was designed to test the feasibility of implementing Multipath TCP inside an operating systems kernel. Costin Raicu ‘s implementation was designed to evaluate the performance of the coupled congestion control scheme [WRGH11]. These two implementations were complementary. Sebastien’s implementation leveraged his earlier experience in implementing shim6 RFC 5533 in the Linux kernel [BarreRB11]. For this reason, it only supported IPv6, while Costin’s implementation only worked over IPv4. Sebastien continued to improve his first implementation. Version 0.2 that was released in March 2010. When he added IPv4 support to this implementation, he sent the kernel sources to colleagues in Finland and UK. The Multipath TCP implementation was working well inside our labs and this was an opportunity to test it over longer distance paths to see how retransmissions and other techniques reacted to longer delays. The first tests were a disaster. This version of Multipath TCP could establish a connection to the remote server, but no data was exchanged. Sebastien looked at all the possible sources of problems and eventually took a packet trace on both servers to manually check the packets that were exchanged. Eventually, he found that a middlebox somewhere on the Internet was changing the TCP sequence numbers of the packets without modifying the Multipath TCP options. He summarized his findings in the email below.
He continued to explore the problem and found that the culprit was our campus firewall… This firewall was configured to configured to rewrite TCP sequence numbers to protect TCP connections from weak machines such as Windows98 …
This was the first time that a middlebox interfered with a Multipath TCP implementation, by far not the the last unfortunately. The protocol designers learned this lesson and looked at different ways to make Multipath TCP much more resilient to middlebox interference. We will explore these issues in more details later in another blog post.
An important lesson that we learned is that by having developed a fully functional implementation early, we could quickly detect operational problems and fix them. Sebastien’s initial implementation later became the reference Multipath TCP implementation and various developers have contributed to the current code base. The global Internet is far more complex than what students learn in textbooks…
References
| [BarreRB11] | Sébastien Barré, John Ronan, and Olivier Bonaventure. Implementation and evaluation of the shim6 protocol in the linux kernel. Computer Communications, 34(14):1685–1695, 2011. URL: https://inl.info.ucl.ac.be/publications/implementation-and-evaluation-shim6-protocol-linux-kernel. |
| [WRGH11] | D. Wischik, C. Raiciu, A. Greenhalgh, and M. Handley. Design, Implementation and Evaluation of Congestion Control for Multipath TCP. In Proceedings of the 8th USENIX Symposium on Networked Systems Design and Implementation (NSDI). 2011. |