Measuring the adoption of Multipath TCP is not so simple…
In September 2015, a google alert announced a new workshop paper entitled An Early Look at Multipath TCP Deployment in the Wild. The paper abstract was intriguing with sentences like We find that less than 0.1% of Alexa unique domains and IP addresses currently support MPTCP. Their geographic distribution is quite different from that of clients reported in other studies, with the majority of domains being in China. Based on the abstract and the results reported in the paper, one could assume that Multipath TCP has already been deployed on various Internet servers, and the paper lists several important websites in China that are supposed to support Multipath TCP.
Unfortunately, the initial measurements reported in this workshop paper were flawed. Most of the studies on the deployment of TCP extensions have used a network scanner (zmap in this one) to send SYN segments containing a specific TCP option. If the server replies with the same option, then it is assumed to support the TCP extension. The authors of this paper applied the same methodology to Multipath TCP. Unfortunately, looking simply at the presence of an option in the SYN+ACK is not sufficient because there are middleboxes that simply echo any option that they receive. This problem has been discussed in the Multipath TCP mailing list in the past and has influenced the design of Multipath TCP. It is notably described in the article that proposed the tracebox debugging tool.
With tracebox, it is easy to check whether a server really supports Multipath TCP. Let us start with multipath-tcp.org
sudo tracebox -n -v -p "IP/TCP/MPCAPABLE" multipath-tcp.org
tracebox to 130.104.230.45 (multipath-tcp.org): 64 hops max
...
...
...
9: 130.104.230.45 TCP::SrcPort (47416 -> 80) TCP::DstPort (80 -> 47416)
TCP::SeqNumber (554672918 -> 3111441317) TCP::AckNumber (0 -> 554672919)
TCP::DataOffset (8 -> 9) TCP::Flags (( SYN ) -> ( SYN ACK ))
TCP::WindowsSize (5840 -> 28800) TCP::CheckSum (0x5eb9 -> 0x206d)
IP::TotalLength (52 -> 56) IP::Identification (0x50ab -> 0x0)
IP::Flags (0 -> 2) IP::TTL (9 -> 57) IP::CheckSum (0x406 -> 0x1879)
IP::SourceIP (192.168.0.9 -> 130.104.230.45)
IP::DestinationIP (130.104.230.45 -> 192.168.0.9)
+TCPOptionMaxSegSize < TCPOptionMaxSegSize (4 bytes) :: Kind = 2 , Length = 4 , MaxSegSize = 1380 , >
TCPOptionMPTCPCapable::Sender's Key (Sender's Key = 692439777126907904 -> Sender's Key = 17898842517462319104)
The tracebox command is used to send a SYN segment with the MP_CAPABLE option. The output above (only the last line, the interesting one is shown) indicates that the server has replied by adding the MSS option that was not present in the SYN. Furthermore, the SYN+ACK includes the MP_CAPABLE option with a different key than the one sent in the SYN segment.
The same test towards a server that does not (yet ?) support Multipath TCP is shown below :
sudo tracebox -n -v -p "IP/TCP/MPCAPABLE" google.com
tracebox to 109.88.203.231 (google.com): 64 hops max
...
...
...
5: 109.88.203.231 TCP::SrcPort (12002 -> 80) TCP::DstPort (80 -> 12002)
TCP::SeqNumber (304251108 -> 752364946) TCP::AckNumber (0 -> 304251109)
TCP::DataOffset (8 -> 6) TCP::Flags (( SYN ) -> ( SYN ACK ))
TCP::WindowsSize (5840 -> 29200) TCP::CheckSum (0x6797 -> 0xf71)
IP::TotalLength (52 -> 44) IP::Identification (0x7c31 -> 0xaeac)
IP::TTL (5 -> 60) IP::CheckSum (0xbd6 -> 0xd62e)
IP::SourceIP (192.168.0.9 -> 109.88.203.231)
IP::DestinationIP (109.88.203.231 -> 192.168.0.9)
+TCPOptionMaxSegSize < TCPOptionMaxSegSize (4 bytes) :: Kind = 2 , Length = 4 , MaxSegSize = 1460 , >
-TCPOptionMPTCPCapable < TCPOptionMPTCPCapable (12 bytes) :: Kind = 30 , Length = 12 , Subtype = 0 , Version = 0 , Checksum = 1 (Checksum Enabled) , Flags = 0 , Crypto = 1 (HMAC-SHA1) , Sender's Key = Sender's Key = 1674215399152943104 , >
Here, the - (minus) sign before the MP_CAPABLE option indicates that this option was not included in the SYN+ACK. This server clearly does not support Multipath TCP.
Now, let us perform the same test to one server located in China :
sudo tracebox -n -v -p "IP/TCP/MPCAPABLE" cnzz.com
tracebox to 42.156.162.55 (cnzz.com): 64 hops max
...
...
...
23: 42.156.162.55 TCP::SrcPort (7487 -> 80) TCP::DstPort (80 -> 7487)
TCP::SeqNumber (915764810 -> 316943796) TCP::AckNumber (0 -> 915764811)
TCP::Flags (( SYN ) -> ( SYN ACK )) TCP::CheckSum (0x61b5 -> 0x210c)
IP::TTL (23 -> 46) IP::CheckSum (0xaf7c -> 0xcc48) IP::SourceIP
(192.168.0.9 -> 42.156.162.55) IP::DestinationIP
(42.156.162.55 -> 192.168.0.9)
Here, there is no - (minus) sign before the MP_CAPABLE option, which indicates that this option has been echoed by this server. A valid Multipath TCP server would not select the same key as the client, but a middlebox that echoes unknown options would…
Note that this behaviour is not specific to the Multipath TCP option. Any unknown option is echoed by these middleboxes :
sudo tracebox -n -v -p "IP/TCP/TCPOption.new{kind=222, length=4, data={249,137}}" baidu.com
tracebox to 111.13.101.208 (baidu.com): 64 hops max
...
...
...
20: 111.13.101.208 TCP::SrcPort (46253 -> 80) TCP::DstPort (80 -> 46253)
TCP::SeqNumber (569044298 -> 733320366) TCP::AckNumber(0 -> 569044299)
TCP::Flags (( SYN ) -> ( SYN ACK )) TCP::CheckSum (0x57be -> 0x9749)
IP::TTL (20 -> 47) IP::CheckSum (0x9426 -> 0xa4fa)
IP::SourceIP (192.168.0.9 -> 111.13.101.208)
IP::DestinationIP (111.13.101.208 -> 192.168.0.9)
While a normal server removes this unknown option in the SYN+ACK.
sudo tracebox -n -v -p "IP/TCP/TCPOption.new{kind=222, length=4, data={249,137}}" multipath-tcp.org
tracebox to 130.104.230.45 (multipath-tcp.org): 64 hops max
...
...
...
9: 130.104.230.45 TCP::SrcPort (51065 -> 80) TCP::DstPort (80 -> 51065)
TCP::SeqNumber (10489234 -> 3886645051) TCP::AckNumber (0 -> 10489235)
TCP::Flags (( SYN ) -> ( SYN ACK )) TCP::WindowsSize (5840 -> 29200)
TCP::CheckSum (0xb23c -> 0xc02c) IP::Identification (0x2b81 -> 0x0)
IP::Flags (0 -> 2) IP::TTL (9 -> 57) IP::CheckSum (0x3130 -> 0x1885)
IP::SourceIP (192.168.0.9 -> 130.104.230.45)
IP::DestinationIP (130.104.230.45 -> 192.168.0.9)
-TCPOption < TCPOption (4 bytes) :: Kind = 222 , Length = 4 , Payload = \xf9\x89>
+TCPOptionMaxSegSize < TCPOptionMaxSegSize (4 bytes) :: Kind = 2 , Length = 4 , MaxSegSize = 1380 , >
Here, the MSS option has been added and the unknown option has been removed.
After some discussions, the authors of this scan have updated their methodology and now correctly distinguish between real Multipath TCP enabled servers and middleboxes that echo Multipath TCP options. They even provide an interesting dashboard that summarises their measurements at
https://academic-network-security.research.nicta.com.au/mptcp/deployment/
