Why are there sometimes long delays before the establishment of MPTCP connections ?
Multipath TCP users sometimes complain that Multipath TCP connections are established after a longer delay than regular TCP connections. This can happen in some networks and the culprit is usually a middlebox hidden on the path between the client and the server. This problem can easily be detected by capturing the packets on the client with tcpdump Such a capture looks like :
11:24:05.225096 IP client.59499 > multipath-tcp.org.http:
Flags [S], seq 270592032, win 29200, options [mss 1460,sackOK,
TS val 7358805 ecr 0,nop,wscale 7,
mptcp capable csum {0xaa7fa775d16fa6bf}], length 0
The client sends a SYN with the MP_CAPABLE option… Since it receives no answer, it retransmits the SYN.
11:24:06.224215 IP client.59499 > multipath-tcp.org.http: Flags [S],
seq 270592032, win 29200, options [mss 1460,sackOK,
TS val 7359055 ecr 0,nop,wscale 7,
mptcp capable csum {0xaa7fa775d16fa6bf}], length 0
And unfortunately two times more…
11:24:08.228242 IP client.59499 > multipath-tcp.org.http: Flags [S],
seq 270592032, win 29200, options [mss 1460,sackOK,
TS val 7359556 ecr 0,nop,wscale 7,
mptcp capable csum {0xaa7fa775d16fa6bf}], length 0
11:24:12.236284 IP client.59499 > multipath-tcp.org.http: Flags [S],
seq 270592032, win 29200, options [mss 1460,sackOK,
TS val 7360558 ecr 0,nop,wscale 7,
mptcp capable csum {0xaa7fa775d16fa6bf}], length 0
At this point, Multipath TCP considers that there could be a middlebox that discards SYN segments with the MP_CAPABLE option on the path to reach the server and disables Multipath TCP.
11:24:20.244351 IP client.59499 > multipath-tcp.org.http: Flags [S],
seq 270592032, win 29200, options [mss 1460,sackOK,
TS val 7362560 ecr 0,nop,wscale 7], length 0
This segment immediately reaches the server that replies :
11:24:20.396718 IP multipath-tcp.org.http > client.59499: Flags [S.],
seq 3954135908, ack 270592033, win 28960, options [mss 1380,sackOK,
TS val 2522075773 ecr 7362560,nop,wscale 7], length 0
11:24:20.396748 IP client.59499 > multipath-tcp.org.http: Flags [.],
ack 1, win 229, options [nop,nop,TS val 7362598 ecr 2522075773], length 0
As shown by the trace, the middlebox, by dropping the SYN segments containing the MP_CAPABLE option has delayed the establishment of the TCP connection by fifteen seconds. This delay is controlled by the initial retransmission timer (one second in this example) and the exponential backoff applied by TCP to successive retransmissions of the same segments.
What can Multipath TCP users do to reduce this delay ?
- the best answer is to contact their sysadmins/network administrators and use a tool like tracebox to detect where packets with the MP_CAPABLE option are dropped and upgrade this middlebox
- if changing the network is not possible, the implementation of Multipath TCP in the Linux kernel can be configured to more aggressively fallback to regular TCP through the net.mptcp.mptcp_syn_retries configuration variable described on http://multipath-tcp.org/pmwiki.php/Users/ConfigureMPTCP. This variable controls the number of retransmissions for the initial SYN before stopping to use the MP_CAPABLE option (the default is 3)