In Korean, Multipath TCP is pronounced GIGA Path

In September 2013, Apple surprised the networking community by enabling Multipath TCP on all iOS devices . The main motivation for this deployment was to support Apple’s voice recognition application and enable it to work seamlessly over both WiFi and cellular networks. Multipath TCP is a good match for this application, but it can also be used for other use cases.

At IETF‘93 in Prague, SungHoon Seo provided several very interesting details of the Gigapath commercial service that is now sold by KT. This service enables smartphone users to reach bandwidth of up to 1 Gbps on existing smartphones. This is probably the fastest commercially deployed mobile network. They achieve this high bandwidth by combining both fast LTE (with carrier aggregation) and fast WiFi networks on Multipath TCP enabled smartphones. At this stage, only the Samsung Galaxy S6 and Galaxy S6 Edge smartphones support the Gigapath service, but KT is working with other vendors to add Multipath TCP on their smartphones. Measurements presented at the MPTCP Working Group meeting revealed that current smartphones are able to reach throughputs of about 800 Mbps out of a theoretical maximum of 1.17 Gbps.

What is more impressive is how the system has been implemented and how the users can benefit from it. The figure below, extracted from SungHoon Seo’s presentation provides the general architecture of the GIGA Path system.

../../../_images/kt.png

On the client side, the smartphones include the open-source Multipath TCP implementation in the Linux kernel. Samsung reused release 0.89.4 and backported it in their Android kernel. The full source code of their Multipath TCP kernel is available online

Enabling Multipath TCP on the smartphone is the first step in deploying it. However, this is not sufficient since there are very few servers that support Multipath TCP today. To enable their users to benefit from Multipath TCP for all the applications that they use, KT has opted for a SOCKSv5 proxy. This proxy is running on x86 servers using release 0.89.5 of the open-source Multipath TCP implementation in the Linux kernel. During the presentation, SungHoon Seo mentioned that despite the recent rollout of the service, there were already 5,500 active users on the SOCKS proxy the last time he checked. Thanks to this proxy, the subscribes of the Giga Path service in Korea can benefit from Multipath TCP with all the TCP-based applications that they use.

At the end of KT’s presentation, another network engineer mentioned that he would come back to his management and propose a similar approach to deploy Multipath TCP in his network. We can thus expect other large scale deployments in the coming months.

A closer look at the scientific literature on Multipath TCP

Some time ago, @ben_pfaff sent a tweet indicating that he was impressed by a google scholar search that returns more than 1,000 hits for “open vswitch”. This triggered my curiosity and I wondered what was the current impact of Multipath TCP in the scientific literature.

Google scholar clearly indicates that there is a Multipath TCP effect in the academic community. For the “Multipath TCP” query, google scholar already lists more than 1500 results.

../../../_images/mptcp1.png

For the “mptcp” query, google scholar reports a similar number of documents.

../../../_images/mptcp2.png

This is a clear indication that there is a growing number of researchers who are adopting Multipath TCP and propose extensions and improvements to it. The researchers who start to work on Multipath TCP will need to understand an already large set of articles. As a first step towards a detailed bibliography on Multipath TCP, I’ve started to collect some notes on IETF documents and scientific paper on this topic. This is a work in progress that will be updated every time I find some time to dig into older papers or new interesting papers are published.

The current version of the annotated bibliography will always be available from https://github.com/obonaventure/mptcp-bib This repository contains all the latex and bibtex files and could be useful to anyone doing research on Multipath TCP. The pdf version (mptcp-bib.pdf) will also be updated after each major change for those who prefer pdf documents.

Interesting Multipath TCP talks

Various tutorials and trainings have been taught on Multipath TCP during the last years. Some of these have been recorded and are available via youtube.com.

The most recent video presentation is the talk given by Octavian Purdila from intel at the netdev‘01 conference In this talk, Octavian first starts with a brief tutorial on Multipath TCP targeted at Linux networking kernel developpers and then describes in details the structure of the current code and the plans for upstreaming it to the official Linux kernel.

A longer tutorial on the Multipath TCP protocol was given by Olivier Bonaventure at IETF‘87 in Berlinin August 2013.

Christoph Paasch gave a shorter Multipath TCP tutorial earlier during FOSDEM‘13 in Brussels.

Earlier, Costin Raiciu and Christoph Paasch gave a one hour Google Research talk on the design of the protocol and several use cases.

Costin : google tech talk

The Google Research talk was given a few days after the presentation of the USENIX NSDI‘12 paper that received the community award. This presentation is available from the USENIX website.

mptcptrace demo, experiment five

This is the fifth post of a series of five. Context is presented in the first post. The second post is here. The third post is here. The fourth post is here.

Fifth experiment

Green at 0s:
  • delay : 10ms
  • bandwidth : 4mbit/s
  • loss : 0%
Green at 5s:
  • delay : 100ms
  • bandwidth : 4mbit/s
  • loss : 10%
Green at 15s:
  • delay : 10ms
  • bandwidth : 4mbit/s
  • loss : 0%
Red:
  • delay : 40ms
  • bandwidth : 4mbit/s
Client:
  • Scheduler : Round robin

Last experiment of this series, we come back to the third experiment, and instead of adding a 1% loss rate after 15 seconds of the red subflow, we change the MPTCP scheduler and we use the round robin. It is worth to note that the round robin scheduler still respects the congestion window of the subflows.

Let’s see the evolution of the sequence number :

../../../_images/sequence4.png

The first thing that we can see are the small steps between 5 and 15 seconds. We also have the impression that we use more the red subflow but if we zoom :

../../../_images/sequence_zoom4.png
../../../_images/sequence_zoom_21.png

we can confirm that we use both subflows. We see 3 lines:

  1. The segments, because they are sent together at the sender, red and green subflows do not form two separate lines
  2. The green acks : we can see that they are closer to the segment line
  3. The red acks : we can see that all red acks are late from the MPTCP point of view. This is normal since the green delay is shorter.

If we look at the evolution of the sequence number between 5 and 15 seconds, we can observe a series of stairs.

../../../_images/sequence_zoom_3.png

If we take a close at one of the this stairs :

../../../_images/sequence_zoom_4.png

because the green subflow is lossy during this period, we have reordering. Because we use the round robin scheduler, MPTCP still decides to send some data over the green path.

If we now take a look at the evolution of the goodput :

../../../_images/gput4.png

We can see the perturbation of the lossy link over the “instantaneous” goodput.

However the impact on the average goodput is somehow mitigated. Depending of the application, these variations may be problematic or not.

../../../_images/flight2.png

If we take a look at the evolution of the MPTCP unacked data, we see a lot of variations during the period 5 to 15 seconds. This is due to the reordering that happens during this period. This is not a big issue as long as the receive window is big enough to absorb these variations. In some scenarios this may be an issue if the window is too small. We may also remark that MPTCP may use more memory in this case on the receiver due to the buffer auto-tuning.

Finally, we can take a look at the evolution of the unacked data at the TCP level.

../../../_images/flightpf2.png

We can observe that we use both subflows during the whole connection but losses between 5 and 15 seconds on the green subflow leads to a bigger usage of the red subflow during this period.

Conclusion

This ends the series of posts that shows some basic MPTCP experiments. mptcptrace has been used to get the values out of the traces and R scripts have been used to produce the graphs. However we did not really post process the data in R. We have more experiments and visualizations that we will present later.