Commercial usage of Multipath TCP

Since the publication of RFC 6824 in January 2013, various companies have started to leverage Multipath TCP‘s unique capabilities to create new innovate products. This post is a short summary of some of the publicly announced utilisations of Multipath TCP .

Multipath networks

Multipath networks is an Irish company that was the first to use Multipath TCP to bond two ADSL links or an ADSL and a wireless link. Their initial product relied on a modified home router that used the Linux Multipath TCP kernel together with OpenVPN and an HTTP proxy. The router intercepts all TCP traffic, sends it to a server running in the cloud over Multipath TCP and the server uses regular TCP to reach the final destination. Unfortunately, the company went bankrupt and the service is not sold anymore as of this writing.


VRT is the Flemish TV broadcaster in Belgium. They have designed their own cars to allow web journalists to capture videos, edit them and upload them to the VRT head quarters. Videos are long files that require a large bandwidth to be uploaded quickly. To allow the journalists to send their video reports as quickly as possibly, the latest VRT car, called The Beast has been equipped with three types of antennas :

  • one satellite antenna
  • several 3G antennas with the corresponding SIMs
  • several WiFi antennas

Once a video is ready, the server running in the car automatically starts all the available network interfaces and combines them thanks to Multipath TCP to upload the entire video to the VRT head quarters. This car has been used in production for more than a year at VRT.


Apple has started to use Multipath TCP on iPhones and iPads in September 2013 to support the Siri voice recognition application. Thanks to Multipath TCP, these mobile devices can better cope with losses and connectivity problems over the wireless interfaces. This deployment uses an implementation written by Apple’s engineers that is now also included in MacOS. Apple’s implementation of Multipath TCP does not include all the features of the protocol defined in RFC 6824 but it is fully interoperable with the Linux implementation.


Tessares is a recent spinoff from UCL that was created with funding from Proximus, the Belgian network operator and the VIVES investment fund. Its objective is to develop new innovative network services on top of Multipath TCP. The first product developed by this company is a solution for Hybrid Access Networks. Such an access network combines two different types of technologies, typically DSL and 3G/4G. It is illustrated in the figure below.

Hybrid access networks with Multipath TCP

This solution is composed of two different network devices :

  • The Hybrid CPE (HCPE)
  • The Hybrid Aggregation Gateway (HAG)

The HCPE is a CPE device that is capable of using two separate access networks. It is typically a home router that has been extended with a 3G/4G interface. Tessares provides a tuned version of the Multipath TCP implementation in the Linux kernel that has been optimised for this platform. It also includes a Multipath TCP proxy that intercepts the TCP connections established by the devices in the home network and converts them into Multipath TCP connections. Thanks to the utilisation of Multipath TCP, the devices used in the home network can use both the DSL and the 3G/4G network. The Hybrid Aggregation Gateway terminates the Multipath TCP connection and converts them into regular TCP connections so that regular servers that have not been upgraded to support Multipath TCP can be contacted.

The BroadBand Forum is working on solutions to support Hybrid Access Networks. During the last Broadband World Forum in London, several companies have demonstrated solutions that include the Multipath TCP implementation in the Linux kernel : Tessares that received a highly commended award, SoftatHome, Sagemcom, Technicolor, Intel and Ericsson.


Gigapath is a commercial service that was launched during the summer 2015 by Korean Telecom. In Korea competition among network operators forces them to provide higher bandwidth mobile services. The cellular networks deployed in this country are among the fastest in the world, but this is still not sufficient. Gigapath allows smartphone uses to combine together their 4G and WiFi networks to reach bandwidths of 800 Mbps and more.

From a technical viewpoint, the solution deployed by KT combines Multipath TCP and the SOCKS protocol. Korean Telecom has convinced Samsung and LG Electronics to port the open-source Multipath TCP implementation in the Linux kernel on their high-end smartphones. As of December 2015, there are about half a dozen different smartphone models from these two vendors that include Multipath TCP. Each smartphone also includes a SOCKS client that intercepts all TCP connection establishments and redirects them to a SOCKS proxy running on one server managed by Korean Telecom. The SOCKS proxy uses the Multipath TCP implementation in the Linux kernel and terminates the Multipath TCP connection.

Architecture of the Gigapath service

In July 2015, 5,000 users had subscribed to the Gigapath service. In November 2015, there were about 20,000 users.


OVH is a French cloud company that also provides DSL services. In September 2015, they announced a new product called Overthebox. This product combines Multipath TCP and SOCKS proxies to enable users to bond different DSL lines together. In contrast with the SOCKS-based solution deployed by KT, OVH did not modify the enduser devices. Instead, they provide a device that is attached to the different DSL routers that need to be combined. This device acts as the default gateway in the home network and serves as the DHCP server. Its SOCKS client can then intercept all established TCP connections and convert them into Multipath TCP towards a SOCKS server running in the cloud. The SOCKS server terminates the Multipath TCP and creates a regular TCP connection to the final destination. In December 2015, more than 300 users already participate in the beta and the commercial deployment is expected in January 2016.

New ways to verify that Multipath TCP works through your network

The design of Multipath TCP has been heavily influenced by the middleboxes that have been deployed in a wide range of networks, notably in cellular and enterprise networks. Some of these middleboxes like regular NATs interact correctly with Multipath TCP and many Multipath TCP users work behind NATs. However, some middleboxes, such as firewalls or TCP optimisers, terminate TCP connections or interfere with TCP options and thus interact badly with Multipath TCP.

Several tools can be used to verify that Multipath TCP works through a given network. If you have installed a Multipath TCP enabled kernel, you can simply use curl and issue the following command :


The webserver that supports has been configured to send a special response to an HTTP request with the curl User-Agent. If the request is sent over a regular TCP connection, the server replies with :

Nay, Nay, Nay, your have an old computer that does not speak MPTCP. Shame on you!

If the HTTP request is sent over a Multipath TCP connection, the server replies with :

Yay, you are MPTCP-capable! You can now rest in peace.

This is a basic test that if often used to validate the correct installation of a Multipath TCP enabled Linux kernel.

However, many users are interested in simpler tests through a web interface or through a smartphone application. Two young researchers have recently released two useful tools. has been the first website created to verify that Multipath TCP was working correctly. Unfortunately, it is not anymore up and running. Alexander Frommgen and his colleagues at TU Darmstadt have posted an updated version of this website. In addition to verifying that the web page is served over a Multipath TCP connection, the new web site also checks whether Multipath TCP passes correctly through other ports. You can test it at Other tests will be added soon.

Screenshot of

Another option is tracebox. This command-line tool allows to perform traceroute-like tests with different TCP options to verify whether they pass through middleboxes. tracebox works well on Linux and MacOS, but not yet on smartphones.

Raffaele Zullo a student at the University of Napoli in Italy has spent several months at the University of Liege to work with Benoit Donnet. During his internship, he developed a new version of tracebox that runs on Android smartphones. It requires a rooted smartphone, but does not need a Multipath TCP kernel on the smartphone. You can download it from

Screenshot of Tracebox on Android

Measuring the adoption of Multipath TCP is not so simple...

In September 2015, a google alert announced a new workshop paper entitled An Early Look at Multipath TCP Deployment in the Wild. The paper abstract was intriguing with sentences like We find that less than 0.1% of Alexa unique domains and IP addresses currently support MPTCP. Their geographic distribution is quite different from that of clients reported in other studies, with the majority of domains being in China. Based on the abstract and the results reported in the paper, one could assume that Multipath TCP has already been deployed on various Internet servers, and the paper lists several important websites in China that are supposed to support Multipath TCP.

Unfortunately, the initial measurements reported in this workshop paper were flawed. Most of the studies on the deployment of TCP extensions have used a network scanner (zmap in this one) to send SYN segments containing a specific TCP option. If the server replies with the same option, then it is assumed to support the TCP extension. The authors of this paper applied the same methodology to Multipath TCP. Unfortunately, looking simply at the presence of an option in the SYN+ACK is not sufficient because there are middleboxes that simply echo any option that they receive. This problem has been discussed in the Multipath TCP mailing list in the past and has influenced the design of Multipath TCP. It is notably described in the article that proposed the tracebox debugging tool.

With tracebox, it is easy to check whether a server really supports Multipath TCP. Let us start with

sudo tracebox -n -v -p "IP/TCP/MPCAPABLE"
tracebox to ( 64 hops max
9: TCP::SrcPort (47416 -> 80) TCP::DstPort (80 -> 47416)
   TCP::SeqNumber (554672918 -> 3111441317) TCP::AckNumber (0 -> 554672919)
   TCP::DataOffset (8 -> 9) TCP::Flags (( SYN ) -> ( SYN ACK ))
   TCP::WindowsSize (5840 -> 28800) TCP::CheckSum (0x5eb9 -> 0x206d)
   IP::TotalLength (52 -> 56) IP::Identification (0x50ab -> 0x0)
   IP::Flags (0 -> 2) IP::TTL (9 -> 57) IP::CheckSum (0x406 -> 0x1879)
   IP::SourceIP ( ->
   IP::DestinationIP ( ->
   +TCPOptionMaxSegSize < TCPOptionMaxSegSize (4 bytes) :: Kind = 2 , Length = 4 , MaxSegSize = 1380 , >
   TCPOptionMPTCPCapable::Sender's Key (Sender's Key = 692439777126907904 -> Sender's Key = 17898842517462319104)

The tracebox command is used to send a SYN segment with the MP_CAPABLE option. The output above (only the last line, the interesting one is shown) indicates that the server has replied by adding the MSS option that was not present in the SYN. Furthermore, the SYN+ACK includes the MP_CAPABLE option with a different key than the one sent in the SYN segment.

The same test towards a server that does not (yet ?) support Multipath TCP is shown below :

sudo tracebox -n -v -p "IP/TCP/MPCAPABLE"
tracebox to ( 64 hops max
5: TCP::SrcPort (12002 -> 80) TCP::DstPort (80 -> 12002)
TCP::SeqNumber (304251108 -> 752364946) TCP::AckNumber (0 -> 304251109)
TCP::DataOffset (8 -> 6) TCP::Flags (( SYN ) -> ( SYN ACK ))
TCP::WindowsSize (5840 -> 29200) TCP::CheckSum (0x6797 -> 0xf71)
IP::TotalLength (52 -> 44) IP::Identification (0x7c31 -> 0xaeac)
IP::TTL (5 -> 60) IP::CheckSum (0xbd6 -> 0xd62e)
IP::SourceIP ( ->
IP::DestinationIP ( ->
+TCPOptionMaxSegSize < TCPOptionMaxSegSize (4 bytes) :: Kind = 2 , Length = 4 , MaxSegSize = 1460 , >
-TCPOptionMPTCPCapable < TCPOptionMPTCPCapable (12 bytes) :: Kind = 30 , Length = 12 , Subtype = 0 , Version = 0 , Checksum = 1 (Checksum Enabled) , Flags = 0 , Crypto = 1 (HMAC-SHA1) , Sender's Key = Sender's Key = 1674215399152943104 , >

Here, the - (minus) sign before the MP_CAPABLE option indicates that this option was not included in the SYN+ACK. This server clearly does not support Multipath TCP.

Now, let us perform the same test to one server located in China :

sudo tracebox -n -v -p "IP/TCP/MPCAPABLE"
tracebox to ( 64 hops max
23: TCP::SrcPort (7487 -> 80) TCP::DstPort (80 -> 7487)
TCP::SeqNumber (915764810 -> 316943796) TCP::AckNumber (0 -> 915764811)
TCP::Flags (( SYN ) -> ( SYN ACK )) TCP::CheckSum (0x61b5 -> 0x210c)
IP::TTL (23 -> 46) IP::CheckSum (0xaf7c -> 0xcc48) IP::SourceIP
( -> IP::DestinationIP
( ->

Here, there is no - (minus) sign before the MP_CAPABLE option, which indicates that this option has been echoed by this server. A valid Multipath TCP server would not select the same key as the client, but a middlebox that echoes unknown options would...

Note that this behaviour is not specific to the Multipath TCP option. Any unknown option is echoed by these middleboxes :

sudo tracebox -n -v -p "IP/TCP/{kind=222, length=4, data={249,137}}"
tracebox to ( 64 hops max
20: TCP::SrcPort (46253 -> 80) TCP::DstPort (80 -> 46253)
TCP::SeqNumber (569044298 -> 733320366) TCP::AckNumber(0 -> 569044299)
TCP::Flags (( SYN ) -> ( SYN ACK )) TCP::CheckSum (0x57be -> 0x9749)
IP::TTL (20 -> 47) IP::CheckSum (0x9426 -> 0xa4fa)
IP::SourceIP ( ->
IP::DestinationIP ( ->

While a normal server removes this unknown option in the SYN+ACK.

sudo tracebox -n -v -p "IP/TCP/{kind=222, length=4, data={249,137}}"
tracebox to ( 64 hops max
9: TCP::SrcPort (51065 -> 80) TCP::DstPort (80 -> 51065)
TCP::SeqNumber (10489234 -> 3886645051) TCP::AckNumber (0 -> 10489235)
TCP::Flags (( SYN ) -> ( SYN ACK )) TCP::WindowsSize (5840 -> 29200)
TCP::CheckSum (0xb23c -> 0xc02c) IP::Identification (0x2b81 -> 0x0)
IP::Flags (0 -> 2) IP::TTL (9 -> 57) IP::CheckSum (0x3130 -> 0x1885)
IP::SourceIP ( ->
IP::DestinationIP ( ->
-TCPOption < TCPOption (4 bytes) :: Kind = 222 , Length = 4 , Payload = \xf9\x89>
+TCPOptionMaxSegSize < TCPOptionMaxSegSize (4 bytes) :: Kind = 2 , Length = 4 , MaxSegSize = 1380 , >

Here, the MSS option has been added and the unknown option has been removed.

After some discussions, the authors of this scan have updated their methodology and now correctly distinguish between real Multipath TCP enabled servers and middleboxes that echo Multipath TCP options. They even provide an interesting dashboard that summarises their measurements at


NetPerfMeter : A Network Performance Metering Tool


A common problem for evaluating multiple transport protocols in a multi-platform environment is to have a test tool that is capable to run in all these environments, and – of course – to support all necessary protocols. Using different evaluation tools is not a good solution, since each tool may introduce its own – and possibly incompatible – parametrisation scheme. In order to overcome this problem, originally for the use case of evaluating the Stream Control Transmission Protocol (SCTP) and compare it to the Transmission Control Protocol (TCP), NetPerfMeter has been designed and developed.

What is NetPerfMeter?

NetPerfMeter [3][5] is an open source, multi-platform transport protocol performance evaluation software. It currently supports the Linux, FreeBSD and MacOS platforms (with possibility to easily extend it to further platforms), and the transport protocols SCTP, TCP including Multi-Path TCP (MPTCP, if supported by the operating system), UDP (User Datagram Protocol) and DCCP (Datagram Congestion Control Protocol, if supported by the operating system). The figure below presents the NetPerfMeter protocol stack.


In each direction, a data channel can be operated in saturated mode (send as much as possible; not available for UDP-based data channels) or non-saturated mode. In case of the non-saturated mode, the traffic is configured in form of frame rates and frame sizes. That is, frames of a given size are generated in given intervals. Both, frame rate and frame size can be randomised as well.

The performance of the data channels (bandwidth, delay, jitter, etc.) is evaluated and recorded at active and passive side. At the end of a measurement, all collected data is transferred over the control channel to the active side. That is, a user can just conveniently collect all results at the active side. Further details on NerPerfMeter can be found in [3]. [4], [3] also introduce an OMNeT++ simulation model for NetPerfMeter that can be used in simulations, in order to easily compare simulation results and real-world measurements. [1], [2] and [6] provide examples of NetPerfMeter usage for SCTP and MPTCP protocol performance analyses.

Testing MPTCP with NetPerfMeter

In the following, a short tutorial for using NetPerfMeter with MPTCP is provided. It explains the basic features to start experimenting. A more detailed overview of all possible options can be found in the manpage of NetPerfMeter.

Starting the Passive Side

First, the passive side of NetPerfMeter needs to be started:

netperfmeter 9000

This command starts NetPerfMeter in server mode, waiting for control channel connections on port 9000.

Starting the Active Side

One Non-Saturated TCP Flow

Let us start with just one TCP flow:

netperfmeter <passive side>:9000 -tcp const10:const1460:const0:const0

This command starts NetPerfMeter in client mode, connecting to the given peer (given by hostname or IP address), port 9000. A TCP data channel is established, sending 10 frames/s with 1460 bytes/frame from the active to the passive side. The second parameter block specifies 0 frames/s with 0 bytes/frame (that is, no data) in the opposite direction. The measurement ends when stopped by SIGINT (i.e. Ctrl+C).

One Saturated TCP Flow

Making the TCP flow saturated (that is, to send as much as possible) is as easy as setting the frame rate to 0 frames/s (meaning to send as much as possible). Each frame again has a size of 1460 bytes:

netperfmeter <passive side>:9000 -tcp const0:const1460:const0:const0

A Bidirectional TCP Flow

In order to also transmit data in the direction from passive side to active side, just update the parameters:

netperfmeter <passive side>:9000 -tcp const0:const1460:const10:const1460

A fixed frame size of 1460 bytes may not be useful in all scenarios. Therefore, NetPerfMeter provides random distributions as well:

  • exp<average>: Negative exponential distribution with given average. Example: exp1000.
  • uniform<lower>,<upper>: Uniform distribution from lower to upper bound. Example: uniform500,25000.

Random distributions can of course be used for the frame rate as well.

Multi-Path Transport

NetPerfMeter is also able to turn MPTCP on or off per socket (given that the implementation supports the TCP_MULTIPATH_ENABLE socket option). Then, MPTCP can be enabled by the “cmt=mptcp” option:

netperfmeter <passive side>:9000 -tcp const0:const1460:const0:const0:cmt=mptcp

MPTCP versus TCP

Of course, NetPerfMeter can use multiple data channels as well:

netperfmeter <passive side>:9000 \
    -tcp const0:const1460:const0:const0:cmt=mptcp \
    -tcp const0:const1460:const0:const0:cmt=off

In this case, NetPerfMeter starts one MPTCP flow and one concurrent TCP flow. “cmt=off” turns MPTCP off (the default; just not specifying the “cmt” option has the same effect).

Recording Statistics

One of the most important features for researchers is of course to easily get machine-readable results files. Two parameters provide the generation of such files:

  • -scalar=<prefix>.<suffix>
: Generates files named <prefix>-<active|passive>-<flow ID>-<stream ID>.<suffix> with aggregates over the whole measurement. The scalar file format is compatible to OMNeT++ scalar files.
  • -vector=<prefix>.<suffix> :

Generates files named <prefix>-<active|passive>-<flow ID>-<stream ID>.<suffix> with per-frame statistics. The vector files are text tables that could for example be processed by GNU R2 or GNU Plot3. For convenience, if the suffix ends with .bz>2, the results file is BZip2-compressed on the fly. To automate measurements, the -runtime=<seconds> option specifies a duration of the measurement. That is, the following example would run a 60s MPTCP versus TCP comparison, and record scalars as well as vectors:

netperfmeter <passive side>:9000 \
   -runtime=60 \ \
   -vector=vectors.vec.bz2 \
   -tcp const0:const1460:const0:const0:cmt=mptcp \
   -tcp const0:const1460:const0:const0:cmt=off


NetPerfMeter is a convenient and flexible open source tool for transport protocol performance analysis. It particularly provides multi-platform support and works with TCP, SCTP, UDP as well as DCCP.



[1] Dreibholz, T.; Zhou, X. and Fa, F.: “Multi-Path TCP in Real-World Setups – An Evaluation in the NorNet Core Testbed”, in 5th International Workshop on Protocols and Applications with Multi-Homing Support (PAMS), pp. 617–622, Gwangju/South Korea, March 2015.

[2] Dreibholz, T.; Adhari, H.; Becke, M. and Rathgeb, E. P.: “Simulation and Experimental Evaluation of Multipath Congestion Control Strategies”, in Proceedings of the 2nd International Workshop on Protocols and Applications with Multi-Homing Support (PAMS), Fukuoka/Japan, March 2012.

[3] Dreibholz, T.: “Evaluation and Optimisation of Multi-Path Transport using the Stream Control Transmission Protocol”, Habilitation Treatise, University of Duisburg-Essen, Faculty of Economics, Institute for Computer Science and Business Information Systems, March 2012.

[4] Dreibholz, T.; Adhari, H.; Becke, M. and Rathgeb, E. P.: “NetPerfMeter – A Versatile Tool for Multi-Protocol Network Performance Evaluations”, OMNeT++ Code Contribution, University of Duisburg-Essen, Institute for Experimental Mathematics, February 2012.

[5] Dreibholz, T.; Becke, M.; Adhari, H. and Rathgeb, E. P.: “Evaluation of A New Multipath Congestion Control Scheme using the NetPerfMeter Tool-Chain”, inProceedings of the 19th IEEE International Conference on Software, Telecommunications and Computer Networks (SoftCOM), pp. 1–6, Hvar/Croatia, September 2011.

[6] Adhari, H.; Dreibholz, T.; Becke, M.; Rathgeb, E. P. and Tüxen, M.: “Evaluation of Concurrent Multipath Transfer over Dissimilar Paths”, in Proceedings of the 1st International Workshop on Protocols and Applications with Multi-Homing Support (PAMS), pp. 708–714, Singapore, March 2011.