Apple Music on iOS13 uses Multipath TCP through load-balancers
Apple Music on iOS13 uses Multipath TCP through load-balancers
Since the publication of RFC6824 in 2013, interest in Multipath TCP has continued to grow and various use cases have been deployed. Apple uses Multipath TCP for its Siri voice recognition application since 2013 to support seamless handovers while walking. Tessares uses Multipath TCP to deploy Hybrid Access Networks that combine xDSL and LTE to provide faster Internet access services in rural areas. Samsung, LG and Huawei use Multipath TCP on their Android smartphones to combine Wi-Fi and 4G. Recently, 3GPPP has selected Multipath TCP to support Wi-Fi/5G co-existence in future 5G networks and a first prototype has been demonstrated.
Despite these growing deployments, web hosting companies and CDNs have complained that Multipath TCP was difficult to deploy because they assume that load balancers would need to be changed to terminate Multipath TCP connections.
It turns out that it is possible to support Multipath TCP on servers with todays’ load-balancers. Fabien Duchêne proposed and evaluated this solution in the Making Multipath TCP Friendlier to Load-Balancers and Anycast paper that he presented at ICNP’17. A simpler load-balancer is illustrated in the figure below. The load-balancer uses IP address 1.2.3.4 and forwards connections to the servers behind it, selecting them e.g. based on a hash function.
With Multipath TCP, this simple approach does not work anymore as the second subflow from the client could be hashed to a different server than the one of the initial subflow.
The solution proposed by Fabien Duchêne is both simple and efficient. The load-balancer has a public IP address that is advertised by the DNS. Furthermore, each server that resides behind this load balancer has it own public IP address. When a client contacts the load-balanced service, its SYN packet reaches the load-balancer that selects one of the servers in the pool. The server confirms the establishment of the connection using the load-balanced IP address. As soon as the Multipath TCP connection is established, the server sends a packet containing its own IP address inside an ADD_ADDR option. Thanks to this address, the client can establish the subsequent subflows directly towards the server address, effectively bypassing the load-balancer.
Given its benefits, this solution has been included in the standards-track version of Multipath TCP that is being finalised by the IETF. However, it had not yet been deployed at a large scale, until the release of iOS13 by Apple in September 2019. Given the benefits of Multipath TCP for Siri, Apple has decided to also use Multipath TCP for its Apple Maps and Apple Music applications. These two applications are often used while walking and thus suffered from interruptions during Wi-Fi/cellular handovers. A closer look at a packet trace collected from an iPhone using Apple Music shows interesting results.
First, Apple Music uses Multipath TCP as shown by the options contained in the SYN packet. It is interesting to point out that both the iPhone and the server use the first version of Multipath TCP defined in RFC6824.
Once the connection has been established, the server sends quickly a packet that advertises its own address in the ADD_ADDR option.
A closer look at this option indicates that this address is advertised as having address identifier 0. According to RFC6824, this identifier is reserved for the address of the initial subflow. This advertisement thus changes the address of the initial subflow, and we can expect that iOS13 will use this new address to establish subsequent subflows on this Multipath TCP connection. We checked by contacting one of Apple Music servers from a Linux client to see how Linux reacts to such an option and it supports it correctly.
According to statista, there were more than 60 million subscribers of the Apple Music service in June 2019. As they upgrade their smartphones to support iOS13, we will observe a huge growth in Multipath TCP traffic during the coming weeks… If you see other servers or CDNs that enable Multipath TCP, let us know…