Leveraging a dataset of almost half a billion packets with high-precision packet times and sizes, we extract characteristics of the bursts emitted over Starlink’s Ethernet interface. The structure of these bursts directly reflects the physical layer reception of OFDMA frames on the satellite link. We study these bursts by analyzing their rates, and thus indirectly also the transition between different physical layer rates. The results highlight that there is definitive structure in the transition behavior, and we note specific behaviors such as particular transition steps associated with rate switching, and that rate switching occurs mainly to neighboring rates. We also study the joint burst rate and burst duration transitions, noting that transitions occur mainly within the same rate, and that changes in burst duration are often performed with an intermediate short burst in-between. Furthermore, we examine the configurations of the three factors burst rate, burst duration, and inter-burst silent time, which together determine the effective throughput of a Starlink connection. We perform pattern mining on these three factors, and we use the patterns to construct a dynamic N-gram model predicting the characteristics of the next upcoming burst, and by extension, the short-term future throughput. We further train a Deep Learning time-series model which shows improved prediction performance. 

Network latency can have a large impact on the performance of networked applications and the quality of experience for the end users. By using eBPF to implement continuous passive network latency monitoring and aggregation in kernel space with epping, we provide an efficient monitoring solution that can be deployed on any existing Linux node in the network. Our solution thus allows network operators to gain a highly granular view of the network latency experienced by the traffic their network is serving without needing to buy and deploy new hardware. This information can help network operators understand what quality of service their network is currently offering, locate issues in their networks, and verify if new solutions aimed at improving network latency have the desired effect, ultimately improving the experience for the end users. In this technical report, we provide a detailed description of how epping works, cover how different features have been implemented, and discuss many of the challenges we have encountered while implementing it.

Leveraging a data set of almost half a billion packets with high-precision packet times and sizes, we process it to extract characteristics of the bursts emitted over Starlink's Ethernet interface. The structure of these bursts directly reflect the physical layer receipt of OFDMA frames on the satellite link. We study these bursts by analyzing their rates, and by proxy the transition between different physical layer rates. The results highlight that  there is definitive structure in the transition behavior, and we note specific behaviors such as  particular transitionsteps associated with rate switching, and that rate switching occurs mainly to neighboring rates. We also study the joint burst rate and burst duration transitions, noting that transitions occur mainly within the same rate, and that changes in burst duration are often performed with an intermediate short burst in-between.Finally, we examine the configurations of the three factors burst rate, burst duration, and inter-burst silent time, which together determine the effective throughput of a Starlink connection.

Although Starlink has been deployed for several years, a detailed understanding of system internals is still lacking. In this work we employ precise per-packet timestamps obtained from a hardware-timestamp capable NIC connected to a Starlink terminal.We find that Starlink frame timing details are readily observable at the network layer by analyzing the packet timing patterns.Based on a one-week measurement campaign we collect around half a billion of packet size and timing observations. Processing these observations yields 2.3 million transmission bursts. To learn details on the radio resource management we develop a methodology to infer the effective physical layer sending rate. Our findings show that although Starlink throughput can vary widely over multiple time-scales, there are a small number of fundamental physical layer transmission rates. We employ Gaussian Mixture Modeling to determine 14 such fundamental transmission rates, and relate the obtained rates to previous knowledge of the Starlink OFDMA frame structure. Our empirical observations provide an excellent match for a radio resource configuration where a Starlink frame employs 1000 subcarriers and 287 symbols per frame for user traffic transmission, which for uniform 4-QAM modulation yields a base rate of 430.5 Mbps. This physical layer base rate appears to mostly be varied by multiples of 27 Mbps, in several instances likely by modifying the modulation of a subset of the symbols in multiples of 18 symbols. 

While Internet Service Providers (ISPs) have traditionally focused on marketing network throughput, it is becoming increasingly recognized that network latency also plays a significant role for the quality of experience. However, many ISPs lack the means to continuously monitor the latency of their network. In this work, we present a method to continuously monitor and aggregate network latency per subnet directly in the Linux kernel by leveraging eBPF. We deploy this solution on a middlebox in an ISP network and collect an extensive dataset of latency measurements for both the internal and external parts of the network. We find that our monitoring solution can monitor all subscriber traffic while maintaining a low overhead of only around 1% additional CPU utilization. Our analysis of the latency data reveals a wide latency tail in the last-mile access, which grows during busy periods in the evening. Furthermore, we dissect the external network latency and uncover the latency profiles for the most popular autonomous systems.

The Internet plays an important role in modern society, and its network performance impacts billions of users every day. For many network applications, network latency has a large impact on the quality of experience for the end user. Due to a lack of extensive network latency monitoring, the observability of network latency in real networks is often limited. This poses a problem for understanding network latency on the Internet today, and for assessing the impact various solutions that aim to reduce network latency have once they are deployed in the wild. This thesis addresses shortcomings with current solutions for monitoring network latency, in particular the performance of passive monitoring solutions on general-purpose commodity hardware, aiming to enable more ubiquitous latency monitoring and ultimately provide a comprehensive view of real-world network latency. We utilize the recently emerging eBPF technology to implement passive network latency monitoring inside the Linux kernel. Through experiments on a testbed, we show that our solution can monitor packets at over an order of magnitude higher rates than comparable previous solutions, allowing it to successfully monitor the latency for multi-gigabit traffic on general-purpose commodity hardware. Additionally, we demonstrate the feasibility of continuously monitoring network latency by deploying our solution inside an Internet Service Provider and monitoring the network latency for all customer traffic. Through an extensive analysis of the collected latency data, we show large differences in how network latency is distributed across different parts of the network. 

The Internet plays a vital role in modern society, and its performance affects billions of users daily. Network latency often has a significant impact on the end users' experience. However, due to limited monitoring of network latency, the observability of latency in real networks is often poor. This hinders our understanding of latency on the Internet today and makes it challenging to assess how the deployment of new networking technologies impacts latency. This thesis uses the emerging eBPF technology to improve the performance of passive network latency monitoring, aiming to enable latency monitoring on more network devices to create a more comprehensive view of latency on the Internet. By conducting controlled experiments on a testbed, we find that our solution is over an order of magnitude faster than previous solutions, making it possible to monitor multi-gigabit traffic on general-purpose commodity hardware. Furthermore, we demonstrate the feasibility of continuously monitoring latency by deploying our solution inside the network of an Internet Service Provider to monitor all their traffic. Our analysis of the latency data reveals large differences in how latency is distributed across different parts of the network.

With a growing user base and the deployment of new systems, such as 5G and Starlink, a deep understanding of the varying link throughput for wireless systems is highly important. While detailed analysis of link throughput can be done on packet traces, collecting extensive packet traces often faces storage and privacy challenges. Instead, we propose using traces of link-wide inter-packet delay (IPD) to enable highly granular link throughput characterization on a wider scale. To this end, we present an eBPF-based tool designed to capture IPDs, and evaluate the accuracy of captured IPDs with the IPD tool and tcpdump, both with and without access to hardware timestamps. While hardware provided timestamps provide accurate IPDs, we find that software based timestamps lead to IPD values which are very inaccurate, but still useful in aggregate form to characterize throughput at millisecond timescales. Furthermore, we show that concurrent packet processing incurs a significant amount of packet reordering, which necessitates the consideration of several previous packets when computing the link-wide IPD. Finally we present an example use case of IPD collection, characterizing frequent silent periods during a speedtest over a 5G link.

Network latency is a critical factor for the perceived quality of experience for many applications. With an increasing focus on interactive and real-time applications, which require reliable and low latency, the ability to continuously and efficiently monitor latency is becoming more important than ever. Always-on passive monitoring of latency can provide continuous latency metrics without injecting any traffic into the network. However, software-based monitoring tools often struggle to keep up with traffic as packet rates increase, especially on contemporary multi-Gbps interfaces. We investigate the feasibility of using eBPF to enable efficient passive network latency monitoring by implementing an evolved Passive Ping (ePPing). Our evaluation shows that ePPing delivers accurate RTT measurements and can handle over 1 Mpps, or correspondingly over 10 Gbps, on a single core, greatly improving on state-of-the-art software based solutions, such as PPing.

Although Starlink has been rolled-out for several years, there is still a lack of knowledge regarding system details and performance characteristics. To address this, we perform a network layer measurement campaign utilizing precise times-Tamping to analyze throughput variations at multiple time scales, and infer system timing details. On larger timescales we quantify the diurnal variations where the throughput varies with the time of day. On the medium timescales we establish the likely frequency allocation and beam switching period to be 15 seconds. The associated connectivity disturbances contribute to severe link underutilization for single long-lived TCP flows, which typically reach only 46% of the estimated link capacity. On the sub-millisecond timescale our network layer measurements corroborate recent physical layer investigations of the Starlink frame timing, which is confirmed to be 1.33 ms. © 2023 Owner/Author(s).

Network latency plays a crucial role for many applications and their perceived quality of experience. With an increasing focus on high network speeds and real time, interactive applications relying on reliable and low latency, the ability to effectively monitor latency is becoming more important than ever. While many available tools rely on active monitoring, this approach relies on traffic injection in the network, which can be a source of latency in itself and have a negative overall network performance impact. This paper presents evolved Passive Ping (ePPing), a tool that leverages eBPF to passively monitor latency of existing network traffic. Preliminary evaluation shows that ePPing delivers RTT reports more reliably and at a lower overhead than other state-of-the-art tools, such as PPing.