Network latency is a critical factor for the perceived quality of experience for many applications. With an increasing focus on interactive and real-time applications, which require reliable and low latency, the ability to continuously and efficiently monitor latency is becoming more important than ever. Always-on passive monitoring of latency can provide continuous latency metrics without injecting any traffic into the network. However, software-based monitoring tools often struggle to keep up with traffic as packet rates increase, especially on contemporary multi-Gbps interfaces. We investigate the feasibility of using eBPF to enable efficient passive network latency monitoring by implementing an evolved Passive Ping (ePPing). Our evaluation shows that ePPing delivers accurate RTT measurements and can handle over 1 Mpps, or correspondingly over 10 Gbps, on a single core, greatly improving on state-of-the-art software based solutions, such as PPing.

Network latency can have a large impact on the performance of networked applications and the quality of experience for the end users. By using eBPF to implement continuous passive network latency monitoring and aggregation in kernel space with epping, we provide an efficient monitoring solution that can be deployed on any existing Linux node in the network. Our solution thus allows network operators to gain a highly granular view of the network latency experienced by the traffic their network is serving without needing to buy and deploy new hardware. This information can help network operators understand what quality of service their network is currently offering, locate issues in their networks, and verify if new solutions aimed at improving network latency have the desired effect, ultimately improving the experience for the end users. In this technical report, we provide a detailed description of how epping works, cover how different features have been implemented, and discuss many of the challenges we have encountered while implementing it.

While Internet Service Providers (ISPs) have traditionally focused on marketing network throughput, it is becoming increasingly recognized that network latency also plays a significant role for the quality of experience. However, many ISPs lack the means to continuously monitor the latency of their network. In this work, we present a method to continuously monitor and aggregate network latency per subnet directly in the Linux kernel by leveraging eBPF. We deploy this solution on a middlebox in an ISP network and collect an extensive dataset of latency measurements for both the internal and external parts of the network. We find that our monitoring solution can monitor all subscriber traffic while maintaining a low overhead of only around 1% additional CPU utilization. Our analysis of the latency data reveals a wide latency tail in the last-mile access, which grows during busy periods in the evening. Furthermore, we dissect the external network latency and uncover the latency profiles for the most popular autonomous systems.

With a growing user base and the deployment of new systems, such as 5G and Starlink, a deep understanding of the varying link throughput for wireless systems is highly important. While detailed analysis of link throughput can be done on packet traces, collecting extensive packet traces often faces storage and privacy challenges. Instead, we propose using traces of link-wide inter-packet delay (IPD) to enable highly granular link throughput characterization on a wider scale. To this end, we present an eBPF-based tool designed to capture IPDs, and evaluate the accuracy of captured IPDs with the IPD tool and tcpdump, both with and without access to hardware timestamps. While hardware provided timestamps provide accurate IPDs, we find that software based timestamps lead to IPD values which are very inaccurate, but still useful in aggregate form to characterize throughput at millisecond timescales. Furthermore, we show that concurrent packet processing incurs a significant amount of packet reordering, which necessitates the consideration of several previous packets when computing the link-wide IPD. Finally we present an example use case of IPD collection, characterizing frequent silent periods during a speedtest over a 5G link.

Low Earth Orbit (LEO) satellite networks, such as Starlink, are transforming global Internet access by delivering high-speed connectivity to underserved and remote regions. Despite extensive research into Starlink’s performance, latency characteristics remain under-explored. This study presents a comprehensive analysis of one-way delay components in the Starlink network using high-frequency, high-precision measurement probes. Over a 10-day period, more than 500 million probe packets were collected and analyzed. The results reveal minor diurnal latency variation and provide means to separate out the delay components contributing to the observed one-way delay, and we sketch a delay model and provide empirical distributions. By measuring both uplink and downlink paths, the study uncovers significant differences in scheduling behavior, with uplink delays more affected by Starlink’s periodic 15-second reconfiguration cycles. The results also highlight the limitations of using too coarse measurement intervals, which can introduce aliasing effects. Our OWD data set and traffic generation tool are made available to support further research in the area.