In Machine Learning (ML) and Deep Learning (DL) research, ablation studies are typically performed to provide insights into the individual contribution of different building blocks and components of an ML/DL system (e.g., a deep neural network), as well as to justify that certain additions or modifications to an existing ML/DL system can result in the proposed improved performance. Although dedicated frameworks for performing ablation studies have been introduced in recent years, conducting such experiments is still associated with requiring tedious, redundant work, typically involving maintaining redundant and nearly identical versions of code that correspond to different ablation trials. Inspired by the recent promising performance of Large Language Models (LLMs) in the generation and analysis of ML/DL code, in this paper we discuss the potential of LLMs as facilitators of ablation study experiments for scientific research projects that involve or deal with ML and DL models. We first discuss the different ways in which LLMs can be utilized for ablation studies and then present the prototype of a tool called AblationMage, that leverages LLMs to semi-automate the overall process of conducting ablation study experiments. We showcase the usability of AblationMage as a tool through three experiments, including one in which we reproduce the ablation studies from a recently published applied DL paper.

Data centers increasingly utilize commodity servers to deploy low-latency Network Functions (NFs). However, the emergence of multi-hundred-gigabit-per-second network interface cards (NICs) has drastically increased the performance expected from commodity servers. Additionally, recently introduced systems that store packet payloads in temporary off-CPU locations (e.g., programmable switches, NICs, and RDMA servers) further increase the load on NF servers, making packet processing even more challenging. This paper demonstrates existing bottlenecks and challenges of state-of-the-art stateful packet processing frameworks and proposes a system, called FAJITA, to tackle these challenges & accelerate stateful packet processing on commodity hardware. FAJITA proposes an optimized processing pipeline for stateful network functions to minimize memory accesses and overcome the overheads of accessing shared data structures while ensuring efficient batch processing at every stage of the pipeline. Furthermore, FAJITA provides a performant architecture to deploy high-performance network functions service chains containing stateful elements with different state granularities. FAJITA improves the throughput and latency of high-speed stateful network functions by ~2.43x compared to the most performant state-of-the-art solutions, enabling commodity hardware to process up to ~178 Million 64-B packets per second (pps) using 16 cores.

Data centers increasingly deploy commodity servers with high-speed network interface cards to enable network services with low latency and high throughput capabilities. However, existing packet processing systems still suffer from high I/O and memory access overheads, especially when deploying stateful network functions, resulting in degraded performance.

This doctoral dissertation describes our attempts to improve the performance of stateful network functions deployed on commodity servers by carefully studying traffic properties in heterogeneous data centers, low-level analysis of existing systems' bottlenecks, and finally proposing solutions to optimize stateful packet processing and alleviating systems' memory and I/O overheads. 

The first contribution of this dissertation studies the impact of temporal and spatial traffic locality on the performance of commodity servers. Accordingly, we propose Reframer as a system that deliberately delays packets and reorders them to increase traffic locality. By deploying Reframer in front of a network function chain the system achieves up to 84% improvement in throughput and reduces the flow completion time of a web server by 11%.

The second contribution of this dissertation focuses on optimizing packet processing frameworks when deploying a chain of stateful network functions with various flow definitions. We identify three commonly practiced principles that are essential to achieve high performance. We propose FAJITA as a cache-friendly stateful packet processing framework that improves the performance of stateful network function service chains compared to existing state-of-the-art solutions, by at least 2.4× & 1.5× when using shared-nothing & shared architectures, respectively.

The third contribution of this dissertation takes one step further in optimizing packet processing frameworks by automatically configuring RSS before deploying a stateful network function chain on a commodity server. We propose FlowMage, a system that leverages Large Language Models (LLMs) to perform code analysis and extract essential information from stateful network functions. FlowMage uses this data to find an efficient configuration of a network function chain while preserving the semantics of the NF chain. FlowMage achieves a significant performance improvement (up to 11×) in comparison to the default configuration of the system.

Datacenter använder i allt högre grad vanliga servrar med högpresterande nätverkskort för att möjliggöra nätverkstjänster med låg latens och hög genomströmningskapacitet. Dock lider befintliga paketbehandlingssystem fortfarande av höga I/O och minnesåtkomstkostnader, särskilt vid implementering av tillståndsberoende nätverksfunktioner, vilket resulterar i försämrad prestanda.

Denna doktorsavhandling beskriver våra försök att förbättra prestandan för tillståndsberoende nätverksfunktioner implementerade på vanliga servrar genom att noggrant studera trafikegenskaper i heterogena datacenter, göra låg-nivå analyser av befintliga systemflaskhalsar och slutligen föreslå lösningar för att optimera tillståndsberoende paketbehandling och minska systemens minnes- och I/O-kostnader.

Den första bidraget i denna avhandling undersöker påverkan av temporal och spatial trafiklokalitet på prestandan hos vanliga servrar. Därmed föreslår vi Reframer som ett system som medvetet fördröjer paket och omordnar dem för att öka trafikens lokalitet. Genom att implementera Reframer framför en nätverksfunktionskedja uppnår systemet upp till 84% förbättring i genomströmning och minskar flödesavslutningstiden för en webbserver med 11%.

Det andra bidraget i denna avhandling fokuserar på att optimera paketbehandlingsramverk vid implementering av en kedja av tillståndsberoende nätverksfunktioner med olika flödesdefinitioner. Vi identifierar tre vanligt förekommande principer som är avgörande för att uppnå hög prestanda. Vi föreslår FAJITA som ett cache-vänligt ramverk för tillståndsberoende paketbehandling som förbättrar prestandan för tjänstekedjor med tillståndsberoende nätverksfunktioner jämfört med befintliga toppmoderna lösningar, med minst 2,4× & 1,5× när man använder respektive icke-delad och delad arkitektur.

Det tredje bidraget i denna avhandling tar ett steg längre i att optimera paketbehandlingsramverk genom att automatiskt konfigurera RSS innan man implementerar en kedja av tillståndsberoende nätverksfunktioner på en vanlig server. Vi föreslår FlowMage, ett system som använder stora språkmodeller (LLMs) för att utföra kodanalys och extrahera väsentlig information från tillståndsberoende nätverksfunktioner. FlowMage använder denna data för att hitta en effektiv konfiguration av en nätverksfunktionskedja samtidigt som den bevarar NF-kedjans semantik. FlowMage uppnår en betydande prestandaförbättring (upp till 11×) jämfört med systemets standardkonfiguration.

High-speed ASIC switches hold great promise for offloading complex packet processing pipelines directly in the highspeed data-plane. Yet, a large variety of today’s packet processing pipelines, including stateful network functions andpacket schedulers, require storing some (or all the) packetsfor short amount of times in a programmatic manner. Such aprogrammable buffer feature is missing on today’s high-speedASIC switches.In this work, we present RIBOSOME, a system that extendsprogrammable switches with external memory (to store packets) and external general-purpose packet processing devicessuch as CPUs or FPGAs (to perform stateful operations). Astoday’s packet processing devices are bottlenecked by theirnetwork interface speeds, RIBOSOME carefully transmits onlythe relevant bits to these devices. RIBOSOME leverages sparebandwidth from any directly connected servers to store theincoming payloads through RDMA. Our evaluation showsthat RIBOSOME can process 300G of traffic through a stateful packet processing pipeline (e.g., firewall, load balancer,packet scheduler) by running the pipeline logic on a singleserver equipped with one 100G interface.

Data centers increasingly deploy commodity servers with high-speed network interfaces to enable low-latency communication. However, achieving low latency at high data rates crucially depends on how the incoming traffic interacts with the system's caches. When packets that need to be processed in the same way are consecutive, i.e., exhibit high temporal and spatial locality, caches deliver great benefits.

In this paper, we systematically study the impact of temporal and spatial traffic locality on the performance of commodity servers equipped with high-speed network interfaces. Our results show that (i) the performance of a variety of widely deployed applications degrades substantially with even the slightest lack of traffic locality, and (ii) a traffic trace from our organization reveals poor traffic locality as networking protocols, drivers, and the underlying switching/routing fabric spread packets out in time (reducing locality). To address these issues, we built Reframer, a software solution that deliberately delays packets and reorders them to increase traffic locality. Despite introducing μs-scale delays of some packets, we show that Reframer increases the throughput of a network service chain by up to 84% and reduces the flow completion time of a web server by 11% while improving its throughput by 20%.

A commonly held belief is that traffic engineering and routing changes are infrequent. However, based on our measurements over a number of years of traffic between data centers in one of the largest cloud provider's networks, we found that it is common for flows to change paths at ten-second intervals or even faster. These frequent path and, consequently, latency variations can negatively impact the performance of cloud applications, specifically, latency-sensitive and geo-distributed applications.

Our recent measurements and analysis focused on observing path changes and latency variations between different Amazon aws regions. To this end, we devised a path change detector that we validated using both ad hoc experiments and feedback from cloud networking experts. The results provide three main insights: (1) Traffic Engineering (TE) frequently moves (TCP and UDP) flows among network paths of different latency, (2) Flows experience unfair performance, where a subset of flows between two machines can suffer large latency penalties (up to 32% at the 95th percentile) or excessive number of latency changes, and (3) Tenants may have incentives to selfishly move traffic to low latency classes (to boost the performance of their applications). We showcase this third insight with an example using rsync synchronization.

To the best of our knowledge, this is the first paper to reveal the high frequency of TE activity within a large cloud provider's network. Based on these observations, we expect our paper to spur discussions and future research on how cloud providers and their tenants can ultimately reconcile their independent and possibly conflicting objectives. Our data is publicly available for reproducibility and further analysis at http://goo.gl/25BKte.