Data centers increasingly utilize commodity servers to deploy low-latency Network Functions (NFs). However, the emergence of multi-hundred-gigabit-per-second network interface cards (NICs) has drastically increased the performance expected from commodity servers. Additionally, recently introduced systems that store packet payloads in temporary off-CPU locations (e.g., programmable switches, NICs, and RDMA servers) further increase the load on NF servers, making packet processing even more challenging. This paper demonstrates existing bottlenecks and challenges of state-of-the-art stateful packet processing frameworks and proposes a system, called FAJITA, to tackle these challenges & accelerate stateful packet processing on commodity hardware. FAJITA proposes an optimized processing pipeline for stateful network functions to minimize memory accesses and overcome the overheads of accessing shared data structures while ensuring efficient batch processing at every stage of the pipeline. Furthermore, FAJITA provides a performant architecture to deploy high-performance network functions service chains containing stateful elements with different state granularities. FAJITA improves the throughput and latency of high-speed stateful network functions by ~2.43x compared to the most performant state-of-the-art solutions, enabling commodity hardware to process up to ~178 Million 64-B packets per second (pps) using 16 cores.

This article explores opportunities to mitigate the performance impact of IOMMU on high-speed network traffic, as used in the Linux kernel. We first characterize IOTLB behavior and its effects on recent Intel Xeon Scalable & AMD EPYC processors at 200 Gbps, by analyzing the impact of different factors contributing to IOTLB misses and causing throughput drop (up to 20% compared to the no-IOMMU case in our experiments). Secondly, we discuss and analyze possible mitigations, including proposals and evaluation of a practical hugepage-aware memory allocator for the network device drivers to employ hugepage IOTLB entries in the Linux kernel. Our evaluation shows that using hugepage-backed buffers can completely recover the throughput drop introduced by IOMMU. Moreover, we formulate a set of guidelines that enable network developers to tune their systems to avoid the “IOTLB wall”, i.e., the point where excessive IOTLB misses cause throughput drop. Our takeaways signify the importance of having a call to arms to rethink Linux-based I/O management at higher data rates.

By virtue of the recent technological developments in cloud computing, more applications are deployed in the cloud. Among these modern cloud-based applications, many societal applications require bounded and predictable low-latency responses. However, the current cloud infrastructure is unsuitable for these applications since it cannot satisfy these requirements due to many limitations in both hardware and software.

This doctoral dissertation describes our attempts to reduce the latency of Internet services by carefully studying the multi-hundred-gigabit-per-second commodity hardware, optimizing it, and improving its performance. The main focus is to improve the performance of packet processing done by the network functions deployed on commodity hardware, known as network functions virtualization (NFV), which is one of the significant sources of latency for Internet services.

The first contribution of this dissertation takes a step toward optimizing the cache performance of time-critical NFV service chains. By doing so, we reduce the tail latencies of such systems running at 100 Gbps. This is an important achievement as it increases the probability of realizing bounded and predictable latency for Internet services.

The second contribution of this dissertation performs whole-stack optimizations on software-based network functions deployed on top of modular packet processing frameworks to further enhance the effectiveness of cache memories. We build a system to efficiently handle metadata and produce a customized binary of NFVservice chains. Our system improves both throughput & latency of per-core hundred-gigabit-per-second packet processing on commodity hardware.

The third contribution of this dissertation studies the efficiency of I/O security solutions provided by commodity hardware at multi-hundred-gigabit-per-second rates. We characterize the performance of IOMMU & IOTLB (i.e., I/O virtual address translation cache) at 200 Gbps and explore the possible opportunities to mitigate its performance overheads in the Linux kernel.

Tack vare den senaste tekniska utvecklingen inom molntjänster används allt fler tillämpningar i molnet. Bland dessa moderna molnbaserade tillämpningar kräver många samhällsorienterade tillämpningar svarstider med låg latens, som är förutsägbara och ligger inom givna gränser. Den nuvarande molninfrastrukturen är dock otillräcklig för sådana tillämpningar eftersom den inte kan uppfylla dessa krav på grund av olika begränsningar i både hårdvara och mjukvara.

I denna doktorsavhandling beskrivs våra försök att minska latenstiden för Internettjänster genom att noggrant studera tillänglig hårdvara med stöd för flera hundra gigabit per sekund, optimera denna och förbättra dess prestanda. Huvudfokus ligger på att förbättra prestandan för den paketbearbetning som utförs av nätverksfunktioner som installeras på allmänt tillgänglig hårdvara, så kallad nätverksfunktionsvirtualisering (NFV), som är en av de betydande källorna till latens för Internettjänster. 

Det första bidraget i den här avhandlingen tar ett steg mot att optimera cache-prestanda för tidskritiska kedjor av NFV-tjänster. Genom att göra detta minskar vi de långa latenstiderna för sådana system som körs vid 100 Gbps. Detta är ett viktigt resultat eftersom det ökar sannolikheten för att uppnå en begränsad och förutsägbar fördröjning hos internettjänster. 

Det andra bidraget i den här avhandlingen är optimeringar av hela stacken av mjukvarubaserade nätverksfunktioner som används ovanpå modulära ramverk för paketbearbetning för att ytterligare förbättra effektiviteten hos cacheminnen. Vi bygger ett system för att effektivt hantera metadata och producera anpassade binärversioner av NFV-tjänstekedjor. Vårt system förbättrar både genomströmning och latens för tillgänglig hårdvara där varje CPU-kärna har kapacitet för paketbearbetning i storleksordningen 100 Gbps. .

I det tredje bidraget i denna avhandling studeras effektiviteten hos I/O-säkerhetslösningar som tillhandahålls av allmänt tillgänglig hårdvara i hastigheter på flera hundra gigabit per sekund. Vi karakteriserar prestandan hos IOMMU and IOTLB (dvs. “I/O memory management unit” och “I/O virtual address translation cache”) vid 200 Gbps och undersöker möjligheterna att minska dess prestanda-overhead i kärnan av operativsystemet Linux.

Data centers increasingly deploy commodity servers with high-speed network interfaces to enable low-latency communication. However, achieving low latency at high data rates crucially depends on how the incoming traffic interacts with the system's caches. When packets that need to be processed in the same way are consecutive, i.e., exhibit high temporal and spatial locality, caches deliver great benefits.

In this paper, we systematically study the impact of temporal and spatial traffic locality on the performance of commodity servers equipped with high-speed network interfaces. Our results show that (i) the performance of a variety of widely deployed applications degrades substantially with even the slightest lack of traffic locality, and (ii) a traffic trace from our organization reveals poor traffic locality as networking protocols, drivers, and the underlying switching/routing fabric spread packets out in time (reducing locality). To address these issues, we built Reframer, a software solution that deliberately delays packets and reorders them to increase traffic locality. Despite introducing μs-scale delays of some packets, we show that Reframer increases the throughput of a network service chain by up to 84% and reduces the flow completion time of a web server by 11% while improving its throughput by 20%.

We present PacketMill, a system for optimizing software packet processing, which (i) introduces a new model to efficiently manage packet metadata and (ii) employs code-optimization techniques to better utilize commodity hardware. PacketMill grinds the whole packet processing stack, from the high-level network function configuration file to the low-level userspace network (specifically DPDK) drivers, to mitigate inefficiencies and produce a customized binary for a given network function. Our evaluation results show that PacketMill increases throughput (up to 36.4 Gbps -- 70%) & reduces latency (up to 101 us -- 28%) and enables nontrivial packet processing (e.g., router) at ~100 Gbps, when new packets arrive >10× faster than main memory access times, while using only one processing core.

A method and device for controlling memory handling in a processing system comprising a cache shared between a plurality of processing units, wherein the cache comprises a plurality of cache portions. The method comprises obtaining first information pertaining to an allocation of a first memory portion of a memory to a first application, an allocation of a first processing unit of the plurality of processing units to the first application, and an association between a first cache portion of the plurality of cache portions and the first processing unit. The method further comprises reconfiguring a mapping configuration based on the obtained first information, and controlling a providing of first data associated with the first application to the first cache portion from the first memory portion using the reconfigured mapping configuration.

Digitalization across society is expected to produce a massive amount of data, leading to the introduction of faster network interconnects. In addition, many Internet services require high throughput and low latency. However, having only faster links does not guarantee throughput or low latency. Therefore, it is essential to perform holistic system optimization to fully take advantage of the faster links to provide high-performance services. Intel Data Direct I/O (DDIO) is a recent technology that was introduced to facilitate the deployment of high-performance services based on fast interconnects. We evaluated the effectiveness of DDIO for multi-hundred-gigabit networks. This paper briefly discusses our findings on DDIO, which show the necessity of optimizing/adapting it to address the challenges of multi-hundred-gigabit-per-second links.

Memory access is the major bottleneck in realizing multi-hundred-gigabit networks with commodity hardware, hence it is essential to make good use of cache memory that is a faster, but smaller memory closer to the processor. Our goal is to study the impact of cache management on the performance of I/O intensive applications. Specifically, this paper looks at one of the bottlenecks in packet processing, i.e., direct cache access (DCA). We systematically studied the current implementation of DCA in Intel processors, particularly Data Direct I/O technology (DDIO), which directly transfers data between I/O devices and the processor's cache. Our empirical study enables system designers/developers to optimize DDIO-enabled systems for I/O intensive applications. We demonstrate that optimizing DDIO could reduce the latency of I/O intensive network functions running at 100 Gbps by up to ~30%. Moreover, we show that DDIO causes a 30% increase in tail latencies when processing packets at 200 Gbps, hence it is crucial to selectively inject data into the cache or to explicitly bypass it.

Embodiments herein relates e.g., to a method performed by a first entity, for handling memory operations of an application in a computer environment, is provided. The first entity obtains position data associated with data of the application being fragmented into a number of positions in a physical memory. The position data indicates one or more positions of the number of positions in the physical memory. The first entity then provides, to a second entity, one or more indications of the one or more positions indicated by the position data for prefetching data from the second entity, using the one or more indications.

In modern (Intel) processors, Last Level Cache (LLC) is divided into multiple slices and an undocumented hashing algorithm (aka Complex Addressing) maps different parts of memory address space among these slices to increase the effective memory bandwidth. After a careful study of Intel’s Complex Addressing, we introduce a slice-aware memory management scheme, wherein frequently used data can be accessed faster via the LLC. Using our proposed scheme, we show that a key-value store can potentially improve its average performance ∼12.2% and ∼11.4% for 100% & 95% GET workloads, respectively. Furthermore, we propose CacheDirector, a network I/O solution which extends Direct Data I/O (DDIO) and places the packet’s header in the slice of the LLC that is closest to the relevant processing core. We implemented CacheDirector as an extension to DPDK and evaluated our proposed solution for latency-critical applications in Network Function Virtualization (NFV) systems. Evaluation results show that CacheDirector makes packet processing faster by reducing tail latencies (90-99th percentiles) by up to 119 µs (∼21.5%) for optimized NFV service chains that are running at 100 Gbps. Finally, we analyze the effectiveness of slice-aware memory management to realize cache isolation