Data centers increasingly deploy commodity servers with high-speed network interfaces to enable low-latency communication. However, achieving low latency at high data rates crucially depends on how the incoming traffic interacts with the system's caches. When packets that need to be processed in the same way are consecutive, i.e., exhibit high temporal and spatial locality, caches deliver great benefits.

In this paper, we systematically study the impact of temporal and spatial traffic locality on the performance of commodity servers equipped with high-speed network interfaces. Our results show that (i) the performance of a variety of widely deployed applications degrades substantially with even the slightest lack of traffic locality, and (ii) a traffic trace from our organization reveals poor traffic locality as networking protocols, drivers, and the underlying switching/routing fabric spread packets out in time (reducing locality). To address these issues, we built Reframer, a software solution that deliberately delays packets and reorders them to increase traffic locality. Despite introducing μs-scale delays of some packets, we show that Reframer increases the throughput of a network service chain by up to 84% and reduces the flow completion time of a web server by 11% while improving its throughput by 20%.

We present PacketMill, a system for optimizing software packet processing, which (i) introduces a new model to efficiently manage packet metadata and (ii) employs code-optimization techniques to better utilize commodity hardware. PacketMill grinds the whole packet processing stack, from the high-level network function configuration file to the low-level userspace network (specifically DPDK) drivers, to mitigate inefficiencies and produce a customized binary for a given network function. Our evaluation results show that PacketMill increases throughput (up to 36.4 Gbps -- 70%) & reduces latency (up to 101 us -- 28%) and enables nontrivial packet processing (e.g., router) at ~100 Gbps, when new packets arrive >10× faster than main memory access times, while using only one processing core.

The cloud is evolving due to additional demands introduced by new technological advancements and the wide movement toward digitalization. Moreover, next-generation Data Centers (DCs) and clouds are expected (and need) to become cheaper, more efficient, and capable of offering more predictable services. Aligned with this, this thesis examines the concept of Software-Defined “Hardware” Infrastructure (SDHI) based on hardware resource disaggregation as one possible way of realizing next-generation DCs. This thesis starts with an overview of the functional architecture of a cloud based on SDHI. Following this, a series of use-cases and deployment scenarios enabled by SDHI are discussed along with an exploration of the role of each functional block of SDHI’s architecture, i.e., cloud infrastructure, cloud platforms, cloud execution environments, and applications.

This thesis proposes a framework to evaluate the impact of SDHI on the techno-economic efficiency of DCs, explicitly focusing on application profiling, hardware dimensioning, and Total Cost of Ownership (TCO). It then shows that combining resource disaggregation and software-defined capabilities makes DCs less expensive and easier to expand; hence, they can rapidly follow the expected exponential demand growth. Additionally, this thesis elaborates the technologies underlying SDHI, its challenges, and its potential future directions.

It is advocated that achieving and maintaining a high level of memory performance is crucial for realizing SDHI & disaggregated DC. Nevertheless, a memory management and Input/Output (I/O) data management scheme suitable for SDHI is proposed and its advantages are shown. This work focuses on the management of Last Level Cache (LLC) in currently available Intel processors, takes advantage of LLC’s Non-Uniform Cache Architectures (NUCA), and investigates how better utilization of LLC can provide higher performance, more predictable response time, and improved isolation between threads. Additionally, this thesis scrutinizes the impact of cache management, specifically Direct Cache Access (DCA), on the performance of I/O intensive applications. The results of an empirical study shows that the proposed memory management scheme enables system designers and developers to optimize systems for I/O intensive applications and highlights some potential changes expected for I/O management in future DC systems.

Nya tekniska framsteg och krav samt den breda digitaliseringen gör att molnet utvecklas i allt snabbare takt. Det gör att nästa generations datacenter förväntas vara billigare, mer effektiva, och dessutom kunna erbjuda robustare tjänster. I linje med detta undersöker vi mjukvarudefinierade hårdvaru-infrastrukturer (MDHI) baserade på disaggregering av hårdvaruresurser som en potentiell teknologi för att realisera nästa generationens datacenter. Vi inleder med en översikt av den funktionella arkitekturen för ett moln baserat på MDHI. Vi analyserar sedan ett antal användningsfall och utplaceringsscenarion som möjliggörs av MDHI, och undersöker den roll som varje funktionellt block i en MDHI arkitektur kan spela: Till exempel molninfrastruktur, molnplattform, molnexekveringsmiljö och tillämpningar.</p><p>Denna avhandling föreslår sedan ett ramverk för att utvärdera den effekt som MDHI kan ha på den tekno-ekonomiska effektiviteten i ett datacenter, med särskilt fokus på profilering av tillämpningar, hårdvaru-dimensionering samt total ägandekostnad. Våra studier visar att datacenter kan bli billigare och lättare att expandera om man använder sig av resurs-disaggregering och mjukvarudefiniering; därmed kan de också snabbare följa den exponentiellt ökande efterfrågan. Vi undersöker också teknologierna bakom MDHI, dess utmaningar och möjliga framtida utvecklingsriktningar.</p><p>Det förespråkas inse MDHI, det är avgörande för att uppnå och bibehålla en hög minnesprestanda. Ändå föreslog vi minneshantering och I/O -datahanteringsschema som är lämpligt för MDHI och visade deras fördelar. Detta reporten fokuserar på hanteringen av sista-nivå-cacheminne i för närvarande tillgängliga Intel -processorer, utnyttjarcacheminne nonuniform architecture och undersöker hur bättre utnyttjandeav sista-nivåcacheminne kan ge högre prestanda, mer förutsägbar svarstidoch förbättrad isolering mellan trådarna. Dessutom granskar denna avhandling effekterna av cachehantering, särskilt direkt cacheadkomst, på prestanda för I/O - intensiva applikationer. Resultaten av en empirisk studie visar att det föreslagna minneshanteringsschemat gör det möjligt för systemdesigners och utvecklare att optimera system för I/O intensiva applikationer och vi belyser några potentiella förändringar som förväntas för I/O hantering i framtida datacenter.

A method and device for controlling memory handling in a processing system comprising a cache shared between a plurality of processing units, wherein the cache comprises a plurality of cache portions. The method comprises obtaining first information pertaining to an allocation of a first memory portion of a memory to a first application, an allocation of a first processing unit of the plurality of processing units to the first application, and an association between a first cache portion of the plurality of cache portions and the first processing unit. The method further comprises reconfiguring a mapping configuration based on the obtained first information, and controlling a providing of first data associated with the first application to the first cache portion from the first memory portion using the reconfigured mapping configuration.

Digitalization across society is expected to produce a massive amount of data, leading to the introduction of faster network interconnects. In addition, many Internet services require high throughput and low latency. However, having only faster links does not guarantee throughput or low latency. Therefore, it is essential to perform holistic system optimization to fully take advantage of the faster links to provide high-performance services. Intel Data Direct I/O (DDIO) is a recent technology that was introduced to facilitate the deployment of high-performance services based on fast interconnects. We evaluated the effectiveness of DDIO for multi-hundred-gigabit networks. This paper briefly discusses our findings on DDIO, which show the necessity of optimizing/adapting it to address the challenges of multi-hundred-gigabit-per-second links.

Memory access is the major bottleneck in realizing multi-hundred-gigabit networks with commodity hardware, hence it is essential to make good use of cache memory that is a faster, but smaller memory closer to the processor. Our goal is to study the impact of cache management on the performance of I/O intensive applications. Specifically, this paper looks at one of the bottlenecks in packet processing, i.e., direct cache access (DCA). We systematically studied the current implementation of DCA in Intel processors, particularly Data Direct I/O technology (DDIO), which directly transfers data between I/O devices and the processor's cache. Our empirical study enables system designers/developers to optimize DDIO-enabled systems for I/O intensive applications. We demonstrate that optimizing DDIO could reduce the latency of I/O intensive network functions running at 100 Gbps by up to ~30%. Moreover, we show that DDIO causes a 30% increase in tail latencies when processing packets at 200 Gbps, hence it is crucial to selectively inject data into the cache or to explicitly bypass it.

Embodiments herein relates e.g., to a method performed by a first entity, for handling memory operations of an application in a computer environment, is provided. The first entity obtains position data associated with data of the application being fragmented into a number of positions in a physical memory. The position data indicates one or more positions of the number of positions in the physical memory. The first entity then provides, to a second entity, one or more indications of the one or more positions indicated by the position data for prefetching data from the second entity, using the one or more indications.

In modern (Intel) processors, Last Level Cache (LLC) is divided into multiple slices and an undocumented hashing algorithm (aka Complex Addressing) maps different parts of memory address space among these slices to increase the effective memory bandwidth. After a careful study of Intel’s Complex Addressing, we introduce a slice-aware memory management scheme, wherein frequently used data can be accessed faster via the LLC. Using our proposed scheme, we show that a key-value store can potentially improve its average performance ∼12.2% and ∼11.4% for 100% & 95% GET workloads, respectively. Furthermore, we propose CacheDirector, a network I/O solution which extends Direct Data I/O (DDIO) and places the packet’s header in the slice of the LLC that is closest to the relevant processing core. We implemented CacheDirector as an extension to DPDK and evaluated our proposed solution for latency-critical applications in Network Function Virtualization (NFV) systems. Evaluation results show that CacheDirector makes packet processing faster by reducing tail latencies (90-99th percentiles) by up to 119 µs (∼21.5%) for optimized NFV service chains that are running at 100 Gbps. Finally, we analyze the effectiveness of slice-aware memory management to realize cache isolation

A method in a multi-core processing system which comprises a processor comprising at least a first and a second processing unit, a cache, common to the first and the second processing unit, comprising a first cache portion associated with the first processing unit and a second cache portion associated with the second processing unit, a memory, comprising a first memory portion associated with the first cache portion and a second memory portion associated with the second cache portion. The method comprises detecting that a data access criteria of the second memory portion is fulfilled, establishing that first data stored in the second memory portion is related to a first application running on the first processing unit, allocating at least a part of the first memory portion to the first application based on cache information, and migrating the first data to the part of first memory portion.

This paper provides an overview of software-defined "hardware" infrastructures (SDHI). SDHI builds upon the concept of hardware (HW) resource disaggregation. HW resource disaggregation breaks today's physical server-oriented model where the use of a physical resource (e.g., processor or memory) is constrained to a physical server's chassis. SDHI extends the definition of of software-defined infrastructures (SDI) and brings greater modularity, flexibility, and extensibility to cloud infrastructures, thus allowing cloud operators to employ resources more efficiently and allowing applications not to be bounded by the physical infrastructure's layout. This paper aims to be an initial introduction to SDHI and its associated technological advancements. This paper starts with an overview of the cloud domain and puts into perspective some of the most prominent efforts in the area. Then, it presents a set of differentiating use-cases that SDHI enables. Next, we state the fundamentals behind SDI and SDHI, and elaborate why SDHI is of great interest today. Moreover, it provides an overview of the functional architecture of a cloud built on SDHI, exploring how the impact of this transformation goes far beyond the cloud infrastructure level in its impact on platforms, execution environments, and applications. Finally, an in-depth assessment is made of the technologies behind SDHI, the impact of these technologies, and the associated challenges and potential future directions of SDHI.