We investigate a binary classification problem in an edge intelligence system where false negatives are more costly than false positives. The system features a compact, locally deployed model, supplemented by a larger, remote model that is accessible via the network, albeit at an offloading cost. For each sample, our system first uses the locally deployed model for inference. Based on the output of the local model, the sample may be offloaded to the remote model. This work aims to understand the fundamental trade-off between classification accuracy and the offloading costs within such a hierarchical inference (HI) system. To optimise this system, we propose an online learning framework that continuously adapts a pair of thresholds on the local model’s confidence scores. These thresholds determine the prediction of the local model and whether a sample is classified locally or offloaded to the remote model. We present a closed-form solution for the setting where the local model is calibrated. For the more general case of uncalibrated models, we introduce H2T2, an online two-threshold hierarchical inference policy, and prove it achieves sublinear regret. H2T2 is model-agnostic, requires no training, and learns during the inference phase using limited feedback. Simulations on real-world datasets show that H2T2 consistently outperforms naive and single-threshold HI policies, sometimes even surpassing single-threshold offline optima. The policy also demonstrates robustness to distribution shifts and adapts effectively to mismatched classifiers.

We investigate a binary classification problem in an edge intelligence system where false negatives are more costly than false positives. The system features a compact, locally deployed model, supplemented by a larger, remote model that is accessible via the network, albeit at an offloading cost. For each sample, our system first uses the locally deployed model for inference. Based on the output of the local model, the sample may be offloaded to the remote model. This work aims to understand the fundamental trade-off between classification accuracy and the offloading costs within such a hierarchical inference (HI) system. To optimise this system, we propose an online learning framework that continuously adapts a pair of thresholds on the local model's confidence scores. These thresholds determine the prediction of the local model and whether a sample is classified locally or offloaded to the remote model. We present a closed-form solution for the setting where the local model is calibrated. For the more general case of uncalibrated models, we introduce H2T2, an online two-threshold hierarchical inference policy, and prove it achieves sublinear regret. H2T2 is model-agnostic, requires no training, and learns during the inference phase using limited feedback. Simulations on real-world datasets show that H2T2 consistently outperforms naive and single-threshold HI policies, sometimes even surpassing single-threshold offline optima. The policy also demonstrates robustness to distribution shifts and adapts effectively to mismatched classifiers.

The growing demand for stringent quality of service (QoS) guarantees in 5G networks requires accurate characterisation of delay performance, often measured using Delay Violation Probability (DVP) for a given target delay. Widely used retransmission schemes like Automatic Repeat reQuest (ARQ) and Hybrid ARQ (HARQ) improve QoS through effective feedback, incremental redundancy (IR), and parallel retransmission processes. However, existing works to quantify the DVP under these retransmission schemes overlook practical aspects such as decoding complexity, feedback delays, and the resulting need for multiple parallel ARQ/HARQ processes that enable packet transmissions without waiting for previous feedback, thus exploiting valuable transmission opportunities. This work proposes a comprehensive multi-server delay model for ARQ/HARQ that incorporates these aspects. Using a finite blocklength error model, we derive closed-form expressions and algorithms for accurate DVP evaluation under realistic 5G configurations aligned with 3GPP standards. Our numerical evaluations demonstrate notable improvements in DVP accuracy over the state-of-the-art, highlight the impact of parameter tuning and resource allocation, and reveal how DVP affects system throughput. 

Despite years of research and the introduction of advanced features in 5G systems, no commercially available 5G device has yet met the stringent latency and reliability requirements defined for ultra-reliable and low-latency communication (URLLC). This raises fundamental questions about whether and how URLLC can be practically implemented under current technological constraints. To answer this question, in this work, we explore the combination of several features defined by the 3GPP standardization, including link adaptation, retransmission schemes, and scheduling algorithms. We evaluate the performance of these techniques to investigate the extent to which the requirements of URLLC can be achieved by using the NS-3 5G LENA simulator. Our results demonstrate that the shortest latency at which a reliability level of 99.999% can be achieved is 4 milliseconds, provided that the wireless channel is of exceptionally high quality. This level of performance is enabled by combining a retransmission scheme with an effective scheduling algorithm. However, in the presence of poor channel quality, it becomes necessary to relax the reliability constraint and/or provide more resources to the transmission to maintain system feasibility.

The ongoing proliferation of intelligent systems, driven by artificial intelligence (AI) and 6G, is leading to a surge in closed-loop inference tasks performed on distributed compute nodes.These systems operate under strict latency and energy constraints, extending the challenge beyond achieving high accuracy to enabling timely and energy-efficient inference.This thesis examines how distributed inference can be optimised through two key decisions: when to sample the environment and when to offload computation to a more accurate remote model.These decisions are guided by the semantics of the underlying environment and its associated costs.The semantics are kept abstract, and pre-trained inference models are employed, ensuring a platform-independent formulation adaptable to the rapid evolution of distributed intelligence and wireless technologies.

Regarding sampling, we studied the trade-off between sampling cost and detection delay in event-detection systems without sufficient local inference capabilities. The problem was posed as an optimisation over sampling instants under a stochastic event sequence and analysed at different levels of modelling complexity, ranging from periodic to aperiodic sampling. Closed-form, algorithmic, and approximate solutions were developed, with some results of independent mathematical interest.Simulations in realistic settings showed marked gains in efficiency over systems that neglect event semantics. In particular, aperiodic sampling achieved a stable improvement of ~10% over optimised periodic policies across parameter variations.

Regarding offloading, we introduced a novel Hierarchical Inference (HI) framework, which makes sequential offload decisions between a low-latency, energy-efficient local model and a high-accuracy remote model using locally available confidence measures. We proposed HI algorithms based on thresholds and ambiguity regions learned online by suitably extending the Prediction with Expert Advice (PEA) approaches to continuous expert spaces and partial feedback. HI algorithms minimise the expected cost across inference rounds, combining offloading and misclassification costs, and are shown to achieve a uniformly sublinear regret of O(T^2/3).The proposed algorithms are agnostic to model architecture and communication systems, do not alter model training, and support model updates during operation. Benchmarks on standard classification tasks using the softmax output as a confidence measure showed that HI adaptively distributes inference based on offloading costs, achieving results close to the offline optimum. HI is shown to add resilience to distribution changes and model mismatches, especially when asymmetric misclassification costs are present.

In summary, this thesis presents efficient approaches for sampling and offloading of inference tasks, where various performance metrics are combined into a single cost structure. The work extends beyond conventional inference problems to areas with similar trade-offs, advancing toward efficient distributed intelligence that infers at the right time and in the right place. Future work includes conceptual extensions like joint sampling-offloading design, and integration with collaborative model-training architectures.

Den pågående spridningen av intelligenta system, drivna av artificiell intelligens (AI) och 6G, leder till en ökning av återkopplade inferensuppgifter som utförs på distribuerade beräkningsnoder. Dessa system verkar under strikta krav på latens och energiförbrukning, vilket gör att utmaningen inte enbart handlar om att uppnå hög noggrannhet utan också om att möjliggöra snabb och energieffektiv inferens. Denna avhandling undersöker hur distribuerad inferens kan optimeras genom två centrala beslut: när miljön ska samplas och när beräkningar ska avlastas till en mer exakt, fjärrbelägen modell. Dessa beslut styrs av miljöns semantiska egenskaper och de kostnader som är förknippade med dessa. Semantiken hålls på en abstrakt nivå, och förtränade inferensmodeller används, vilket möjliggör en plattformsoberoende formulering som är anpassningsbar till den snabba utvecklingen inom distribuerad intelligens och trådlös kommunikation.

Angående sampling studerades avvägningen mellan samplingskostnad och detektionsfördröjning i händelsedetekteringssystem som saknar tillräcklig lokal inferenskapacitet. Ett optimeringsproblem över samplingstidpunkter formuleras för stokastiska händelser och analyserades på olika nivåer av modelleringskomplexitet, från periodisk till aperiodisk sampling. Slutna, algoritmiska, och approximativa lösningar utvecklades, varav vissa resultat även är av allmänt matematiskt intresse. Simuleringar i realistiska system visade tydliga effektivitetsvinster jämfört med system som bortser från händelsernas semantik. Särskilt aperiodisk sampling uppnådde en stabil förbättring på cirka 10% jämfört med periodiska strategier över olika systemparametrar.

Angående avlastning introducerades ett nytt ramverk för hierarkisk inferens (HI), som fattar sekventiella avlastningsbeslut mellan en lokal modell med låg fördröjning och energiförbrukning, och en fjärrmodell med högre noggrannhet, baserat på lokala konfidensmått. Vi föreslog HI-algoritmer baserade på tröskelvärden och ambiguitetsregioner som lärs in online genom att utvidga metoder för expertbaserad prediktion (Prediction with Expert Advice, PEA) till kontinuerliga expertrum med partiell återkoppling. HI-algoritmerna minimerar den förväntade kostnaden över flera inferensomgångar genom att kombinera kostnader för avlastning och felklassificering, och uppnår O(T^2/3) sublinjär ånger. De föreslagna algoritmerna är oberoende av modellarkitektur och kommunikationssystem, kräver ingen ändring av modellträningen, och stödjer modelluppdateringar under drift. Jämförelser på standardiserade klassificeringsuppgifter med softmax-värde som konfidensmått visade att HI fördelar inferens adaptivt beroende på avlastningskostnader och når resultat nära det offline-optimum som beräknats i efterhand. HI visade sig dessutom öka robustheten mot distributionsförändringar och modellavvikelser, särskilt i fall med asymmetriska felklassificeringskostnader.

Sammanfattningsvis presenterar avhandlingen effektiva metoder för sampling och avlastning av inferensuppgifter där olika prestandamått kombineras i en gemensam kostnadsstruktur. Arbetet sträcker sig bortom konventionella inferensproblem till områden med liknande avvägningar, och bidrar till utvecklingen av effektiv distribuerad intelligens som tar beslut vid rätt tidpunkt och på rätt plats. Framtida arbete inkluderar konceptuella utvidgningar såsom gemensam design av sampling och avlastning, samt integration med kollaborativa modellträningsarkitekturer.

We consider a resource-constrained Edge Device (ED), such as an IoT sensor or a microcontroller unit, embedded with a small-size ML model (S-ML) for a generic classification application and an Edge Server (ES) that hosts a large-size ML model (L-ML). Since the inference accuracy of S-ML is lower than that of the L-ML, offloading all the data samples to the ES results in high inference accuracy, but it defeats the purpose of embedding S-ML on the ED and deprives the benefits of reduced latency, bandwidth savings, and energy efficiency of doing local inference. In order to get the best out of both worlds, i.e., the benefits of doing inference on the ED and the benefits of doing inference on ES, we explore the idea of Hierarchical Inference (HI), wherein S-ML inference is only accepted when it is correct, otherwise the data sample is offloaded for L-ML inference. However, the ideal implementation of HI is infeasible as the correctness of the S-ML inference is not known to the ED. We thus propose an online meta-learning framework that the ED can use to predict the correctness of the S-ML inference. In particular, we propose to use the probability corresponding to the maximum probability class output by S-ML for a data sample and decide whether to offload it or not. The resulting online learning problem turns out to be a Prediction with Expert Advice (PEA) problem with continuous expert space. For a full feedback scenario, where the ED receives feedback on the correctness of the S-ML once it accepts the inference, we propose the HIL-F algorithm and prove a sublinear regret bound √ n ln(1/λ min )/2 without any assumption on the smoothness of the loss function, where n is the number of data samples and λ min is the minimum difference between any two distinct maximum probability values across the data samples. For a no-local feedback scenario, where the ED does not receive the ground truth for the classification, we propose the HIL-N algorithm and prove that it has O ( n 2/3 ln 1/3 (1/λ min )) regret bound. We evaluate and benchmark the performance of the proposed algorithms for image classification application using four datasets, namely, Imagenette and Imagewoof [1], MNIST [2], and CIFAR-10 [3].

Deep learning (DL) applications have rapidly evolved to address increasingly complex tasks by leveraging large-scale, resource-intensive models. However, deploying such models on low-power devices is not practical or economically scalable. While cloud-centric solutions satisfy these computational demands, they present challenges in terms of communication costs and latencies for real-Time applications when every computation task is offloaded. To mitigate these concerns, hierarchical inference (HI) frameworks have been proposed, enabling edge devices equipped with small ML models to collaborate with edge servers by selectively offloading complex tasks. Existing HI approaches depend on immediate offloading of data upon selection, which can lead to inefficiencies due to frequent communication, especially in time-varying wireless environments. In this work, we introduce Batch HI, an approach that offloads samples in batches, thereby reducing communication overhead and improving system efficiency while achieving similar performance as existing HI methods. Additionally, we find the optimal batch size that attains a crucial balance between responsiveness and system time, tailored to specific user requirements. Numerical results confirm the effectiveness of our approach, highlighting the scenarios where batching is particularly beneficial.

We study the problem of finding efficient sampling policies in an edge-based feedback system, where sensor samples are offloaded to a back-end server that processes them and generates feedback to a user. Sampling the system at maximum frequency results in the detection of events of interest with minimum delay but incurs higher energy costs due to the communication and processing of redundant samples. On the other hand, lower sampling frequency results in higher delay in detecting the event, thus increasing the idle energy usage and degrading the quality of experience. We quantify this trade-off as a weighted function between the number of samples and the sampling interval. We solve the minimisation problem for exponential and Rayleigh distributions, for the random time to the event of interest. We prove the convexity of the objective functions by using novel techniques, which can be of independent interest elsewhere. We argue that adding an initial offset to the periodic sampling can further reduce the energy consumption and jointly compute the optimum offset and sampling interval. We apply our framework to two practically relevant applications and show energy savings of up to 36% when compared to an existing periodic scheme. 

This paper presents ExPECA, an edge computing and wireless communication research testbed designed to tackle two pressing challenges: comprehensive end-to-end experimentation and high levels of experimental reproducibility. Leveraging OpenStack-based Chameleon Infrastructure (CHI) framework for its proven flexibility and ease of operation, ExPECA is located in a unique, isolated underground facility, providing a highly controlled setting for wireless experiments. The testbed is engineered to facilitate integrated studies of both communication and computation, offering a diverse array of Software-Defined Radios (SDR) and Commercial Off-The-Shelf (COTS) wireless and wired links, as well as containerized computational environments. We exemplify the experimental possibilities of the testbed using OpenRTiST, a latency-sensitive, bandwidthintensive application, and analyze its performance. Lastly, we highlight an array of research domains and experimental setups that stand to gain from ExPECA's features, including closed-loop applications and time-sensitive networking.

We consider a resource-constrained Edge Device (ED), such as an IoT sensor or a microcontroller unit, embedded with a small-size ML model (S-ML) for a generic classification application and an Edge Server (ES) that hosts a large-size ML model (L-ML). Since the inference accuracy of S-ML is lower than that of the L-ML, offloading all the data samples to the ES results in high inference accuracy, but it defeats the purpose of embedding S-ML on the ED and deprives the benefits of reduced latency, bandwidth savings, and energy efficiency of doing local inference. In order to get the best out of both worlds, i.e., the benefits of doing inference on the ED and the benefits of doing inference on ES, we explore the idea of Hierarchical Inference (HI), wherein S-ML inference is only accepted when it is correct, otherwise the data sample is offloaded for L-ML inference. However, the ideal implementation of HI is infeasible as the correctness of the S-ML inference is not known to the ED. We propose an online meta-learning framework that the ED can use to predict the correctness of the S-ML inference. In particular, we propose to use the maximum softmax value output by S-ML for a data sample and decide whether to offload it or not. The resulting online learning problem turns out to be a Prediction with Expert Advice (PEA) problem with continuous expert space. We propose two different algorithms and prove sublinear regret bounds for them without any assumption on the smoothness of the loss function. We evaluate and benchmark the performance of the proposed algorithms for image classification application using four datasets, namely, Imagenette and Imagewoof, MNIST, and CIFAR-10.