There is a lot of unexploited potential in using data-driven and self-learning methods to dramatically improve automatic decision-making and control in complex industrial systems. So far, and on a relatively small scale, these methods have demonstrated some potential to achieve performance gains for the automated tuning of complex distributed systems. However, many difficult questions and challenges remain in relation to how to design methods and organise their deployment and operation into large-scale real-world systems. For systematic and scalable integration of state-of-the-art machine learning into such systems, we propose a structured architectural approach.

To understand the essential elements of this architecture, we identify a set of foundational challenges and then derive a set of five research questions. These questions drill into the essential and complex interdependency between data streams, self-learning algorithms that never stop learning and the supporting reference and run-time architectural structures. While there is a need for traditional one-shot supervised models, pushing the technical boundaries of automating all classes of machine learning model training will require a continual approach. 

To support continual learning, real-time data streams are complemented with accurate synthetic data generated for use in model training. By developing and integrating advanced simulations, models can be trained before deployment into a live system, for which system accuracy is then measured quantitatively in realistic scenarios. Reinforcement learning, exploring an action space and qualifying effective dynamic action combinations, is here employed for effective network policy learning. While single-agent and centralised model training may be appropriate in some cases, distributed multi-agent self-learning is essential in industrial scale systems, and thus such a scalable and energy-efficient approach is developed, implemented and analysed in detail. 

Energy usage minimisation in software and hardware intense communication systems, such as the 5G radio access system, is an important and difficult problem in its own right. Our work has focused on energy-aware approaches to applying self-learning methods both to energy reduction applications and algorithms. Using this approach, we can demonstrate clear energy savings while at the same time improving system performance.

Perhaps most importantly, our work attempts to form an understanding of the broader industrial system issues of applying self-learning approaches at scale. Our results take some clear, formative, steps towards large-scale industrialisation of self-learning approaches in communication systems such as 5G.

Datadrivna och självlärande system besitter en mycket stor outnyttjad potential för att förbättra automatisk kontroll och automatiskt beslutsfattande i komplexa industriella system. I mindre skala så har dessa metoder visats ha en viss potential rörande förbättrad prestanda för  automatisk justering av komplexa distribuerande system. Trots detta återstår många svåra frågor och utmaningar kring hur man utformar metoder och hur man organiserar implementering och drift för dessa i storskaliga realtidssystem. 

För systematisk och skalbar integrering av moderna maskininlärningstekniker i dessa verkliga och kommersiellt fungerande system föreslår vi här en strukturerad metod. För att förstå de viktigaste beståndsdelarna och arkitektoniska utmaningarna så namnger och förklarar vi en uppsättning sådana. Ur dessa härleder vi sedan fem forskningsfrågor, vilka undersöker det komplexa beroendeförhållandet mellan dataströmmar, självlärande algoritmer med kontinuerlig inlärning, samt stödjande referens- och driftstrukturer.Det finns fortfarande ett behov av övervakade ''one-shot''-modeller, men för att tänja på de tekniska gränserna avseende automatiserad träning av alla olika slags självlärande system så krävs en kontinuerlig metod. För att främja kontinuerlig inlärning kompletteras realtidsdataströmmar med adekvata syntetiska data, genererade för att möjliggöra träning av modellerna.Genom att utveckla och integrera avancerade simuleringar kan system och modeller tränas innan de implementeras för att användas ''live'', där systemets prestanda and korrekthet kan mätas kvantitativt i realistiska scenarier. För effektiv inlärning av en policy för nätverk så används förstärkningsinlärning (''reinforcement learning''), som utforskar en rymd av möjliga handlingar, ofta i kvalificerade kombinationer.

Medan centraliserad träning kan vara lämpligt i vissa fall så är distribuerade och självlärande agenter nödvändiga komponenter i industriellt storskaliga system. Därför utvecklar, implementerar och detaljanalyserar vi en sådan skalbar och energieffektiv metod.Att minska energianvändningen i mjuk- och hårdvaruintensiva kommunikationssystem, som 5G-radiosystemet, är en svår och viktig utmaning i sig. Vårt arbete har fokuserat på en energimedveten ansats med självlärande metoder, både  för tillämpningarna och för de grundläggande algoritmerna. Genom denna ansats har vi lyckats påvisa avsevärda energibesparingar samtidigt som systemets prestanda förbättrats. Till sist så är nyckelresultatet i vårt arbete analysen av de största utmaningarna för självlärande system i industriell skala och vi har därmed  tagit ett stort steg emot storskalig industrialisering av självlärande metoder inom kommunikationssystem 

As design, deployment and operation complexity increase in mobile systems, adaptive self-learning techniques have become essential enablers in mitigation and control of the complexity problem. Artificial intelligence and, in particular, reinforcement learning has shown great potential in learning complex tasks through observations. The majority of ongoing reinforcement learning research activities focus on single-agent problem settings with an assumption of accessibility to a globally observable state and action space. In many real-world settings, such as LTE or 5G, decision making is distributed and there is often only local accessibility to the state space. In such settings, multi-agent learning may be preferable, with the added challenge of ensuring that all agents collaboratively work towards achieving a common goal. We present a novel cooperative and distributed actor-critic multi-agent reinforcement learning algorithm. We claim the approach is sample efficient, both in terms of selecting observation samples and in terms of assignment of credit between subsets of collaborating agents.

Communication service providers need a greater degree of RAN automation to cope with the increasingly advanced RAN. Getting there will require an increased use of artificial intelligence and machine-learning techniques.

A significant and growing portion of communication service providers’ (CSPs) opex relates to the manual tuning of algorithms in RANs that do not exploit the full potential of the networks in the field. As 5G and cloud-native RAN implementations continue, the skill level needed to operate the RAN will continue to rise. Our AI-centered approach to RAN automation is designed to overcome both of these challenges. 

Modern mobile networks are increasingly complex from a resource management perspective, with diverse combinations of software, infrastructure elements and services that need to be configured and tuned for correct and efficient operation. It is well accepted in the communications community that appropriately dimensioned, efficient and reliable configurations of systems like 5G or indeed its predecessor 4G is a massive technical challenge. One promising avenue is the application of machine learning methods to apply a data-driven and continuous learning approach to automated system performance tuning. We demonstrate the effectiveness of policy-gradient reinforcement learning as a way to learn and apply complex interleaving patterns of radio resource block usage in 4G and 5G, in order to automate the reduction of cell edge interference. We show that our method can increase overall spectral efficiency up to 25% and increase the overall system energy efficiency up to 50% in very challenging scenarios by learning how to do more with less system resources. We also introduce a flexible phased and continuous learning approach that can be used to train a bootstrap model in a simulated environment after which the model is transferred to a live system for continuous contextual learning. 

Artificial intelligence and its subfield machine learning offer well-established techniques for solving historically difficult multi-parameterization problems. Used correctly, these techniques have tremendous potential to overcome complex cross-domain automation challenges in radio networks.

Our ongoing research reveals that an integrated framework of software enablers will be essential to success.

Modern telecommunication and mobile networks are increasingly complex from a resource management perspective, with diverse combinations of software and infrastructure elements that need to be configured and tuned for efficient operation with high quality of service. Increased real-time automation at all levels and time-frames is a critical tool in controlling this complexity. A key component in automation is practical and accurate simulation methods that can be used in live traffic scenarios. This paper introduces a new method with supporting algorithms for sampling key parameters from live or recorded traffic which can be used to generate large volumes of synthetic traffic with very similar rate distributions and temporal characteristics. Multiple spatial renewal processes are used to generate fractional Gaussian noise, which is scaled and transformed into a log-normal rate distribution with discrete arrival events, fitted to the properties observed in given recorded traces. This approach works well for modelling large user aggregates but is especially useful for medium sized and relatively small aggregates, where existing methods struggle to reproduce the most important properties of recorded traces. The technique is demonstrated through experimental comparisons with data collected from an operational LTE network to be highly useful in supporting self-learning and automation algorithms which can ultimately reduce complexity, increase energy efficiency, and reduce total network operation costs.

The number of connected mobile devices is increasing rapidly with more than 10 billion expected by 2022. Their total aggregate energy consumption poses a significant concern to society. The current 3gpp (3rd Generation Partnership Project) LTE/LTE-Advanced standard incorporates an energy saving technique called discontinuous reception (DRX). It is expected that 5G will use an evolved variant of this scheme. In general, the single selection of DRX parameters per device is non trivial. This paper describes how to improve energy efficiency of mobile devices by selecting DRX based on the traffic profile per device. Our particular approach uses a two phase data-driven strategy which tunes the selection of DRX parameters based on a smart fast energy model. The first phase involves the off-line selection of viable DRX combinations for a particular traffic mix. The second phase involves an on-line selection of DRX from this viable list. The method attempts to guarantee that latency is not worse than a chosen threshold. Alternatively, longer battery life for a device can be traded against increased latency. We built a lab prototype of the system to verify that the technique works and scales on a real LTE system. We also designed a sophisticated traffic generator based on actual user data traces. Complementary method verification has been made by exhaustive off-line simulations on recorded LTE network data. Our approach shows significant device energy savings, which has the aggregated potential over billions of devices to make a real contribution to green, energy efficient networks.

Model-driven engineering has got a foothold in industry as an effective way to tame the complexity of modern software, which is meant to run on embedded systems with real-time constraints by promoting abstraction, in terms of prescriptive models, and automation, in terms of model manipulations. In the plethora of modeling languages, the unified modeling language (UML) has emerged and established itself as a de facto standard in industry, the most widely used architectural description language and an ISO/IEC standard. In the SMARTCore project, we have provided solutions for the UML-based development of software to run on multicore embedded real-time systems with the specific focus of automating the generation of executable code and the optimization of task allocation based on a unique combination of model-based and execution-based mechanisms. In this paper, we describe the lessons learned in the research work carried out within SMARTCore and provide a set of perspectives that we consider to be highly relevant for the forthcoming future of this research area to enable a wider adoption of UML-based development in industry in general, and in the multicore embedded real-time domain in particular.

This paper presents a method for load balancing heterogeneous networks by dynamically assigning values to the LTE cell range expansion (CRE) parameter. The method records hand-over events online and adapts flexibly to changes in terminal traffic and mobility by maintaining statistical estimators that are used to support autonomous assignment decisions. The proposed approach has low overhead and is highly scalable due to a modularised and completely distributed design that exploits self-organisation based on local inter-cell interactions. An advanced simulator that incorporates terminal traffic patterns and mobility models with a radio access network simulator has been developed to validate and evaluate the method. 

Thanks to continuous advances in both software and hardware technologies the power of modern embedded systems is ever increasing along with their complexity. Among the others, Model-Driven Engineering has grown consideration for mitigating this complexity through its ability to shift the focus of the development from hand-written code to models from which correct-by-construction implementation is automatically generated. However, the path towards correctness-by-construction is often twisted by the inability of current MDE approaches to preserve certain extra-functional properties such as CPU and memory usage, execution time and power consumption. With SMART Core we address open challenges, described in this paper together with an overview of possible solutions, in modelling, generating code from models, and exploiting back-propagated extra-functional properties observed at runtime for deployment optimisation of embedded systems on multicore. SMART Core brings together world leading competence in software engineering, model-driven engineering for embedded systems (Mälardalen University), and market leading expertise in the development of these systems in different business areas (ABB Corporate Research, Ericsson AB, Alten Sweden AB).