The deepening of human fatigue will lead to the reduction of exercise ability and work efficiency, the increase of errors and accidents, and even the occurrence of organic diseases. Obviously, it is significant to understand the impact of human fatigue on the health, safe production and safe work of different people. At present, fatigue detection is mostly carried out through EEG and EMG signals. These methods usually have the disadvantages of contact and non-realtime. In response to the aforementioned issues in the process of human fatigue detection, this article effectively applies the visual image analysis method of spectrograms to human fatigue detection and proposes a cross-modal human fatigue detection method based on speech spectral image recognition. First, Mel spectrograms of speech segments in the corpus are extracted, and a fatigue spectrogram data set is established. A deep learning model is established through convolutional neural network (CNN) and extreme learning machine (ELM) for spectral image recognition and fatigue detection. CNN is used to extract features from the input image. The feature mapping will eventually be encoded into a one-dimensional vector and sent to ELM for classification. The experimental results indicate that the speech spectrum image features extracted by this method have better fatigue characterization ability than traditional acoustic features.

To address the issue of low accuracy in speech fatigue recognition (SFR) under small samples, a method for small-sample SFR based on generative adversarial networks (GANs) is proposed. First, we enable the generator and discriminator to adversarially train and learn the features of the samples, and use the generator to generate high-quality simulated samples to expand our dataset. Then, we transfer discriminator parameters to fatigue identification network to accelerate network training speed. Furthermore, we use a bidirectional long short-term memory network (BLSTM) to further learn temporal fatigue features and improve the recognition rate of fatigue. 720 speech samples from a self-made Chinese speech database (SUSP-SFD) were chosen for training and testing. The results indicate that compared with traditional SFR methods, like convolutional neural networks (CNNs) and long short-term memory network (LSTM), our method improved the SFR rate by about 2.3-6.7%, verifying the effectiveness of the method.

Machine learning (ML) is a key technique for big-data-driven modelling and analysis of massive Internet of Things (IoT) based intelligent and ubiquitous computing. For fast-increasing applications and data amounts, distributed learning is a promising emerging paradigm since it is often impractical or inefficient to share/aggregate data to a centralized location from distinct ones. This paper studies the problem of training an ML model over decentralized systems, where data are distributed over many user devices and the learning algorithm run on-device, with the aim of relaxing the burden at a central entity/server. Although gossip-based approaches have been used for this purpose in different use cases, they suffer from high communication costs, especially when the number of devices is large. To mitigate this, incremental-based methods are proposed. We first introduce incremental block-coordinate descent (I-BCD) for the decentralized ML, which can reduce communication costs at the expense of running time. To accelerate the convergence speed, an asynchronous parallel incremental BCD (API-BCD) method is proposed, where multiple devices/agents are active in an asynchronous fashion. We derive convergence properties for the proposed methods. Simulation results also show that our API-BCD method outperforms state of the art in terms of running time and communication costs.

To leverage massive distributed data and computation resources, machine learning in the network edge is considered to be a promising technique especially for large-scale model training. Federated learning (FL), as a paradigm of collaborative learning techniques, has obtained increasing research attention with the benefits of communication efficiency and improved data privacy. Due to the lossy communication channels and limited communication resources (e.g., bandwidth and power), it is of interest to investigate fast responding and accurate FL schemes over wireless systems. Hence, we investigate the problem of jointly optimized communication efficiency and resources for FL over wireless Internet of things (IoT) networks. To reduce complexity, we divide the overall optimization problem into two sub-problems, i.e., the client scheduling problem and the resource allocation problem. To reduce the communication costs for FL in wireless IoT networks, a new client scheduling policy is proposed by reusing stale local model parameters. To maximize successful information exchange over networks, a Lagrange multiplier method is first leveraged by decoupling variables including power variables, bandwidth variables and transmission indicators. Then a linear-search based power and bandwidth allocation method is developed. Given appropriate hyper-parameters, we show that the proposed communication-efficient federated learning (CEFL) framework converges at a strong linear rate. Through extensive experiments, it is revealed that the proposed CEFL framework substantially boosts both the communication efficiency and learning performance of both training loss and test accuracy for FL over wireless IoT networks compared to a basic FL approach with uniform resource allocation.

In this paper, data-driven self-calibration algorithms for the Internet-of-Things-based gas concentration monitoring systems embedded with low-cost gas sensors are designed. The measurement errors are assumed to be caused by imperfect compensation for the variation of sensor component behavior. Specifically, the calibration procedure for the non-dispersive infrared CO2 sensors is developed, for which the temperature dependency is the most dominant drift source. For a single sensor, the hidden Markov model is used to characterize the statistical relationship between different quantities introduced by the physical model that builds on the Beer-Lambert law. For the calibration in the Internet-of-Things-based system, sensors first transmit their belief functions of the true gas concentration level to the cloud. Then the cloud fusion center computes a fused belief function according to certain rules. This belief function is then used as reference for calibrating the sensors. To deal with the case where belief functions highly conflict with each other, a Wasserstein distance based weighted average belief function fusion approach is first proposed as networked calibration algorithm. To achieve more long-term stable calibration results, the networked calibration problem is further formulated as a partially observed Markov decision process problem, and the calibration strategies are derived in a sequential manner. Correspondingly, the deep Q-network approach is applied as a computationally efficient method to solve the proposed Markov decision process problem. The performance of practical designs of the proposed self-calibration algorithms is finally illustrated in numerical experiments utilizing real data.

With the ever-increasing penetration and proliferation of various smart Internet of Things (IoT) applications, machine learning (ML) is envisioned to be a key technique for big-data-driven modelling and analysis. Since massive data generated from these IoT devices are commonly collected and stored in a distributed manner, ML at the networks, e.g., distributed machine learning (DML), has been a promising emerging paradigm, especially for large-scale model training. In this thesis, we explore the optimization and design of DML algorithms under different network conditions. Our main research with regards to DML can be sorted into the following four aspects/papers as detailed below.

In the first part of the thesis, we explore fully-decentralized ML by utilizing alternating direction method of multipliers (ADMM). Specifically, to address the two main critical challenges in DML systems, i.e., communication bottleneck and stragglers (nodes/devices with slow responses), an error-control-coding-based stochastic incremental ADMM (csI-ADMM) is proposed. Given an appropriate mini-batch size, it is proved that the proposed csI-ADMM method has a $O( 1/\sqrt{k})$) convergence rate and $O(1/{\mu ^2})$ communication cost, where $k$ denotes the number of iterations and $\mu$ is the target accuracy.  In addition, tradeoff between the convergence rate and the number of stragglers, as well as the relationship between mini-batch size and number of stragglers, are both theoretically and experimentally analyzed. 

In the second part of the thesis, we investigate the asynchronous approach for fully-decentralized federated learning (FL). Specifically, an asynchronous parallel incremental block-coordinate descent (API-BCD) algorithm is proposed, where multiple nodes/devices are active in an asynchronous fashion to accelerate the convergence speed. The solution convergence of API-BCD is theoretically proved and simulation results demonstrate its superior performance in terms of both running speed and communication costs compared with state-of-the-art algorithms.

The third part of the thesis is devoted to the study of jointly optimizing communication efficiency and wireless resources for FL over wireless networks. Accordingly, an overall optimization problem is formulated, which is divided into two sub-problems, i.e., the client scheduling problem and the resource allocation problem for tractability. More specifically, to reduce the communication costs, a communication-efficient client scheduling policy is proposed by limiting communication exchanges and reusing stale local models. To optimize resource allocation at each communication round of FL training, an optimal solution based on linear search method is derived. The proposed communication-efficient FL (CEFL) algorithm is evaluated both analytically and by simulation. The final part of the thesis is a case study of implementing FL in low Earth orbit (LEO) based satellite communication networks. We investigate four possible architectures of combining ML in LEO-based computing networks. The learning performance of the proposed strategies is evaluated by simulation and results validate that FL-based computing networks can significantly reduce communication overheads and latency.

Med ökat genomslag och spridning av olika smarta Internet of Things (IoT) applikationer, förväntas maskininlärning (ML) bli en nyckelteknik för modellering ochanalys av stora data mängder. Eftersom data från dessa IoT enheter vanligtvis sparaslokalt har ML på nätverksnivå, t.ex. distribuerad maskininlärning (DML), blivit en lovande ny paradigm, särskilt för storskalig modellträning. I denna avhandling utforskarvi optimeringen och designen av DML algoritmer under olika förutsättningar i nätverken. Vår huvudsakliga forskning i hänseende till DML är fördelad i fyra papper,beskrivna enligt nedan.I första delen av denna avhandling tittar vi på fullt decentraliserad ML genom nyttjandet av älternating direction method of multipliers"(ADMM). Mer specifikt föreslårvi en "error-control-coding-baserad stochastic incremental ADMM"(csl-ADMM) föratt tackla de två mest kritiska utmaningarna i DML system, dvs. flaskhalsar i kommunikation och eftersläpare (noder/enheter med långsam respons). Givet en lämpligmini-batch storlek visar vi att den föreslagna csl-ADMM metoden konvergerar medO(1/√k) med en kommunikationskostnad på O(1/µ2), där k är antalet iterationer ochv är sökt noggrannhet. Vi ger även en teoretisk och experimentell analys av sambandetmellan konvergenshastighet och antalet eftersläpare samt sambandet mellan mini-batchstorlek och antalet eftersläpare.I avhandlingens andra del undersöker vi den asynkrona hanteringen av fullt decentraliserad kollaborativ inlärning (FL, eng. Federated Learning). Specifikt föreslår vi enalgoritm för äsynchronous parallel incremental block-coordinate descent"(API-BCD),där flera enheter/noder är asynkront aktiva för att öka konvergens hastigheten. Vi gerteoretiskt bevis för API-BCD lösningens konvergens samt visar simuleringar som demonstrerar dess överlägsna prestanda i termer av både hastighet och kommunikationskostnader jämfört med state-of-the-art algoritmer.Avhandlingens tredje del är en studie i att simultant optimera kommunikations effektivitet och hanteringen av trådlösa resurser för FL över trådlösa nätverk. Ett övergripande optimeringsproblem formuleras, som delas upp i två delproblem, schemaläggning av klienter och ett resursallokerings problem. För att reducera kommunikationskostnaderna, föreslås en effektiv kommunikations policy för schemaläggning av klienter som begränsar kommunikation och återanvändandet av lokala modeller som blivitmindre relevanta med tiden. För att optimera resurs allokeringen i varje kommunikations runda av FL träning, härleds en optimal lösning baserad på en linjär sök metod.Den föreslagna kommunikationseffektiva FL (CEFL, eng. Communication EfficientFL) algoritmen utvärderas både analytiskt och med simulering.Den sista delen av avhandlingen är en fallstudie där FL implementeras i satellitkommunikationsnätverk i låg omloppsbana (LEO, eng. Low Earh Orbit). Vi undersökerfyra möjliga arkitekturer för kombinering av ML i satellitburna datornätverk. Prestandan av de föreslagna strategierna utvärderas via simuleringar och resultaten visar attFL-baserade datornätverk kan anmärkningsvärt minska kommunikationsoverhead ochlatens.

Driven by the ever increasing penetration and proliferation of data-driven applications, a new generation of wireless communication, the sixth generation (6G) mobile system enhanced by artificial intelligence, has attracted substantial research interests. Among various candidate technologies of 6G, low Earth orbit (LEO) satellites have appealing characteristics of ubiquitous wireless access. However, the costs of satellite communication (SatCom) are still high, relative to their counterparts of ground mobile networks. To support massively interconnected devices with intelligent adaptive learning and reduce expensive traffic in SatCom, we propose federated learning (FL) in LEO-based satellite communication networks. We first review the state-of-the-art LEO-based SatCom and related machine learning (ML) techniques, and then analyze four possible ways of combining ML with satellite networks. The learning performance of the proposed strategies is evaluated by simulation and results reveal that FL-based computing networks improve the performance of communication overheads and latency. Finally, we discuss future research topics along this research direction.

Big data, including applications with high security requirements, are often collected and stored on multiple heterogeneous devices, such as mobile devices, drones, and vehicles. Due to the limitations of communication costs and security requirements, it is of paramount importance to analyze information in a decentralized manner instead of aggregating data to a fusion center. To train large-scale machine learning models, edge/fog computing is often leveraged as an alternative to centralized learning. We consider the problem of learning model parameters in a multiagent system with data locally processed via distributed edge nodes. A class of minibatch stochastic alternating direction method of multipliers (ADMMs) algorithms is explored to develop the distributed learning model. To address two main critical challenges in distributed learning systems, i.e., communication bottleneck and straggler nodes (nodes with slow responses), error-control-coding-based stochastic incremental ADMM is investigated. Given an appropriate minibatch size, we show that the minibatch stochastic ADMM-based method converges in a rate of O(1/root k), where k denotes the number of iterations. Through numerical experiments, it is revealed that the proposed algorithm is communication efficient, rapidly responding, and robust in the presence of straggler nodes compared with state-of-the-art algorithms.

The dynamic provisioning of Service Function Chain (SFC) using Virtual Network Functions (VNFs) is a challenging problem, especially for availability-constrained services. The provisioning of backup resources is often used to ensure that availability requirements are fulfilled. However, the assignment of backup resources should be carefully designed to avoid resource inefficiencies as much as possible. This paper proposes the Optional Backup with Shared Path and Shared Function (OBSPSF) strategy, which aims at improving resource efficiency while fulfilling the availability requirements of SFC requests. The strategy uses optional backup provisioning to ensure that backup resources are assigned only when strictly needed (i.e., when the SFC alone does not meet the availability constraint). Moreover, OBSPSF encourages backup sharing (among both connectivity and backup VNFs) to reduce the backup resource overhead. Results show that the strategy can accommodate orders-of-magnitude more services than benchmark heuristics from the literature.

The alternating direction method of multipliers (ADMM) has recently been recognized as a promising approach for large-scale machine learning models. However, very few results study ADMM from the aspect of communication costs, especially jointly with running time. In this letter, we investigate the communication efficiency and running time of ADMM in solving the consensus optimization problem over decentralized networks. We first review the effort of random walk ADMM (W-ADMM), which reduces communication costs at the expense of running time. To accelerate the convergence speed of W-ADMM, we propose the parallel random walk ADMM (PW-ADMM) algorithm, where multiple random walks are active at the same time. Moreover, to further reduce the running time of PW-ADMM, the intelligent parallel random walk ADMM (IPW-ADMM) algorithm is proposed through integrating the Random Walk with Choice with PW-ADMM. By numerical results from simulations, we demonstrate that the proposed algorithms can be both communication efficient and fast in running speed compared with state-of-the-art methods.