Mean opinion score (MOS) is a bounded speech quality measure, ranging between 1 and 5. We propose using a Beta distribution to model the posterior of the bounded MOS for a given speech clip. We use a deep neural network (DNN), trained using a maximum-likelihood principle, providing the parameters of the posterior Beta distribution. A self-teacher learning setup is used to achieve robustness against the inherent challenge of training on a noisy dataset. The dataset noise comes from the subjective nature of the MOS labels, and only a handful of quality score ratings are provided for each speech clip. To compare with existing state-of-the-art methods, we use the mean of Beta posterior as a point estimate of the MOS. The proposed method shows competitive performance vis-a-vis several existing DNN-based methods that provide MOS point estimates, and an ablation study shows the importance of various components of the proposed method.

This study investigates the generalization ability of non-intrusive speech quality assessment models on unseen datasets, focusing on architectural and training strategies. We evaluate two neural network designs: convolutional neural networks (CNNs) with global pooling layers, and with more complex recurrent neural networks (RNNs). Our findings reveal three key insights: (1) CNNs with global pooling layers adapt better to unseen data compared to CNN-RNN architecture, demonstrating stronger generalization ability; (2) student-teacher learning, where a student model learns from a teacher model, enhances generalization performance; and (3) smaller models are both efficient and effective, showing robust behavior across diverse datasets and test runs. These results underscore the potential of lightweight architectures and advanced training frameworks for improving the reliability of the mean-opinion-score (MOS) prediction models.

Drone-based system is an emerging technology for advanced applications in Intelligent Transport Systems (ITS). This paper presents our latest developments of a visual perception and analysis system, called AVIATOR, for drone-based road traffic management. The system advances from the previous SeeFar system in several aspects. For visual perception, deep-learning based computer vision models still play the central role but the current system development focuses on fast and efficient detection and tracking performance during real-time image processing. To achieve that, YOLOv7 and ByteTrack models have replaced the previous perception modules to gain better computational performance. Meanwhile, a lane-based traffic steam detection module is added for recognizing detailed traffic flow per lane, enabling more detailed estimation of traffic flow patterns. The traffic analytics module has been modified to estimate traffic states using lane-based data collection. This includes detailed lane-based traffic flow counting as well as traffic density estimation according to vehicle arrival patterns per lane.

We propose a deep neural network (DNN) based method that provides a posterior distribution of mean-opinion-score (MOS) for an input speech signal. The DNN outputs parameters of the posterior, mainly the posterior's mean and variance. The proposed method is referred to as deep posterior MOS (DeePMOS). The relevant training data is inherently limited in size (limited number of labeled samples) and noisy due to the subjective nature of human listeners. For robust training of DeePMOS, we use a combination of maximum-likelihood learning, stochastic gradient noise, and a student-teacher learning setup. Using the mean of the posterior as a point estimate, we evaluate standard performance measures of the proposed DeePMOS. The results show comparable performance with existing DNN-based methods that only provide point estimates of the MOS. Then we provide an ablation study showing the importance of various components in DeePMOS.

We consider a decentralized learning problem where training data samples are distributed over agents (processing nodes) of an underlying communication network topology without any central (master) node. Due to information privacy and security issues in a decentralized setup, nodes are not allowed to share their training data and only parameters of the neural network are allowed to be shared. This article investigates decentralized learning of randomization-based neural networks that provides centralized equivalent performance as if the full training data are available at a single node. We consider five randomization-based neural networks that use convex optimization for learning. Two of the five neural networks are shallow, and the others are deep. The use of convex optimization is the key to apply alternating-direction-method-of-multipliers with decentralized average consensus. This helps us to establish decentralized learning with centralized equivalence. For the underlying communication network topology, we use a doubly-stochastic network policy matrix and synchronous communications. Experiments with nine benchmark datasets show that the five neural networks provide good performance while requiring low computational and communication complexity for decentralized learning. The performance rankings of five neural networks using Friedman rank are also enclosed in the results, which are ELM < RVFL< dRVFL < edRVFL < SSFN.

Drone-based system has a big potential to be applied for traffic monitoring and other advanced applications in Intelligent Transport Systems (ITS). This paper introduces our latest efforts of digitalising road traffic by various types of sensing systems, among which visual detection by drones provides a promising technical solution. A platform, called METRIC, is under recent development to carry out real-time traffic measurement and prediction using drone-based data collection. The current system is designed as a cyber-physical system (CPS) with essential functions aiming for visual traffic detection and analysis, real-time traffic estimation and prediction as well as decision supports based on simulation. In addition to the computer vision functions developed in the earlier stage, this paper also presents the CPS system architecture and the current implementation of the drone front-end system and a simulation-based system being used for further drone operations.

In a communication network, decentralized learning refers to the knowledge collaboration between the different local agents (processing nodes) to improve the local estimation performance without sharing private data. The ideal case is that the decentralized solution approximates the centralized solution, as if all the data are available at a single node, and requires low computational power and communication overhead. In this work, we propose a decentralized learning of randomization-based neural networks with asynchronous communication and achieve centralized equivalent performance. We propose an ARock-based alternating-direction-method-of-multipliers (ADMM) algorithm that enables individual node activation and one-sided communication in an undirected connected network, characterized by a doubly-stochastic network policy matrix. Besides, the proposed algorithm reduces the computational cost and communication overhead due to its asynchronous nature. We study the proposed algorithm on different randomization-based neural networks, including ELM, SSFN, RVFL, and its variants, to achieve the centralized equivalent performance under efficient computation and communication costs. We also show that the proposed asynchronous decentralized learning algorithm can outperform a synchronous learning algorithm regarding computational complexity, especially when the network connections are sparse.

Machine learning and artificial intelligence have been wildly explored and developed very fast to adapt to the expanding need for almost every aspect of human development. When stepping into the big data era, siloed data localization has become a big challenge for machine learning. Restricted by scattered locations and privacy regulations of information sharing, recent studies aim to develop collaborated machine learning techniques for local models to approximate the centralized performance without sharing real data. Privacy preservation is as important as the model performance and the model complexity. This thesis aims to investigate the scopes of the low computational complexity learning model, randomization-based feed-forward neural networks (RFNs). As a class of artificial neural networks (ANNs), RFNs enjoy the favorable balance between low computational complexity and satisfying performance, especially for non-image data. Driven by the advantages of RFNs and the need for distributed learning resolutions, we aim to study the potential and applicability of RFNs and distributed optimization methods that may lead to the design of the decentralized variant of RFNs to deliver desired results.

Firstly, we provide the decentralized learning algorithms based on RFN architectures for undirected network topology using synchronous communication. We investigate decentralized learning of five RFNs that provides centralized equivalent performance as if the total training data samples are available at a single node. Two of the five neural networks are shallow, and the others are deep. Experiments with nine benchmark datasets show that the five neural networks provide good performance while requiring low computational and communication complexity for decentralized learning. 

Then we are motivated to design an asynchronous decentralized learning application that achieves centralized equivalent performance with low computational complexity and communication overhead. We propose an asynchronous decentralized learning algorithm using ARock-based ADMM to realize the decentralized variants of a variety of RFNs. The proposed algorithm enables single node activation and one-sided communication in an undirected communication network, characterized by a doubly-stochastic network policy matrix. Besides, the proposed algorithm obtains the centralized solution with reduced computational cost and improved communication efficiency. 

Finally, We consider the problem of training a neural net over a decentralized scenario with a high sparsity level in connections. The issue is addressed by adapting a recently proposed incremental learning approach, called `learning without forgetting.' While an incremental learning approach assumes data availability in a sequence, nodes of the decentralized scenario can not share data between them, and there is no master node. Nodes can communicate information about model parameters among neighbors. Communication of model parameters is the key to adapt the `learning without forgetting' approach to the decentralized scenario.

Maskininlärning och artificiell intelligens har utforskats vilt och utvecklats mycket snabbt för att anpassa sig till det växande behovet av nästan alla aspekter av mänsklig utveckling. När man går in i big data-eran har lokaliserad datalokalisering blivit en stor utmaning för maskininlärning. Begränsat av spridda platser och sekretessregler för informationsdelning, syftar nya studier till att utveckla samarbetade maskininlärningstekniker för lokala modeller för att approximera den centraliserade prestandan utan att dela verkliga data. Sekretessbevarande är lika viktigt som modellens prestanda och modellens komplexitet. Denna avhandling syftar till att undersöka omfattningen av den inlärningsmodell med låg beräkningskomplexitet, randomiseringsbaserade feed-forward neurala nätverk (RFN). Som en klass av artificiella neurala nätverk (ANN) har RFN: er den gynnsamma balansen mellan låg beräkningskomplexitet och tillfredsställande prestanda, särskilt för icke-bilddata. Drivs av RFN: s fördelar och behovet av distribuerade inlärningsupplösningar, syftar vi till att studera RFN: s potential och användbarhet och distribuerade optimeringsmetoder som kan leda till utformningen av den decentraliserade varianten av RFN för att leverera önskade resultat.

För det första tillhandahåller vi de decentraliserade inlärningsalgoritmerna baserade på RFN-arkitekturer för oriktad nätverkstopologi med synkron kommunikation. Vi undersöker decentraliserad inlärning av fem RFN som ger centraliserad ekvivalent prestanda som om de totala träningsdataproverna är tillgängliga i en enda nod. Två av de fem neurala nätverken är grunda, och de andra är djupa. Experiment med nio benchmarkdatauppsättningar visar att de fem neurala nätverken ger bra prestanda samtidigt som de kräver låg beräknings- och kommunikationskomplexitet för decentraliserat lärande.

Då är vi motiverade att designa en asynkron decentraliserad inlärningsapplikation som uppnår central motsvarande prestanda med låg beräkningskomplexitet och kommunikationsomkostnader. Vi föreslår en asynkron decentraliserad inlärningsalgoritm med ARock-baserad ADMM för att förverkliga de decentraliserade varianterna av en mängd olika RFN. Den föreslagna algoritmen möjliggör aktivering av enstaka noder och ensidig kommunikation i ett oriktat kommunikationsnätverk, kännetecknat av en dubbelstokastisk nätverkspolitisk matris. Dessutom erhåller den föreslagna algoritmen den centraliserade lösningen med minskad beräkningskostnad och förbättrad kommunikationseffektivitet.

Slutligen betraktar vi problemet med att träna ett neuralt nät över ett decentraliserat scenario med hög sparsitetsnivå i anslutningar. Frågan hanteras genom att anpassa en nyligen föreslagen inkrementell inlärningsmetod, kallad 'lärande utan att glömma.' Medan en inkrementell inlärningsmetod antar datatillgänglighet i en sekvens, kan noder i det decentraliserade scenariot inte dela data mellan dem, och det finns ingen masternod. Noder kan kommunicera information om modellparametrar bland grannar. Kommunikation av modellparametrar är nyckeln till att anpassa inlärningsmetoden till det decentraliserade scenariot.

We propose a feature reuse approach for an existing multi-layer randomization based feedforward neural network. The feature representation is directly linked among all the necessary hidden layers. For the feature reuse at a particular layer, we concatenate features from the previous layers to construct a large-dimensional feature for the layer. The large-dimensional concatenated feature is then efficiently used to learn a limited number of parameters by solving a convex optimization problem. Experiments show that the proposed model improves the performance in comparison with the original neural network without a significant increase in computational complexity.

We consider the problem of training a neural net over a decentralized scenario with a low communication over-head. The problem is addressed by adapting a recently proposed incremental learning approach, called `learning without forgetting'. While an incremental learning approach assumes data availability in a sequence, nodes of the decentralized scenario can not share data between them and there is no master node. Nodes can communicate information about model parameters among neighbors. Communication of model parameters is the key to adapt the `learning without forgetting' approach to the decentralized scenario. We use random walk based communication to handle a highly limited communication resource.