Condition-awareness regarding electrical and electronic components is not only significant for predictive maintenance of automotive vehicles but also plays a crucial role in ensuring the operational safety by supporting the detection of anomalies, faults, and degradations over lifetime. In this paper, we present a novel control strategy that combines stochastic dynamic control method with condition-awareness for safe automated driving. In particular, the effectiveness of condition-awareness is supported by two distinct condition-monitoring functions. The first function involves the monitoring of a vehicle's internal health condition using model-based approaches. The second function involves the monitoring of a vehicle's external surrounding conditions, using machine learning and artificial intelligence approaches. For the quantification of current conditions, the results from these monitoring functions are used to create system health indices, which are then utilized by a safety control function for dynamic behavior regulation. The design of this safety control function is based on a chance-constrained model predictive control model, combined with a control barrier function for ensuring safe operation. The novelty of the proposed method lies in a systematic integration of monitored external and internal conditions, estimated component degradation, and remaining useful life, with the controller's dynamic responsiveness. The efficacy of the proposed strategy is evaluated with adaptive cruise control in the presence of various sensory uncertainties.

This paper presents a reconfigurable near-sensor anomaly detection processor to real-time monitor the potential anomalous behaviors of amputees with limb prostheses. The processor is low-power, low-latency, and suitable for equipment on the prostheses and comprises a reconfigurable Variational Autoencoder (VAE), a scalable Self-Organizing Map (SOM) Array, and a window-size-adjustable Markov Chain, which can implement an integrated miniaturized anomaly detection system. With the reconfigurable VAE, the proposed processor can support up to 64 sensor sampling channels programmable by global configuration, which can meet the anomaly detection requirements in different scenarios. A scalable SOM array allows for the selection of different sizes based on the complexity of the data. Unlike traditional time accumulation-based anomaly detection methods, the Markov Chain is utilized to detect time-series-based anomalous data. The processor is designed and fabricated in a UMC 40-nm LP technology with a core area of 1.49 mm2 and a power consumption of 1.81 mW. It achieves real-time detection performance with 0.933 average F1 Score for the FSP dataset within 24.22 s, and 0.956 average F1 Score for the SFDLA-12 dataset within 30.48 s, respectively. The energy dissipation of detection for each input feature is 43.84 nJ with the FSP dataset, and 55.17 nJ with the SFDLA-12 dataset. Compared with ARM Cortex-M4 and ARM Cortex-M33 microcontrollers, the processor achieves energy and area efficiency improvements ranging from 257×, 193× and 11×, 8×, respectively. IEEE

Human Activity Recognition (HAR) refers to a field that aims to identify human activitiesby adopting multiple techniques. In this field, different applications, such as smart homes andassistive robots, are introduced to support individuals in their Activities of Daily Living (ADL)by analyzing data collected from various sensors. Apart from wearable sensors, the adoption ofcamera frames to analyze and classify ADL has emerged as a promising trend for achieving theidentification and classification of ADL. To accomplish this, the existing approaches typically rely onobject classification with pose estimation using the image frames collected from cameras. Given theexistence of inherent correlations between human–object interactions and ADL, further efforts areoften needed to leverage these correlations for more effective and well justified decisions. To this end,this work proposes a framework where Graph Neural Networks (GNN) are adopted to explicitlyanalyze human–object interactions for more effectively recognizing daily activities. By automaticallyencoding the correlations among various interactions detected through some collected relational data,the framework infers the existence of different activities alongside their corresponding environmentalobjects. As a case study, we use the Toyota Smart Home dataset to evaluate the proposed framework.Compared with conventional feed-forward neural networks, the results demonstrate significantlysuperior performance in identifying ADL, allowing for the classification of different daily activitieswith an accuracy of 0.88. Furthermore, the incorporation of encoded information from relational dataenhances object-inference performance compared to the GNN without joint prediction, increasingaccuracy from 0.71 to 0.77. 

Gait analysis helps clinical assessment and achieves comfortable prosthetic designs for lower limb amputees, in which accurate gait phase recognition is a key component. However, gait phase detection remains a challenge due to the individual nature of prosthetic sockets and limbs. For the first time, we present a gait phase recognition approach for transfemoral amputees based on intra-socket pressure measurement. We proposed a multiple GMM-HMM (Hidden Markov Model with Gaussian Mixture Model emissions) method to label the gait events during walking. For each of the gait phases in the gait cycle, a separate GMM-HMM model is trained from the collected pressure data. We use gait phase recognition accuracy as a primary metric. The evaluation of six human subjects during walking shows a high accuracy of over 99% for single-subject, around 97.4% for multiple-subject, and up to 84.5% for unseen-subject scenarios. We compare our approach with the widely used CHMM (Continuous HMM) and LSTM (Long Short-term Memory) based methods, demonstrating better recognition accuracy performance across all scenarios.

Prognostics and health management (PHM) is an engineering discipline that aims to maintain system behaviour and function and ensure mission success, safety and effectiveness. Addressing the challenges in prognostics and health management for modern intelligent systems, especially automated driving systems, is complex due to the contextual nature of faults. This complexity necessitates a thorough understanding of spatial, and temporal conditions, and relationships within operational scenarios and life-cycle stages. This paper introduces a framework designed to automatically recognize driving scenarios in automated driving systems using graph neural networks (GNNs). The framework extracts relational data from image frames, constructing graph-based models and transforming unstructured sensory data into structured data with diverse node types and relationships. A specific graph neural network processes the graph model to reveal and detect operational conditions and relationships. The proposed framework is evaluated using the KITTI dataset, demonstrating superior performance compared to conventional feed-forward networks such as MLP, particularly in handling relational data.

Object detection is one of the most fundamental problems in computer vision, and image sensors are commonly used for this. In this paper, we present the impact of image sensor faults on pruned neural networks for object detection. We compare the error sensitivities of networks after network slimming, networks after magnitude-based pruning, and native compact models. We also explore different spatial fault types with three intensities. Furthermore, we have developed a temporal error model based on realistic aging image sensor faults. The results illuminate that the performance on clean images is important as the mean Average Precision (mAP) experiences a decrease with an increase in injected faults. Additionally, we demonstrate that the size of the model does not invariably yield a decisive impact on error tolerance when comparing small models such as pruned models and native compact models.

Anomaly detection is essential in Industrial Cyber-Physical Systems (ICPS) for monitoring both system and environmental conditions. However, effective anomaly detection remains a continuous challenge due to the complexity and diversity of operational features in both spatial and temporal domains of such systems. Current data-driven approaches utilize Artificial Intelligence (AI)-enabled methods, such as Recurrent Neural Networks (RNN) and Convolutional Neural Networks (CNN), to process and analyze the collected operational data. From the industrial perspective, a key challenge lies in enhancing the effectiveness and explainability, as these methods often exhibit opaque and sophisticated network structures when dealing with complex operational data. Meanwhile, recent advancements in Graph Neural Networks (GNN) demonstrate its effectiveness of supporting the analysis of complex relationships and dependencies. The adoption of GNN poses however inherent challenges when the data are unstructured. This paper designs a novel framework that integrates Self-Organizing Maps (SOM) and GNN for the analysis of complex operational relationships and dependencies. In particular, the adoption of SOM allows the generation graph-structured data from unstructured raw datasets and thereby enhances a GNN’s ability to differentiate normal and anomalous conditions effectively. As case studies, we select multiple public datasets to compare the performance of the proposed framework with other benchmark methods. The results show that the proposed methods present promising results. Additionally, compared to other baseline methods that use GNN-based structures to detect anomalies, the proposed framework achieves the highest F1-Score.

Artificial Intelligence (AI) techniques through Learning-Enabled Components (LEC) are widely employed in Automated Driving Systems (ADS) to support operation perception and other driving tasks relating to planning and control. Therefore, the risk management plays a critical role in assuring the operational safety of ADS. However, the probabilistic and nondeterministic nature of LEC challenges the safety analysis. Especially, the impacts of their functional faults and incompatible external conditions are often difficult to identify. To address this issue, this article presents a simulation-aided approach as follows: 1) A simulation-aided operational data generation service with the operational parameters extracted from the corresponding system models and specifications; 2) A Fault Injection (FI) service aimed at high-dimensional sensor data to evaluate the robustness and residual risks of LEC. 3) A Variational Bayesian (VB) method for encoding the collected operational data and supporting an effective estimation of the likelihood of operational conditions. As a case study, the paper presents the results of one experiment, where the behaviour of an Autonomous Emergency Braking (AEB) system is simulated under various weather conditions based on the CARLA driving simulator. A set of fault types of cameras, including solid occlusion, water drop, salt and pepper, are modelled and injected into the perception module of the AEB system in different weather conditions. The results indicate that our framework enables to identify the critical faults under various operational conditions. To approximate the critical faults in undefined weather, we also propose Variational Autoencoder (VAE) to encode the pixel-level data and estimate the likelihood.

Anomaly detection plays a critical role in condition monitors to support the trustworthiness of Cyber-Physical Systems (CPS). Detecting multivariate anomalous data in such systems is challenging due to the lack of a complete comprehension of anomalous behaviors and features. This paper proposes a framework to address time series multivariate anomaly detection problems by combining the Self-Organizing Map (SOM) with Deep Reinforcement Learning (DRL). By clustering the multivariate data, SOM creates an environment to enable the DRL agents interacting with the collected system  operational data in terms of a tabular dataset. In this environment, Markov chains reveal the likely anomalous features to support the DRL agent exploring and exploiting the state-action space to maximize anomaly detection performance. We use a time series dataset, Skoltech Anomaly Benchmark (SKAB), to evaluate our framework. Compared with the best results by some currently applied methods, our framework improves the F1 score by 9%, from 0.67 to 0.73.