This paper introduces Synthetic Active Learning (SAL), a fully automatic model refinement strategy for manufacturing parts detection using only synthetic data actively generated with domain randomization for training. SAL iteratively updates the detection model by identifying its weaknesses, such as in specific categories, materials, or object sizes, using custom evaluators, and generating targeted synthetic data to address them; it selectively synthesizes new useful data with respect to active learning, where traditionally humans in the loop select data to label. During each iteration, model training and data generation occur simultaneously to improve efficiency. Evaluated on four use cases from two industrial datasets, SAL achieved mAP@50 improvements of 2 to 6% percentage points over static learning, which refers to training on a fixed, pre-generated dataset. It also showed notable gains in underperforming categories, leading to more balanced performance across classes. Another benefit is that it uses a consistent configuration across multiple use cases, avoiding the need for extensive hyperparameter tuning common in prior domain randomization studies. Given its encouraging performance across diverse scenarios, we believe that SAL can scale to broader industrial applications where training can be fully or mostly based on synthetic data.

Human–robot collaboration (HRC) relies on accurate and timely recognition of human intentions to ensure seamless interactions. Among common HRC tasks, human-to-robot object handovers have been studied extensively for planning the robot's actions during object reception, assuming the human intention for object handover. However, distinguishing handover intentions from other actions has received limited attention. Most research on handovers has focused on visually detecting motion trajectories, which often results in delays or false detections when trajectories overlap. This paper investigates whether human intentions for object handovers are reflected in non-movement-based physiological signals. We conduct a multimodal analysis comparing three data modalities: electroencephalogram (EEG), gaze, and hand-motion signals. Our study aims to distinguish between handover-intended human motions and non-handover motions in an HRC setting, evaluating each modality's performance in predicting and classifying these actions before and after human movement initiation. We develop and evaluate human intention detectors based on these modalities, comparing their accuracy and timing in identifying handover intentions. To the best of our knowledge, this is the first study to systematically develop and test intention detectors across multiple modalities within the same experimental context of human–robot handovers. Our analysis reveals that handover intention can be detected from all three modalities. Nevertheless, gaze signals are the earliest as well as the most accurate to classify the motion as intended for handover or non-handover.

Coordinated control of Flexible AC Transmission Systems (FACTS) setpoints can significantly enhance power flow and voltage control. However, optimizing the setpoints of multiple FACTS devices in real-world systems remains uncommon, partly due to challenges in model-based control. Data-driven approaches, such as reinforcement learning (RL), offer a promising alternative for coordinated control. In this work, we address a setting where a useful real-time network model is unavailable. Recognizing the increasing deployment of Phasor Measurement Units (PMUs) for advanced monitoring and control, we consider having access to a few but reliable measurements and a constraint violation signal. Under these assumptions, we demonstrate on several scenarios on the IEEE 14-bus and IEEE 57-bus systems that an RL-based optimization of FACTS setpoints can substantially reduce voltage deviations compared to a fixed-setpoint baseline. To improve robustness and prevent unobserved constraint violations, we show that a complete, albeit simple, constraint violation signal is necessary. As an alternative to relying on such a signal, Dynamic Mode Decomposition is proposed to determine new PMU placements, thereby reducing the risk of unobserved constraint violations. Finally, to assess the gap to an optimal policy, we benchmark the RL-based agent against a model-based optimal controller with perfect information.

This work aims to interpret human behavior to anticipate potential user confusion when a robot provides explanations for failure, allowing the robot to adapt its explanations for more natural and efficient collaboration. Using a dataset [1] that included facial emotion detection, eye gaze estimation, and gestures from 55 participants in a user study [2], we analyzed how human behavior changed in response to different types of failures and varying explanation levels. Our goal is to assess whether human collaborators are ready to accept less detailed explanations without inducing confusion. We formulate a data-driven predictor to predict human confusion during robot failure explanations. We also propose and evaluate a mechanism, based on the predictor, to adapt the explanation level according to observed human behavior. The promising results from this evaluation indicate the potential of this research in adapting a robot's explanations for failures to enhance the collaborative experience.

In the manufacturing industry, automatic quality inspections can lead to improved product quality and productivity. Deep learning-based computer vision technologies, with their superior performance in many applications, can be a possible solution for automatic quality inspections. However, collecting a large amount of annotated training data for deep learning is expensive and time-consuming, especially for processes involving various products and human activities such as assembly. To address this challenge, we propose a method for automated assembly quality inspection using synthetic data generated from computer-aided design (CAD) models. The method involves two steps: automatic data generation and model implementation. In the first step, we generate synthetic data in two formats: two-dimensional (2D) images and three-dimensional (3D) point clouds. In the second step, we apply different state-of-the-art deep learning approaches to the data for quality inspection, including unsupervised domain adaptation, i.e., a method of adapting models across different data distributions, and transfer learning, which transfers knowledge between related tasks. We evaluate the methods in a case study of pedal car front-wheel assembly quality inspection to identify the possible optimal approach for assembly quality inspection. Our results show that the method using Transfer Learning on 2D synthetic images achieves superior performance compared with others. Specifically, it attained 95% accuracy through fine-tuning with only five annotated real images per class. With promising results, our method may be suggested for other similar quality inspection use cases. By utilizing synthetic CAD data, our method reduces the need for manual data collection and annotation. Furthermore, our method performs well on test data with different backgrounds, making it suitable for different manufacturing environments.

Robotic systems for manipulation tasks are increasingly expected to be easy to configure for new tasks or unpredictable environments, while keeping a transparent policy that is readable and verifiable by humans. We propose the method BEhavior TRee eXPansion with Large Language Models (BETR-XP-LLM) to dynamically and automatically expand and configure Behavior Trees as policies for robot control. The method utilizes an LLM to resolve errors outside the task planner's capabilities, both during planning and execution. We show that the method is able to solve a variety of tasks and failures and permanently update the policy to handle similar problems in the future.

This paper addresses key aspects of domain randomization in generating synthetic data for manufacturing object detection applications. To this end, we present a comprehensive data generation pipeline that reflects different factors: object characteristics, background, illumination, camera settings, and post-processing. We also introduce the Synthetic Industrial Parts Object Detection dataset (SIP15-OD) consisting of 15 objects from three industrial use cases under varying environments as a test bed for the study, while also employing an industrial dataset publicly available for robotic applications. In our experiments, we present more abundant results and insights into the feasibility as well as challenges of sim-toreal object detection. In particular, we identified material properties, rendering methods, post-processing, and distractors as important factors. Our method, leveraging these, achieves top performance on the public dataset with Yolov8 models trained exclusively on synthetic data; mAP@50 scores of 96.4% for the robotics dataset, and 94.1%, 99.5%, and 95.3% across three of the SIP15-OD use cases, respectively. The results showcase the effectiveness of the proposed domain randomization, potentially covering the distribution close to real data for the applications.

In this study, we explore the feasibility of single-trial predictions of odor registration in the brain using olfactory bio-signals. We focus on two main aspects: input data modality and the processing model. For the first time, we assess the predictability of odor registration from novel electrobulbogram (EBG) recordings, both in sensor and source space, and compare these with commonly used electroencephalogram (EEG) signals. Despite having fewer data channels, EBG shows comparable performance to EEG. We also examine whether breathing patterns contain relevant information for this task. By comparing a logistic regression classifier, which requires hand-crafted features, with an end-to-end convolutional deep neural network, we find that end-to-end approaches can be as effective as classic methods. However, due to the high dimensionality of the data, the current dataset is insufficient for either classifier to robustly differentiate odor and non-odor trials. Finally, we identify key challenges in olfactory BCIs and suggest future directions for improving odor detection systems.

Decoding visual images from brain activity has significant potential for advancing brain-computer interaction and enhancing the understanding of human perception. Recent approaches align the representation spaces of images and brain activity to enable visual decoding. In this paper, we introduce the use of human-aligned image encoders to map brain signals to images. We hypothesize that these models more effectively capture perceptual attributes associated with the rapid visual stimuli presentations commonly used in visual brain data recording experiments. Our empirical results support this hypothesis, demonstrating that this simple modification improves image retrieval accuracy by up to 21% compared to state-of-the-art methods. Comprehensive experiments confirm consistent performance improvements across diverse EEG architectures, image encoders, alignment methods, participants, and brain imaging modalities.

Decoding visual images from brain activity has significant potential for advancing brain-computer interaction and enhancing the understanding of human perception. Recent approaches align the representation spaces of images and brain activity to enable visual decoding. In this paper, we introduce the use of human-aligned image encoders to map brain signals to images. We hypothesize that these models more effectively capture perceptual attributes associated with the rapid visual stimuli presentations commonly used in visual brain data recording experiments. Our empirical results support this hypothesis, demonstrating that this simple modification improves image retrieval accuracy by up to 21% compared to state-of-the-art methods. Comprehensive experiments confirm consistent performance improvements across diverse EEG architectures, image encoders, alignment methods, participants, and brain imaging modalities(1).