A deep learning based sensor fusion method to diagnose tightening errorsShow others and affiliations
2023 (English)In: Journal of manufacturing systems, ISSN 0278-6125, E-ISSN 1878-6642, Vol. 71, p. 59-69Article in journal (Refereed) Published
Abstract [en]
Modern smart assembly lines commonly include electric tools with built-in sensors to tighten safety-critical joints. These sensors generate data that are subsequently analyzed by human experts to diagnose potential tightening errors. Previous research aimed to automate the diagnosing process by developing diagnosing models based on tightening theory and calibration of the friction coefficient in specific lab setups. Generalizing these results is difficult and often unsuccessful since friction coefficients vary between lab and production environments. To overcome this problem, this paper presents a novel methodology that builds multi-label classification deep learning models for diagnosing tightening errors using production data. The proposed methodology comprises three key contributions, i.e., the Labrador method, the Model Combo (MoBo) framework, and a heuristic evaluation method. Labrador is an elastic deep learning based sensor fusion method that (1) uses feature encoders to extract features; (2) conducts data-level and/or feature-level sensor fusion in both time and frequency domains; and (3) performs multi-label classification to detect and diagnose tightening errors. MoBo is a configurable and modular framework that supports Labrador in identifying optimal feature encoders. With MoBo and Labrador, one can easily explore and design a bounded search space for sensor fusion strategies (SFSs) and feature encoders. In order to identify the optimal solution within the defined search space, this paper introduces a heuristic method. By evaluating the trade-off between machine learning (ML) metrics (e.g., accuracy, subset accuracy, and F1) and operational (OP) metrics (e.g., inference latency), the proposed method identifies the most suitable solution depending on the requirements of individual use cases. In the experimental evaluation, we adopt the proposed methodology to identify the most suitable multi-label classification solutions for diagnosing tightening errors. To optimize ML metrics, the identified solution achieved 99.69% accuracy, 93.39% subset accuracy, 97.39% F1, and 6.68ms inference latency. To optimize OP metrics, the identified solution achieved 99.66% accuracy, 92.65% subset accuracy, 97.28% F1, and 2.41ms inference latency.
Place, publisher, year, edition, pages
Elsevier, 2023. Vol. 71, p. 59-69
Keywords [en]
Tightening error diagnosing, Smart manufacturing, Deep learning, CNN, Sensor data fusion, Multi-label classification
National Category
Control Engineering Signal Processing Production Engineering, Human Work Science and Ergonomics
Research subject
Computer Science; Industrial Information and Control Systems; Production Engineering
Identifiers
URN: urn:nbn:se:kth:diva-336587DOI: 10.1016/j.jmsy.2023.08.015ISI: 001073676700001Scopus ID: 2-s2.0-85170228617OAI: oai:DiVA.org:kth-336587DiVA, id: diva2:1797235
Projects
TECoSA
Funder
Vinnova, Tecosa
Note
QC 20230915
2023-09-142023-09-142023-10-24Bibliographically approved