kth.sePublications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
A deep learning based sensor fusion method to diagnose tightening errors
KTH, School of Industrial Engineering and Management (ITM), Engineering Design, Mechatronics and Embedded Control Systems.ORCID iD: 0000-0002-4211-3770
KTH, School of Industrial Engineering and Management (ITM), Centres, Innovative Centre for Embedded Systems, ICES. KTH, School of Industrial Engineering and Management (ITM), Engineering Design, Mechatronics and Embedded Control Systems.ORCID iD: 0000-0001-5703-5923
Atlas Copco Industrial Technique AB.
Show others and affiliations
2023 (English)In: Journal of manufacturing systems, ISSN 0278-6125, E-ISSN 1878-6642, Vol. 71, p. 59-69Article in journal (Refereed) Published
Abstract [en]

Modern smart assembly lines commonly include electric tools with built-in sensors to tighten safety-critical joints. These sensors generate data that are subsequently analyzed by human experts to diagnose potential tightening errors. Previous research aimed to automate the diagnosing process by developing diagnosing models based on tightening theory and calibration of the friction coefficient in specific lab setups. Generalizing these results is difficult and often unsuccessful since friction coefficients vary between lab and production environments. To overcome this problem, this paper presents a novel methodology that builds multi-label classification deep learning models for diagnosing tightening errors using production data. The proposed methodology comprises three key contributions, i.e., the Labrador method, the Model Combo (MoBo) framework, and a heuristic evaluation method. Labrador is an elastic deep learning based sensor fusion method that (1) uses feature encoders to extract features; (2) conducts data-level and/or feature-level sensor fusion in both time and frequency domains; and (3) performs multi-label classification to detect and diagnose tightening errors. MoBo is a configurable and modular framework that supports Labrador in identifying optimal feature encoders. With MoBo and Labrador, one can easily explore and design a bounded search space for sensor fusion strategies (SFSs) and feature encoders. In order to identify the optimal solution within the defined search space, this paper introduces a heuristic method. By evaluating the trade-off between machine learning (ML) metrics (e.g., accuracy, subset accuracy, and F1) and operational (OP) metrics (e.g., inference latency), the proposed method identifies the most suitable solution depending on the requirements of individual use cases. In the experimental evaluation, we adopt the proposed methodology to identify the most suitable multi-label classification solutions for diagnosing tightening errors. To optimize ML metrics, the identified solution achieved 99.69% accuracy, 93.39% subset accuracy, 97.39% F1, and 6.68ms inference latency. To optimize OP metrics, the identified solution achieved 99.66% accuracy, 92.65% subset accuracy, 97.28% F1, and 2.41ms inference latency.

Place, publisher, year, edition, pages
Elsevier, 2023. Vol. 71, p. 59-69
Keywords [en]
Tightening error diagnosing, Smart manufacturing, Deep learning, CNN, Sensor data fusion, Multi-label classification
National Category
Control Engineering Signal Processing Production Engineering, Human Work Science and Ergonomics
Research subject
Computer Science; Industrial Information and Control Systems; Production Engineering
Identifiers
URN: urn:nbn:se:kth:diva-336587DOI: 10.1016/j.jmsy.2023.08.015ISI: 001073676700001Scopus ID: 2-s2.0-85170228617OAI: oai:DiVA.org:kth-336587DiVA, id: diva2:1797235
Projects
TECoSA
Funder
Vinnova, Tecosa
Note

QC 20230915

Available from: 2023-09-14 Created: 2023-09-14 Last updated: 2024-12-18Bibliographically approved
In thesis
1. AI Driven Smart Tightening and Feature Management System for Agile Assembly
Open this publication in new window or tab >>AI Driven Smart Tightening and Feature Management System for Agile Assembly
2025 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

The convergence of Industry 4.0, digitization, and the concepts of agile andsmart manufacturing is shaping a future driven by cyber-physical systems (CPS),the industrial internet of things (IIoT), and artificial intelligence (AI). This evo-lution promises unprecedented efficiency, agility, and intelligence in manufac-turing. Correspondingly, these advancements also present new challenges forthe assembly industry. This thesis tackles these challenges in two key areas:trustworthy feature management for agile assembly, and AI-powered tighteningdiagnosis for smart assembly.Agile manufacturing prioritizes flexibility for uncertain markets. To achievethis, assembly device maker are increasingly shifting focus to software thatlargely defines hardware functionality. This shift allows companies to offer plat-forms with customizable features rather than just hardware, which in tern,reduces operational expenses and allows customers to dynamically configuretheir assembly lines via software. This transition also necessitates a trustwor-thy Feature Management System (FMS) to control feature activation throughsoftware licensing. However, existing server-based solutions pose trust issues:sellers fear license abuse, while buyers worry about single points of failure duringserver outages.This thesis contributes a novel permissioned blockchain system designedto address trust concerns in feature management for assembly devices. Theproposed solution combines software licensing for feature control with secureownership transaction records on the blockchain. By leveraging the trust, trans-parency, and security of permissioned blockchain technology, the system ensuressecure and controlled access to license information for authorized parties.Integrating software and physical assembly devices into CPSs enables seam-less data acquisition through communication protocols. Once this data is col-lected, AI becomes a powerful tool for decision-making. In smart tighteningsystems, accurately diagnosing tightening results is critical. Modern assemblylines use advanced electric tightening tools equipped with torque and angletransducers to capture detailed data after each operation, enabling the evalua-tion of tightening performance.Currently, accurate evaluation still requires manual analysis by tighteningexperts, resulting in inefficient quality checks that are limited to small samplesof production units. This thesis introduces innovative deep learning methods toautomate the tightening quality assessment process, achieving expert-level ac-curacy across all manufactured units and reducing the risk of defective productsreaching consumers.Towards AI-powered smart assembly, our initial research contribution fo-cused on developing a sensor fusion approach using convolutional neural networkand transformer-based architectures for diagnosing tightening results throughsupervised learning. However supervised learning requires labeled data, andlabeling tightening results requires significant manual work from tightening ex-perts, leading to a scarcity of labeled datasets. Furthermore, in sensitive assem-bly applications, regulatory constraints may prevent sensor data from leaving theshop floor, where computational resources are often limited. To address thesechallenges, this thesis also contributes novel self-supervised learning, data aug-mentation and data augmentation scheduling methods that reduce reliance onlabeled data and computational resources. These innovations ultimately resultin a robust deep learning solution for diagnosing tightening results in real-worldmanufacturing environments.

Abstract [sv]

Sammansmältningen av Industri 4.0, digitalisering och koncepten kring agil ochsmart tillverkning formar en framtid som drivs av cyber-fysiska system (CPS),industriella sakernas internet (IIoT) och artificiell intelligens (AI). Denna utveck-ling lovar oöverträffad effektivitet, smidighet och intelligens inom tillverkn-ingsindustrin. Samtidigt medför dessa framsteg nya utmaningar för monter-ingsindustrin. Denna avhandling tar itu med dessa utmaningar inom två ny-ckelområden: tillförlitlig funktionshantering för agil montering och AI-drivetåtdragningsdiagnos för smart montering.Agil tillverkning prioriterar flexibilitet för osäkra marknader. För att uppnådetta skiftar tillverkarna av monteringsutrustning i allt högre grad fokus motmjukvara som i stor utsträckning definierar maskinvarans funktionalitet. Dennaförändring gör det möjligt för företag att erbjuda plattformar med anpassnings-bara funktioner snarare än bara maskinvara, vilket i sin tur minskar driftskost-naderna och gör det möjligt för kunder att dynamiskt konfigurera sina mon-teringslinjer via mjukvara. Denna övergång kräver dock också ett tillförlitligtfunktionshanteringssystem (FMS) för att kontrollera funktionsaktivering genommjukvarulicensering. Befintliga serverbaserade lösningar medför förtroendeprob-lem: säljare fruktar licensmissbruk, medan köpare oroar sig för enskilda felpunk-ter vid serveravbrott.Denna avhandling bidrar med ett nytt tillståndsbaserat blockkedjesystemsom är utformat för att hantera förtroendeproblem inom funktionshanteringför monteringsenheter. Den föreslagna lösningen kombinerar mjukvarulicenser-ing för funktionskontroll med säkra ägartransaktionsregister på blockkedjan.Genom att utnyttja den tillit, transparens och säkerhet som finns i tillståndsbase-rad blockkedjeteknik säkerställer systemet säker och kontrollerad åtkomst tilllicensinformation för behöriga parter.Integreringen av mjukvara och fysiska monteringsenheter i CPS möjliggörsömlös datainsamling genom kommunikationsprotokoll. När dessa data samlasin blir AI ett kraftfullt verktyg för beslutsfattande. I smarta åtdragningssystemär korrekt diagnos av åtdragningsresultat avgörande. Moderna monteringslinjeranvänder avancerade elektriska åtdragningsverktyg utrustade med moment ochvinkelgivare för att fånga detaljerad data efter varje operation, vilket möjliggören utvärdering av åtdragningsprestanda.För närvarande kräver en korrekt utvärdering fortfarande manuell analys avåtdragningsexperter, vilket leder till ineffektiva kvalitetskontroller som begränsastill små prover av produktionsenheter. Denna avhandling introducerar innova-tiva metoder för djupinlärning för att automatisera kvalitetsbedömningsprocessenför åtdragning, uppnår expertliknande noggrannhet över alla tillverkade enheteroch minskar risken för att defekta produkter når konsumenterna.Mot AI-drivet smart montering fokuserade vårt initiala forskningsbidrag påatt utveckla en sensorfusionsmetod med hjälp av konvolutionella neurala nätverkoch arkitekturer baserade på transformatorer för att diagnostisera åtdragnings-resultat genom övervakat lärande. Övervakat lärande kräver dock märktadata, och märkning av åtdragningsresultat kräver betydande manuellt arbetefrån åtdragningsexperter, vilket leder till brist på märkta dataset. Dessutomkan regleringsbegränsningar i känsliga monteringsapplikationer hindra sensor-data från att lämna fabriken, där datorkapaciteten ofta är begränsad. Föratt hantera dessa utmaningar bidrar denna avhandling även med nya metoderför självövervakat lärande, dataaugmentation och schemaläggning av dataaug-mentation som minskar beroendet av märkta data och datorkapacitet. Dessainnovationer resulterar slutligen i en robust djupinlärningslösning för att diag-nostisera åtdragningsresultat i verkliga tillverkningsmiljöer.

Place, publisher, year, edition, pages
Stockholm: KTH Royal Institute of Technology, 2025. p. 101
Series
TRITA-ITM-AVL ; 2024:27
Keywords
Smart Tightening, Agile Tightening, Permissioned Blockchain, Feature Management System, AI, Deep Learning, Sensor Fusion, Tightening Result Diagnosis
National Category
Engineering and Technology
Research subject
Machine Design
Identifiers
urn:nbn:se:kth:diva-357850 (URN)978-91-8106-122-2 (ISBN)
Public defence
2025-01-24, Sal Gladan https://kth-se.zoom.us/j/66216944295, Brinellvägen 85, Stockholm, Stockholm, 09:00 (English)
Opponent
Supervisors
Projects
Tecosa
Available from: 2024-12-18 Created: 2024-12-18 Last updated: 2025-02-04Bibliographically approved

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full textScopus

Authority records

Tang, LifeiFeng, LeiTörngren, Martin

Search in DiVA

By author/editor
Tang, LifeiFeng, LeiTörngren, Martin
By organisation
Mechatronics and Embedded Control SystemsInnovative Centre for Embedded Systems, ICES
In the same journal
Journal of manufacturing systems
Control EngineeringSignal ProcessingProduction Engineering, Human Work Science and Ergonomics

Search outside of DiVA

GoogleGoogle Scholar

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 186 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf