kth.sePublications
Change search
Link to record
Permanent link

Direct link
Publications (10 of 119) Show all publications
Zhu, X., Mårtensson, P., Hanson, L., Björkman, M. & Maki, A. (2025). Automated assembly quality inspection by deep learning with 2D and 3D synthetic CAD data. Journal of Intelligent Manufacturing, 36(4), 2567-2582, Article ID e222.
Open this publication in new window or tab >>Automated assembly quality inspection by deep learning with 2D and 3D synthetic CAD data
Show others...
2025 (English)In: Journal of Intelligent Manufacturing, ISSN 0956-5515, E-ISSN 1572-8145, Vol. 36, no 4, p. 2567-2582, article id e222Article in journal (Refereed) Published
Abstract [en]

In the manufacturing industry, automatic quality inspections can lead to improved product quality and productivity. Deep learning-based computer vision technologies, with their superior performance in many applications, can be a possible solution for automatic quality inspections. However, collecting a large amount of annotated training data for deep learning is expensive and time-consuming, especially for processes involving various products and human activities such as assembly. To address this challenge, we propose a method for automated assembly quality inspection using synthetic data generated from computer-aided design (CAD) models. The method involves two steps: automatic data generation and model implementation. In the first step, we generate synthetic data in two formats: two-dimensional (2D) images and three-dimensional (3D) point clouds. In the second step, we apply different state-of-the-art deep learning approaches to the data for quality inspection, including unsupervised domain adaptation, i.e., a method of adapting models across different data distributions, and transfer learning, which transfers knowledge between related tasks. We evaluate the methods in a case study of pedal car front-wheel assembly quality inspection to identify the possible optimal approach for assembly quality inspection. Our results show that the method using Transfer Learning on 2D synthetic images achieves superior performance compared with others. Specifically, it attained 95% accuracy through fine-tuning with only five annotated real images per class. With promising results, our method may be suggested for other similar quality inspection use cases. By utilizing synthetic CAD data, our method reduces the need for manual data collection and annotation. Furthermore, our method performs well on test data with different backgrounds, making it suitable for different manufacturing environments.

Place, publisher, year, edition, pages
Springer Nature, 2025
Keywords
Assembly quality inspection, Computer vision, Point cloud, Synthetic data, Transfer learning, Unsupervised domain adaptation
National Category
Computer Sciences Production Engineering, Human Work Science and Ergonomics
Identifiers
urn:nbn:se:kth:diva-363099 (URN)10.1007/s10845-024-02375-6 (DOI)001205028300001 ()2-s2.0-105002924620 (Scopus ID)
Note

QC 20250506

Available from: 2025-05-06 Created: 2025-05-06 Last updated: 2025-05-19Bibliographically approved
Sabel, D., Westin, T. & Maki, A. (2023). 3D Point Cloud Registration for GNSS-denied Aerial Localization over Forests. In: Image Analysis - 23rd Scandinavian Conference, SCIA 2023, Proceedings: . Paper presented at 23nd Scandinavian Conference on Image Analysis, SCIA 2023, Lapland, Finland, Apr 18 2023 - Apr 21 2023 (pp. 396-411). Springer Nature
Open this publication in new window or tab >>3D Point Cloud Registration for GNSS-denied Aerial Localization over Forests
2023 (English)In: Image Analysis - 23rd Scandinavian Conference, SCIA 2023, Proceedings, Springer Nature , 2023, p. 396-411Conference paper, Published paper (Refereed)
Abstract [en]

This paper presents a vision-based localization approach for Unmanned Aerial Vehicles (UAVs) flying at low altitude over forested areas. We address the task as a point cloud registration problem using local 3D features with the intention to exploit the shape and relative arrangement of the trees. We propose a 3D descriptor called SHOT-N which is an adaptation of the state-of-the-art SHOT 3D descriptor. SHOT-N leverages constraints in the extrinsic parameters of a gimballed, nadir-looking camera. Extensive experiments were performed with semi-simulated point cloud data based on real aerial images over four forested areas. SHOT-N is shown to outperform two state-of-the-art 3D descriptors in terms of the rate of successful registrations. The results suggest a high potential of the approach for aerial localization over forested areas.

Place, publisher, year, edition, pages
Springer Nature, 2023
Keywords
aerial navigation, GNSS-denied, natural environments, point cloud registration, visual navigation
National Category
Robotics and automation Computer graphics and computer vision
Identifiers
urn:nbn:se:kth:diva-338612 (URN)10.1007/978-3-031-31435-3_27 (DOI)2-s2.0-85161464639 (Scopus ID)
Conference
23nd Scandinavian Conference on Image Analysis, SCIA 2023, Lapland, Finland, Apr 18 2023 - Apr 21 2023
Note

Part of ISBN 9783031314346

QC 20231106

Available from: 2023-11-06 Created: 2023-11-06 Last updated: 2025-02-05Bibliographically approved
Fukui, K., Sogi, N., Kobayashi, T., Xue, J.-H. & Maki, A. (2023). Discriminant feature extraction by generalized difference subspace. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(2), 1618-1635
Open this publication in new window or tab >>Discriminant feature extraction by generalized difference subspace
Show others...
2023 (English)In: IEEE Transactions on Pattern Analysis and Machine Intelligence, ISSN 0162-8828, E-ISSN 1939-3539, Vol. 45, no 2, p. 1618-1635Article in journal (Refereed) Published
Abstract [en]

This paper reveals the discriminant ability of the orthogonal projection of data onto a generalized difference subspace (GDS) both theoretically and experimentally. In our previous work, we have demonstrated that GDS projection works as the quasi-orthogonalization of class subspaces. Interestingly, GDS projection also works as a discriminant feature extraction through a similar mechanism to the Fisher discriminant analysis (FDA). A direct proof of the connection between GDS projection and FDA is difficult due to the significant difference in their formulations. To avoid the difficulty, we first introduce geometrical Fisher discriminant analysis (gFDA) based on a simplified Fisher criterion. gFDA can work stably even under few samples, bypassing the small sample size (SSS) problem of FDA. Next, we prove that gFDA is equivalent to GDS projection with a small correction term. This equivalence ensures GDS projection to inherit the discriminant ability from FDA via gFDA. Furthermore, we discuss two useful extensions of these methods, 1) nonlinear extension by kernel trick, 2) the combination of convolutional neural network (CNN) features. The equivalence and the effectiveness of the extensions have been verified through extensive experiments on the extended Yale B+, CMU face database, ALOI, ETH80, MNIST and CIFAR10, focusing on the SSS problem. IEEE

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE), 2023
Keywords
Discriminant analysis, Face recognition, Feature extraction, Fisher criterion, Image recognition, Kernel, Lighting, PCA without data centering, Principal component analysis, subspace representation, Task analysis, Extraction, Fisher information matrix, Job analysis, Neural networks, Discriminant feature extraction, Features extraction, Fisher discriminant analysis, Principal-component analysis, Subspace projection
National Category
Computer graphics and computer vision
Identifiers
urn:nbn:se:kth:diva-323273 (URN)10.1109/TPAMI.2022.3168557 (DOI)000912386000019 ()35439128 (PubMedID)2-s2.0-85128699946 (Scopus ID)
Note

QC 20230124

Available from: 2023-01-24 Created: 2023-01-24 Last updated: 2025-02-07Bibliographically approved
Zhu, X., Björkman, M., Maki, A., Hanson, L. & Mårtensson, P. (2023). Surface Defect Detection with Limited Training Data: A Case Study on Crown Wheel Surface Inspection. In: 56th CIRP International Conference on Manufacturing Systems, CIRP CMS 2023: . Paper presented at 56th CIRP International Conference on Manufacturing Systems, CIRP CMS 2023, Cape Town, South Africa, Oct 24 2023 - Oct 26 2023 (pp. 1333-1338). Elsevier BV
Open this publication in new window or tab >>Surface Defect Detection with Limited Training Data: A Case Study on Crown Wheel Surface Inspection
Show others...
2023 (English)In: 56th CIRP International Conference on Manufacturing Systems, CIRP CMS 2023, Elsevier BV , 2023, p. 1333-1338Conference paper, Published paper (Refereed)
Abstract [en]

This paper presents an approach to automatic surface defect detection by a deep learning-based object detection method, particularly in challenging scenarios where defects are rare, i.e., with limited training data. We base our approach on an object detection model YOLOv8, preceded by a few steps: 1) filtering out irrelevant information, 2) enhancing the visibility of defects, namely brightness contrast, and 3) increasing the diversity of the training data through data augmentation. We evaluated the method in an industrial case study of crown wheel surface inspection in detecting Unclean Gear as well as Deburring defects, resulting in promising performances. With the combination of the three preprocessing steps, we improved the detection accuracy by 22.2% and 37.5% respectively while detecting those two defects. We believe that the proposed approach is also adaptable to various applications of surface defect detection in other industrial environments as the employed techniques, such as image segmentation, are available off the shelf.

Place, publisher, year, edition, pages
Elsevier BV, 2023
Keywords
Automatic Quality Inspection, Computer Vision, Deep Learning, Image Processing, Surface Defect Detection
National Category
Computer graphics and computer vision
Identifiers
urn:nbn:se:kth:diva-343752 (URN)10.1016/j.procir.2023.09.172 (DOI)2-s2.0-85184602644 (Scopus ID)
Conference
56th CIRP International Conference on Manufacturing Systems, CIRP CMS 2023, Cape Town, South Africa, Oct 24 2023 - Oct 26 2023
Note

QC 20240222

Available from: 2024-02-22 Created: 2024-02-22 Last updated: 2025-02-07Bibliographically approved
Zhu, X., Bilal, T., Mårtensson, P., Hanson, L., Björkman, M. & Maki, A. (2023). Towards sim-to-real industrial parts classification with synthetic dataset. In: Proceedings: 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2023. Paper presented at 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2023, Vancouver, Canada, Jun 18 2023 - Jun 22 2023 (pp. 4454-4463). Institute of Electrical and Electronics Engineers (IEEE)
Open this publication in new window or tab >>Towards sim-to-real industrial parts classification with synthetic dataset
Show others...
2023 (English)In: Proceedings: 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2023, Institute of Electrical and Electronics Engineers (IEEE) , 2023, p. 4454-4463Conference paper, Published paper (Refereed)
Abstract [en]

This paper is about effectively utilizing synthetic data for training deep neural networks for industrial parts classification, in particular, by taking into account the domain gap against real-world images. To this end, we introduce a synthetic dataset that may serve as a preliminary testbed for the Sim-to-Real challenge; it contains 17 objects of six industrial use cases, including isolated and assembled parts. A few subsets of objects exhibit large similarities in shape and albedo for reflecting challenging cases of industrial parts. All the sample images come with and without random backgrounds and post-processing for evaluating the importance of domain randomization. We call it Synthetic Industrial Parts dataset (SIP-17). We study the usefulness of SIP-17 through benchmarking the performance of five state-of-the-art deep network models, supervised and self-supervised, trained only on the synthetic data while testing them on real data. By analyzing the results, we deduce some insights on the feasibility and challenges of using synthetic data for industrial parts classification and for further developing larger-scale synthetic datasets. Our dataset † and code ‡ are publicly available.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE), 2023
National Category
Computer graphics and computer vision
Identifiers
urn:nbn:se:kth:diva-337847 (URN)10.1109/CVPRW59228.2023.00468 (DOI)2-s2.0-85170821045 (Scopus ID)
Conference
2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2023, Vancouver, Canada, Jun 18 2023 - Jun 22 2023
Note

Part of ISBN 9798350302493

QC 20231010

Available from: 2023-10-10 Created: 2023-10-10 Last updated: 2025-02-07Bibliographically approved
Maki, A., Kragic, D., Kjellström, H., Azizpour, H., Sullivan, J., Björkman, M., . . . Sundblad, Y. (2022). In Memoriam: Jan-Olof Eklundh. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(9), 4488-4489
Open this publication in new window or tab >>In Memoriam: Jan-Olof Eklundh
Show others...
2022 (English)In: IEEE Transactions on Pattern Analysis and Machine Intelligence, ISSN 0162-8828, E-ISSN 1939-3539, Vol. 44, no 9, p. 4488-4489Article in journal (Refereed) Published
Place, publisher, year, edition, pages
IEEE COMPUTER SOC, 2022
National Category
Computer and Information Sciences
Identifiers
urn:nbn:se:kth:diva-316696 (URN)10.1109/TPAMI.2022.3183266 (DOI)000836666600005 ()
Note

QC 20220905

Available from: 2022-09-05 Created: 2022-09-05 Last updated: 2022-09-05Bibliographically approved
Rixon Fuchs, L., Maki, A. & Gällström, A. (2022). Optimization Method for Wide Beam Sonar Transmit Beamforming. Sensors, 22(19), 7526, Article ID 7526.
Open this publication in new window or tab >>Optimization Method for Wide Beam Sonar Transmit Beamforming
2022 (English)In: Sensors, E-ISSN 1424-8220, Vol. 22, no 19, p. 7526-, article id 7526Article in journal (Refereed) Published
Abstract [en]

Imaging and mapping sonars such as forward-looking sonars (FLS) and side-scan sonars (SSS) are sensors frequently used onboard autonomous underwater vehicles. To acquire information from around the vehicle, it is desirable for these sonar systems to insonify a large area; thus, the sonar transmit beampattern should have a wide field of view. In this work, we study the problem of the optimization of wide transmission beampatterns. We consider the conventional phased-array beampattern design problem where all array elements transmit an identical waveform. The complex weight vector is adjusted to create the desired beampattern shape. In our experiments, we consider wide transmission beampatterns (>= 20 degrees) with uniform output power. In this paper, we introduce a new iterative-convex optimization method for narrowband linear phased arrays and compare it to existing approaches for convex and concave-convex optimization. In the iterative-convex method, the phase of the weight parameters is allowed to be complex as in disciplined convex-concave programming (DCCP). Comparing the iterative-convex optimization method and DCCP to the standard convex optimization, we see that the former methods archive optimized beampatterns closer to the desired beampatterns. Furthermore, for the same number of iterations, the proposed iterative-convex method achieves optimized beampatterns, which are closer to the desired beampattern than the beampatterns achieved by optimization with DCCP.

Place, publisher, year, edition, pages
MDPI AG, 2022
Keywords
autonomous underwater vehicles, sonar, phased antenna arrays, transmit beamforming, convex optimization, beampattern, side-scan sonar, forward-looking sonar, seabed mapping
National Category
Computer graphics and computer vision Vehicle and Aerospace Engineering
Identifiers
urn:nbn:se:kth:diva-320686 (URN)10.3390/s22197526 (DOI)000867028100001 ()36236625 (PubMedID)2-s2.0-85139948286 (Scopus ID)
Note

QC 20221031

Available from: 2022-10-31 Created: 2022-10-31 Last updated: 2025-02-14Bibliographically approved
Rixon Fuchs, L., Gallstrom, A. & Maki, A. (2022). Towards Dense Point Correspondence with PatchMatch in Low-Resolution Sonar Images. In: 2022 IEEE/OES AUTONOMOUS UNDERWATER VEHICLES SYMPOSIUM (AUV): . Paper presented at IEEE/OES Autonomous Underwater Vehicles Symposium (AUV), SEP 19-21, 2022, Singapore, SINGAPORE. Institute of Electrical and Electronics Engineers (IEEE)
Open this publication in new window or tab >>Towards Dense Point Correspondence with PatchMatch in Low-Resolution Sonar Images
2022 (English)In: 2022 IEEE/OES AUTONOMOUS UNDERWATER VEHICLES SYMPOSIUM (AUV), Institute of Electrical and Electronics Engineers (IEEE) , 2022Conference paper, Oral presentation only (Refereed)
Abstract [en]

Robust feature correspondences between 2D sonar imagery are important for perception tasks in the underwater domain such as 3D reconstruction but involve open challenges, in particular, low-resolution as well as the fact that object appearance is view-dependent. Although sonars in the MHz range would allow for higher resolution imagery, in this paper we focus on scenarios with a lower frequency kHz sensor, in which the longer visual range is gained at the sacrifice of image resolution. To this end, we first propose to solve the correspondence task using the PatchMatch algorithm for the first time in sonar imagery, and then propose a method for feature extraction based on IC. We then compare the proposed methods against conventional methods from computer vision. We evaluate our method on data from a lake experiment with objects captured with an FLS sensor. Our results show that the proposed combination of IC together with PatchMatch is well-suited for point feature extraction and correspondence in sonar imagery. Further, we also evaluate the different methods for point correspondence with a 3D object reconstruction task.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE), 2022
Series
IEEE OES Autonomous Underwater Vehicles, ISSN 1522-3167
Keywords
PatchMatch, FLS, feature correspondence, 3D reconstruction
National Category
Computer graphics and computer vision
Identifiers
urn:nbn:se:kth:diva-323585 (URN)10.1109/AUV53081.2022.9965885 (DOI)000896331200018 ()2-s2.0-85143972442 (Scopus ID)
Conference
IEEE/OES Autonomous Underwater Vehicles Symposium (AUV), SEP 19-21, 2022, Singapore, SINGAPORE
Note

QC 20230208

Available from: 2023-02-08 Created: 2023-02-08 Last updated: 2025-02-07Bibliographically approved
Zhu, X., Maki, A. & Hanson, L. (2022). Unsupervised domain adaptive object detection for assembly quality inspection. In: Proceedings 15th CIRP Conference on Intelligent Computation in Manufacturing Engineering, ICME 2021: . Paper presented at 15th CIRP Conference on Intelligent Computation in Manufacturing Engineering, ICME 2021, Naples, 14-16 July 2021 (pp. 477-482). Elsevier BV, 112
Open this publication in new window or tab >>Unsupervised domain adaptive object detection for assembly quality inspection
2022 (English)In: Proceedings 15th CIRP Conference on Intelligent Computation in Manufacturing Engineering, ICME 2021, Elsevier BV , 2022, Vol. 112, p. 477-482Conference paper, Oral presentation with published abstract (Refereed)
Abstract [en]

A challenge to apply deep learning-based computer vision technologies for assembly quality inspection lies in the diverse assembly approaches and the restricted annotated training data. This paper describes a method for overcoming the challenge by training an unsupervised domain adaptive object detection model on annotated synthetic images generated from CAD models and unannotated images captured from cameras. On a case study of pedal car front-wheel assembly, the model achieves promising results compared to other state-of-the-art object detection methods. Besides, the method is efficient to implement in production as it does not require manually annotated data.

Place, publisher, year, edition, pages
Elsevier BV, 2022
Series
Procedia CIRP, ISSN 2212-8271
National Category
Computer Sciences Control Engineering
Identifiers
urn:nbn:se:kth:diva-327337 (URN)10.1016/j.procir.2022.09.038 (DOI)2-s2.0-85142641837 (Scopus ID)
Conference
15th CIRP Conference on Intelligent Computation in Manufacturing Engineering, ICME 2021, Naples, 14-16 July 2021
Note

QC 20230525

Available from: 2023-05-24 Created: 2023-05-24 Last updated: 2023-05-25Bibliographically approved
Sabel, D., Maki, A., Westin, T. & Åsvärn, D. (2022). VISION-BASED LOCALISATION FOR AUTONOMOUS AERIAL NAVIGATION IN GNSS-DENIED SITUATIONS. In: 33rd Congress of the International Council of the Aeronautical Sciences, ICAS 2022: . Paper presented at 33rd Congress of the International Council of the Aeronautical Sciences, ICAS 2022, Stockholm, Sweden, Sep 4 2022 - Sep 9 2022 (pp. 5973-5987). International Council of the Aeronautical Sciences
Open this publication in new window or tab >>VISION-BASED LOCALISATION FOR AUTONOMOUS AERIAL NAVIGATION IN GNSS-DENIED SITUATIONS
2022 (English)In: 33rd Congress of the International Council of the Aeronautical Sciences, ICAS 2022, International Council of the Aeronautical Sciences , 2022, p. 5973-5987Conference paper, Published paper (Refereed)
Abstract [en]

Flight safety is compromised when positioning with Global Navigation Satellite Systems (GNSS) is rendered unusable or unreliable. The cause of such GNSS-denied situations can be signal obstruction, multipath issues from mountains or tall buildings as well as malicious jamming or spoofing. The Swedish company Spacemetric is working together with the Royal Institute of Technology (KTH) in Stockholm to address the challenge of accurate and reliable aerial localisation, which is a key aspect to ensure flight safety for autonomous aerial navigation. The project, called “Autonomous Navigation Support from Real-Time Visual Mapping”, focusses on vision-based methods for aerial positioning in Unmanned Aerial Vehicles navigating in natural environments. This paper present results from the project's ongoing research on vision-based approaches with deep learning as well as a novel approach to localisation which exploits the three-dimensional structure of vegetation and terrain.

Place, publisher, year, edition, pages
International Council of the Aeronautical Sciences, 2022
Keywords
deep learning, GNSS-denied navigation, natural environments, UAV, visual methods
National Category
Robotics and automation
Identifiers
urn:nbn:se:kth:diva-333304 (URN)2-s2.0-85159675534 (Scopus ID)
Conference
33rd Congress of the International Council of the Aeronautical Sciences, ICAS 2022, Stockholm, Sweden, Sep 4 2022 - Sep 9 2022
Note

Part of ISBN 9781713871163

QC 20230801

Available from: 2023-08-01 Created: 2023-08-01 Last updated: 2025-02-09Bibliographically approved
Organisations
Identifiers
ORCID iD: ORCID iD iconorcid.org/0000-0002-4266-6746

Search in DiVA

Show all publications