Submit to Special Issue Submit Abstract to Special Issue Review for Sensors Propose a Special Issue

Journal Menu

Journal Browser

► Journal Browser

Deep-Learning-Based Defect Detection for Smart Manufacturing

Special Issue Editors
Special Issue Information
Published Papers
Planned Papers

A special issue of Sensors (ISSN 1424-8220). This special issue belongs to the section "Fault Diagnosis & Sensors".

Deadline for manuscript submissions: 25 August 2024 | Viewed by 11417

Share This Special Issue

Special Issue Editor

Dr. Iñigo Barandiaran

E-Mail Website
Guest Editor

Vicomtech Foundation, Basque Research and Technology Alliance (BRTA), Mikeletegi 57, 20009 Donostia-San Sebastián, Spain
Interests: computer vision; artificial intelligence; image processing and image understanding; simulation and 3D visualization; photography

Special Issue Information

Dear Colleagues,

Nowadays, artificial intelligence (AI) is becoming more widely used in the smart industry field, due to its ability to enhance production process efficiency, lower expenses, and improve production quality. The smart industry trend relies on the advanced integration of information and communication technologies, such as robotics, AI, Big Data, and the Internet of Things (IoT).

Within the smart industry, defect detection in production systems is one of the most popular applications of AI. By utilizing AI algorithms, such as deep learning, smart production systems are capable of analysing images and videos of production processes, detecting deviations, identifying problems in a timely manner, improving product quality, and predicting maintenance needs.

Implementing smart inspection systems presents unique challenges, including the complexity of the components to be inspected, the availability of training data, the design of agile and robust AI algorithms, and the deployment of these systems within real industrial scenarios. This Special Issue aims to highlight novel and cutting-edge research focused on artificial intelligence applied to industry and production processes.

In particular, submitted papers should clearly show novel contributions and innovative applications covering, among others, any of the following topics:

Machine vision and pattern recognition techniques;
The use of sensors in intelligent industrial quality control applications;
Data augmentation techniques in unfavourable scenarios of the lack or imbalance of data;
Artificial Intelligence techniques for surface defect detection and characterization;
Deployment and integration of intelligent quality control systems using machine vision in real industrial environments.

Dr. Iñigo Barandiaran
Guest Editor

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Sensors is an international peer-reviewed open access semimonthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2600 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Published Papers (11 papers)

Download All Papers

Order results

Result details

Show export options Show export options

Select all

Export citation of selected articles as:

Research

30 pages, 8488 KiB

Open AccessArticle

Improving Rebar Twist Prediction Exploiting Unified-Channel Attention-Based Image Restoration and Regression Techniques

by Jong-Chan Park and Gun-Woo Kim

Sensors 2024, 24(14), 4757; https://doi.org/10.3390/s24144757 - 22 Jul 2024

Viewed by 545

Abstract

Recent research has made significant progress in automated unmanned systems utilizing Artificial Intelligence (AI)-based image processing to optimize the rebar manufacturing process and minimize defects such as twisting during production. Despite various studies, including those employing data augmentation through Generative Adversarial Networks (GANs), the performance of rebar twist prediction has been limited due to image quality degradation caused by environmental noise, such as insufficient image quality and inconsistent lighting conditions in rebar processing environments. To address these challenges, we propose a novel approach for real-time rebar twist prediction in manufacturing processes. Our method involves restoring low-quality grayscale images to high resolution and employing an object detection model to identify and track rebar endpoints. We then apply regression analysis to the coordinates obtained from the bounding boxes to estimate the error rate of the rebar endpoint positions, thereby determining the occurrence of twisting. To achieve this, we first developed a Unified-Channel Attention (UCA) module that is robust to changes in intensity and contrast for grayscale images. The UCA can be integrated into image restoration models to more accurately detect rebar endpoint characteristics in object detection models. Furthermore, we introduce a method for predicting the future positions of rebar endpoints using various linear and non-linear regression models. The predicted positions are used to calculate the error rate in rebar endpoint locations, determined by the distance between the actual and predicted positions, which is then used to classify the presence of rebar twisting. Our experimental results demonstrate that integrating the UCA module with our image restoration model significantly improved existing models in Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index Measure (SSIM) metrics. Moreover, employing regression models to predict future rebar endpoint positions enhances the F1 score for twist prediction. As a result, our approach offers a practical solution for rapid defect detection in rebar manufacturing processes. Full article

(This article belongs to the Special Issue Deep-Learning-Based Defect Detection for Smart Manufacturing)

► Show Figures

Figure 1

17 pages, 5024 KiB

Open AccessArticle

SCAE—Stacked Convolutional Autoencoder for Fault Diagnosis of a Hydraulic Piston Pump with Limited Data Samples

by Oybek Eraliev, Kwang-Hee Lee and Chul-Hee Lee

Sensors 2024, 24(14), 4661; https://doi.org/10.3390/s24144661 - 18 Jul 2024

Viewed by 684

Abstract

Deep learning (DL) models require enormous amounts of data to produce reliable diagnosis results. The superiority of DL models over traditional machine learning (ML) methods in terms of feature extraction, feature dimension reduction, and diagnosis performance has been shown in various studies of fault diagnosis systems. However, data acquisition can sometimes be compromised by sensor issues, resulting in limited data samples. In this study, we propose a novel DL model based on a stacked convolutional autoencoder (SCAE) to address the challenge of limited data. The innovation of the SCAE model lies in its ability to enhance gradient information flow and extract richer hierarchical features, leading to superior diagnostic performance even with limited and noisy data samples. This article describes the development of a fault diagnosis method for a hydraulic piston pump using time–frequency visual pattern recognition. The proposed SCAE model has been evaluated on limited data samples of a hydraulic piston pump. The findings of the experiment demonstrate that the suggested approach can achieve excellent diagnostic performance with over 99.5% accuracy. Additionally, the SCAE model has outperformed traditional DL models such as deep neural networks (DNN), standard stacked sparse autoencoders (SSAE), and convolutional neural networks (CNN) in terms of diagnosis performance. Furthermore, the proposed model demonstrates robust performance under noisy data conditions, further highlighting its effectiveness and reliability. Full article

(This article belongs to the Special Issue Deep-Learning-Based Defect Detection for Smart Manufacturing)

► Show Figures

Figure 1

20 pages, 8845 KiB

Open AccessArticle

Gearbox Fault Diagnosis Method in Noisy Environments Based on Deep Residual Shrinkage Networks

by Jianhui Cao, Jianjie Zhang, Xinze Jiao, Peibo Yu and Baobao Zhang

Sensors 2024, 24(14), 4633; https://doi.org/10.3390/s24144633 - 17 Jul 2024

Viewed by 302

Abstract

Gearbox fault diagnosis is essential in the maintenance and preventive repair of industrial systems. However, in actual working environments, noise frequently interferes with fault signals, consequently reducing the accuracy of fault diagnosis. To effectively address this issue, this paper incorporates the noise attenuation of the DRSN-CW model. A compound fault detection method for gearboxes, integrated with a cross-attention module, is proposed to enhance fault diagnosis performance in noisy environments. First, frequency domain features are extracted from the public dataset by using the fast Fourier transform (FFT). Furthermore, the cross-attention mechanism model is inserted in the optimal position to improve the extraction and recognition rate of global and local fault features. Finally, noise-related features are filtered through soft thresholds within the network structure to efficiently mitigate noise interference. The experimental results show that, compared to existing network models, the proposed model exhibits superior noise immunity and high-precision fault diagnosis performance. Full article

(This article belongs to the Special Issue Deep-Learning-Based Defect Detection for Smart Manufacturing)

► Show Figures

Figure 1

20 pages, 4037 KiB

Open AccessArticle

Temporal-Quality Ensemble Technique for Handling Image Blur in Packaging Defect Inspection

by Guk-Jin Son, Hee-Chul Jung and Young-Duk Kim

Sensors 2024, 24(14), 4438; https://doi.org/10.3390/s24144438 - 9 Jul 2024

Viewed by 484

Abstract

Despite achieving numerous successes with surface defect inspection based on deep learning, the industry still faces challenges in conducting packaging defect inspections that include critical information such as ingredient lists. In particular, while previous achievements primarily focus on defect inspection in high-quality images, they do not consider defect inspection in low-quality images such as those containing image blur. To address this issue, we proposed a noble inference technique named temporal-quality ensemble (TQE), which combines temporal and quality weights. Temporal weighting assigns weights to input images by considering the timing in relation to the observed image. Quality weight prioritizes high-quality images to ensure the inference process emphasizes clear and reliable input images. These two weights improve both the accuracy and reliability of the inference process of low-quality images. In addition, to experimentally evaluate the general applicability of TQE, we adopt widely used convolutional neural networks (CNNs) such as ResNet-34, EfficientNet, ECAEfficientNet, GoogLeNet, and ShuffleNetV2 as the backbone network. In conclusion, considering cases where at least one low-quality image is included, TQE has an F1-score approximately 17.64% to 22.41% higher than using single CNN models and about 1.86% to 2.06% higher than an average voting ensemble. Full article

(This article belongs to the Special Issue Deep-Learning-Based Defect Detection for Smart Manufacturing)

► Show Figures

Figure 1

14 pages, 3566 KiB

Open AccessArticle

LESM-YOLO: An Improved Aircraft Ducts Defect Detection Model

by Runyuan Wen, Yong Yao, Zijian Li, Qiyang Liu, Yijing Wang and Yizhuo Chen

Sensors 2024, 24(13), 4331; https://doi.org/10.3390/s24134331 - 3 Jul 2024

Viewed by 515

Abstract

Aircraft ducts play an indispensable role in various systems of an aircraft. The regular inspection and maintenance of aircraft ducts are of great significance for preventing potential failures and ensuring the normal operation of the aircraft. Traditional manual inspection methods are costly and inefficient, especially under low-light conditions. To address these issues, we propose a new defect detection model called LESM-YOLO. In this study, we integrate a lighting enhancement module to improve the accuracy and recognition of the model under low-light conditions. Additionally, to reduce the model’s parameter count, we employ space-to-depth convolution, making the model more lightweight and suitable for deployment on edge detection devices. Furthermore, we introduce Mixed Local Channel Attention (MLCA), which balances complexity and accuracy by combining local channel and spatial attention mechanisms, enhancing the overall performance of the model and improving the accuracy and robustness of defect detection. Finally, we compare the proposed model with other existing models to validate the effectiveness of LESM-YOLO. The test results show that our proposed model achieves an mAP of 96.3%, a 5.4% improvement over the original model, while maintaining a detection speed of 138.7, meeting real-time monitoring requirements. The model proposed in this paper provides valuable technical support for the detection of dark defects in aircraft ducts. Full article

(This article belongs to the Special Issue Deep-Learning-Based Defect Detection for Smart Manufacturing)

► Show Figures

Figure 1

18 pages, 1812 KiB

Open AccessArticle

Failure Mode Classification for Rolling Element Bearings Using Time-Domain Transformer-Based Encoder

by Minh Tri Vu, Motoaki Hiraga, Nanako Miura and Arata Masuda

Sensors 2024, 24(12), 3953; https://doi.org/10.3390/s24123953 - 18 Jun 2024

Viewed by 422

Abstract

In this paper, we propose a Transformer-based encoder architecture integrated with an unsupervised denoising method to learn meaningful and sparse representations of vibration signals without the need for data transformation or pre-trained data. Existing Transformer models often require transformed data or extensive computational resources, limiting their practical adoption. We propose a simple yet competitive modification of the Transformer model, integrating a trainable noise reduction method specifically tailored for failure mode classification using vibration data directly in the time domain without converting them into other domains or images. Furthermore, we present the key architectural components and algorithms underlying our model, emphasizing interpretability and trustworthiness. Our model is trained and validated using two benchmark datasets: the IMS dataset (four failure modes) and the CWRU dataset (four and ten failure modes). Notably, our model performs competitively, especially when using an unbalanced test set and a lightweight architecture. Full article

(This article belongs to the Special Issue Deep-Learning-Based Defect Detection for Smart Manufacturing)

► Show Figures

Figure 1

16 pages, 2868 KiB

Open AccessArticle

Unsupervised Feature-Construction-Based Motor Fault Diagnosis

by Tsatsral Amarbayasgalan and Keun Ho Ryu

Sensors 2024, 24(10), 2978; https://doi.org/10.3390/s24102978 - 8 May 2024

Cited by 1 | Viewed by 659

Abstract

Any bearing faults are a leading cause of motor damage and bring economic losses. Fast and accurate identification of bearing faults is valuable for preventing damaging the whole equipment and continuously running industrial processes without interruption. Vibration signals from a running motor can be utilized to diagnose a bearing health condition. This study proposes a detection method for bearing faults based on two types of neural networks from motor vibration data. The proposed method uses an autoencoder neural network for constructing a new motor vibration feature and a feed-forward neural network for the final detection. The constructed signal feature enhances the prediction performance by focusing more on a fault type that is difficult to detect. We conducted experiments on the CWRU bearing datasets. The experimental study shows that the proposed method improves the performance of the feed-forward neural network and outperforms the other machine learning algorithms. Full article

(This article belongs to the Special Issue Deep-Learning-Based Defect Detection for Smart Manufacturing)

► Show Figures

Figure 1

24 pages, 3978 KiB

Open AccessArticle

LSTM-Autoencoder Based Anomaly Detection Using Vibration Data of Wind Turbines

by Younjeong Lee, Chanho Park, Namji Kim, Jisu Ahn and Jongpil Jeong

Sensors 2024, 24(9), 2833; https://doi.org/10.3390/s24092833 - 29 Apr 2024

Viewed by 1169

Abstract

The problem of energy depletion has brought wind energy under consideration to replace oil- or chemical-based energy. However, the breakdown of wind turbines is a major concern. Accordingly, unsupervised learning was performed using the vibration signal of a wind power generator to achieve an outlier detection performance of 97%. We analyzed the vibration data through wavelet packet conversion and identified a specific frequency band that showed a large difference between the normal and abnormal data. To emphasize these specific frequency bands, high-pass filters were applied to maximize the difference. Subsequently, the dimensions of the data were reduced through principal component analysis, giving unique characteristics to the data preprocessing process. Normal data collected from a wind farm located in northern Sweden was first preprocessed and trained using a long short-term memory (LSTM) autoencoder to perform outlier detection. The LSTM Autoencoder is a model specialized for time-series data that learns the patterns of normal data and detects other data as outliers. Therefore, we propose a method for outlier detection through data preprocessing and unsupervised learning, utilizing the vibration signals from wind generators. This will facilitate the quick and accurate detection of wind power generator failures and provide alternatives to the problem of energy depletion. Full article

(This article belongs to the Special Issue Deep-Learning-Based Defect Detection for Smart Manufacturing)

► Show Figures

Figure 1

18 pages, 42942 KiB

Open AccessArticle

PCB Defect Detection via Local Detail and Global Dependency Information

by Bixian Feng and Jueping Cai

Sensors 2023, 23(18), 7755; https://doi.org/10.3390/s23187755 - 8 Sep 2023

Cited by 3 | Viewed by 2516

Abstract

Due to the impact of the production environment, there may be quality issues on the surface of printed circuit boards (PCBs), which could result in significant economic losses during the application process. As a result, PCB surface defect detection has become an essential step for managing PCB production quality. With the continuous advancement of PCB production technology, defects on PCBs now exhibit characteristics such as small areas and diverse styles. Utilizing global information plays a crucial role in detecting these small and variable defects. To address this challenge, we propose a novel defect detection framework named Defect Detection TRansformer (DDTR), which combines convolutional neural networks (CNNs) and transformer architectures. In the backbone, we employ the Residual Swin Transformer (ResSwinT) to extract both local detail information using ResNet and global dependency information through the Swin Transformer. This approach allows us to capture multi-scale features and enhance feature expression capabilities.In the neck of the network, we introduce spatial and channel multi-head self-attention (SCSA), enabling the network to focus on advantageous features in different dimensions. Moving to the head, we employ multiple cascaded detectors and classifiers to further improve defect detection accuracy. We conducted extensive experiments on the PKU-Market-PCB and DeepPCB datasets. Comparing our proposed DDTR framework with existing common methods, we achieved the highest F1-score and produced the most informative visualization results. Lastly, ablation experiments were performed to demonstrate the feasibility of individual modules within the DDTR framework. These experiments confirmed the effectiveness and contributions of our approach. Full article

(This article belongs to the Special Issue Deep-Learning-Based Defect Detection for Smart Manufacturing)

► Show Figures

Figure 1

19 pages, 9344 KiB

Open AccessArticle

Optimization of Gearbox Fault Detection Method Based on Deep Residual Neural Network Algorithm

by Zhaohua Wang, Yingxue Tao, Yanping Du, Shuihai Dou and Huijuan Bai

Sensors 2023, 23(17), 7573; https://doi.org/10.3390/s23177573 - 31 Aug 2023

Cited by 2 | Viewed by 1190

Abstract

Because of its long running time, complex working environment, and for other reasons, a gear is prone to failure, and early failure is difficult to detect by direct observation; therefore, fault diagnosis of gears is very necessary. Neural network algorithms have been widely used to realize gear fault diagnosis, but the structure of the neural network model is complicated, the training time is long and the model is not easy to converge. To solve the above problems and combine the advantages of the ResNeXt50 model in the extraction of image features, this paper proposes a gearbox fault detection method that integrates the convolutional block attention module (CBAM). Firstly, the CBAM is embedded in the ResNeXt50 network to enhance the extraction of image channels and spatial features. Secondly, the different time–frequency analysis method was compared and analyzed, and the method with the better effect was selected to convert the one-dimensional vibration signal in the open data set of the gearbox into a two-dimensional image, eliminating the influence of the redundant background noise, and took it as the input of the model for training. Finally, the accuracy and the average training time of the model were obtained by entering the test set into the model, and the results were compared with four other classical convolutional neural network models. The results show that the proposed method performs well both in fault identification accuracy and average training time under two working conditions, and it also provides some references for existing gear failure diagnosis research. Full article

(This article belongs to the Special Issue Deep-Learning-Based Defect Detection for Smart Manufacturing)

► Show Figures

Figure 1

17 pages, 2378 KiB

Open AccessArticle

STMS-YOLOv5: A Lightweight Algorithm for Gear Surface Defect Detection

by Rui Yan, Rangyong Zhang, Jinqiang Bai, Huijuan Hao, Wenjie Guo, Xiaoyan Gu and Qi Liu

Sensors 2023, 23(13), 5992; https://doi.org/10.3390/s23135992 - 28 Jun 2023

Cited by 4 | Viewed by 1712

Abstract

Most deep-learning-based object detection algorithms exhibit low speeds and accuracy in gear surface defect detection due to their high computational costs and complex structures. To solve this problem, a lightweight model for gear surface defect detection, namely STMS-YOLOv5, is proposed in this paper. Firstly, the ShuffleNetv2 module is employed as the backbone to reduce the giga floating-point operations per second and the number of parameters. Secondly, transposed convolution upsampling is used to enhance the learning capability of the network. Thirdly, the max efficient channel attention mechanism is embedded in the neck to compensate for the accuracy loss caused by the lightweight backbone. Finally, the SIOU_Loss is adopted as the bounding box regression loss function in the prediction part to speed up the model convergence. Experiments show that STMS-YOLOv5 achieves frames per second of 130.4 and 133.5 on the gear and NEU-DET steel surface defect datasets, respectively. The number of parameters and GFLOPs are reduced by 44.4% and 50.31%, respectively, while the [email protected] reaches 98.6% and 73.5%, respectively. Extensive ablation and comparative experiments validate the effectiveness and generalization capability of the model in industrial defect detection. Full article

(This article belongs to the Special Issue Deep-Learning-Based Defect Detection for Smart Manufacturing)

► Show Figures

Figure 1

Show export options Show export options

Select all

Export citation of selected articles as:

Displaying articles 1-11

Planned Papers

The below list represents only planned manuscripts. Some of these manuscripts have not been received by the Editorial Office yet. Papers submitted to MDPI journals are subject to peer-review.

Title: Failure modes classification for rolling element bearings using time-domain transformer-based encoder
Authors: Minh Vu Tri, Motoaki Hiraga, Nanako Miura, Arata Masuda* (*Corresponding author)
Affiliation: Kyoto Institute of Technology
Abstract: Existing Transformer models often require transformed data or extensive computational resources, limiting their practical adoption. we propose a simple yet competitive modification of the Transformer model, integrating a trainable noise reduction method specifically tailored for failure mode classification using vibration data in the time domain. Furthermore, we present the key architectural components and algorithms underlying our model, emphasizing interpretability and trustworthiness. Our model is trained and validated using two benchmark datasets: the IMS dataset (4 failure modes) and the CWRU dataset (4 and 10 failure modes). Notably, our model performs competitively, especially when using an unbalanced test set and a lightweight architecture.

Title: Optimizing Automated Optical Inspection: An Adaptive Fusion and Semi-Supervised Self-Learning Approach for Elevated Accuracy and Efficiency in Scenarios with Scarce Labeled Data
Authors: Yu-Shu Ni and Jiun-In Guo
Affiliation: Institute of Electronics, National Yang Ming Chiao Tung University, Hsinchu, Taiwan
Abstract: In the realm of Automatic Optical Inspection (AOI), this study introduces two innovative technical strategies aimed at enhancing the accuracy of object detection models while reducing reliance on extensive annotated datasets. Initially, by establishing a preliminary defect detection workflow and utilizing a dataset collaboratively assembled with a major panel manufacturer in Taiwan, we developed and refined a defect detection model. This process commenced with a preliminary set of 3,579 images spanning 24 categories to construct the model. Subsequently, the model was evaluated on 12,000 ambiguously labeled images to assess its initial performance and verify the accuracy of the annotations. Through data augmentation, annotation refinement, and defect classification techniques, we enhanced the model's accuracy and generalizability, thereby expanding the defect dataset on unlabelled datasets and retraining the model. Moreover, addressing the self-learning needs of AOI inspection, we introduced an Adaptive-Fused Semi-Supervised Self-learning (AFSL) method. This approach, rooted in semi-supervised learning and tailored for Anchor-based object detection models, facilitates the model's self-learning and continuous optimization through a minimal set of labeled data and a larger volume of unlabeled data. The proposed AFSL technique, with its modules of Bounding Box Assigner, Adaptive Training Scheduler, and Data Allocator, enables dynamic threshold adjustments, balanced training between labeled and unlabeled data, and efficient data allocation, significantly boosting the model's accuracy on AOI datasets. This methodology not only elevates the precision and efficiency of AOI object detection but also provides an effective approach for achieving efficient model training with limited labeled data.

Title: Deep learning-based method for fault detection-correction in fused deposition modeling additive manufacturing
Authors: Young Ho Park, Saeed Behseresht, Omar Alejandro Valdez Pastrana, Allen Love
Affiliation: Mechanical Engineering Department, New Mexico State University (NMSU).
Abstract: Additive manufacturing (AM) , also commonly known as 3D printing, is an advanced technique for manufacturing complex three-dimensional (3D) parts by depositing raw material layer by layer. Fused Deposition Modeling (FDM) has gained widespread adoption as a popular method for manufacturing 3D parts, even for heavy-duty industrial applications. However, challenges remain, particularly regarding part quality. Print parameters such as print speed, nozzle temperature, and flow rate can significantly impact the final product’s quality. To address this, implementing a closed-loop quality control system is essential. The proposed system will consistently monitor part surface quality during printing and adjust print parameters upon defect detection. In the proposed study, we try to develop a deep learning-based closed-loop control system, utilizing serial communication and Python programming language.

Journal Menu

Journal Browser

Deep-Learning-Based Defect Detection for Smart Manufacturing

Share This Special Issue

Special Issue Editor

Special Issue Information

Published Papers (11 papers)

Research

Planned Papers

Further Information

Guidelines

MDPI Initiatives

Follow MDPI