Next Article in Journal
Computational Simulation of a Femoral Nail Fracture
Next Article in Special Issue
Study on Friction Characteristics of AA7075 Aluminum Alloy under Pulse Current-Assisted Hot Stamping
Previous Article in Journal
Failure Analysis of a Femoral Cephalomedullary Nail
Previous Article in Special Issue
Research on the 2A11 Aluminum Alloy Sheet Cyclic Tension–Compression Test and Its Application in a Mixed Hardening Model
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

An Algorithm for Real-Time Aluminum Profile Surface Defects Detection Based on Lightweight Network Structure

1
School of Physics and Electronic Science, Changsha University of Science and Technology, Changsha 410114, China
2
School of Computer and Communications Engineering, Changsha University of Science and Technology, Changsha 410114, China
*
Author to whom correspondence should be addressed.
Metals 2023, 13(3), 507; https://doi.org/10.3390/met13030507
Submission received: 9 February 2023 / Revised: 24 February 2023 / Accepted: 27 February 2023 / Published: 2 March 2023
(This article belongs to the Special Issue Aluminum Alloys and Aluminum-Based Matrix Composites)

Abstract

:
Surface defects, which often occur during the production of aluminum profiles, can directly affect the quality of aluminum profiles, and should be monitored in real time. This paper proposes an effective, lightweight detection method for aluminum profiles to realize real-time surface defect detection with ensured detection accuracy. Based on the YOLOv5s framework, a lightweight network model is designed by adding the attention mechanism and depth-separable convolution for the detection of aluminum. The lightweight network model improves the limitations of the YOLOv5s framework regarding to its detection accuracy and detection speed. The backbone network GCANet is built based on the Ghost module, in which the Attention mechanism module is embedded in the AC3Ghost module. A compression of the backbone network is achieved, and more channel information is focused on. The model size is further reduced by compressing the Neck network using a deep separable convolution. The experimental results show that, compared to YOLOv5s, the proposed method improves the mAP by 1.76%, reduces the model size by 52.08%, and increases the detection speed by a factor of two. Furthermore, the detection speed can reach 17.4 FPS on Nvidia Jeston Nano’s edge test, which achieves real-time detection. It also provides the possibility of embedding devices for real-time industrial inspection.

1. Introduction

Due to their excellent thermal conductivity and moisture resistance, aluminum profiles have become an important primary material for buildings, vehicles, ships, houses, and other fields. With the rapid development of related industries, the demand for high-quality aluminum profiles is also increasing. Surface defects on aluminum profiles directly affect the quality of products. Therefore, it is significant to detect those defects during their production.
It is difficult to use traditional manual visual inspection to ensure the accuracy of inspection results and inspection efficiency because manual processes can produce a series of problems, such as inefficiency and human physiological fatigue [1]. Some scholars have applied machine learning methods for industrial defect recognition to solve those problems. Yu et al. [2] utilized SVM (support vector machine) to classify wood surface defects. The recognition accuracy of the back propagation neural network model proposed by the authors was 92.7% and 92.0% in the training and test sets, respectively. Hu et al. [3] proposed an algorithm based on ellipse fitting with distance thresholding to detect crater defects on steel shell surface. Elliptical fitting of the extracted inner circle curve was performed, and thus there was high accuracy and detection efficiency for crater defects. You et al. [4] identified crack defects of 0.15 mm using the C-scanning method. This approach can only identify crack defects and has its limitations. K et al. [5] realized the detection of internal defects in carbon-fiber-reinforced plastics and glass-fiber-reinforced plastics using recurrence methods and C-scans. Chen et al. [6] presented smooth filtering to detect steel plate surface defects. Wang et al. [7] use SUSAN operator to detect the edges of the foil image and obtain the threshold aluminum foil image to determine the effective area of the foil in the image. The localization and identification of defects on the surface of aluminum foil were achieved. Although the abovementioned works have achieved some good results in surface defect detection, there are still some limitations, such as poor robustness and weak adaptability.
With the convolutional neural networks (CNNs) proposed, deep learning, which overcomes the limitations of machine learning methods, has been widely used for surface defect detection [8]. Deep-learning-based target detection algorithms are mainly divided into two categories. One is a two-stage classification, and the representative algorithms include R-CNN (regions with CNN features) [9], Fast R-CNN(fast region-based CNN) [10], and Faster R-CNN [11]. These algorithms are applied to the creation and the classification of candidate boxes. The other is single-stage classification, and the representative algorithms include SSD (Single Shot MultiBox Detector) [12], YOLO (You Only Look Once) [13,14,15,16], CenterNet [17], and Retinanet [18]. These algorithms generate class probability values and coordinates of the position of the target object during the creation of a candidate frame. The final detection result can be obtained directly after detecting the target. Fu et al. [19] proposed an end-to-end model based on SqueezeNet to achieve steel strip detection under inhomogeneous illumination with a detection speed of more than 100 fps. However, a dataset with insignificant difference in defect target size was used. Li et al. [20] have improved the network structure of YOLO and achieved an accurate detection of steel strips with 95.86% mAP for defects. Yang et al. [21] have realized the detection of surface defects in automotive pipe joints based on wavelet decomposition and convolutional neural networks. Amin et al. [22] fulfilled the detection of surface defects in steel based on U-NET [23]. Defects can be detected quickly, but the detection accuracy is only 0.731. Zhang et al. [24] proposed the MRSDI-CNN algorithm, which combined SSD with YOLOv3 for the recognition of surface defects on steel rails. The detection speed was improved to a certain extent, but the real-time detection is not realized on embedded devices. Chen et al. [25] have fulfilled the recognition of steel rail surface defects based on Faster R-CNN with 97.8% mAP of blemishes. However, it is not real-time to detection of surface defects. Y et al. [26] implemented the defect depth detection of 3D woven composites using Fully Convolutional Neural Network and recurrence methods. Zhou et al. [27] implemented microtubule defect detection on wafer surface by embedding DA attention module in YOLOv5. The algorithm model has good detection capability on small target defects; however, it is large in size and cannot achieve real-time detection. A CNN-based detection algorithm can be accurate regarding aluminum surface defects, but the detection speed cannot meet industrial inspection needs.
With the development of lightweight network technology, many scholars have realized the real-time detection of surface defects of aluminum profiles by adding a lightweight network to the detection algorithm. Ma et al. [28] implemented the detection of surface defects on aluminum strips by embedding a Ghost module with union attention mechanism in YOLOv4 network. The method achieved an mAP value of 94.68%, a model volume reduction of 80.41%, a threefold increase in detection speed, and better performance than the YOLOv4 model. However, the model size of the algorithm is 48.5 MB and the detection speed is 20.749 f/s, so there is still room for improvement. Wang et al. [29] proposed an improved MS-YOLOv5 model based on the YOLOv5 algorithm. A multi-stream network was present as the first detection head of the algorithm, and the Neck layer was optimized. The model recognition ability and model localization extraction at different sizes were improved. However, the mAP of the model is 87.4%, which cannot meet the needs of industrial inspection. Yang et al. [30] achieved accurate identification of dirty spots by embedding FPN structures in Faster R-CNN to improve the model’s ability to extract feature information about defects. There were strong limitations for the algorithm. It was used to only identify defects within a single category under specific conditions, and is not suitable for surface defect detection of aluminum profiles in industrial manufacturing processes. Li et al. [31] implemented the detection of 10 kinds of aluminum profile surface defects based on migration learning, and the classification accuracy reached 98.47%. However, the speed of detection was not mentioned. Wu et al. [32] proposed a defect detection model based on YOLO X, which replaced the original CSP-DarkNet with CSP-ResNeXt and integrated an attention mechanism. The algorithm achieved 90.69% mAP with a detection speed of 33.6 FPS. Although the above work has achieved some good results in the detection of aluminum surface defects, the detection speed and detection accuracy still need to be further improved.
Deep-learning-based defect detection algorithms can achieve high accuracy rates for specific datasets. As the size of defects on the surface of aluminum profiles varies greatly, those methods fail to achieve good results, and real-time detection is difficult to implement in embedded devices. Therefore, this study designs a novel lightweight network based on the YOLOv5s algorithm to realize a real-time defect detection of aluminum profile surfaces on an embedded system with promising detection accuracy. Depth-separable convolution and Ghostconv are used to compress the lightweight network model, and an attention mechanism is embedded in the backbone network to increase the attention of the network to channel information. As a result, the detection accuracy of the algorithm is improved. The proposed algorithm of the novel lightweight network can reach an accurate real-time detection on embedded devices in this study. Specific innovation points are as follows:
(1)
A lightweight model based on YOLOv5s is proposed. Compared with the YOLOv5s algorithm, the proposed model greatly improves the speed and accuracy, while the model size is greatly reduced to facilitate the deployment of edge devices.
(2)
GCANet is constructed by combining a Ghost module and attention mechanism, which significantly improves the model detection speed, reduces memory consumption, and ensures model accuracy. Moreover, the Attention mechanism module is embedded in the backbone network, which mainly enhances the ability of the backbone network to focus on the channel information.
(3)
In the neck network of the lightweight model, the regular convolution is replaced by the depthwise separable convolution, which greatly compresses the model size and further improves the detection speed.

2. Image Preprocessing and Datasets

Insufficient training samples during the training process can lead to low detection accuracy, overfitting, and low robustness. The number of images is increased by appropriate enhancement of the original images to effectively solve the problem of insufficient training samples [33]. Four typical types of defects on the surface of an aluminum profile are scuffing, soiling, folds, and pinholes, as shown in Figure 1. For the dataset, the pixel size of each image is 640 × 480. A pinhole is caused by the formation of tiny pores during the solidification of aluminum, with an average pixel size of 20 × 20. According to the definition in the literature [34], a pinhole with less than 1.23% of annotated pixels is a small object. Dirt is introduced by the contamination of equipment lubricants. Scratches are caused by relative friction between aluminum and equipment during processing and production. Folds are caused by unbalanced forces during aluminum processing and production. Lin et al. [35] have improved the robustness of defect detection using Gaussian filtering for noise reduction regarding the target. Simonyan et al. [36] processed the images by random flip, rotate, and crop to effectively expand the dataset. This easily leads to missed detection and false detection for low-resolution small targets, the presence of few available features, and high positioning accuracy requirements and aggregation. To enhance the semantic information of small targets, this paper adopts noise reduction by utilizing Gaussian low-pass filtering during copy-pasting in specific regions. The pinholes generated by copy-pasting and Gaussian blur techniques have more effective feature information. Following this, the images are expanded by random flip, rotation, and cutout. The cutout is able to further enhance the localization capability of the model by requiring the model to identify objects from a local view and adding information about other samples to the cut region. Color space transformation generally eliminates lighting, luminance and color differences. These image preprocessing operations increase the number of training datasets to make them as diverse as possible, which in turn improves the generalization ability and robustness of the model. The results of the expanded images are shown in Figure 2. After image enhancement and expansion, the dataset reached 4400 images. Among the dataset, 3600 images are randomly selected for the training set, 400 images for the testing set, and the remaining 400 images for the validation set.

3. Description of Methodology

3.1. Network Architecture

In this study, a lightweight model network structure based on the four basic structural frameworks of YOLOv5s is proposed, and consists of Input, GCANet backbone, Neck, and Prediction. Through the lightweight modules with embedded attention mechanisms, real-time accurate detection of surface defects on aluminum profiles is achieved.
Figure 3 shows the lightweight model network structure based on YOLOv5s. In the Input layer, the input image is resized to 640 × 640 × 3, and is input to the GCANet backbone. The attention mechanism is embedded in the C3Ghost module to improve ability of the model to focus on channel information and spatial information. Three scale feature maps, (80 × 80) (40 × 40) (20 × 20), are extracted at different levels. Following this, based on the DwConv module, the images are inputted to Neck for further compression of the model. Finally, detection is performed in Prediction.

3.2. GCANet Backbone Structure

The GCANet backbone architecture consists of the CBL module, Ghost module, and AC3Ghost module. The CBL module consists of Conv, BatchNorm, and Leaky relu. The ghost module is from GhostNet, proposed by Huawei in 2020. Compared to traditional convolution, Ghost convolution is divided into two steps, which can effectively reduce the amount of computation and number of parameters. Firstly, the standard convolution is used to compute and obtain m feature maps with fewer channel features, then s feature maps are generated using cheap linear operations. Secondly, the two feature maps are concatenated to obtain the new output of m s feature maps. The structure of the Ghost module is shown in Figure 4. In standard convolution, the number of convolution kernels is assumed to be n , the size of the input feature map is h w c , the output feature map is n h w , and the convolution kernel is k k . The model floating-point computations for standard convolution and Ghost convolution are q 1 and q 2 , respectively.
q 1 = n h w c k k
q 2 = n s h w c k k + s 1 n s h w d d
where c denotes the number of channels of the input image, kk denotes the size of the convolution kernel of the standard convolution operation, h and w are the height and width of the original feature map by Ghost convolution, h and w denotes the height and width of the original feature map generated by Ghost convolution, dd is the size of the convolution kernel of the linear operation, and s<<c.
The comparison of the computation of the standard convolution operation and the Ghost module is shown in (3). A comparison of the parametric quantities of the two convolutions is shown in (4). From Equations (3) and (4), it can be seen that when k and d are equal in size, the number of parameters and the computational effort for feature extraction of Ghost convolution is about 1/ s for that of the standard convolution.
r s = n h w c k k n s h w c k k + s 1 n s h w d d = c k k 1 s c k k + s 1 s d d s c s + c 1 s
r c = n c k k n s c k k + s 1 n s d d s c s + c 1 s

3.3. AC3Ghost Structure

Adding attention mechanisms to neural networks can effectively improve the performance of network feature extraction. Hu et al. [37] proposed an SE attention mechanism to establish spatial correlation in feature maps. Hou et al. [38] proposed the CA attention mechanism to integrate spatial coordinate information into feature maps effectively. Woo [39] proposed the CBAM attention mechanism to pay attention to channel and spatial information. To effectively utilize the channel and spatial information, this paper proposes an AC3Ghost module consisting of the CBAM attention mechanism and C3Ghost module, and the CBAM attention mechanism is embedded in C3Ghost. The structure of the AC3Ghost module is shown in Figure 5. When the data processed by the Ghost are input to AC3Ghost, the AC3Ghost module is divided into two branches to process in parallel, one for hierarchical feature fusion by multiple Ghost Bottleneck stacks and three 1 × 1 convolution modules, and the other for reducing the number of channels by only one 1 × 1 convolution module. Following this, feature maps of the two branches are fused as output feature maps by concat, and the CBAM attention mechanism focuses on the channel and spatial information. Finally, it passes through a 1 × 1 convolution module.

3.4. DwConv Module

Depthwise separable convolution (DwConv) was proposed in MobileNet [40] in 2017. DwConv reduces the number of parameters needed during the convolution calculation and improves the efficiency of convolution by splitting the standard convolution in the spatial dimension and channel dimension. As shown in Figure 6, the DwConv module structure is divided into two main processes of Depthwise Convolution and Pointwise Convolution. One convolution kernel of Depthwise Convolution is responsible for one channel. One channel is convolved by only one convolution kernel. The process producing the Pointwise Convolution is very similar to regular convolution. It has a convolution kernel size of 1 × 1, and is weighted in the direction of the map depth from the previous step to generate a new feature map. The computational complexity of a regular convolution C C o n v is shown in Equation (5), and the computational complexity of a depth-separable convolution C s e p a r a b l e C o n v is shown in Equation (6). The ratio of the computational cost of deep separable convolution to that of standard convolution is shown in Equation (7). Experiments show [32] that the computation is 8–9 times less than the standard convolution when the convolution kernel size of DwConv is set to 3 × 3.
C C o n v = D o u t   1 D o u t   2 D k 1 D k 2 C o u t   C i n  
C s e p a r a b l e C o n v   = D o u t   1 D o u t   2 D k 1 D k 2 C i n   + D o u t   1 D o u t   2 C o u t   C i n  
C S e p a r a b l e   C o n v   C C o n v   = D o u t   1 D o u t   2 D k 1 D k 2 C i n   + D o u t   1 D o u t t   C o u t   C i n   D o u t l   D o u t   2 D k 1 D k 2 C o u t   C i n  
where   D in 1 , D in 2 are the input dimensions, D out   1 , D out   2 are the output dimensions, D k 1 , D k 2 are the convolution kernel size, C in   is the number of input channels, and C out   is the number of output channels.

4. Experiments and Discussion

4.1. Experimental Environment

All experiments were performed on a CPU with NVIDIA GeForce RTX 3090 24 GB GPU and Intel i7–12700. The computing software environment was set to python 3.8, CUDA Version 11.6, and the compiler was PyTorch 1.11. In the network training, this study took a batch size of 64, a learning rate of 0.001, an epoch of 500, and an SGD momentum is 0.937.

4.2. Evaluation of Model Performance

Average precision (AP) indicates the accuracy of categories. The mAP is the average of AP, representing the averaged accuracy of all categories. [email protected] indicates the mAP with an IOU greater than 0.5, while [email protected]:0.95 indicates the mAP with an IOU at [0.5,0.95]. The IOU is shown in Equation (8). Precision measures the exactness of classification, as shown in Equation (9). AP and mAP are shown in Equations (10) and (11), respectively. In Equations (8)–(11), b o x g t is the ground truth of the defect; b o x p is the predicted area of the defect; TP and FP are, respectively, the numbers of true-positive cases and false-positive cases; and n is the number of detection classes. TN and FN are, respectively, the numbers of true-negative cases and false-negative cases, and FPS is used to evaluate the detection speed of the model. In this paper, [email protected], [email protected]:0.95, FPS, and model size are chosen to evaluate the experimental method.
I O U   b o x   g t ,   b o x   p = b o x g t b o x p b o x g t b o x p
Precision   = T P T P + F P
A P = i = 1 n P i n
m A P = i = 1 k A P i k

4.3. Test Result of Defect Detection

This paper adopts a fivefold cross-validation method to test the algorithm’s accuracy. The dataset is divided into five groups, four of them are used as training data, and the remaining one as the test data. Experiments were conducted, with the results shown in Table 1. The experiments with the best mAP in Table 1 will also be used for the subsequent comparison experiments. The experiments show that the Precision, Recall and [email protected] of the fold reach 99.45, 100, and 99.37, respectively, which has a better performance, because the folds are distinctive, fixed in shape, and easy to locate. Similarly, the accuracy and detection rate of this paper’s method for both types of scratches and dirt are very high, and there are few cases of missed detection. These two defects have distinctive features and slight shape variations, and are less subject to interference from the background and other defects. However, for Pinhole, which has fewer pixel points, the algorithm captures and displays less texture information. In this paper, copy-pasting and Gaussian blur techniques are used to generate more feature information to meet the model training requirements. The results showed that the AP of the pinhole reached 82.45%. The various defect detection visual results of image detection are shown in Figure 7. All four defects can be detected accurately with a confidence score above 0.8.

4.4. Comparison of the Effect of Different Defect Detection Algorithms

The proposed method in this paper is compared with the following single-stage algorithms: SSD, YOLOv3-tiny, YOLOv4-tiny, YOLOv5-Mobilenetv3, YOLOv5-Shufflenetv2, and YOLOv5. The visualization results of the detection results of different algorithms are shown in Figure 8. The features of [email protected], [email protected]:0.95, Model Size, and Detection Speed are compared, as shown in Table 2. Compared with the other six algorithms, the proposed algorithm has the best performance in [email protected] and [email protected]:0.95 with 94.85 and 73.36. YOLOv5-Mobilenetv3 and YOLOv5-Shufflenetv2 have good detection speed, but the [email protected] is below 90% and cannot achieve the accurate detection of surface defects in aluminum profiles. From the analysis of the results, it can be seen that the proposed method possesses great advantages in the accuracy and real-time detection of surface defects in aluminum profiles, and the detection speed reaches 136 FPS.

4.5. Ablation Study

To understand well the contribution of the improved module to the defect detection effect, a large number of ablation experiments were performed to further verify the YOLOv5s. The results of the ablation study are exhibited in Table 3, where the baseline is the YOLOv5s after data enhancement is performed.
As shown in Table 3, image preprocessing has a good effect on the model’s accuracy. Image preprocessing can increase the number and diversity of images and enrich the characteristics of defects. The [email protected] has been increased by 2.77%. Adding a GCANet Backbone Network Structure on top of the proposed lightweight model can significantly reduce the model’s size and improve the model’s detection capability, and mAP is increased by 0.81%, because ghostconv in GCANet network can reduce a lot of redundant information by linear transformation. Continually adding the AC3Ghost structure after the GCANet Backbone, the detection accuracy of the model is improved further, and the model mAP is increased by 1.06%, but the model size is also slightly increased by 0.3 MB. The DwConv module improves the standard convolution by splitting the correlation between spatial and channel dimensions, reducing the number of parameters needed for the convolution calculation and improving the efficiency of the convolution. For DwConv Module, the model size is reduced by 0.4 MB, and the mAP was slightly dropped from 94.96% to 94.84%, only decreased by 0.12%. At the end, the overall mAP of the model was increased by 0.94%. These results verify that the proposed network model is effective for detecting surface defects in aluminum strips, and that the detection accuracy can be ensured while the model’s size is significantly reduced.

4.6. Edge Testing

To verify the detection of the improved algorithm on the embedded platform, an edge surface defect-detection system was built, as shown in Figure 9 and Table 4. The system consists of an LED light source, a CCD image sensor, a 7-inch touch screen, an Nvidia Jeston Nano, an encoder, a conveyor belt, and two power supplies. The testing sample arrives at the center of the CCD image sensor via a conveyor belt. The CCD image sensor detects the sample and displays the results on the Touch screen. The experimental results show that the edge surface defect-detection system can achieve real-time detection of surface defects on aluminum profiles with a detection speed of 17.4 FPS with good robustness. In summary, the proposed method takes into account both accuracy and real-time operation, and can achieve good detection results in the detection of defects in the industrial production of aluminum profiles.

5. Conclusions

Surface defects on aluminum profiles directly affect their quality. Advanced inspection processes and methods can ensure the accuracy of detection results with high-efficiency detection process. In this paper, a lightweight network model was proposed by adding an attention mechanism and depth-separable convolution for the detection of surface defects in an aluminum strip. By combining the ghost module and the attention mechanism, a new backbone was built. The model size was reduced by 6.2 MB and the mAP was increased by 4.64%. In the neck network of the lightweight model, the regular convolution was replaced by the deeply separable convolution. The model size was further compressed to 7.8 MB. Compared with other object detections and lightweight model experiments, the proposed algorithm has better real-time performance and accuracy than other single-stage detection algorithms. The detection accuracy [email protected] is 94.85, [email protected]:0.95 is 73.36, and the detection speed is 136.98 FPS. Furthermore, in edge testing, the proposed algorithm in the present work can achieve real-time detection. Experiments show that it can detect the surface defects of aluminum profiles in real time with guaranteed high accuracy. In addition, the algorithm has high scalability and can be extended to other fields such as PCB surface defects. In the future, this research work will be focused on continuing to improve pinhole detection accuracy in complex contexts. At the same time, we will continue to carry out research efforts on the types of aluminum profile surface defects to achieve the identification of more defect types.

Author Contributions

Conceptualization, J.T. and S.L.; methodology, J.T.; software, J.T.; validation, J.T., S.L. and D.Z.; formal analysis, J.T.; investigation, J.T.; resources, W.Z.; data curation, J.T.; writing—original draft preparation, B.Z.; writing—review and editing, D.Z.; visualization, J.T.; supervision, L.T.; project administration, J.T.; funding acquisition, J.T. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by the Open Research Fund of Hunan Provincial Key Laboratory of Flexible Electronic Materials Genome Engineering under grant (No. 202015).

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

The data presented in this study are available on request from the corresponding author.

Acknowledgments

The authors would like to acknowledge the anonymous reviewers and editors whose thoughtful comments helped to improve this manuscript.

Conflicts of Interest

The authors declare they have no conflict of interest.

References

  1. Thomas, S.S.; Gupta, S.; Subramanian, V.K. Smart surveillance based on video summarization. In Proceedings of the IEEE Region 10 Symposium (TENSYMP), Cochin, India, 14–16 July 2017; pp. 1–5. [Google Scholar]
  2. Yu, H.; Liang, Y.; Liang, H.; Zhang, Y. Recognition of wood surface defects with near infrared spectroscopy and machine vision. J. For. Res. 2019, 30, 2379–2386. [Google Scholar] [CrossRef]
  3. Hu, H.; Xu, D.; Zheng, X.; Zhang, B. Pit defect detection on steel shell end face based on machine vision. In Proceedings of the 2020 IEEE 4th Information Technology, Networking, Electronic and Automation Control Conference (ITNEC), Chongqing, China, 12–14 June 2020. [Google Scholar]
  4. You, Y.; Xiao, Z.; Chun-Guang, X.U.; Xiao, D.G.; Yi-Xin, H.U.; Pei-Lu, L.I.; Guo, C.Z. Robotic NDT for Turbine Blades Based on the Transverse Waves. In Proceedings of the 2017 Far East NDT New Technology & Application Forum (FENDT), Xi’an, China, 22–24 June 2017; pp. 18–22. [Google Scholar]
  5. Ciecielag, K.; Kecik, K.; Skoczylas, A.; Matuszak, J.; Korzec, I.; Zaleski, R. Non-Destructive Detection of Real Defects in Polymer Composites by Ultrasonic Testing and Recurrence Analysis. Materials 2022, 15, 7335. [Google Scholar] [CrossRef] [PubMed]
  6. Chen, N.; Sun, J.; Wang, X.; Huang, Y.; Li, Y.; Guo, C. Research on surface defect detection and grinding path planning of steel plate based on machine vision. In Proceedings of the 2019 14th IEEE Conference on Industrial Electronics and Applications (ICIEA), Xi’an, China, 19–21 June 2019; pp. 1748–1753. [Google Scholar]
  7. Hui, W.; Chunhua, G.; Xiangxu, X.; Yongfa, L.; Yuji, W. Study on Edge Detection Method of Aluminum Foil Image. In Proceedings of the 2017 International Conference on Computer Systems, Electronics and Control (ICCSEC), Dalian, China, 25–27 December 2017; pp. 1008–1010. [Google Scholar]
  8. Indolia, S.; Goswami, A.; Mishra, S.; Asopa, P. Conceptual understanding of convolutional neural network-a deep learning approach. Proc. Comput. Sci. 2018, 132, 679–688. [Google Scholar] [CrossRef]
  9. Girshick, R.; Donahue, J.; Darrell, T.; Malik, J. Rich feature hierarchies for accurate object detection and semantic segmentation. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA, 23–28 June 2014; pp. 580–587. [Google Scholar]
  10. Girshick, R. Fast R-CNN. In Proceedings of the IEEE International Conference on Computer Vision, Washington, DC, USA, 7–13 December 2015; pp. 1440–1448. [Google Scholar]
  11. Ren, S.; He, K.; Girshick, R.; Sun, J. Faster R-CNN: Towards real-time object detection with region proposal networks. IEEE Trans. Pattern Anal. Mach. Intell. 2017, 39, 1137–1149. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  12. Liu, W.; Anguelov, D.; Erhan, D.; Szegedy, C.; Reed, S.; Fu, C.; Berg, A.C. SSD: Single shot multibox detector. In Proceedings of the Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, 11–14 October 2016; pp. 21–37. [Google Scholar]
  13. Redmon, J.; Divvala, S.; Girshick, R.; Farhadi, A. You only look once: Unified real-time object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 26 June–1 July 2016; pp. 779–788. [Google Scholar]
  14. Redmon, J.; Farhadi, A. YOLO9000: Better faster stronger. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, San Juan, PR, USA, 17–19 June 2017; pp. 7263–7271. [Google Scholar]
  15. Redmon, J.; Farhadi, A. YOLOv3: An incremental improvement. arXiv 2018, arXiv:1804.02767. [Google Scholar]
  16. Bochkovskiy, A.; Wang, C.-Y.; Liao, H.-Y.M. YOLOv4: Optimal speed and accuracy of object detection. arXiv 2020, arXiv:2004.10934. [Google Scholar]
  17. Duan, K.; Bai, S.; Xie, L.; Qi, H.; Huang, Q.; Tian, Q. CenterNet: Keypoint triplets for object detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Republic of Korea, 28 October 2019; pp. 6569–6578. [Google Scholar]
  18. Lin, T.Y.; Goyal, P.; Girshick, R.; He, K.; Dollár, P. Focal loss fordense object detection. In Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy, 22–29 October 2017; pp. 2980–2988. [Google Scholar]
  19. Cheng, X.; Yu, J. RetinaNet with difference channel attention and adaptively spatial feature fusion for steel surface defect detection. IEEE Trans. Instrum. Meas. 2021, 70, 2503911. [Google Scholar] [CrossRef]
  20. Fu, G.; Sun, P.; Zhu, W.; Yang, J.; Cao, Y.; Yang, M.Y.; Cao, Y. A deep-learning-based approach for fast and robust steel surface defects classification. Opt. Lasers Eng. 2019, 121, 397–405. [Google Scholar] [CrossRef]
  21. Li, J.; Su, Z.; Geng, J.; Yin, Y. Real-time detection of steel strip surface defects based on improved YOLO detection network. IFAC-PapersOnLine 2018, 51, 76–81. [Google Scholar] [CrossRef]
  22. Yang, Z.; Zhang, M.; Li, C.; Meng, Z.; Li, Y.; Chen, Y.; Liu, L. Image Classification for Automobile Pipe Joints Surface Defect Detection Using Wavelet Decomposition and Convolutional Neural Network. IEEE Access 2022, 10, 77191–77204. [Google Scholar] [CrossRef]
  23. Amin, D.; Akhter, S. Deep Learning-Based Defect Detection System in Steel Sheet Surfaces. In 2020 IEEE Region 10 Symposium (TENSYMP); IEEE: Piscataway, NJ, USA, 2020; pp. 444–448. [Google Scholar]
  24. Ronneberger, O.; Fischer, P.; Brox, T. U-NET: Convolutional networks for biomedical image segmentation. In Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Munich, Germany, 5–9 October 2015; Volume 2, pp. 234–241. [Google Scholar]
  25. Zhang, H.; Song, Y.; Chen, Y.; Zhong, H.; Liu, L.; Wang, Y.; Wu, Q.J. MRSDI-CNN: Multi-Model Rail Surface Defect Inspection System Based on Convolutional Neural Networks. In IEEE Transactions on Intelligent Transportation Systems; IEEE: Piscataway, NJ, USA, 2022; Volume 23, pp. 11162–11177. [Google Scholar]
  26. Chen, X.; Zhang, H. Rail Surface Defects Detection Based on Faster R-CNN. In Proceedings of the 2020 International Conference on Artificial Intelligence and Electromechanical Automation (AIEA), Tianjin, China, 26–28 June 2020; pp. 819–822. [Google Scholar]
  27. Guo, Y.; Xiao, Z.; Geng, L.; Wu, J.; Zhang, F.; Liu, Y.; Wang, W. Fully Convolutional Neural Network With GRU for 3D Braided Composite Material Flaw Detection. IEEE Access 2019, 7, 151180–151188. [Google Scholar] [CrossRef]
  28. Zhou, N.; Liu, Z.; Zhou, J. Yolov5-based defect detection for wafer surface micropipe. In Proceedings of the 3rd International Conference on Information Science, Parallel and Distributed Systems, ISPDS, Guangzhou, China, 22–24 July 2022; pp. 165–169. [Google Scholar]
  29. Ma, Z.; Li, Y.; Huang, M.; Huang, Q.; Cheng, J.; Tang, S. Automated real-time detection of surface defects in manufacturing processes of aluminum alloy strip using a lightweight network architecture. J. Intell. Manuf. 2022. [Google Scholar] [CrossRef]
  30. Wang, T.; Su, J.; Xu, C.; Zhang, Y. An Intelligent Method for Detecting Surface Defects in Aluminium Profiles Based on the Improved YOLOv5 Algorithm. Electronics 2022, 11, 2304. [Google Scholar] [CrossRef]
  31. Yang, Y.; Sun, Q.; Zhang, D.; Shao, L.; Song, X.; Li, X. Improved Method Based on Faster R-CNN Network Optimization for Small Target Surface Defects Detection of Aluminum Profile. In Proceedings of the 2021 IEEE 15th International Conference on Electronic Measurement & Instruments (ICEMI), Nanjing, China, 29–31 October 2021; pp. 465–470. [Google Scholar]
  32. Li, B.; Ren, F.; Ni, H.; Kang, X.; Lv, S.; Hao, Z. Classification Method of Surface Defects of Aluminum Profile Based on Transfer Learning. In Proceedings of the 2022 International Conference on Machine Learning and Intelligent Systems Engineering (MLISE), Guangzhou, China, 5–7 August 2022; pp. 1–5. [Google Scholar]
  33. Wu, D.; Shen, X.; Chen, L. Detection of Defects on Aluminum Profile Surface Based on Improved YOLO. In Proceedings of the 2022 Prognostics and Health Management Conference (PHM-2022 London), London, UK, 27–29 May 2022; pp. 468–472. [Google Scholar]
  34. Mushtaq, Z.; Su, S. Environmental sound classification using a regularized deep convolutional neural network with data augmentation. Appl. Acoust. 2020, 167, 107389–107401. [Google Scholar] [CrossRef]
  35. Kisantal, M.; Wojna, Z.; Murawski, J.; Naruniec, J.; Cho, K. Augmentation for small object detection. arXiv 2019, arXiv:1902.07296. [Google Scholar]
  36. Lin, L. An effective denoising method for images contaminated with mixed noise based on adaptive median filtering and wavelet threshold denoising. J. Inf. Process. Syst. 2018, 14, 539–551. [Google Scholar]
  37. Simonyan, K.K.C.; Vedaldi, A.; Zisserman, A. Return of the devil in the details: Delving deep into convolutional nets. arXiv 2014, arXiv:1405.3531. [Google Scholar]
  38. Hu, J.; Shen, L.; Sun, G. Squeeze-and-excitation networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–23 June 2018. [Google Scholar]
  39. Hou, Q.; Zhou, D.; Feng, J. Coordinate attention for efficient mobile network design. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA, 20–25 June 2021. [Google Scholar]
  40. Woo, S.; Park, J.; Lee, J.Y.; Kweon, I.S. Cbam: Convolutional block attention module. In Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany, 8–14 September 2018. [Google Scholar]
Figure 1. The four defects of aluminum profiles: (a) pinhole; (b) dirt; (c) fold; (d) scratch.
Figure 1. The four defects of aluminum profiles: (a) pinhole; (b) dirt; (c) fold; (d) scratch.
Metals 13 00507 g001
Figure 2. Images were obtained using the expansion technique.
Figure 2. Images were obtained using the expansion technique.
Metals 13 00507 g002aMetals 13 00507 g002b
Figure 3. The lightweight model network structure based on YOLOv5s.
Figure 3. The lightweight model network structure based on YOLOv5s.
Metals 13 00507 g003
Figure 4. Ghost module structure.
Figure 4. Ghost module structure.
Metals 13 00507 g004
Figure 5. AC3Ghost module Structure.
Figure 5. AC3Ghost module Structure.
Metals 13 00507 g005
Figure 6. DwConv module Structure.
Figure 6. DwConv module Structure.
Metals 13 00507 g006
Figure 7. Visual inspection results for four types of defects.
Figure 7. Visual inspection results for four types of defects.
Metals 13 00507 g007
Figure 8. Visual comparison of the results of the five algorithms on the dataset.
Figure 8. Visual comparison of the results of the five algorithms on the dataset.
Metals 13 00507 g008
Figure 9. Edge surface defect detection system.
Figure 9. Edge surface defect detection system.
Metals 13 00507 g009
Table 1. Test results for four types of defect detection.
Table 1. Test results for four types of defect detection.
PinholeScratchDirtFold
Precision83.2498.8699.1199.45
Recall77.2199.3999.22100.00
[email protected]:0.9551.9479.8081.1080.60
[email protected]82.4598.7798.8099.37
[email protected]:0.9573.36
[email protected]94.85
Table 2. Results of different object detection models.
Table 2. Results of different object detection models.
Algorithm[email protected]%[email protected]:0.95%Model Size (MB)Detection Speed (FPS)
SSD67.8642.7691.0939
YOLOv3-tiny85.8265.8933.1966
YOLOv4-tiny89.3167.8623.0986
YOLOv5s93.0972.5814.4075
YOLOv5-Mobilenetv389.7167.785.96158
YOLOv5-Shufflenetv289.6566.863.30176
Ours94.8573.367.80136
Table 3. Effects of various design modules on surface image.
Table 3. Effects of various design modules on surface image.
DescriptionModel Size (MB)[email protected] (%)Change (%)
YOLOv5s14.4090.32--
Baseline14.4093.09+2.77
+GCANet Backbone Network Structure7.9093.90+0.81
+AC3Ghost Structural8.2094.96+1.06
+DwConv Module7.8094.84−0.12
Table 4. Serial number and name of component in detection system.
Table 4. Serial number and name of component in detection system.
NumberComponent Name
1Conveyor belt
2Encoder
3Power supply
4Nvidia Jeston Nano
5Touch screen
6CCD
7LED light source
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Tang, J.; Liu, S.; Zhao, D.; Tang, L.; Zou, W.; Zheng, B. An Algorithm for Real-Time Aluminum Profile Surface Defects Detection Based on Lightweight Network Structure. Metals 2023, 13, 507. https://doi.org/10.3390/met13030507

AMA Style

Tang J, Liu S, Zhao D, Tang L, Zou W, Zheng B. An Algorithm for Real-Time Aluminum Profile Surface Defects Detection Based on Lightweight Network Structure. Metals. 2023; 13(3):507. https://doi.org/10.3390/met13030507

Chicago/Turabian Style

Tang, Junlong, Shenbo Liu, Dongxue Zhao, Lijun Tang, Wanghui Zou, and Bin Zheng. 2023. "An Algorithm for Real-Time Aluminum Profile Surface Defects Detection Based on Lightweight Network Structure" Metals 13, no. 3: 507. https://doi.org/10.3390/met13030507

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop