Industrial Laser Welding Defect Detection and Image Defect Recognition Based on Deep Learning Model Developed
Abstract
:1. Introduction
2. Recent Studies
2.1. Research Progress of Weld Defect Detection
2.2. Research Progress of Weld Defect Image Recognition
2.3. Summary of Related Studies
3. Materials and Methods
3.1. Analysis of Steel Plate’s Surface Defects
3.2. Detection Technologies for Steel Plate’s Surface Defects
3.2.1. Traditional Detection Technology
3.2.2. Deep Learning Detection Technology
3.3. Image Preprocessing
3.3.1. Image Denoising
3.3.2. Image Enhancement
3.4. Deep Learning Neural Networks
3.4.1. Convolution Neural Networks
3.4.2. Transfer Learning
3.5. Deep Learning-Based Image Defect Recognition Model
3.6. Experimental Data and Performance Evaluation
3.6.1. Experimental Environment and Data
3.6.2. Image Partition and Labeling
3.6.3. Model Performance Evaluation
4. Results and Discussion
4.1. Image Processing Results of Weld Defects
4.2. Performance Comparison of Different Training Models
4.3. Comparative Experiment of Industrial Weld Defect Images
4.4. Results of Other Performance Indicators
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Qiao, J.; Yu, P.; Wu, Y.; Chen, T.; Du, Y.; Yang, J. A Compact Review of Laser Welding Technologies for Amorphous Alloys. Metals 2020, 12, 1690. [Google Scholar] [CrossRef]
- Zapata, J.; Vilar, R.; Ruiz, R. An adaptive-network-based fuzzy inference system for classification of welding defects. NDT E Int. 2010, 43, 191–199. [Google Scholar] [CrossRef]
- Gruse, J.N.; Streeter, M.J.V.; Thornton, C.; Armstrong, C.D.; Baird, C.D.; Bourgeois, N.; Cipiccia, S.; Finlay, O.J.; Gregory, C.D.; Katzir, Y.; et al. Application of compact laser-driven accelerator X-ray sources for industrial imaging. Nucl. Instrum. Methods Phys. Res. Sect. A Accel. Spectrom. Detect. Assoc. Equip. 2020, 983, 164369. [Google Scholar]
- Drotár, A.; Zubko, P.; Mašlejová, A.; Kalmár, P.; Vranec, P.; Hockicková, S.; Demčáková, M.; Hrabčáková, L. Defects of Steel Sheets Joined by Laser Welding. Defect Diffus. Forum 2020, 405, 240–244. [Google Scholar] [CrossRef]
- Schmid, M.; Bhogaraju, S.K.; Liu, E.; Elger, G. Comparison of Nondestructive Testing Methods for Solder, Sinter, and Adhesive Interconnects in Power and Opto-Electronics. Appl. Sci. 2020, 10, 8516. [Google Scholar] [CrossRef]
- Ricketts, J. Film-Screen Radiography in Bachelor’s Degree Program Curriculum. Radiol. Technol. 2016, 88, 234–236. [Google Scholar]
- Vorobeychikov, S.E.; Chakhlov, S.V.; Udod, V.A. A Cumulative sums algorithm for segmentation of digital X-ray images. J. Nondestruct. Eval. 2019, 38, 78–86. [Google Scholar] [CrossRef]
- Roshani, M.; Phan, G.T.; Ali, P.J.M.; Roshani, G.H.; Hanus, R.; Duong, T.; Corniani, E.; Nazemi, E.; Kalmoun, E.M. Evaluation of flow pattern recognition and void fraction measurement in two phase flow independent of oil pipeline’s scale layer thickness. Alex. Eng. J. 2021, 60, 1955–1966. [Google Scholar] [CrossRef]
- Nisar, M.U.; Voghoei, S.; Ramaswamy, L. Caching for pattern matching queries in time evolving graphs: Challenges and approaches. In Proceedings of the 2017 IEEE 37th International Conference on Distributed Computing Systems (ICDCS), Atlanta, GA, USA, 5–8 June 2017; pp. 2352–2357. [Google Scholar]
- Tasoujian, S.; Salavati, S.; Franchek, M.A.; Grigoriadis, K.M. Robust delay-dependent LPV synthesis for blood pressure control with real-time Bayesian parameter estimation. IET Control. Theory Appl. 2020, 14, 1334–1345. [Google Scholar] [CrossRef]
- Karimi, M.; Jahanshahi, A.; Mazloumi, A.; Sabzi, H.Z. Border gateway protocol anomaly detection using neural network. In Proceedings of the 2019 IEEE International Conference on Big Data (Big Data), Los Angeles, CA, USA, 9–12 December 2019; pp. 6092–6094. [Google Scholar]
- Moaveni, B.; Fathabadi, F.R.; Molavi, A. Supervisory predictive control for wheel slip prevention and tracking of desired speed profile in electric trains. ISA Trans. 2020, 101, 102–115. [Google Scholar] [CrossRef] [PubMed]
- Voghoei, S.; Tonekaboni, N.H.; Yazdansepas, D.; Soleymani, S.; Farahani, A.; Arabnia, H.R. Personalized Feedback Emails: A Case Study on Online Introductory Computer Science Courses. In Proceedings of the 2020 ACM Southeast Conference, Tampa, FL, USA, 2–4 April 2020; pp. 18–25. [Google Scholar]
- Shih, P.C.; Hsu, C.C.; Tien, F.C. Automatic Reclaimed Wafer Classification Using Deep Learning Neural Networks. Symmetry 2020, 12, 705. [Google Scholar] [CrossRef]
- Brachmann, A.; Redies, C. Using Convolutional Neural Network Filters to Measure Left-Right Mirror Symmetry in Images. Symmetry 2016, 8, 144. [Google Scholar] [CrossRef] [Green Version]
- Zhang, M.; Diao, M.; Gao, L.; Liu, L. Neural Networks for Radar Waveform Recognition. Symmetry 2017, 9, 75. [Google Scholar] [CrossRef] [Green Version]
- Hou, W.; Wei, Y.; Jin, Y.; Zhu, C. Deep features based on a DCNN model for classifying imbalanced weld flaw types. Measurement 2019, 131, 482–489. [Google Scholar] [CrossRef]
- Shevchik, S.; Le-Quang, T.; Meylan, B.; Farahani, F.V.; Olbinado, M.P.; Rack, A.; Masinelli, G.; Leinenbach, C.; Wasmer, K. Supervised deep learning for real-time quality monitoring of laser welding with X-ray radiographic guidance. Sci. Rep. 2020, 10, 3389. [Google Scholar] [CrossRef] [Green Version]
- Ajmi, C.; Zapata, J.; Martínez-Álvarez, J.J.; Doménech, G.; Ruiz, R. Using Deep Learning for Defect Classification on a Small Weld X-ray Image Dataset. J. Nondestruct. Eval. 2020, 39, 68. [Google Scholar] [CrossRef]
- Ajmi, C.; Zapata, J.; Elferchichi, S.; Zaafouri, A.; Laabidi, K. Deep Learning Technology for Weld Defects Classification Based on Transfer Learning and Activation Features. Adv. Mater. Sci. Eng. 2020, 2020, 1574350. [Google Scholar] [CrossRef]
- Boikov, A.; Payor, V.; Savelev, R.; Kolesnikov, A. Synthetic Data Generation for Steel Defect Detection and Classification Using Deep Learning. Symmetry 2021, 13, 1176. [Google Scholar] [CrossRef]
- Javadi, Y.; Sweeney, N.E.; Mohseni, E.; MacLeod, C.N.; Lines, D.; Vasilev, M.; Qiu, Z.; Vithanage, R.K.; Mineo, C.; Stratoudaki, T. In-process calibration of a non-destructive testing system used for in-process inspection of multi-pass welding. Mater. Des. 2020, 195, 108981. [Google Scholar] [CrossRef]
- Boaretto, N.; Centeno, T.M. Automated detection of welding defects in pipelines from radiographic images DWDI. NDT E Int. 2017, 86, 7–13. [Google Scholar] [CrossRef]
- Bestard, G.A.; Sampaio, R.C.; Vargas, J.A.; Alfaro, S.C.A. Sensor fusion to estimate the depth and width of the weld bead in real time in GMAW processes. Sensors 2018, 18, 962. [Google Scholar] [CrossRef] [Green Version]
- Vasilev, M.; MacLeod, C.; Galbraith, W.; Pierce, G.; Gachagan, A. In-process ultrasonic inspection of thin mild steel plate GMAW butt welds using non-contact guided waves. In Proceedings of the 46th Annual Review of Progress in Quantitative Nondestructive Evaluation QNDE2019, Portland, OR, USA, 14–19 July 2019; pp. 115–124. [Google Scholar]
- Gao, Y.; Gao, L.; Li, X.; Yan, X. A semi-supervised convolutional neural network-based method for steel surface defect recognition. Robot. Comput.-Integr. Manuf. 2020, 61, 101825. [Google Scholar] [CrossRef]
- Li, C.; Wang, Q.; Jiao, W.; Johnson, M.; Zhang, Y. Deep Learning-Based Detection of Penetration from Weld Pool Reflection Images. Weld. J. 2020, 99, 239s–345s. [Google Scholar] [CrossRef]
- Sony, S.; Dunphy, K.; Sadhu, A.; Capretz, M. A systematic review of convolutional neural network-based structural condition assessment techniques. Eng. Struct. 2021, 226, 111347. [Google Scholar] [CrossRef]
- Chen, F.-C.; Jahanshahi, M.R. NB-CNN: Deep learning-based crack detection using convolutional neural network and Naïve Bayes data fusion. IEEE Trans. Ind. Electron. 2017, 65, 4392–4400. [Google Scholar] [CrossRef]
- Davis, D.S. Object-based image analysis: A review of developments and future directions of automated feature detection in landscape archaeology. Archaeol. Prospect. 2019, 26, 155–163. [Google Scholar] [CrossRef]
- Li, L.; Xiao, L.; Liao, H.; Liu, S.; Ye, B. Welding quality monitoring of high frequency straight seam pipe based on image feature. J. Mater. Process. Technol. 2017, 246, 285–290. [Google Scholar] [CrossRef]
- Gao, X.; Du, L.; Xie, Y.; Chen, Z.; Zhang, Y.; You, D.; Gao, P.P. Identification of weld defects using magneto-optical imaging. Int. J. Adv. Manuf. Technol. 2019, 105, 1713–1722. [Google Scholar] [CrossRef]
- Bashar, A. Survey on evolving deep learning neural network architectures. J. Artif. Intell. 2019, 1, 73–82. [Google Scholar]
- Malarvel, M.; Singh, H. An autonomous technique for weld defects detection and classification using multi-class support vector machine in X-radiography image. Optik 2021, 231, 166342. [Google Scholar] [CrossRef]
- Chu, M.; Zhao, J.; Liu, X.; Gong, R. Multi-class classification for steel surface defects based on machine learning with quantile hyper-spheres. Chemom. Intell. Lab. Syst. 2017, 168, 15–27. [Google Scholar] [CrossRef]
- Zhou, S.; Chen, Y.; Zhang, D.; Xie, J.; Zhou, Y. Classification of surface defects on steel sheet using convolutional neural networks. Mater. Technol. 2017, 51, 123–131. [Google Scholar]
- Gong, R.; Wu, C.; Chu, M. Steel surface defect classification using multiple hyper-spheres support vector machine with additional information. Chemom. Intell. Lab. Syst. 2018, 172, 109–117. [Google Scholar] [CrossRef]
- Chu, M.; Liu, X.; Gong, R.; Liu, L. Multi-class classification method using twin support vector machines with multi-information for steel surface defects. Chemom. Intell. Lab. Syst. 2018, 176, 108–118. [Google Scholar] [CrossRef]
- Li, J.; Su, Z.; Geng, J.; Yin, Y. Real-time detection of steel strip surface defects based on improved yolo detection network. IFAC-PapersOnLine 2018, 51, 76–81. [Google Scholar] [CrossRef]
- AbdAlla, A.N.; Faraj, M.A.; Samsuri, F.; Rifai, D.; Ali, K.; Al-Douri, Y. Challenges in improving the performance of eddy current testing. Meas. Control. 2019, 52, 46–64. [Google Scholar] [CrossRef] [Green Version]
- Wu, P.; Chen, Z.; Jile, H.; Zhang, C.; Xu, D.; Lv, L. An infrared perfect absorber based on metal-dielectric-metal multi-layer films with nanocircle holes arrays. Results Phys. 2020, 16, 102952. [Google Scholar] [CrossRef]
- Wu, D.; Liu, Z.; Wang, X.; Su, L. Composite magnetic flux leakage detection method for pipelines using alternating magnetic field excitation. NDT E Int. 2017, 91, 148–155. [Google Scholar] [CrossRef]
- Yoo, Y.; Baek, J.-G. A novel image feature for the remaining useful lifetime prediction of bearings based on continuous wavelet transform and convolutional neural network. Appl. Sci. 2018, 8, 1102. [Google Scholar] [CrossRef] [Green Version]
- Dung, C.V.; Sekiya, H.; Hirano, S.; Okatani, T.; Miki, C. A vision-based method for crack detection in gusset plate welded joints of steel bridges using deep convolutional neural networks. Autom. Constr. 2019, 102, 217–229. [Google Scholar] [CrossRef]
- Shin, S.; Jin, C.; Yu, J.; Rhee, S. Real-time detection of weld defects for automated welding process base on deep neural network. Metals 2020, 10, 389. [Google Scholar] [CrossRef] [Green Version]
- Yang, M.; Ma, H.-h.; Shen, Z.-W.; Xu, J. Application of Colloid Water Covering on Explosive Welding of AA1060 Foil to Q235 Steel Plate. Propellants Explos. Pyrotech. 2020, 45, 453–462. [Google Scholar]
- Elhoseny, M.; Shankar, K. Optimal bilateral filter and convolutional neural network based denoising method of medical image measurements. Measurement 2019, 143, 125–135. [Google Scholar] [CrossRef]
- Xiao, B.; Tang, H.; Jiang, Y.; Li, W.; Wang, G. Brightness and contrast controllable image enhancement based on histogram specification. Neurocomputing 2018, 275, 2798–2809. [Google Scholar] [CrossRef]
- Yao, P.; Wu, H.; Gao, B.; Tang, J.; Zhang, Q.; Zhang, W.; Yang, J.J.; Qian, H. Fully hardware-implemented memristor convolutional neural network. Nature 2020, 577, 641–646. [Google Scholar] [CrossRef] [PubMed]
- Fu, Y.; Aldrich, C. Flotation froth image recognition with convolutional neural networks. Miner. Eng. 2019, 132, 183–190. [Google Scholar] [CrossRef]
- Choi, E.; Jo, H.; Kim, J. A Comparative Study of Transfer Learning–based Methods for Inspection of Mobile Camera Modules. IEIE Trans. Smart Process. Comput. 2018, 7, 70–74. [Google Scholar] [CrossRef]
- Ferguson, M.K.; Ronay, A.; Lee, Y.-T.T.; Law, K.H. Detection and segmentation of manufacturing defects with convolutional neural networks and transfer learning. Smart Sustain. Manuf. Syst. 2018, 2, 137–164. [Google Scholar] [CrossRef]
- Chrabaszcz, P.; Loshchilov, I.; Hutter, F. A Downsampled Variant of ImageNet as an Alternative to the CIFAR Datasets. arXiv 2017, arXiv:1707.08819. [Google Scholar]
- Luo, X.; Lam, K.P.; Chen, Y.; Hong, T. Performance evaluation of an agent-based occupancy simulation model. Build. Environ. 2017, 115, 42–53. [Google Scholar] [CrossRef] [Green Version]
Software and Hardware | Specific Configuration Information |
---|---|
Central Processing Unit | Inter(R)-Core (TM) I5-2400 |
RAM | DDR4 16 G |
CPU Hertz | 3.10 GHz |
Operating system | Ubuntu 16.04 |
Programming environment | Python 3.5 |
Type of Defect | Gas Pores (GP) | Flawless (FL) | Lack of Fusion (LOF) | Lack of Penetration (LOP) | Cracks (CK) |
---|---|---|---|---|---|
Training set | 1200 | 1040 | 920 | 680 | 320 |
Validation set | 150 | 130 | 115 | 85 | 40 |
Test set | 150 | 130 | 115 | 85 | 40 |
Label | 1 | 2 | 3 | 4 | 5 |
Type of Defect | GP | FL | LOF | LOP | CK | ACC/% |
---|---|---|---|---|---|---|
CNN-Train_All | 20 | 20 | 20 | 19 | 18 | 97 |
CNN-Frozen_C1C2 | 20 | 20 | 18 | 18 | 18 | 94 |
CNN-Frozen_C1C2C3 | 20 | 19 | 18 | 17 | 17 | 91 |
CNN-Frozen_C1C2C3C4 | 19 | 19 | 17 | 17 | 16 | 86 |
ACC/% | 98.75 | 97.50 | 91.25 | 88.75 | 86.25 | - |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Deng, H.; Cheng, Y.; Feng, Y.; Xiang, J. Industrial Laser Welding Defect Detection and Image Defect Recognition Based on Deep Learning Model Developed. Symmetry 2021, 13, 1731. https://doi.org/10.3390/sym13091731
Deng H, Cheng Y, Feng Y, Xiang J. Industrial Laser Welding Defect Detection and Image Defect Recognition Based on Deep Learning Model Developed. Symmetry. 2021; 13(9):1731. https://doi.org/10.3390/sym13091731
Chicago/Turabian StyleDeng, Honggui, Yu Cheng, Yuxin Feng, and Junjiang Xiang. 2021. "Industrial Laser Welding Defect Detection and Image Defect Recognition Based on Deep Learning Model Developed" Symmetry 13, no. 9: 1731. https://doi.org/10.3390/sym13091731