Deep Learning-Based Automatic Defect Detection of Additive Manufactured Stainless Steel
Abstract
:1. Introduction
- (1)
- An end-to-end architecture for microstructural defect detection using YOLOv8 models was proposed where the input images to the models vary in terms of color space.
- (2)
- The proposed method achieved adequate results while evaluated on a publicly available metallographic dataset.
2. Related Work
3. Research Methodology
3.1. Dataset Preparation
3.2. Dataset Annotation
3.3. Data Augmentation
3.4. Splitting the Dataset
3.5. Pre-Processing
3.6. Training Process and Performance Evaluation
4. Results and Discussion
5. Conclusions
Author Contributions
Funding
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Caggiano, A.; Zhang, J.; Alfieri, V.; Caiazzo, F.; Gao, R.; Teti, R. Machine learning-based image processing for on-line defect recognition in additive manufacturing. CIRP Ann. 2019, 68, 451–454. [Google Scholar] [CrossRef]
- Åkerfeldt, P.; Antti, M.-L.; Pederson, R. Influence of microstructure on mechanical properties of laser metal wire-deposited Ti-6Al-4V. Mater. Sci. Eng. A 2016, 674, 428–437. [Google Scholar] [CrossRef]
- Sun, G.; Zhou, R.; Lu, J.; Mazumder, J. Evaluation of defect density, microstructure, residual stress, elastic modulus, hardness and strength of laser-deposited AISI 4340 steel. Acta Mater. 2015, 84, 172–189. [Google Scholar] [CrossRef]
- Sarkar, S.S.; Sheikh, K.H.; Mahanty, A.; Mali, K.; Ghosh, A.; Sarkar, R. A harmony search-based wrapper-filter feature selection approach for microstructural image classification. Integr. Mater. Manuf. Innov. 2021, 10, 1–19. [Google Scholar] [CrossRef]
- Khan, A.H.; Sarkar, S.S.; Mali, K.; Sarkar, R. A genetic algorithm based feature selection approach for microstructural image classification. Exp. Tech. 2021, 46, 335–347. [Google Scholar] [CrossRef]
- Taheri, H.; Shoaib MR, B.M.; Koester, L.W.; Bigelow, T.A.; Collins, P.C.; Bond, L.J. Powder-based additive manufacturing-a review of types of defects, generation mechanisms, detection, property evaluation and metrology. Int. J. Addit. Subtractive Mater. Manuf. 2017, 1, 172–209. [Google Scholar] [CrossRef]
- Reichardt, A.; Dillon, R.P.; Borgonia, J.P.; Shapiro, A.A.; McEnerney, B.W.; Momose, T.; Hosemann, P. Development and characterization of Ti-6Al-4V to 304L stainless steel gradient components fabricated with laser deposition additive manufacturing. Mater. Des. 2016, 104, 404–413. [Google Scholar] [CrossRef]
- Huang, C.; Zhang, Y.; Vilar, R.; Shen, J. Dry sliding wear behavior of laser clad TiVCrAlSi high entropy alloy coatings on Ti–6Al–4V substrate. Mater. Des. 2012, 41, 338–343. [Google Scholar] [CrossRef]
- Cui, W.; Karnati, S.; Zhang, X.; Burns, E.; Liou, F. Fabrication of AlCoCrFeNi high-entropy alloy coating on an AISI 304 substrate via a CoFe2Ni intermediate layer. Entropy 2018, 21, 2. [Google Scholar] [CrossRef]
- Li, W.; Ghazanfari, A.; McMillen, D.; Leu, M.C.; Hilmas, G.E.; Watts, J. Characterization of zirconia specimens fabricated by ceramic on-demand extrusion. Ceram. Int. 2018, 44, 12245–12252. [Google Scholar] [CrossRef]
- Ahsan, M.N.; Pinkerton, A.J.; Moat, R.J.; Shackleton, J. A comparative study of laser direct metal deposition characteristics using gas and plasma-atomized Ti–6Al–4V powders. Mater. Sci. Eng. A 2011, 528, 7648–7657. [Google Scholar] [CrossRef]
- Ackermann, M. Metal Additive Manufacturing Porosity Images. Figshare. Dataset. 2023. Available online: https://figshare.com/articles/dataset/Metal_additive_manufacturing_porosity_images/22324993 (accessed on 1 October 2022).
- Åkerfeldt, P.; Pederson, R.; Antti, M.-L. A fractographic study exploring the relationship between the low cycle fatigue and metallurgical properties of laser metal wire deposited Ti–6Al–4V. Int. J. Fatigue 2016, 87, 245–256. [Google Scholar] [CrossRef]
- Barua, S.; Liou, F.; Newkirk, J.; Sparks, T. Vision-based defect detection in laser metal deposition process. Rapid Prototyp. J. 2014, 20, 77–85. [Google Scholar] [CrossRef]
- Guan, S. Strip steel defect detection based on saliency map construction using Gaussian pyramid decomposition. ISIJ Int. 2015, 55, 1950–1955. [Google Scholar] [CrossRef]
- LeCun, Y.; Bengio, Y.; Hinton, G. Deep learning. Nature 2015, 521, 436–444. [Google Scholar] [CrossRef] [PubMed]
- Krizhevsky, A.; Sutskever, I.; Hinton, G. Imagenet classification with deep convolutional neural networks. Commun. ACM 2017, 60, 84–90. [Google Scholar] [CrossRef]
- Qi, X.; Chen, G.; Li, Y.; Cheng, X.; Li, C. Applying neural-network-based machine learning to additive manufacturing: Current applications, challenges, and future perspectives. Engineering 2019, 5, 721–729. [Google Scholar] [CrossRef]
- Zhao, W.; Chen, F.; Huang, H.; Li, D.; Cheng, W. A new steel defect detection algorithm based on deep learning. Comput. Intell. Neurosci. 2021, 2021, 5592878. [Google Scholar] [CrossRef]
- Lin, J.; Ma, L.; Yao, Y. Segmentation of casting defect regions for the extraction of microstructural properties. Eng. Appl. Artif. Intell. 2019, 85, 150–163. [Google Scholar] [CrossRef]
- Jin, H.; Jiao, T.; Clifton, R.J.; Kim, K.-S. Dynamic fracture of a bicontinuously nanostructured copolymer: A deep-learning analysis of big-data-generating experiment. J. Mech. Phys. Solids 2022, 164, 104898. [Google Scholar] [CrossRef]
- Roberts, G.; Haile, S.Y.; Sainju, R.; Edwards, D.J.; Hutchinson, B.; Zhu, Y. Deep learning for semantic segmentation of defects in advanced STEM images of steels. Sci. Rep. 2019, 9, 12744. [Google Scholar] [CrossRef] [PubMed]
- Ma, B.; Ban, X.; Huang, H.; Chen, Y.; Liu, W.; Zhi, Y. Deep learning-based image segmentation for al-la alloy microscopic images. Symmetry 2018, 10, 107. [Google Scholar] [CrossRef]
- Barik, S.; Bhandari, R.; Mondal, M.K. Optimization of Wire Arc Additive Manufacturing Process Parameters for Low Carbon Steel and Properties Prediction by Support Vector Regression Model. Steel Res. Int. 2023. [Google Scholar] [CrossRef]
- Jang, J.; Van, D.; Jang, H.; Baik, D.H.; Yoo, S.D.; Park, J.; Mhin, S.; Mazumder, J.; Lee, S.H. Residual neural network-based fully convolutional network for microstructure segmentation. Sci. Technol. Weld. Join. 2019, 25, 282–289. [Google Scholar] [CrossRef]
- Rosenberger, J.; Tlatlik, J.; Münstermann, S. Deep learning based initial crack size measurements utilizing macroscale fracture surface segmentation. Eng. Fract. Mech. 2023, 293, 109686. [Google Scholar] [CrossRef]
- Mutiargo, B.; Pavlovic, M.; Malcolm, A.; Goh, B.; Krishnan, M.; Shota, T.; Shaista, H.; Jhinaoui, A.; Putro, M. Evaluation of X-Ray computed tomography (CT) images of additively manufactured components using deep learning. In Proceedings of the 3rd Singapore International Non-Destructive Testing Conference and Exhibition (SINCE2019), Singapore, 2–5 December 2019. [Google Scholar]
- Neuhauser, F.M.; Bachmann, G.; Hora, P. Surface defect classification and detection on extruded aluminum profiles using convolutional neural networks. Int. J. Mater. Form. 2020, 13, 591–603. [Google Scholar] [CrossRef]
- Redmon, J.; Divvala, S.; Girshick, R.; Farhadi, A. You only look once: Unified, real-time object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 26 June–1 July 2016. [Google Scholar]
- Wang, C.-Y.; Liao, H.-Y.M.; Wu, Y.-H.; Chen, P.-Y.; Hsieh, J.-W.; Yeh, I.-H. CSPNet: A new backbone that can enhance learning capability of CNN. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Seattle, WA, USA, 14–19 June 2020. [Google Scholar]
- Lin, T.-Y.; Dollár, P.; Girshick, R.; He, K.; Hariharan, B.; Belongie, S. Feature pyramid networks for object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 21–26 July 2017. [Google Scholar]
- Liu, S.; Qi, L.; Qin, H.; Shi, J.; Jia, J. Path aggregation network for instance segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–23 June 2018. [Google Scholar]
- Wang, C.-Y.; Bochkovskiy, A.; Liao, H.-Y.M. YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada, 18–22 June 2023. [Google Scholar]
- Bolya, D.; Zhou, C.; Xiao, F.; Lee, Y.J. Yolact: Real-time instance segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Republic of Korea, 27 October–2 November 2019. [Google Scholar]
- Cao, Y.; Chen, K.; Loy, C.C.; Lin, D. Prime sample attention in object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, 13–19 June 2020. [Google Scholar]
- Tamang, S.; Sen, B.; Pradhan, A.; Sharma, K.; Singh, V.K. Enhancing COVID-19 Safety: Exploring YOLOv8 Object Detection for Accurate Face Mask Classification. Int. J. Intell. Syst. Appl. Eng. 2023, 11, 892–897. [Google Scholar]
- Hussain, M. YOLO-v1 to YOLO-v8, the Rise of YOLO and Its Complementary Nature toward Digital Manufacturing and Industrial Defect Detection. Machines 2023, 11, 677. [Google Scholar] [CrossRef]
- Srivastava, S.; Divekar, A.V.; Anilkumar, C.; Naik, I.; Kulkarni, V.; Pattabiraman, V. Comparative analysis of deep learning image detection algorithms. J. Big Data 2021, 8, 66. [Google Scholar] [CrossRef]
- Kim, J.-a.; Sung, J.-Y.; Park, S.-h. Comparison of Faster-RCNN, YOLO, and SSD for real-time vehicle type recognition. In Proceedings of the 2020 IEEE International Conference on Consumer Electronics-ASIA (ICCE-Asia), Seoul, Republic of Korea, 1–3 November 2020. [Google Scholar]
- Luengo, J.; Moreno, R.; Sevillano, I.; Charte, D.; Peláez-Vegas, A.; Fernández-Moreno, M.; Mesejo, P.; Herrera, F. A tutorial on the segmentation of metallographic images: Taxonomy, new MetalDAM dataset, deep learning-based ensemble model, experimental analysis and challenges. Inf. Fusion 2021, 78, 232–253. [Google Scholar] [CrossRef]
- Zhao, W.; Huang, H.; Li, D.; Chen, F.; Cheng, W. Pointer defect detection based on transfer learning and improved cascade-RCNN. Sensors 2020, 20, 4939. [Google Scholar] [CrossRef] [PubMed]
Hardware | Software |
---|---|
CPU | Intel(R) Core(TM) i7-8565U |
GPU | NVIDIA NVTX 11.6 |
Operating system | Linux Ubuntu 16.04 |
Deep learning framework | PyTorch 1.7 |
Language | Python 3.9.13 |
Class | Instances | Box (Precision) | Box (Recall) | mAP | mAP 50–95 |
---|---|---|---|---|---|
All | 51 | 0.932 | 0.895 | 0.927 | 0.656 |
Crack | 35 | 0.964 | 0.914 | 0.945 | 0.731 |
Porosity | 16 | 0.901 | 0.875 | 0.909 | 0.582 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Zubayer, M.H.; Zhang, C.; Wang, Y. Deep Learning-Based Automatic Defect Detection of Additive Manufactured Stainless Steel. Metals 2023, 13, 1987. https://doi.org/10.3390/met13121987
Zubayer MH, Zhang C, Wang Y. Deep Learning-Based Automatic Defect Detection of Additive Manufactured Stainless Steel. Metals. 2023; 13(12):1987. https://doi.org/10.3390/met13121987
Chicago/Turabian StyleZubayer, Md Hasib, Chaoqun Zhang, and Yafei Wang. 2023. "Deep Learning-Based Automatic Defect Detection of Additive Manufactured Stainless Steel" Metals 13, no. 12: 1987. https://doi.org/10.3390/met13121987