FSNB-YOLOV8: Improvement of Object Detection Model for Surface Defects Inspection in Online Industrial Systems
Abstract
:1. Introduction
- To address the challenge of the long-tailed distribution of industrial product packaging bag data in practical industrial scenarios, we introduce an interactive data augmentation algorithm based on fuzzy search. This algorithm effectively increases the number of defective samples by intelligently generating and enhancing them, thereby alleviating the issue of data imbalance and enhancing the generalization capabilities of the model. By leveraging this approach, we can improve the performance of defect detection systems in real-world industrial settings, where data distribution is often imbalanced.
- To address the high computational complexity of the C2F module, we have designed a joint network incorporating FasterNet and SPD-Conv, which replaces the original backbone network of YOLOv8. This improvement aims to reduce the computational load and network redundancy while enhancing the accuracy of small target defect detection. By optimizing the network structure, FSNB_YOLOv8 can better adapt to the computational resource constraints of edge devices while maintaining its performance.
- To further enhance the performance of multiscale feature fusion, we introduce the weighted bi-directional feature pyramid network (BiFPN). This network effectively enhances the model’s ability to detect defects at different scales by integrating deep and shallow information, thereby improving the accuracy and stability of the detection process.
- To reduce the sensitivity to defects’ positional deviations, this paper employs the Normalized Wasserstein Distance (NWD) loss function. This function can more accurately measure the distance between the predicted bounding boxes and actual boxes, optimizing the model’s localization performance and reducing detection errors caused by positional deviations.
2. Related Work
3. Proposed Method
3.1. Interactive Fuzzy Search Data Augmentation Algorithm
3.2. FasterNet Lightweight Network
3.2.1. The Architecture of FasterNet Network
3.2.2. Partial Convolution (PConv)
3.3. Spatial-to-Depth Convolution
3.3.1. SPD-Conv Network Structure
3.3.2. Space to Depth
3.3.3. Non-Stride Convolutional Layer
3.4. Weighted Bidirectional Feature Pyramid Network (BiFPN)
3.4.1. BiFPN Network Structure
3.4.2. Cross-Scale Connections
3.5. Improve the Loss Function
3.5.1. NWD Loss Function
3.5.2. Bounding Box Gaussian Distribution Modeling
3.5.3. Normalized Gaussian Wasserstein Distance
4. Experimental Results
4.1. Experimental Parameters
4.2. Experimental Data
4.3. Experimental Results
4.4. Ablation Experiment
4.5. Comparative Experiment
4.6. Discussion
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Ma, B.; Li, Q. High Speed Pharmaceutical Packaging Detection System Based on Genetic Algorithm and Memory Optimization. In Proceedings of the International Conference on Cloud Computing and Security, Nicosia, Cyprus, 10–13 December 2018; Springer: Cham, Switzerland, 2018; pp. 356–368. [Google Scholar]
- He, Y.; Song, K.; Meng, Q.; Yan, Y. An end-to-end steel surface defect detection approach via fusing multiple hierarchical features. IEEE Trans. Instrum. Meas. 2019, 69, 1493–1504. [Google Scholar] [CrossRef]
- Zhao, Z.B.; Wang, C.X.; Zhang, X.W.; Chen, X.F.; Li, Y.H. Research progress and challenges in process intelligent monitoring of laser powder bed fusion additive manufacturing. Engineering 2023, 59, 253–276. [Google Scholar]
- Fu, X.; Yang, X.; Zhang, N.; Zhang, R.; Zhang, Z.; Jin, A.; Ye, R.; Zhang, H. Bearing surface defect detection based on improved convolutional neural network. Math. Biosci. Eng. 2023, 20, 12341–12359. [Google Scholar] [CrossRef]
- Jun, L.; Jiajie, Z.; Jinglei, W.; Yanzhao, L. Improved Drug Traceability Code Detection Algorithm Based on YOLOv5. Proc. J. Phys. Conf. Ser. 2023, 2589, 012003. [Google Scholar] [CrossRef]
- Sun, Y.; Chen, Y.; Wu, P.; Wang, X.; Wang, Q. DRL: Dynamic rebalance learning for adversarial robustness of UAV with long-tailed distribution. Comput. Commun. 2023, 205, 14–23. [Google Scholar] [CrossRef]
- Chu, P.; Bian, X.; Liu, S.; Ling, H. Feature space augmentation for long-tailed data. In Proceedings of the Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, 23–28 August 2020; Proceedings, Part XXIX 16. Springer International Publishing: Cham, Switzerland, 2020; pp. 694–710. [Google Scholar]
- Shakir, S.; Topal, C. Unsupervised fabric defect detection with local spectra refinement (LSR). Neural Comput. Appl. 2024, 36, 1091–1103. [Google Scholar] [CrossRef]
- Birla, S.; Alya, S.; Singh, R. An integrated image processing approach for 3D scanning and micro-defect detection. J. Micromanuf. 2023, 6, 172–181. [Google Scholar] [CrossRef]
- Davidson, J.L. Classification of Lattice Transformations in Image Processing. Comput. Vis. Image Underst. 1993, 57, 283–306. [Google Scholar] [CrossRef]
- Shutaro, Y. Image Processing Device, Control Method, and Program. WO2018088238A1, 17 May 2018. [Google Scholar]
- Li, D.; Wu, H.; Zhang, J.; Huang, K. Fast a3rl: Aesthetics-aware adversarial reinforcement learning for image cropping. IEEE Trans. Image Process. 2019, 28, 5105–5120. [Google Scholar] [CrossRef]
- He, Y.; Ye, Y.; Hanhart, P.; Xiu, X. Motion compensated prediction with geometry padding for 360 video coding. In Proceedings of the 2017 IEEE Visual Communications and Image Processing (VCIP), St. Petersburg, FL, USA, 10–13 December 2017; IEEE: New York, NY, USA, 2017; pp. 1–4. [Google Scholar]
- Cruz-Roa, A.A.; Arevalo Ovalle, J.E.; Madabhushi, A.; González Osorio, F.A. A deep learning architecture for image representation, visual interpretability and automated basal-cell carcinoma cancer detection. In Proceedings of the Medical Image Computing and Computer-Assisted Intervention–MICCAI 2013: 16th International Conference, Nagoya, Japan, 22–26 September 2013; Proceedings, Part II 16. Springer: Berlin/Heidelberg, Germany, 2013; pp. 403–410. [Google Scholar]
- Jozef, G.; Cassano, J.; Dahlke, S.; de Boer, G. Testing the efficacy of atmospheric boundary layer height detection algorithms using uncrewed aircraft system data from MOSAiC. Atmos. Meas. Tech. Discuss. 2022, 15, 4001–4022. [Google Scholar] [CrossRef]
- Zou, D.; Cao, Y.; Li, Y.; Gu, Q. The benefits of mixup for feature learning. In Proceedings of the International Conference on Machine Learning, Honolulu, HI, USA, 23–29 July 2023; PMLR: New York, NY, USA, 2023; pp. 43423–43479. [Google Scholar]
- Huaizhou, Y.; Danyang, X. Research and Analysis of Image Enhancement Algorithm in the Classification of Rock Thin Section Images. In Proceedings of the 2021 3rd International Conference on Intelligent Control, Measurement and Signal Processing and Intelligent Oil Field (ICMSP), Xi’an, China, 23–25 July 2021; IEEE: New York, NY, USA, 2021; pp. 125–128. [Google Scholar]
- Ren, S.; He, K.; Girshick, R.; Sun, J. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. IEEE Trans. Pattern Anal. Mach. Intell. 2017, 39, 1137–1149. [Google Scholar] [CrossRef]
- Duan, K.; Bai, S.; Xie, L.; Qi, H.; Huang, Q.; Tian, Q. CenterNet: Keypoint Triplets for Object Detection. In Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Republic of Korea, 27 October–2 November 2019; pp. 6568–6577. [Google Scholar]
- Redmon, J.; Divvala, S.; Girshick, R.; Farhadi, A. You only look once: Unified, real-time object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 26 June–1 July 2016; pp. 779–788. [Google Scholar]
- Stergiou, A.; Poppe, R. Adapool: Exponential adaptive pooling for information-retaining downsampling. IEEE Trans. Image Process. 2022, 32, 251–266. [Google Scholar] [CrossRef] [PubMed]
- Cheng, W.; Liang, P.; Cao, S. Texture Detect on Rotary-Veneer Surface Based on Semi-Fuzzy Clustering Algorithm. In Proceedings of the Computer and Computing Technologies in Agriculture III: Third IFIP TC 12 International Conference, CCTA 2009, Beijing, China, 14–17 October 2009; Revised Selected Papers 3. Springer: Berlin/Heidelberg, Germany, 2010; pp. 369–374. [Google Scholar]
- Zhang, W.; Zeng, Y.; Wang, J.; Ma, H.; Zhang, Q.; Fan, S. Multi-scale feature pyramid approach for melt track classification in laser powder bed fusion via coaxial high-speed imaging. Comput. Ind. 2023, 151, 103975. [Google Scholar] [CrossRef]
- Zhan, C.; Duan, X.; Xu, S.; Song, Z.; Luo, M. An improved UAV object detection algorithm based on ASFF-YOLOv5s. Math. Biosci. Eng. MBE 2023, 20, 10773–10789. [Google Scholar]
- Zhou, S.; Zhao, J.; Shi, Y.S.; Wang, Y.F.; Mei, S.Q. Research on improving YOLOv5s algorithm for fabric defect detection. Int. J. Cloth. Sci. Technol. 2023, 35, 88–106. [Google Scholar] [CrossRef]
- Bawankule, R.; Gaikwad, V.; Kulkarni, I.; Kulkarni, S.; Jadhav, A.; Ranjan, N. Visual Detection of Waste using YOLOv8. In Proceedings of the 2023 International Conference on Sustainable Computing and Smart Systems (ICSCSS), Coimbatore, India, 14–16 June 2023; pp. 869–873. [Google Scholar]
- Hobert, J.P. The data augmentation algorithm: Theory and methodology. In Handbook of Markov Chain Monte Carlo; CRC Press: Boca Raton, FL, USA, 2011; pp. 253–293. [Google Scholar]
- Guo, A.; Sun, K.; Zhang, Z. A lightweight YOLOv8 integrating FasterNet for real-time underwater object detection. J. Real-Time Image Process. 2024, 21, 49. [Google Scholar] [CrossRef]
- Sunkara, R.; Luo, T. No more strided convolutions or pooling: A new CNN building block for low-resolution images and small objects. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases; Springer Nature: Cham, Switzerland, 2022; pp. 443–459. [Google Scholar]
- Tan, M.; Pang, R.; Le, Q.V. EfficientDet: Scalable and Efficient Object Detection. In Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, 13–19 June 2020; pp. 10778–10787. [Google Scholar]
- Wang, J.; Xu, C.; Yang, W.; Yu, L. A Normalized Gaussian Wasserstein Distance for Tiny Object Detection. arXiv 2021, arXiv:2110.13389. [Google Scholar]
- Xu, H.; Gao, Y.; Yu, F.; Darrell, T. End-to-end learning of driving models from large-scale video datasets. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 21–26 July 2017; pp. 2174–2182. [Google Scholar]
- Feng, J.; Wang, Z.; Wang, S.; Tian, S.; Xu, H. MSDD-YOLOX: An enhanced YOLOX for real-time surface defect detection of oranges by type. Eur. J. Agron. 2023, 149, 126918. [Google Scholar] [CrossRef]
- Ling, Z.; Zhang, A.; Ma, D.; Shi, Y.; Wen, H. Deep Siamese Semantic Segmentation Network for PCB Welding Defect Detection. IEEE Trans. Instrum. Meas. 2022, 71, 5006511. [Google Scholar] [CrossRef]
- Li, C.; Huang, Y.; Li, H.; Zhang, X. A weak supervision machine vision detection method based on artificial defect simulation. Knowl.-Based Syst. 2020, 208, 106466. [Google Scholar] [CrossRef]
- Liu, J.; Guo, F.; Gao, H.; Li, M.; Zhang, Y.; Zhou, H. Defect detection of injection molding products on small datasets using transfer learning. J. Manuf. Process. 2021, 70, 400–413. [Google Scholar] [CrossRef]
- Li, J.; Su, Z.; Geng, J.; Yin, Y. Real-time detection of steel strip surface defects based on improved yolo detection network. IFAC-Pap. 2018, 51, 76–81. [Google Scholar] [CrossRef]
- Liu, J.; Wang, C.; Su, H.; Du, B.; Tao, D. Multistage GAN for Fabric Defect Detection. IEEE Trans. Image Process. 2020, 29, 3388–3400. [Google Scholar] [CrossRef] [PubMed]
- Pan, Y.; Zhang, L. Dual attention deep learning network for automatic steel surface defect segmentation. Comput.-Aided Civ. Infrastruct. Eng. 2022, 37, 1468–1487. [Google Scholar] [CrossRef]
- He, K.; Liu, X.; Liu, J.; Wu, P. A multitask learning-based neural network for defect detection on textured surfaces under weak supervision. IEEE Trans. Instrum. Meas. 2021, 70, 5016914. [Google Scholar] [CrossRef]
Configuration | Type |
---|---|
OS | Windows10 |
CPU | i9-11900k |
GPU | 4090 |
CUDA | 11.8 |
Python | 3.10 |
PyTorch | 2.0.1 |
Visual Studio | 2019 |
Parameters | Value |
---|---|
Batch Size | 32 |
Lr | 0.01 |
Mixup | 0.5 |
Mosaic | 0.8 |
Copy_Paste | 0.5 |
Epoch | 300 |
Momentum | 0.937 |
FasterNet (F) | SPD (S) | BiFPN (B) | NWD (N) | mAP (%) | Precision (%) | Recall (%) | FPS | |
---|---|---|---|---|---|---|---|---|
YOLOv8 | 92.6 | 96.5 | 91.7 | 65 | ||||
YOLOv8+F | √ | 92.8 | 96.8 | 92.1 | 76 | |||
YOLOV8+FS | √ | √ | 96.2 | 98.5 | 94.4 | 73 | ||
YOLOV8+FSB | √ | √ | √ | 97.3 | 98.8 | 95.2 | 71 | |
YOLOV8+FSBN | √ | √ | √ | √ | 98.8 | 99.2 | 97.6 | 74 |
FPS | mAP (%) | |
---|---|---|
FasterR-CNN | 5 | 81.3 |
CenterNet | 28 | 79.8 |
YOLOv4 | 40 | 88.6 |
YOLOv5 | 56 | 91.4 |
FSNB_YOLOv8 | 74 | 98.8 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Li, J.; Wu, J.; Shao, Y. FSNB-YOLOV8: Improvement of Object Detection Model for Surface Defects Inspection in Online Industrial Systems. Appl. Sci. 2024, 14, 7913. https://doi.org/10.3390/app14177913
Li J, Wu J, Shao Y. FSNB-YOLOV8: Improvement of Object Detection Model for Surface Defects Inspection in Online Industrial Systems. Applied Sciences. 2024; 14(17):7913. https://doi.org/10.3390/app14177913
Chicago/Turabian StyleLi, Jun, Jinglei Wu, and Yanhua Shao. 2024. "FSNB-YOLOV8: Improvement of Object Detection Model for Surface Defects Inspection in Online Industrial Systems" Applied Sciences 14, no. 17: 7913. https://doi.org/10.3390/app14177913