Advanced Defect Detection in Wrap Film Products: A Hybrid Approach with Convolutional Neural Networks and One-Class Support Vector Machines with Variational Autoencoder-Derived Covariance Vectors
Abstract
:1. Introduction
- Detection of Micro-Defects: Micro-defects, often missed by traditional inspection systems, can significantly impact the quality of wrap film products. The integration of CNNs, particularly AlexNet and VGG19, allows for detailed feature extraction, capturing subtle anomalies critical for identifying micro-defects [19,20]. VAE-derived covariance vectors enhance this capability by robustly representing normal data, making deviations more detectable [21].
- Handling Variations in Film Texture and Thickness: Variations in texture and thickness can introduce noise and inconsistencies in defect detection. Our method leverages CNNs’ adaptability to learn and generalize from these variations, while the OCSVM, trained on normal images, distinguishes true defects from variations. This combination ensures that the system remains sensitive to defects while being robust against benign variations in the film [10].
- Training Without Defect Images: One of the significant advancements of this study is the ability to train OCSVM models using only normal images. This approach addresses the common limitation of requiring defect images for training, which are often scarce and difficult to obtain in sufficient quantities. The use of Sequential Minimal Optimization (SMO), as described by Platt (1998), provides a fast and efficient algorithm for training support vector machines (SVMs), further enhancing the practicality of training OCSVM models in such scenarios [22]. This capability allows for effective anomaly detection even with limited defect data, providing a practical advantage in real-world industrial applications.
2. Image Dataset and Training
2.1. Image Dataset for Training and Testing
2.2. Training of the One-Class SVM
2.3. Combined Strengths of AlexNet, VGG19, and VAE in Feature Extraction
2.3.1. Individual Contributions
2.3.2. Synergistic Integration
2.3.3. Role of Covariance Vectors in VAE’s Latent Variables
2.3.4. Maximizing Defect Detection Efficiency
3. One-Class SVMs (OCSVMs)
3.1. Overview of OCSVMs
3.2. Overview of VAEs
3.3. Training of the VAE
3.4. An OCSVM Using a VAE as a Feature Extractor
3.4.1. Data Distribution and Experimental Setup
- a.
- Training Data: The wrap film manufacturer provided 9512 normal images, which were used to optimize the OCSVM using a solver called sequential minimal optimization (SMO) [19].
- b.
- Test Data: The test data contained 320 normal images and 320 anomaly images. The anomaly images consisted of 160 originals and 160 augmented by horizontal flipping. As shown in Figure 5, the 320 anomaly test images include various defects.
3.4.2. Examples of Defects and Image Processing
3.4.3. VAE Reconstruction and Anomaly Detection
3.5. OCSVMs Using AlexNet as Feature Extractor
3.6. OCSVMs Using VGG19 as a Feature Extractor
4. Results and Discussion
5. Conclusions and Future Work
5.1. Summary of Findings
5.2. Key Innovations and Advantages
5.3. Practical Implications
5.4. Comparison to Initial Goals
5.5. Future Research Directions
- a.
- Enhancing dataset quality and size, encompassing various defect types, lighting conditions, and production stages.
- b.
- Exploring methods like noise reduction, illumination normalization, and data augmentation to improve model robustness.
- c.
- Combining strengths of different models to enhance overall accuracy and minimize individual model weaknesses.
- d.
- Bridging the gap between labeled datasets and real-world anomalies.
- e.
- Implementing strategies that dynamically adjust decision thresholds based on input data characteristics.
- a.
- Investigating attention mechanisms or self-supervised learning to capture finer details and enhance discriminatory power.
- g.
- Evaluating newer CNN architectures such as ResNet, WideResNet, and MobileNet for feature extraction to potentially improve anomaly detection performance by leveraging their advanced design and feature extraction capabilities.
- h.
- Exploring hybrid structures that combine the strengths of multiple architectures to further enhance anomaly detection performance.
Author Contributions
Funding
Data Availability Statement
Conflicts of Interest
References
- Nakashima, K.; Nagata, F.; Watanabe, K. Detecting defects in a wrapped roll using convolutional neural networks. In Proceedings of the 36th Fuzzy System Symposium, Virtual, 7–9 September 2020; pp. 111–115. (In Japanese). [Google Scholar]
- Ehara, F.; Muto, Y. Bearing abnormal sound detection system using a one-class support vector machine. Jpn. J. Inst. Ind. Appl. Eng. 2021, 9, 31–37. (In Japanese) [Google Scholar]
- Ulger, F.; Yuksel, S.E.; Yilmaz, A. Anomaly detection for solder joints using β-VAE. IEEE Trans. Compon. Packag. Manuf. Technol. 2021, 11, 2214–2221. [Google Scholar] [CrossRef]
- Tang, H.; Liang, S.; Yao, D.; Qiao, Y. Improved STMask R-CNN-based Defect Detection Model for Automatic Visual Inspection. Appl. Opt. J. 2023, 62, 8869–8881. [Google Scholar] [CrossRef] [PubMed]
- Srilakshmi, V.; Kiran, G.U.; Yashwanth, G.; Gayathri Ch, N.; Raju, A.S. Automatic Visual Inspection—Defects Detection using CNN. In Proceedings of the 6th International Conference of Electronics, Communication and Aerospace Technology (ICECA 2022), Coimbatore, India, 1–3 December 2022. [Google Scholar] [CrossRef]
- Lee, K.-H.; Lee, W.H.; Yun, G.J. A Defect Detection Framework Using Three-Dimensional Convolutional Neural Network (3D-CNN) with in-situ Monitoring Data in Laser Powder Bed Fusion Process. Opt. Laser Technol. 2023, 165, 109571. [Google Scholar] [CrossRef]
- Jiang, X.; Guo, K.; Lu, Y.; Yan, F.; Liu, H.; Cao, J.; Xu, M.; Tao, D. CINFormer: Transformer Network with Multi-Stage CNN Backbones for Defect Detection. Computer Vision and Pattern Recognition. aXiv 2023, arXiv:2309.12639. [Google Scholar]
- Choi, N.-H.; Sohn, J.W.; Oh, J.-S. Defect Detection Model Using CNN and Image Augmentation for Seat Foaming Process. Mathematics 2023, 11, 4894. [Google Scholar] [CrossRef]
- Pimentel MA, F.; Clifton, D.A.; Clifton, L.; Tarassenko, L. A Review of Novelty Detection. Signal Process. 2014, 99, 215–249. [Google Scholar] [CrossRef]
- Schölkopf, B.; Platt, J.C.; Shawe-Taylor, J.; Smola, A.J.; Williamson, R.C. Estimating the Support of a High-Dimensional Distribution. Neural Comput. 2001, 13, 1443–1471. [Google Scholar] [CrossRef] [PubMed]
- Liu, Y.; Zhang, C.; Dong, X. A Survey of Real-time Surface Defect Inspection Methods Based on Deep Learning. Artif. Intell. Rev. 2023, 56, 12131–12170. [Google Scholar] [CrossRef]
- He, H.; Erfani, S.; Gong, M.; Ke, Q. Learning Transferable Representations for Image Anomaly Localization Using Dense Pretraining. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, Waikoloa, HI, USA, 3–8 January 2024; pp. 1113–1122. [Google Scholar]
- Wang, J.; Xu, G.; Li, C.; Gao, G.; Cheng, Y. Multi-feature Reconstruction Network using Crossed-mask Restoration for Unsupervised Anomaly Detection. arXiv 2024, arXiv:2404.13273. [Google Scholar]
- Kingma, D.P.; Welling, M. Auto-encoding Variational Bayes. The Latest Revised Version is v11, 10 December 2022. Available online: https://arxiv.org/abs/1312.6114v11 (accessed on 20 August 2024).
- Simonyan, K.; Zisserman, A. Very Deep Convolutional Networks for Large-scale Image Recognition. International Conference on Learning Representations (ICLR2015). arXiv 2015, arXiv:1409.1556. The revised version is also available (v6, 2015). [Google Scholar]
- Krizhevsky, A.; Sutskever, I.; Hinton, G.E. ImageNet Classification with Deep Convolutional Neural Networks. Commun. ACM 2017, 60, 84–90. [Google Scholar] [CrossRef]
- Bhushan Jha, S.; Babiceanu, R.F. Deep CNN-based Visual Defect Detection: Survey of Current Literature. Comput. Ind. 2023, 148, 103911. [Google Scholar] [CrossRef]
- Erik, W.; Seitz, H. A Machine Learning Method for Defect Detection and Visualization in Selective Laser Sintering Based on Convolutional Neural Networks. Addit. Manuf. 2021, 41, 101965. [Google Scholar] [CrossRef]
- Krizhevsky, A.; Sutskever, I.; Hinton, G.E. ImageNet Classification with Deep Convolutional Neural Networks. Adv. Neural Inf. Process. Syst. 2012, 25, 1097–1105. [Google Scholar] [CrossRef]
- Simonyan, K.; Zisserman, A. Very Deep Convolutional Networks for Large-Scale Image Recognition. arXiv 2014, arXiv:1409.1556. [Google Scholar]
- Kingma, D.P.; Welling, M. Auto-Encoding Variational Bayes. arXiv 2013, arXiv:1312.6114. [Google Scholar]
- Platt, J. Sequential Minimal Optimization: A Fast Algorithm for Training Support Vector Machines. Technical Report MSR–TR–98–14. 1998, pp. 1–24. Available online: https://www.microsoft.com/en-us/research/uploads/prod/1998/04/sequential-minimal-optimization.pdf (accessed on 20 August 2024).
- Chandola, V.; Banerjee, A.; Kumar, V. Anomaly Detection: A Survey. ACM Comput. Surv. CSUR 2009, 41, 15. [Google Scholar] [CrossRef]
- Kornblith, S.; Shlens, J.; Le, Q.V. Do better ImageNet models transfer better? In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, 15–20 June 2019; pp. 2656–2666. [Google Scholar] [CrossRef]
- Li, H.; Wang, Y. Understanding feature learning in Adversarial Settings. In Proceedings of the 2016 IEEE European Symposium on Security and Privacy (EuroS&P), Saarbrucken, Germany, 21–24 March 2016. [Google Scholar] [CrossRef]
- Razavi, A.; van den Oord, A.; Vinyals, O. Generating diverse high-fidelity images with VQ-VAE-2. In Proceedings of the Advances in Neural Information Processing Systems (NeurIPS 2019), Vancouver, BC, Canada, 8–14 December 2019. [Google Scholar] [CrossRef]
- Mathworks. MATLAB Documentation R2023b, fitcsvm. 2023. Available online: https://jp.mathworks.com/help/stats/fitcsvm.html (accessed on 1 May 2024).
- Mathworks. MATLAB Documentation R2023b, “Statistics and Machine Learning Toolbox”. 2023. Available online: https://jp.mathworks.com/help/stats/classreg.learning.classif.compactclassificationsvm.predict.html (accessed on 1 May 2024).
- Rätsch, G.; Mika, S.; Schölkopf, B.; Müller, K. Constructing Boosting Algorithms from SVMs: An Application to One-Class Classification. IEEE Trans. Pattern Anal. Mach. Intell. 2002, 24, 1184–1199. [Google Scholar] [CrossRef]
- Nakashima, K.; Nagata, F.; Otsuka, A.; Watanabe, K.; Habib, M.K. Defect Detection in Wrap Film Product Using Compact Convolutional Neural Network. Artif. Life Robot. 2021, 26, 360–366. [Google Scholar] [CrossRef]
- Ruff, L.; Vandermeulen, R.A.; Binder, A.; Görnitz, N.; Müller, E.; Müller, K.-R.; Kloft, M. Deep Semi-Supervised Anomaly Detection. In Proceedings of the Eighth International Conference on Learning Representations (ICLR 2020), Addis Ababa, Ethiopia, 26–30 April 2020. [Google Scholar]
Normal | Anomaly | |
---|---|---|
Normal | 279 | 41 |
Anomaly | 40 | 280 |
Normal | Anomaly | |
---|---|---|
Normal | 293 | 27 |
Anomaly | 42 | 278 |
Normal | Anomaly | |
---|---|---|
Normal | 288 | 32 |
Anomaly | 32 | 288 |
Method | Training Data | Accuracy | Notes |
---|---|---|---|
CNN (Nakashima et al., 2021) [30] | Normal and defective images | 95% | Requires extensive defect data |
Deep SAD (Ruff et al., 2020) [31] | Mainly normal images with minimal labeled anomaly data | Promising results with no indication of the accuracy % | Semi-supervised method |
OCSVM (Our approach) | Only normal images | 91% | Effective without needing defect images, practical advantage |
Method | Accuracy | Precision | Recall | F1 Score |
---|---|---|---|---|
OCSVM with VAE | 0.867 | 0.873 | 0.859 | 0.866 |
OCSVM with AlexNet | 0.836 | 0.841 | 0.828 | 0.835 |
OCSVM with VGG19 | 0.898 | 0.905 | 0.891 | 0.898 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Shimizu, T.; Nagata, F.; Habib, M.K.; Arima, K.; Otsuka, A.; Watanabe, K. Advanced Defect Detection in Wrap Film Products: A Hybrid Approach with Convolutional Neural Networks and One-Class Support Vector Machines with Variational Autoencoder-Derived Covariance Vectors. Machines 2024, 12, 603. https://doi.org/10.3390/machines12090603
Shimizu T, Nagata F, Habib MK, Arima K, Otsuka A, Watanabe K. Advanced Defect Detection in Wrap Film Products: A Hybrid Approach with Convolutional Neural Networks and One-Class Support Vector Machines with Variational Autoencoder-Derived Covariance Vectors. Machines. 2024; 12(9):603. https://doi.org/10.3390/machines12090603
Chicago/Turabian StyleShimizu, Tatsuki, Fusaomi Nagata, Maki K. Habib, Koki Arima, Akimasa Otsuka, and Keigo Watanabe. 2024. "Advanced Defect Detection in Wrap Film Products: A Hybrid Approach with Convolutional Neural Networks and One-Class Support Vector Machines with Variational Autoencoder-Derived Covariance Vectors" Machines 12, no. 9: 603. https://doi.org/10.3390/machines12090603