Optimizing Automated Optical Inspection: An Adaptive Fusion and Semi-Supervised Self-Learning Approach for Elevated Accuracy and Efficiency in Scenarios with Scarce Labeled Data
Abstract
:1. Introduction
- Bounding Box Assigner: This component is responsible for generating and refining the bounding boxes used to identify defects within images. It uses an intersection over union (IOU) filter to ensure that only the most reliable bounding boxes are passed on for further training [13]. This step is critical for reducing errors in the early stages of the model’s learning process [14];
- Adaptive Training Scheduler: To further refine the training process, this scheduler dynamically adjusts the learning rate and other parameters based on the confidence levels of the pseudo-labels generated during training. By doing so, it helps to prevent the model from being misled by unreliable data, which is a common challenge in semi-supervised learning [15,16,17];
- Data Allocator: This module applies various data augmentation techniques to both labeled and unlabeled datasets, increasing the diversity of the training data. The goal is to expose the model to a wide range of possible defect scenarios, thereby improving its ability to generalize across different types of products and manufacturing conditions [18,19].
2. Related Work
2.1. Traditional Machine Learning Approaches
2.2. Deep-Learning-Based Methods
2.3. Semi-Supervised Learning
2.4. Adaptive Learning and Self-Learning Techniques
2.5. Hybrid Approaches
2.6. Challenges and Future Directions
3. Research Methodology
3.1. Initial Dataset Assembly
- Input Layer: High-resolution images (input size 512 × 512 pixels);
- Feature Extraction Layers: Multiple convolutional layers with ReLU activation functions and max-pooling layers;
- Region Proposal Network (RPN): Generates region proposals where defects are likely to be located;
- Classification and Regression Heads: Predicts defect categories and refines bounding box coordinates.
3.2. Dataset Adjustment
3.3. Data Augmentation and Annotation Refinement
- Scaling: Random scaling within a range of 0.8 to 1.2 times the original size as shown in Figure 4;
- Flipping: Horizontal and vertical flips;
- Brightness and Contrast Adjustment: Random changes in brightness and contrast levels;
- Color Adjustment: Applying different color mappings to simulate various imaging conditions.
3.4. Implementation of Adaptive Fusion Semi-Supervised Self-Learning
3.5. Semi-Supervised Learning Object Detection
3.5.1. Complex Model Selection
3.5.2. Bounding Box Assigner
3.5.3. Data Allocator and Adaptive Training Scheduler
4. Performance Evaluation and Validation
4.1. Data Augmentation and Annotation Refinement
4.2. AFSL Validation on COCO and AOI Datasets
4.2.1. COCO Dataset Validation
Methodology | FLOPs | AP@50:95 (%) | EMAFSL | |
---|---|---|---|---|
Two-stage anchor-based | Baseline | 202.31G | 23.6 | |
STAC [24] | 28.64 | 2.49 | ||
Instant Teaching [49] | 30.40 | 3.36 | ||
Humble Teacher [50] | 31.61 | 3.96 | ||
Unbiased Teacher [51] | 31.50 | 3.90 | ||
Soft Teacher [52] | 34.04 | 5.16 | ||
LabelMatch [53] | 35.49 | 5.87 | ||
PseCo [54] | 36.06 | 6.15 | ||
One-stage anchor-free | Baseline | 200.59G | 23.07 | |
Unbiased Teacher v2 [55] | 32.61 | 4.76 | ||
DSL [56] | 36.22 | 6.56 | ||
Dense Teacher [48] | 37.13 | 7.01 | ||
One-stage anchor-free | Baseline | 169.61G | 24.04 | - |
Unbiased Teacher * [57] | 30.34 | 3.71 | ||
Efficient Teacher [47] | 34.09 | 5.92 | ||
Baseline | 104.7G | 25.16 | - | |
AFSL (ours) | 34.20 | 8.63 | ||
Fully Supervised | 49.7 | - |
4.2.2. AOI Dataset Validation
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
Appendix A
Defect Name | Original | First Edition | Second Edition | Final Edition |
---|---|---|---|---|
Type1 | 79.70% | 76.60% | 84% | 87.1% |
Type2 | 87.50% | 85.20% | 100% | 100% |
Type3 | 57.20% | 82.10% | 88% | 97% |
Type4 | 97.80% | 82.70% | 93% | 93% |
Type5 | 95.30% | 82.90% | 86% | 100% |
Type6 | 89.60% | 99.40% | 100% | 98.1% |
Type7 | 100.00% | 100.00% | 100% | 100% |
Type8 | 97.70% | 60.90% | 74% | 91.3% |
Type9 | 89.20% | 86.70% | 100% | 100% |
Type10 | 89.60% | 92.50% | 93% | 93% |
Type11 | 85.40% | 85.30% | 94% | 94% |
Type12 | 83.80% | 93.30% | 100% | 100% |
Type13 | 83.70% | 96.10% | 100% | 100% |
Type14 | 77.40% | 83.70% | 100% | 100% |
Type15 | 94.50% | 94.00% | 98% | 98% |
Type16 | 92.70% | 100.00% | 100% | 100% |
Type17 | 81.10% | 75.90% | 93% | 93% |
Type18 | 88.20% | 30.80% | 56% | 75% |
Type19 | 78.70% | 75.50% | 77% | 84% |
Type20 | 74.20% | 76.30% | 76% | 76% |
Type21 | 74.40% | 82.80% | 86% | 86% |
Type22 | 75.80% | 87.10% | 93% | 93% |
Type23 | 68.80% | 88.00% | 82% | 82% |
Type24 | 72.70% | 70.20% | 84% | 84% |
Type25 | 86.70% | 86.90% | 97% | 97% |
Type26 | 88.50% | 93.30% | 94.6% | 94.6% |
Type27 | 67.30% | 58.90% | 82% | 87% |
Type28 | 87.30% | 89.20% | 95% | 95% |
Type29 | 84.70% | 87.30% | 98% | 98% |
Type30 | 85.60% | 69.10% | 83% | 92% |
Type31 | 100.00% | 100.00% | 100% | 100% |
Type32 | 71.70% | 78.40% | 92% | 93.5% |
Total | 84.00% | 83.50% | 91.30% | 93.88% |
Appendix B
Defect Name | Precision | Precision | AP (0.5) | AP (0.5) | AP (0.5-0.95) | AP (0.5-0.95) |
---|---|---|---|---|---|---|
Baseline | AFSL | Baseline | AFSL | Baseline | AFSL | |
Total | 0.921 | 0.927 | 0.968 | 0.97 | 0.704 | 0.73 |
Type1 | 0.843 | 0.869 | 0.966 | 0.966 | 0.588 | 0.633 |
Type2 | 0.879 | 0.91 | 0.986 | 0.993 | 0.672 | 0.711 |
Type3 | 0.967 | 0.95 | 0.993 | 0.992 | 0.992 | 0.987 |
Type4 | 0.972 | 1 | 0.992 | 0.995 | 0.711 | 0.746 |
Type5 | 0.978 | 0.988 | 0.992 | 0.993 | 0.988 | 0.988 |
Type6 | 0.977 | 0.978 | 0.995 | 0.995 | 0.695 | 0.732 |
Type7 | 0.992 | 0.992 | 0.995 | 0.995 | 0.802 | 0.842 |
Type8 | 0.999 | 0.998 | 0.995 | 0.995 | 0.995 | 0.994 |
Type9 | 0.767 | 0.823 | 0.966 | 0.97 | 0.737 | 0.775 |
Type10 | 0.929 | 0.934 | 0.976 | 0.981 | 0.584 | 0.607 |
Type11 | 0.943 | 0.947 | 0.963 | 0.971 | 0.656 | 0.712 |
Type12 | 0.788 | 0.819 | 0.89 | 0.912 | 0.567 | 0.594 |
Type13 | 1 | 0.997 | 0.995 | 0.95 | 0.832 | 0.847 |
Type14 | 0.95 | 967 | 0.973 | 0.982 | 0.521 | 0.554 |
Type15 | O.927 | 0.936 | 0.99 | 0.993 | 0.75 | 0.773 |
Type16 | 0.805 | 0.788 | 0.751 | 0.748 | 0.523 | 0.521 |
Type17 | 0.737 | 0.769 | 0.826 | 0.859 | 0.486 | 0.524 |
Type18 | 0.937 | 0.936 | 0.978 | 0.981 | 0.803 | 0.822 |
Type19 | 0.903 | 0.915 | 0.976 | 0.974 | 0.58 | 0.616 |
Type20 | 0.934 | 0.964 | 0.978 | 0.982 | 0.813 | 0.837 |
Type21 | 0.859 | 0.862 | 0.973 | 0.976 | 0.641 | 0.66 |
Type22 | 1 | 1 | 0.995 | 0.995 | 0.837 | 0.859 |
Type23 | 0.994 | 0.993 | 0.995 | 0.995 | 0.752 | 0.762 |
Type24 | 0.902 | 0.905 | 0.978 | 0.985 | 0.682 | 0.728 |
Type25 | 0.881 | 0.89 | 0.979 | 0.939 | 0.612 | 0.63 |
Type26 | 0.957 | 0.942 | 0.991 | 0.992 | 0.827 | 0.846 |
Type27 | 0.95 | 0.959 | 0.985 | 0.994 | 0.655 | 0.696 |
Type28 | 0.997 | 0.3998 | 0.995 | 0.995 | 0.754 | 0.804 |
Type29 | 0.919 | 0.877 | 0.984 | 0.973 | 0.59 | 0.622 |
Type30 | 0.742 | 0.933 | 0.946 | 0.949 | 0.585 | 0.608 |
Type31 | 1 | 1 | 0.995 | 0.995 | 0.546 | 0.597 |
Type32 | 0.849 | 0.839 | 0.986 | 0.981 | 0.694 | 0.733 |
References
- Noroozi, M.; Ghadermazi, J.; Shah, A.; Zayas-Castro, J.L. Toward Optimal Defect Detection in Assembled Printed Circuit Boards Under Adverse Conditions. IEEE Access 2023, 11, 127119–127131. [Google Scholar] [CrossRef]
- Rezaei, A.; Nau, J.; Streitferdt, D.; Schambach, J.; Vangelov, T. ReProInspect: Framework for Reproducible Defect Datasets for Improved AOI of PCBAs. In Engineering of Computer-Based Systems: 8th International Conference, ECBS 2023, Västerås, Sweden, 16–18 October 2023, Proceedings; Springer: Berlin/Heidelberg, Germany, 2023; pp. 205–214. [Google Scholar] [CrossRef]
- Lin, H.-I.; Wibowo, F.S. Image Data Assessment Approach for Deep Learning-Based Metal Surface Defect-Detection Systems. IEEE Access 2021, 9, 47621–47638. [Google Scholar] [CrossRef]
- Ye, R.; Chang, M.; Pan, C.-S.; Chiang, C.A.; Gabayno, J.L. High-resolution optical inspection system for fast detection and classification of surface defects. Int. J. Optomechatronics 2018, 12, 1–10. [Google Scholar] [CrossRef]
- Phoulady, A.; Suleiman, Y.; Choi, H.; Moore, T.; May, N.; Shahbazmohamadi, S.; Tavousi, P. Synthetic data augmentation to enhance manual and automated defect detection in microelectronics. Microelectron. Reliab. 2023, 150, 115220. [Google Scholar] [CrossRef]
- Dai, W.; Mujeeb, A.; Erdt, M.; Sourin, A. Towards Automatic Optical Inspection of Soldering Defects. In Proceedings of the 2018 International Conference on Cyberworlds (CW), Singapore, 3–5 October 2018; pp. 375–382. [Google Scholar] [CrossRef]
- Dupre, R.; Fajtl, J.; Argyriou, V.; Remagnino, P. Improving Dataset Volumes and Model Accuracy With Semi-Supervised Iterative Self-Learning. IEEE Trans. Image Process. 2020, 29, 4337–4348. [Google Scholar] [CrossRef]
- Dópido, I.; Li, J.; Marpu, P.R.; Plaza, A.; Dias, J.M.B.; Benediktsson, J.A. Semisupervised Self-Learning for Hyperspectral Image Classification. IEEE Trans. Geosci. Remote Sens. 2023, 51, 4032–4044. [Google Scholar] [CrossRef]
- Ahmad, H.M.; Rahimi, A. Deep learning methods for object detection in smart manufacturing: A survey. J. Manuf. Syst. 2022, 64, 181–196. [Google Scholar] [CrossRef]
- Ghamisi, A.; Charter, T.; Ji, L.; Rivard, M.; Lund, G.; Najjaran, H. Anomaly detection in automated fibre placement: Learning with data limitations. Front. Manuf. Technol. 2024, 4, 1277152. [Google Scholar] [CrossRef]
- Lv, X.; Duan, F.; Jiang, J.-J.; Fu, X.; Gan, L. Deep Active Learning for Surface Defect Detection. Sensors 2020, 20, 6. [Google Scholar] [CrossRef] [PubMed]
- Pham, T.T.A.; Thoi, D.K.T.; Choi, H.; Park, S. Defect Detection in Printed Circuit Boards Using Semi-Supervised Learning. Sensors 2023, 23, 6. [Google Scholar] [CrossRef]
- Hurtik, P.; Vajgl, M. BBRefinement: An Universal Scheme to Improve Precision of Box Object Detectors. October 2020. Available online: https://openreview.net/forum?id=RB0iNPXIj60 (accessed on 22 August 2024).
- Cruciata, G.; Presti, L.L.; La Cascia, M. Iterative Multiple Bounding-Box Refinements for Visual Tracking. J. Imaging 2022, 8, 61. [Google Scholar] [CrossRef] [PubMed]
- Li, K.; Lian, Q.; Gao, C.; Zhang, F. AatMatch: Adaptive Adversarial Training in Semi-Supervised Learning Based on Data-Driven Decision-Making Models. Systems 2023, 11, 5. [Google Scholar] [CrossRef]
- Xie, Q.; Luong, M.-T.; Hovy, E.; Le, Q.V. Self-training with Noisy Student improves ImageNet classification. arXiv 2020, arXiv:1911.04252. [Google Scholar] [CrossRef]
- Oh, S.; Jeong, J.; Lee, C.-G.; Yoo, J.; Nam, G. Synergistic Training: Harnessing Active Learning and Pseudo-Labeling for Enhanced Model Performance in Deep Learning. WSEAS Trans. Comput. 2023, 22, 114–119. [Google Scholar] [CrossRef]
- Mertes, S.; Margraf, A.; Kommer, C.; Geinitz, S.; André, E. Data Augmentation for Semantic Segmentation in the Context of Carbon Fiber Defect Detection using Adversarial Learning. Delta 2020. Available online: https://publica.fraunhofer.de/handle/publica/410037 (accessed on 29 July 2024).
- Kim, E.K.; Lee, H.; Kim, J.Y.; Kim, S. Data Augmentation Method by Applying Color Perturbation of Inverse PSNR and Geometric Transformations for Object Recognition Based on Deep Learning. Appl. Sci. 2020, 10, 3755. [Google Scholar] [CrossRef]
- Rizve, M.N.; Duarte, K.; Rawat, Y.S.; Shah, M. In Defense of Pseudo-Labeling: An Uncertainty-Aware Pseudo-label Selection Framework for Semi-Supervised Learning. arXiv 2021, arXiv:2101.06329. [Google Scholar] [CrossRef]
- Chen, H.; Tao, R.; Fan, Y.; Wang, Y.; Wang, J.; Schiele, B.; Xie, X.; Raj, B.; Savvides, M. SoftMatch: Addressing the Quantity-Quality Trade-off in Semi-supervised Learning. arXiv 2023, arXiv:2301.10921. [Google Scholar] [CrossRef]
- Jin, Z.; Zhong, D.; Song, X.; Liu, Z.; Ye, N.; Zeng, Q. Filter and evolve: Progressive pseudo label refining for semi-supervised automatic speech recognition. arXiv 2022, arXiv:2210.16318. [Google Scholar] [CrossRef]
- Sun, Y.; Gao, C. NCMatch: Semi-supervised Learning with Noisy Labels via Noisy Sample Filter and Contrastive Learning. In Pattern Recognition and Computer Vision; Liu, Q., Wang, H., Ma, Z., Zheng, W., Zha, H., Chen, X., Wang, L., Ji, R., Eds.; Springer Nature: Singapore, 2024; pp. 15–27. [Google Scholar] [CrossRef]
- Sohn, K.; Zhang, Z.; Li, C.-L.; Zhang, H.; Lee, C.-Y.; Pfister, T. A Simple Semi-Supervised Learning Framework for Object Detection. arXiv 2020, arXiv:2005.04757. [Google Scholar] [CrossRef]
- Rosenberg, C.; Hebert, M.; Schneiderman, H. Semi-Supervised Self-Training of Object Detection Models. In Proceedings of the 2005 Seventh IEEE Workshops on Applications of Computer Vision (WACV/MOTION’05)–Volume 1, Breckenridge, CO, USA, 5–7 January 2005; pp. 29–36. [Google Scholar] [CrossRef]
- Kang, S.B.; Lee, J.H.; Song, K.Y.; Pahk, H.J. Automatic defect classification of TFT-LCD panels using machine learning. In Proceedings of the 2009 IEEE International Symposium on Industrial Electronics, Seoul, Republic of Korea, 5–8 July 2009; pp. 2175–2177. [Google Scholar] [CrossRef]
- Iwahori, Y. Defect Detection and Classification of Electronic Circuit Boards Using Keypoint Extraction and CNN Features. 2017. Available online: https://www.semanticscholar.org/paper/Defect-Detection-and-Classification-of-Electronic-Iwahori/c7771f41941cf3a146ff61022594f9e5348bcf70 (accessed on 22 August 2024).
- Akbar, H.; Suryana, N.; Akbar, F. Surface Defect Detection and Classification Based on Statistical Filter and Decision Tree. Int. J. Comput. Theory Eng. 2013, 5, 774–779. [Google Scholar] [CrossRef]
- Reshadat, V.; Kapteijns, R.A.J.W. Improving the Performance of Automated Optical Inspection (AOI) Using Machine Learning Classifiers. In Proceedings of the 2021 International Conference on Data and Software Engineering (ICoDSE), Bandung, Indonesia, 3–4 November 2021; pp. 1–5. [Google Scholar] [CrossRef]
- Ren, R.; Hung, T.; Tan, K.C. A Generic Deep-Learning-Based Approach for Automated Surface Inspection. IEEE Trans. Cybern. 2018, 48, 929–940. [Google Scholar] [CrossRef] [PubMed]
- Yuan-Fu, Y. A Deep Learning Model for Identification of Defect Patterns in Semiconductor Wafer Map. In Proceedings of the 2019 30th Annual SEMI Advanced Semiconductor Manufacturing Conference (ASMC), Saratoga Springs, NY, USA, 6–9 May 2019; pp. 1–6. [Google Scholar] [CrossRef]
- Zheng, X.; Wang, H.; Chen, J.; Kong, Y.; Zheng, S. A Generic Semi-Supervised Deep Learning-Based Approach for Automated Surface Inspection. IEEE Access 2020, 8, 114088–114099. [Google Scholar] [CrossRef]
- Bhatt, P.M.; Malhan, R.K.; Rajendran, P.; Shah, B.C.; Thakar, S.; Yoon, Y.J.; Gupta, S.K. Image-Based Surface Defect Detection Using Deep Learning: A Review. J. Comput. Inf. Sci. Eng. 2021, 21, 040801. [Google Scholar] [CrossRef]
- Paliulis, E.; Zemblys, R.; Daunys, G. IMAGE ANALYSIS PROBLEMS IN AOI SYSTEMS. Inf. Technol. Control. 2008, 37, 3. Available online: https://itc.ktu.lt/index.php/ITC/article/view/11947 (accessed on 22 August 2024).
- Richter, J.; Streitferdt, D.; Rozova, E. On the development of intelligent optical inspections. In Proceedings of the 2017 IEEE 7th Annual Computing and Communication Workshop and Conference (CCWC), Las Vegas, NV, USA, 9–11 January 2017; pp. 1–6. [Google Scholar] [CrossRef]
- Hung, Y.-H. Developing an Anomaly Detection System for Automatic Defective Products’ Inspection. Processes 2022, 10, 8. [Google Scholar] [CrossRef]
- Ebayyeh, A.A.R.M.A.; Mousavi, A. A Review and Analysis of Automatic Optical Inspection and Quality Monitoring Methods in Electronics Industry. IEEE Access 2020, 8, 183192–183271. [Google Scholar] [CrossRef]
- Yin, A.; Fung, A. Effective analysis of optical inspection machines (IMPACT 2011). In Proceedings of the 2011 6th International Microsystems, Packaging, Assembly and Circuits Technology Conference (IMPACT), Taipei, Taiwan, 19–21 October 2011; pp. 408–410. [Google Scholar] [CrossRef]
- Sun, W.; Kontar, A.R.; Jin, J.; Chang, T.-S. A continual learning framework for adaptive defect classification and inspection. J. Qual. Technol. 2023, 55, 598–614. [Google Scholar] [CrossRef]
- Landgraf, C.; Meese, B.; Pabst, M.; Martius, G.; Huber, M.F. A Reinforcement Learning Approach to View Planning for Automated Inspection Tasks. Sensors 2021, 21, 6. [Google Scholar] [CrossRef]
- Jing, W.; Goh, C.F.; Rajaraman, M.; Gao, F.; Park, S.; Liu, Y.; Shimada, K. A Computational Framework for Automatic Online Path Generation of Robotic Inspection Tasks via Coverage Planning and Reinforcement Learning. IEEE Access 2018, 6, 54854–54864. [Google Scholar] [CrossRef]
- Schlosser, T.; Friedrich, M.; Beuth, F.; Kowerko, D. Improving automated visual fault inspection for semiconductor manufacturing using a hybrid multistage system of deep neural networks. J. Intell. Manuf. 2022, 33, 1099–1123. [Google Scholar] [CrossRef]
- Chu, Y.; Feng, D.; Liu, Z.; Zhao, Z.; Wang, Z.; Xia, X.G.; Quek, T.Q. Hybrid-Learning-Based Operational Visual Quality Inspection for Edge-Computing-Enabled IoT System. IEEE Internet Things J. 2022, 9, 4958–4972. [Google Scholar] [CrossRef]
- Schlosser, T.; Beuth, F.; Friedrich, M.; Kowerko, D. A Novel Visual Fault Detection and Classification System for Semiconductor Manufacturing Using Stacked Hybrid Convolutional Neural Networks. In Proceedings of the 2019 24th IEEE International Conference on Emerging Technologies and Factory Automation (ETFA), Zaragoza, Spain, 10–13 September 2019; pp. 1511–1514. [Google Scholar] [CrossRef]
- Wang, C.-Y.; Bochkovskiy, A.; Liao, H.-Y.M. YOLOv7: Trainable Bag-of-Freebies Sets New State-of-the-Art for Real-Time Object Detectors. In Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada, 17–24 June 2023; pp. 7464–7475. [Google Scholar] [CrossRef]
- Tarvainen, A.; Valpola, H. Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results. Adv. Neural Inf. Process. Syst. 2017, 30, 1195–1240. [Google Scholar]
- Xu, B.; Chen, M.; Guan, W.; Hu, L. Efficient Teacher: Semi-Supervised Object Detection for YOLOv5. arXiv 2023, arXiv:2302.07577. [Google Scholar] [CrossRef]
- Zhou, H.; Ge, Z.; Liu, S.; Mao, W.; Li, Z.; Yu, H.; Sun, J. Dense Teacher: Dense Pseudo-Labels for Semi-supervised Object Detection. In Computer Vision—ECCV 2022; Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T., Eds.; Springer Nature: Cham, Switzerland, 2022; pp. 35–50. [Google Scholar] [CrossRef]
- Zhou, Q.; Yu, C.; Wang, Z.; Qian, Q.; Li, H. Instant-Teaching: An End-to-End Semi-Supervised Object Detection Framework. In Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA, 20–25 June 2021; pp. 4079–4088. [Google Scholar] [CrossRef]
- Tang, Y.; Chen, W.; Luo, Y.; Zhang, Y. Humble Teachers Teach Better Students for Semi-Supervised Object Detection. In Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), IEEE Computer Society, Nashville, TN, USA, 20–25 June 2021; pp. 3131–3140. [Google Scholar] [CrossRef]
- Liu, Y.-C.; Ma, C.Y.; He, Z.; Kuo, C.W.; Chen, K.; Zhang, P.; Wu, B.; Kira, Z.; Vajda, P. Unbiased Teacher for Semi-Supervised Object Detection. arXiv 2021, arXiv:2102.09480. [Google Scholar] [CrossRef]
- Xu, M.; Zhang, Z.; Hu, H.; Wang, J.; Wang, L.; Wei, F.; Bai, X.; Liu, Z. End-to-End Semi-Supervised Object Detection with Soft Teacher. In Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada, 10–17 October 2021; pp. 3040–3049. [Google Scholar] [CrossRef]
- Chen, B.; Chen, W.; Yang, S.; Xuan, Y.; Song, J.; Xie, D.; Pu, S.; Song, M.; Zhuang, Y. Label Matching Semi-Supervised Object Detection. In Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA, 18–24 June 2022; pp. 14361–14370. [Google Scholar] [CrossRef]
- Li, G.; Li, X.; Wang, Y.; Wu, Y.; Liang, D.; Zhang, S. PseCo: Pseudo Labeling and Consistency Training for Semi-Supervised Object Detection. In Computer Vision—ECCV 2022; Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T., Eds.; Springer Nature: Cham, Switzerland, 2022; pp. 457–472. [Google Scholar] [CrossRef]
- Liu, Y.-C.; Ma, C.-Y.; Kira, Z. Unbiased Teacher v2: Semi-supervised Object Detection for Anchor-free and Anchor-based Detectors. In Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), IEEE Computer Society, New Orleans, LA, USA, 18–24 June 2022; pp. 9809–9818. [Google Scholar] [CrossRef]
- Chen, B.; Li, P.; Chen, X.; Wang, B.; Zhang, L.; Hua, X.-S. Dense Learning based Semi-Supervised Object Detection. In Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA, 18–24 June 2022; pp. 4805–4814. [Google Scholar] [CrossRef]
- Schmutz, H.; Humbert, O.; Mattei, P.-A. Don’t fear the unlabelled: Safe semi-supervised learning via simple debiasing. arXiv 2022, arXiv:2203.07512. [Google Scholar] [CrossRef]
Method | Resolution | Mosaic | Param. | FLOPs | AP50:95 (%) | Method | Resolution | Mosaic | Param. |
---|---|---|---|---|---|---|---|---|---|
FRCNN | [1333,800] | X | 39.8M | 202.31G | 40.3 | FRCNN | [1333,800] | X | 39.8M |
FCOS | [1333,800] | X | 32.02M | 200.59G | 38.5 | FCOS | [1333,800] | X | 32.02M |
YOLOv5w/o | [640,640] | X | 42.56M | 109.59G | 41.2 | YOLOv5w/o | [640,640] | X | 42.56M |
YOLOv5 | [640,640] | ✓ | 42.56M | 109.59G | 49 | YOLOv5 | [640,640] | ✓ | 42.56M |
YOLOv7 | [640,640] | ✓ | 37.62M | 106.59G | 51.5 | YOLOv7 | [640,640] | ✓ | 37.62M |
RetinaNet | [1333,800] | X | 37.74M | 239.32G | 39.5 | RetinaNet | [1333,800] | X | 37.74M |
DenseNet | [640,640] | ✓ | 42.13M | 169.61G | 44.86 | DenseNet | [640,640] | ✓ | 42.13M |
Datasets | Model Accuracy |
---|---|
No modifications | 84.00% |
First Edition (Category Refinement) | 83.50% |
Second Edition (Category Refinement and Data Augmentation) | 91.30% |
Final Edition (Category Refinement, Data Augmentation, and Full Data Reorganization) | 93.88% |
Precision | Recall | AP @0.5 | AP @0.5:0.95 | |
---|---|---|---|---|
Yolov7_640 × 640 (Baseline 10% labeled) | 35.40% | 31.80% | 43.50% | 25% |
Yolov7_640 × 640 (AFSL 10% labeled) | 51.70% | 45.30% | 57.10% | 38.10% |
Yolov7_640 × 640 (AFSL 100% labeled, additional) | 62.50% | 58.70% | 70.4% | 50.5% |
Yolov7_640 × 640 (Fully Supervised) | 63.20% | 56.20% | 69.70% | 49% |
Labeled Ratio | Precision | Recall | AP @0.5 | AP @0.5-0.95 |
---|---|---|---|---|
Yolov7_640 × 640(Baseline 10% labeled) | 70.70% | 76.50% | 78.80% | 46.70% |
Yolov7_640 × 640(Semi-supervised 10% labeled) | 79.60% | 83.70% | 84.7% (+5.9%) | 61.4% (+14.7%) |
Yolov7_640 × 640(Semi-supervised 10% labeled) | 81.40% | 82.20% | 86.70% | 53.20% |
Yolov7_640 × 640(Semi-supervised 25% labeled) | 92.70% | 95.40% | 97% (+0.2%) | 73% (+2.6%) |
Yolov7_640 × 640 (Fully Supervised) | 92.10% | 94.60% | 96.80% | 70.40% |
Item | Precision | Precision | AP@ 0.5 | AP@ 0.5 | AP@ 0.5:0.95 | AP@ 0.5:0.95) |
---|---|---|---|---|---|---|
Method | Baseline | AFSL | Baseline | AFSL | Baseline | AFSL |
Accuracy | 0.921 | 0.927 | 0.968 | 0.97 | 0.704 | 0.73 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Ni, Y.-S.; Chen, W.-L.; Liu, Y.; Wu, M.-H.; Guo, J.-I. Optimizing Automated Optical Inspection: An Adaptive Fusion and Semi-Supervised Self-Learning Approach for Elevated Accuracy and Efficiency in Scenarios with Scarce Labeled Data. Sensors 2024, 24, 5737. https://doi.org/10.3390/s24175737
Ni Y-S, Chen W-L, Liu Y, Wu M-H, Guo J-I. Optimizing Automated Optical Inspection: An Adaptive Fusion and Semi-Supervised Self-Learning Approach for Elevated Accuracy and Efficiency in Scenarios with Scarce Labeled Data. Sensors. 2024; 24(17):5737. https://doi.org/10.3390/s24175737
Chicago/Turabian StyleNi, Yu-Shu, Wei-Lun Chen, Yi Liu, Ming-Hsuan Wu, and Jiun-In Guo. 2024. "Optimizing Automated Optical Inspection: An Adaptive Fusion and Semi-Supervised Self-Learning Approach for Elevated Accuracy and Efficiency in Scenarios with Scarce Labeled Data" Sensors 24, no. 17: 5737. https://doi.org/10.3390/s24175737