Classification of Apple Color and Deformity Using Machine Vision Combined with CNN
Abstract
:1. Introduction
2. Materials and Methods
2.1. Machine Vision System
2.2. Image Dataset
2.2.1. Apple Samples
2.2.2. Image Acquisition
2.2.3. Image Processing
2.2.4. Image Data Augmentation
2.3. Grading Criteria
2.3.1. Apple Deformity Index
2.3.2. Apple Color
2.4. Convolutional Neural Networks
2.4.1. AlexNet
2.4.2. GoogLeNet
2.4.3. VGG16
2.4.4. Experimental Environment
2.5. Evaluation Indicators
- Precision evaluation indices
- 2.
- Complexity assessment indicators
3. Results and Discussion
3.1. System Evaluation and Image Processing
3.2. Performance Comparison
3.3. Testing Results
3.4. Ablation Experiment
3.5. Discussion
4. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Data Availability Statement
Conflicts of Interest
References
- National Bureau of Statistics of China in 2023. Available online: https://www.stats.gov.cn/sj/ndsj/2023/indexch.htm (accessed on 30 April 2023).
- Sun, Z.; Hu, D.; Xie, L.; Ying, Y. Detection of early stage bruise in apples using optical property mapping. Comput. Electron. Agric. 2022, 194, 106725. [Google Scholar] [CrossRef]
- Liu, T.; He, J.; Yao, W.; Jiang, H.; Chen, Q. Determination of aflatoxin B1 value in corn based on Fourier transform near-infrared spectroscopy: Comparison of optimization effect of characteristic wavelengths. LWT 2022, 164, 113657. [Google Scholar] [CrossRef]
- Shi, Y.; Wang, Y.; Hu, X.; Li, Z.; Huang, X.; Liang, J.; Zhang, X.; Zheng, K.; Zou, X.; Shi, J. Nondestructive discrimination of analogous density foreign matter inside soy protein meat semi-finished products based on transmission hyperspectral imaging. Food Chem. 2023, 411, 135431. [Google Scholar] [CrossRef] [PubMed]
- Xu, Q.; Cai, J.-R.; Zhang, W.; Bai, J.-W.; Li, Z.-Q.; Tan, B.; Sun, L. Detection of citrus Huanglongbing (HLB) based on the HLB-induced leaf starch accumulation using a home-made computer vision system. Biosyst. Eng. 2022, 218, 163–174. [Google Scholar] [CrossRef]
- Zhang, B.; Huang, W.; Li, J.; Zhao, C.; Fan, S.; Wu, J.; Liu, C. Principles, developments and applications of computer vision for external quality inspection of fruits and vegetables: A review. Food Res. Int. 2014, 62, 326–343. [Google Scholar] [CrossRef]
- Hu, D.; Jia, T.; Sun, X.; Zhou, T.; Huang, Y.; Sun, Z.; Zhang, C.; Sun, T.; Zhou, G. Applications of optical property measurement for quality evaluation of agri-food products: A review. Crit. Rev. Food Sci. Nutr. 2023, 1–21. [Google Scholar] [CrossRef] [PubMed]
- Zou, X.; Zhao, J.; Li, Y.; Holmes, M. In-line detection of apple defects using three color cameras system. Comput. Electron. Agric. 2010, 70, 129–134. [Google Scholar]
- Hu, G.; Zhang, E.; Zhou, J.; Zhao, J.; Gao, Z.; Sugirbay, A.; Jin, H.; Zhang, S.; Chen, J. Infield Apple Detection and Grading Based on Multi-Feature Fusion. Horticulturae 2021, 7, 276. [Google Scholar] [CrossRef]
- Song, T.-H.; Sanchez, V.; Daly, H.E.; Rajpoot, N.M. Simultaneous cell detection and classification in bone marrow histology images. IEEE J. Biomed. Health Inform. 2019, 23, 1469–1476. [Google Scholar] [CrossRef]
- Zhou, J.; Wu, Z.; Jiang, Z.; Huang, K.; Guo, K.; Zhao, S. Background selection schema on deep learning-based classification of dermatological disease. Comput. Biol. Med. 2022, 149, 105966. [Google Scholar] [CrossRef]
- Zhang, C.; Xia, K.; Feng, H.; Yang, Y.; Du, X. Tree species classification using deep learning and RGB optical images obtained by an unmanned aerial vehicle. J. For. Res. 2021, 32, 1879–1888. [Google Scholar] [CrossRef]
- Chen, Y.; Wang, J. Deep learning for crown profile modelling of Pinus yunnanensis secondary forests in Southwest China. Front. Plant Sci. 2023, 14, 1093905. [Google Scholar] [CrossRef] [PubMed]
- Sun, X.; Xu, L.; Zhou, Y.; Shi, Y. Leaves and twigs image recognition based on deep learning and combined classifier algorithms. Forests 2023, 14, 1083. [Google Scholar] [CrossRef]
- Feng, J.; Hou, B.; Yu, C.; Yang, H.; Wang, C.; Shi, X.; Hu, Y. Research and validation of potato late blight detection method based on deep learning. Agronomy 2023, 13, 1659. [Google Scholar] [CrossRef]
- Hu, D.; Qiu, D.; Yu, S.; Jia, T.; Zhou, T.; Yan, X. Integration of optical property mapping and machine learning for real-time classification of early bruises of apples. Food Bioprocess Technol. 2023, 1–12. [Google Scholar] [CrossRef]
- Tao, K.; Wang, A.; Shen, Y.; Lu, Z.; Peng, F.; Wei, X. Peach flower density detection based on an improved CNN incorporating attention mechanism and multi-scale feature fusion. Horticulturae 2022, 8, 904. [Google Scholar] [CrossRef]
- Shuprajhaa, T.; Raj, J.M.; Paramasivam, S.K.; Sheeba, K.; Uma, S. Deep learning based intelligent identification system for ripening stages of banana. Postharvest Biol. Technol. 2023, 203, 112410. [Google Scholar] [CrossRef]
- Deng, L.; Li, J.; Han, Z. Online defect detection and automatic grading of carrots using computer vision combined with deep learning methods. LWT 2021, 149, 111832. [Google Scholar] [CrossRef]
- Sun, Z.; Xie, L.; Hu, D.; Ying, Y. An artificial neural network model for accurate and efficient optical property mapping from spatial-frequency domain images. Comput. Electron. Agric. 2021, 188, 106340. [Google Scholar] [CrossRef]
- Li, J.; Xie, S.; Chen, Z.; Liu, H.; Kang, J.; Fan, Z.; Li, W. A shallow Convolutional Neural Network for apple classification. IEEE Access 2020, 8, 111683–111692. [Google Scholar] [CrossRef]
- Li, Y.; Feng, X.; Liu, Y.; Han, X. Apple quality identification and classification by image processing based on convolutional neural networks. Sci. Rep. 2021, 11, 16618. [Google Scholar] [CrossRef] [PubMed]
- Fan, S.; Li, J.; Zhang, Y.; Tian, X.; Wang, Q.; He, X.; Zhang, C.; Huang, W. On line detection of defective apples using computer vision system combined with deep learning methods. J. Food Eng. 2020, 286, 110102. [Google Scholar] [CrossRef]
- Shi, X.; Chai, X.; Yang, C.; Xia, X.; Sun, T. Vision-based apple quality grading with multi-view spatial network. Comput. Electron. Agric. 2022, 195, 106793. [Google Scholar] [CrossRef]
- Ünal, Z.; Kızıldeniz, T.; Özden, M.; Aktaş, H.; Karagöz, Ö. Detection of bruises on red apples using deep learning models. Sci. Hortic. 2024, 329, 113021. [Google Scholar] [CrossRef]
- Fu, Y.; Song, J.; Xie, F.; Bai, Y.; Zheng, X.; Gao, P.; Wang, Z.; Xie, S. Circular fruit and vegetable classification based on optimized GoogLeNet. IEEE Access 2021, 9, 113599–113611. [Google Scholar]
- Ni, J.; Gao, J.; Deng, L.; Han, Z. Monitoring the change process of banana freshness by GoogLeNet. IEEE Access 2020, 8, 228369–228376. [Google Scholar] [CrossRef]
- McCamy, C.S.; Marcus, H.; Davidson, J.G. A color-rendition chart. J. Appl. Photogr. Eng. 1976, 2, 95–99. [Google Scholar]
- Shorten, C.; Khoshgoftaar, T.M. A survey on image data augmentation for deep learning. J. Big Data 2019, 6, 60. [Google Scholar] [CrossRef]
- Krizhevsky, A.; Ilya, S.; Geoffrey, E.H. ImageNet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems 25; Neural Information Processing Systems Foundation: La Jolla, CA, USA, 2012. [Google Scholar]
- Gayathri, D.; Kishore, B.; Senthikumar, C. Feature analysis and classification of maize crop diseases employing AlexNet-inception network. Multimed. Tools Appl. 2024, 83, 26971–26999. [Google Scholar]
- Ni, J.; Gao, J.; Li, J.; Yang, H.; Hao, Z.; Han, Z. E-AlexNet: Quality evaluation of strawberry based on machine learning. J. Food Meas. Charact. 2021, 15, 4530–4541. [Google Scholar] [CrossRef]
- Szegedy, C.; Vanhoucke, V.; Ioffe, S.; Shlens, J.; Wojna, Z. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016; pp. 2818–2826. [Google Scholar]
- Swarup, C.; Singh, K.U.; Kumar, A.; Pandey, S.K.; Varshney, N.; Singh, T. Brain tumor detection using CNN, AlexNet & GoogLeNet ensembling learning approaches. Electron. Res. Arch. 2023, 31, 2900–2924. [Google Scholar]
- Yang, L.; Yu, X.; Zhang, S.; Long, H.; Zhang, H.; Xu, S.; Liao, Y. GoogLeNet based on residual network and attention mechanism identification of rice leaf diseases. Comput. Electron. Agric. 2023, 204, 107543. [Google Scholar] [CrossRef]
- Szegedy, C.; Liu, W.; Jia, Y.; Sermanet, P.; Reed, S.; Anguelov, D.; Erhan, D.; Vanhoucke, V.; Rabinovich, A. Going deeper with convolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, 7–12 June 2015; pp. 1–9. [Google Scholar]
- Liu, Z.; Wu, J.; Fu, L.; Majeed, Y.; Feng, Y.; Li, R.; Cui, Y. Improved kiwifruit detection using pre-trained VGG16 with RGB and NIR information fusion. IEEE Access 2020, 8, 2327–2336. [Google Scholar] [CrossRef]
- Yang, H.; Ni, J.; Gao, J.; Han, Z.; Luan, T. A novel method for peanut variety identification and classification by Improved VGG16. Sci. Rep. 2021, 11, 15756. [Google Scholar] [CrossRef] [PubMed]
- Ji, W.; Wang, J.; Xu, B.; Zhang, T. Apple Grading based on multi-dimensional view processing and deep learning. Foods 2023, 12, 2117. [Google Scholar] [CrossRef]
Label | Training Set | Validation Set | Test Set | |||
---|---|---|---|---|---|---|
Before | After | Before | After | Before | After | |
Deformed apples | 208 | 1248 | 52 | 312 | 100 | 100 |
Non-deformed stripe–red apples | 216 | 1296 | 54 | 324 | 50 | 50 |
Non-deformed slice–red apples | 216 | 1296 | 54 | 324 | 50 | 50 |
Total | 640 | 3840 | 160 | 960 | 200 | 200 |
Configuration | Parameter |
---|---|
GPU | RTX 4090 |
CPU | Xeon(R) Platinum 8352V |
RAM | 90 G |
Accelerated framework | CUDA 11.0 |
Deep learning framework | Pytorch 1.7.0 |
Programming language | Python 3.8 |
Model | Training Accuracy (%) | Validation Accuracy (%) | Loss | Parameter (×107) | Model Size (MB) | Infer Time (ms) |
---|---|---|---|---|---|---|
AlexNet | 94.78 | 91.66 | 0.73 | 6.1 | 217 | 4.81 |
VGG16 | 94.84 | 92.29 | 0.52 | 13.8 | 512.22 | 5.77 |
GoogLeNet | 91.15 | 88.96 | 0.65 | 1.3 | 39.3 | 4.91 |
Category | Precision (%) | Recall (%) | F1-Score (%) |
---|---|---|---|
0 | 95.21 | 82.69 | 88.54 |
1 | 91.03 | 96.91 | 93.90 |
2 | 91.30 | 96.91 | 94.02 |
Model | Accuracy (%) | Parameter (×107) | Model Size (MB) | Infer Time (ms) |
---|---|---|---|---|
VGG16 | 92.29 | 13.8 | 512.22 | 5.77 |
VGG16-1 | 86.35 | 13.8 | 512.08 | 5.56 |
VGG16-2 | 89.68 | 13.8 | 511.66 | 5.68 |
VGG16-3 | 89.58 | 13.7 | 507.72 | 5.65 |
VGG16-4 | 90.52 | 13.4 | 494.22 | 5.71 |
VGG16-5 | 88.54 | 13.4 | 494.22 | 5.73 |
Feature | Model | Accuracy (%) | Reference |
---|---|---|---|
Diameter, defect | Multi-view spatial network | 99.24 | Shi et al. (2022) [24] |
Variety | shallow CNN | 92.00 | Li et al. (2020) [21] |
Defect | CNN | 95.33 | Li et al. (2021) [22] |
Color, shape, diameter, defect | Improved YOLOv5s | 94.46 | Ji et al. (2023) [39] |
Defect | CNN | 92.15 | Fan et al. (2020) [23] |
Color, deformity | VGG16 | 92.29 | This study |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Qiu, D.; Guo, T.; Yu, S.; Liu, W.; Li, L.; Sun, Z.; Peng, H.; Hu, D. Classification of Apple Color and Deformity Using Machine Vision Combined with CNN. Agriculture 2024, 14, 978. https://doi.org/10.3390/agriculture14070978
Qiu D, Guo T, Yu S, Liu W, Li L, Sun Z, Peng H, Hu D. Classification of Apple Color and Deformity Using Machine Vision Combined with CNN. Agriculture. 2024; 14(7):978. https://doi.org/10.3390/agriculture14070978
Chicago/Turabian StyleQiu, Dekai, Tianhao Guo, Shengqi Yu, Wei Liu, Lin Li, Zhizhong Sun, Hehuan Peng, and Dong Hu. 2024. "Classification of Apple Color and Deformity Using Machine Vision Combined with CNN" Agriculture 14, no. 7: 978. https://doi.org/10.3390/agriculture14070978