A VGG-19 Model with Transfer Learning and Image Segmentation for Classification of Tomato Leaf Disease
Abstract
:1. Introduction
2. Materials and Methods
2.1. Plant Village Tomato Leaf Image Datasets
2.2. Proposed Model for Classifying Tomato Leaf Diseases
2.3. Tomato Leaf Image Segmentation Using the HSV Color Space
- Step 1: The RGB image is converted into the image with the HSV color space. The HSV components (three sub-images) can easily distinguish colors related to the type of color, the shade of color, the purity of color, or the brightness of color.
- Step 2: Histograms for all three HSV components are plotted to choose their lower and upper threshold values.
- Step 3: Masking is to segment and convert the HSV images to binary images based on the histogram HSV thresholds before extracting the leaf region. Therefore, we can fill holes in one binary image to create a leaf mask using the morphological operation.
- Step 4: The white mask can map to the RGB image for collecting the original leaf region and the black background.
2.4. VGG-19 Model
2.5. Transfer Learning
2.6. Evaluation of Classification System
3. Results
3.1. Image Segmentation Result
3.2. Classification Performance Using the VGG-19 with Transfer Learning
3.3. Effect of Learning Rate Parameter on Classification Performance
3.4. Effect of Epoch Parameter on Classification Performance
3.5. Comparison of Performance Results with Difference Models
3.6. Comparison of Time for Training and Testing between Segmented and Non-Segmented Images
4. Discussion
5. Conclusions
Author Contributions
Funding
Data Availability Statement
Conflicts of Interest
References
- Mukhtar, T.; Rehman, S.U.; Smith, D.; Sultan, T.; Seleiman, M.F.; Alsadon, A.A.; Amna; Ali, S.; Chaudhary, H.J.; Solieman, T.H.I.; et al. Mitigation of Heat Stress in Solanum lycopersicum L. by ACC-deaminase and Exopolysaccharide Producing Bacillus cereus: Effects on Biochemical Profiling. Sustainability 2020, 12, 2159. [Google Scholar] [CrossRef] [Green Version]
- Duffy, S.; Holmes, E.C. Multiple introductions of the Old World begomovirus Tomato yellow leaf curl virus into the New World. Appl. Environ. Microbiol. 2007, 73, 7114–7117. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Moriones, E.; Praveen, S.; Chakraborty, S. Tomato Leaf Curl New Delhi Virus: An Emerging Virus Complex Threatening Vegetable and Fiber Crops. Viruses 2017, 9, 264. [Google Scholar] [CrossRef] [Green Version]
- Cañizares, M.C.; Rosas-Díaz, T.; Rodríguez-Negrete, E.; Hogenhout, S.A.; Bedford, I.D.; Bejarano, E.R.; Navas-Castillo, J.; Moriones, E. Arabidopsis thaliana, an experimental host for tomato yellow leaf curl disease-associated begomoviruses by agroinoculation and whitefly transmission. Plant Pathol. 2015, 64, 265–271. [Google Scholar] [CrossRef]
- Yin, C.; Zeng, T.; Zhang, H.; Fu, W.; Wang, L.; Yao, S. Maize Small Leaf Spot Classification Based on Improved Deep Convolutional Neural Networks with a Multi-Scale Attention Mechanism. Agronomy 2022, 12, 906. [Google Scholar] [CrossRef]
- Bahrami, H.; Homayouni, S.; Safari, A.; Mirzaei, S.; Mahdianpari, M.; Reisi-Gahrouei, O. Deep Learning-Based Estimation of Crop Biophysical Parameters Using Multi-Source and Multi-Temporal Remote Sensing Observations. Agronomy 2021, 11, 1363. [Google Scholar] [CrossRef]
- Hassan, S.M.; Jasinski, M.; Leonowicz, Z.; Jasinska, E.; Maji, A.K. Plant Disease Identification Using Shallow Convolutional Neural Network. Agronomy 2021, 11, 2388. [Google Scholar] [CrossRef]
- Suman, T.; Dhruvakumar, T. Classification of Paddy Leaf Diseases Using Shape and Color Features. Int. J. Electr. Electron. Eng. 2015, 7, 239–250. [Google Scholar]
- Sanyal, P.; Patel, S.C. Pattern recognition method to detect two diseases in rice plants. Imaging Sci. J. 2008, 56, 319–325. [Google Scholar] [CrossRef]
- Suresh, M.V.; Gopinath, D.; Hemavarthini, M.; Jayanthan, K.; Krishnan, M. Plant Disease Detection using Image Processing. Int. J. Eng. Res. Technol. (IJERT) 2020, 9, 78–82. [Google Scholar] [CrossRef]
- Zhou, C.; Zhou, S.; Xing, J.; Song, J. Tomato Leaf Disease Identification by Restructured Deep Residual Dense Network. IEEE Access 2021, 9, 28822–28831. [Google Scholar] [CrossRef]
- Liu, J.; Wang, X. Early recognition of tomato gray leaf spot disease based on MobileNetv2-YOLOv3 model. Plant Methods 2020, 16, 83–99. [Google Scholar] [CrossRef] [PubMed]
- Ahmed, S.; Hasan, M.B.; Ahmed, T.; Sony, R.K.; Kabir, M.H. Less is More: Lighter and Faster Deep Neural Architecture for Tomato Leaf Disease Classification. IEEE Access 2022, 10, 68868–68884. [Google Scholar] [CrossRef]
- Tan, L.; Lu, J.; Jiang, H. Tomato Leaf Diseases Classification Based on Leaf Images: A Comparison between Classical Machine Learning and Deep Learning Methods. AgriEngineering 2021, 3, 542–558. [Google Scholar] [CrossRef]
- Mohanty, S.P.; Hughes, D.P.; Salathé, M. Using Deep Learning for Image-Based Plant Disease Detection. Front. Plant Sci. 2016, 7, 1419. [Google Scholar] [CrossRef] [Green Version]
- Wu, Q.; Chen, Y.; Meng, J. DCGAN-Based Data Augmentation for Tomato Leaf Disease Identification. IEEE Access 2020, 8, 98716–98728. [Google Scholar] [CrossRef]
- Sladojevic, S.; Arsenovic, M.; Anderla, A.; Culibrk, D.; Stefanovic, D. Deep Neural Networks Based Recognition of Plant Diseases by Leaf Image Classification. Comput. Intell. Neurosci. 2016, 2016, 3289801–3289813. [Google Scholar] [CrossRef] [Green Version]
- Liu, B.; Zhang, Y.; He, D.; Li, Y. Identification of Apple Leaf Diseases Based on Deep Convolutional Neural Networks. Symmetry 2018, 10, 11. [Google Scholar] [CrossRef] [Green Version]
- Lu, Y.; Yi, S.; Zeng, N.; Liu, Y.; Zhang, Y. Identification of rice diseases using deep convolutional neural networks. Neurocomputing 2017, 267, 378–384. [Google Scholar] [CrossRef]
- Fuentes, A.; Yoon, S.; Kim, S.C.; Park, D.S. A Robust Deep-Learning-Based Detector for Real-Time Tomato Plant Diseases and Pests Recognition. Sensors 2017, 17, 2022. [Google Scholar] [CrossRef] [Green Version]
- Saranya, S.M.; Rajalaxmi, R.R.; Prabavathi, R.; Suganya, T.; Mohanapriya, S.; Tamilselvi, T. Deep Learning Techniques in Tomato Plant—A Review. J. Phys. Conf. Ser. 2021, 1767, 012010–012024. [Google Scholar] [CrossRef]
- Yang, G.; Chen, G.; He, Y.; Yan, Z.; Guo, Y.; Ding, J. Self-Supervised Collaborative Multi-Network for Fine-Grained Visual Categorization of Tomato Diseases. IEEE Access 2020, 8, 211912–211923. [Google Scholar] [CrossRef]
- Kaggle: Dataset of diseased plant leaf images and corresponding labels. Tomato Leaf Disease. 2019. Available online: https://www.kaggle.com/emmarex/plantdisease (accessed on 12 July 2022).
- Gadekallu, T.R.; Rajput, D.S.; Reddy, M.P.K.; Lakshmanna, K.; Bhattacharya, S.; Singh, S.; Jolfaei, A.; Alazab, M. A novel PCA–whale optimization-based deep neural network model for classification of tomato plant diseases using GPU. J. Real-Time Image Process. 2021, 18, 1383–1396. [Google Scholar] [CrossRef]
- Wen, J.; Shi, Y.; Zhou, X.; Xue, Y. Crop Disease Classification on Inadequate Low-Resolution Target Images. Sensors 2020, 20, 4601. [Google Scholar] [CrossRef] [PubMed]
- Aravind, K.R.; Raja, P. Automated disease classification in (Selected) agricultural crops using transfer learning. Automatika 2020, 61, 260–272. [Google Scholar] [CrossRef]
- Moyazzoma, R.; Hossain, M.A.A.; Anuz, M.H.; Sattar, A. Transfer Learning Approach for Plant Leaf Disease Detection Using CNN with Pre-Trained Feature Extraction Method Mobilnetv2. In Proceedings of the 2021 2nd International Conference on Robotics, Electrical and Signal Processing Techniques (ICREST), DHAKA, Bangladesh, 5–7 January 2021; pp. 526–529. [Google Scholar]
- Agarwal, M.; Gupta, S.K.; Biswas, K.K. Development of Efficient CNN model for Tomato crop disease identification. Sustain. Comput. Inform. Syst. 2020, 28, 100407–100432. [Google Scholar] [CrossRef]
- Verma, S.; Chug, A.; Singh, A.P. Application of convolutional neural networks for evaluation of disease severity in tomato plant. J. Discret. Math. Sci. Cryptogr. 2020, 23, 273–282. [Google Scholar] [CrossRef]
- Rangarajan, A.K.; Purushothaman, R.; Ramesh, A. Tomato crop disease classification using pre-trained deep learning algorithm. Procedia Comput. Sci. 2018, 133, 1040–1047. [Google Scholar] [CrossRef]
- Brahimi, M.; Boukhalfa, K.; Moussaoui, A. Deep Learning for Tomato Diseases: Classification and Symptoms Visualization. Appl. Artif. Intell. 2017, 31, 299–315. [Google Scholar] [CrossRef]
- Kaur, R. A Brief Review on Plant Disease Detection using in Image Processing. Int. J. Comput. Sci. Mob. Comput. 2017, 6, 101–106. [Google Scholar]
- Kumar, S.; Kaur, R. Plant Disease Detection using Image Processing—A Review. Int. J. Comput. Appl. 2015, 124, 6–9. [Google Scholar] [CrossRef]
- Halder, M.; Sarkar, A.; Bahar, H. Plant Disease Detection By Image Processing: A Literature Review. J. Food Sci. Technol. 2019, 3, 534–538. [Google Scholar] [CrossRef]
- Jagtap, S.B.; Hambarde, S.M. Agricultural Plant Leaf Disease Detection and Diagnosis Using Image Processing Based on Morphological Feature Extraction. IOSR J. VLSI Signal Process. 2014, 4, 24–30. [Google Scholar] [CrossRef]
- Kamlapurkar, S.R. Detection of Plant Leaf Disease Using Image Processing Approach. Int. J. Sci. Res. Publ. 2016, 6, 73–76. [Google Scholar]
- Lu, J.; Ehsani, R.; Shi, Y.; de Castro, A.I.; Wang, S. Detection of multi-tomato leaf diseases (late blight, target and bacterial spots) in different stages by using a spectral-based sensor. Sci. Rep. 2018, 8, 2793–2804. [Google Scholar] [CrossRef] [Green Version]
- Hlaing, C.S.; Zaw, S.M.M. Tomato Plant Diseases Classification Using Statistical Texture Feature and Color Feature. In Proceedings of the 2018 IEEE/ACIS 17th International Conference on Computer and Information Science (ICIS), Singapore, 6–8 June 2018; pp. 439–444. [Google Scholar]
- Barbedo, J.G.A. Digital image processing techniques for detecting, quantifying and classifying plant diseases. SpringerPlus 2013, 2, 660–672. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Almadhor, A.; Rauf, H.T.; Lali, M.I.U.; Damaševičius, R.; Alouffi, B.; Alharbi, A. AI-Driven Framework for Recognition of Guava Plant Diseases through Machine Learning from DSLR Camera Sensor Based High Resolution Imagery. Sensors 2021, 21, 3830. [Google Scholar] [CrossRef]
- Zhang, S.; You, Z.; Wu, X. Plant disease leaf image segmentation based on superpixel clustering and EM algorithm. Neural Comput. Appl. 2019, 31, 1225–1232. [Google Scholar] [CrossRef]
- Singh, V.; Misra, A.K. Detection of plant leaf diseases using image segmentation and soft computing techniques. Inf. Process. Agric. 2017, 4, 41–49. [Google Scholar] [CrossRef] [Green Version]
- Zhang, S.; Wang, H.; Huang, W.; You, Z. Plant diseased leaf segmentation and recognition by fusion of superpixel, K-means and PHOG. Optik 2018, 157, 866–872. [Google Scholar] [CrossRef]
- Storey, G.; Meng, Q.; Li, B. Leaf Disease Segmentation and Detection in Apple Orchards for Precise Smart Spraying in Sustainable Agriculture. Sustainability 2022, 14, 1458. [Google Scholar] [CrossRef]
- Hughes, D.P.; Salathé, M. An Open Access Repository of Images on Plant Health to Enable the Development of Mobile Disease Diagnostics. arXiv 2015, arXiv:1511.08060, 13. [Google Scholar]
- Kumar, J.P.; Domnic, S. Image based leaf segmentation and counting in rosette plants. Inf. Process. Agric. 2019, 6, 233–246. [Google Scholar]
- Krizhevsky, A.; Sutskever, I.; Hinton, G.E. ImageNet Classification with Deep Convolutional Neural Networks. NIPS 2012, 60, 84–90. [Google Scholar] [CrossRef] [Green Version]
- Szegedy, C.; Liu, W.; Jia, Y.; Sermanet, P.; Reed, S.; Anguelov, D.; Erhan, D.; Vanhoucke, V.; Rabinovich, A. Going deeper with convolutions. In Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA, 7–12 June 2015; pp. 1–9. [Google Scholar]
- He, K.; Zhang, X.; Ren, S.; Sun, J. Deep Residual Learning for Image Recognition. arXiv 2016, arXiv:1512.03385. Available online: https://arxiv.org/abs/1512.03385 (accessed on 12 July 2022).
- Simonyan, K.; Zisserman, A. Very Deep Convolutional Networks for Large-Scale Image Recognition. In Proceedings of the 3rd International Conference on Learning Representations, San Diego, CA, USA, 7–9 May 2015; pp. 1–14. [Google Scholar]
- Hai, N.T.; Nguyen, N.T.; Nguyen, M.H.; Livatino, S. Wavelet-Based Kernel Construction for Heart Disease Classification. Adv. Electr. Electron. Eng. J. 2019, 17, 306–319. [Google Scholar] [CrossRef]
- Wang, Z.; Bovik, A.C.; Sheikh, H.R.; Simoncelli, E.P. Image quality assessment: From error visibility to structural similarity. IEEE Trans. Image Process. 2004, 13, 600–612. [Google Scholar] [CrossRef] [Green Version]
- Maeda-Gutiérrez, V.; Galván-Tejada, C.E.; Zanella-Calzada, L.A.; Celaya-Padilla, J.M.; Galván-Tejada, J.I.; Gamboa-Rosales, H.; Luna-García, H.; Magallanes-Quintanar, R.; Guerrero Méndez, C.A.; Olvera-Olvera, C.A. Comparison of Convolutional Neural Network Architectures for Classification of Tomato Plant Diseases. Appl. Sci. 2020, 10, 1245. [Google Scholar] [CrossRef] [Green Version]
- Trivedi, N.K.; Gautam, V.; Anand, A.; Aljahdali, H.M.; Villar, S.G.; Anand, D.; Goyal, N.; Kadry, S. Early Detection and Classification of Tomato Leaf Disease Using High-Performance Deep Neural Network. Sensors 2021, 21, 7987. [Google Scholar] [CrossRef]
No. | Class of Tomato Leaf Images | Images |
---|---|---|
1 | Tomato bacterial spot disease | 2127 |
2 | Tomato Septoria leaf spot disease | 1771 |
3 | Mosaic virus disease | 373 |
4 | Leaf mold disease | 952 |
5 | Target spot disease | 1404 |
6 | Early blight disease | 1000 |
7 | Yellow leaf curl virus disease | 3208 |
8 | Tomato late blight disease | 1908 |
9 | Two-spotted spider mites | 1676 |
10 | Healthy leaf | 1591 |
Total of tomato leaf images | 16,010 |
Predicted Classes | |||
---|---|---|---|
True classes | i | ||
No. | Image Name | SSIM |
1 | Bacterial spot disease | 0.914 |
2 | Early blight disease | 0.905 |
3 | Late blight disease | 0.929 |
4 | Leaf mold disease | 0.908 |
5 | Septoria leaf spot disease | 0.912 |
6 | Two-spotted spider mites | 0.946 |
7 | Target spot disease | 0.924 |
8 | Yellow leaf curl virus disease | 0.917 |
9 | Mosaic virus disease | 0.922 |
10 | Healthy leaf | 0.924 |
Learning Rate | 0.00001 | 0.0001 | 0.001 |
---|---|---|---|
ACC | 99.72 | 99.34 | 99.02 |
SEN | 99.69 | 99.36 | 98.48 |
SPE | 99.90 | 99.77 | 99.66 |
PPV | 99.49 | 99.23 | 98.75 |
F1S | 99.59 | 99.29 | 99.00 |
Epochs | 50 | 100 | 200 | 300 | 400 |
---|---|---|---|---|---|
ACC | 98.50 | 99.13 | 99.53 | 99.72 | 99.66 |
SEN | 98.61 | 99.18 | 99.54 | 99.69 | 99.59 |
SPE | 99.32 | 99.66 | 99.83 | 99.90 | 99.89 |
PPV | 98.16 | 98.75 | 99.38 | 99.49 | 99.43 |
F1S | 98.39 | 98.97 | 99.46 | 99.59 | 99.51 |
Preprocessing | ACC (%) | SEN (%) | SPE (%) | PPV (%) | F1S (%) | Training Time (second) | Testing Time (second) |
---|---|---|---|---|---|---|---|
Segmented images | 99.72 | 99.69 | 99.90 | 99.49 | 99.59 | 2.75 × 105 | 29.30 |
Non-segmented images | 99.63 | 99.58 | 99.82 | 99.37 | 99.47 | 2.98 × 105 | 35.27 |
Work | Model | Transfer Learning | Classes | Preprocessing | ACC |
---|---|---|---|---|---|
Maeda-Gutiérrez et al. [53] | GoogleNet | Yes | 10 | No | 99.39 % |
Brahimi et al. [31] | GoogleNet | Yes | 10 | No | 99.35 % |
Agarwal et al. [28] | CNN model | No | 10 | No | 98.40 % |
Gadekallu et al. [24] | MLP | No | 10 | PCA-WOA | 94.00 % |
Trivedi et al. [54] | CNN model | No | 10 | Transformed into grey images | 98.49% |
Our proposed model | VGG-19 | Yes | 10 | HSV color space for image segmentation | 99.72 % |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Nguyen, T.-H.; Nguyen, T.-N.; Ngo, B.-V. A VGG-19 Model with Transfer Learning and Image Segmentation for Classification of Tomato Leaf Disease. AgriEngineering 2022, 4, 871-887. https://doi.org/10.3390/agriengineering4040056
Nguyen T-H, Nguyen T-N, Ngo B-V. A VGG-19 Model with Transfer Learning and Image Segmentation for Classification of Tomato Leaf Disease. AgriEngineering. 2022; 4(4):871-887. https://doi.org/10.3390/agriengineering4040056
Chicago/Turabian StyleNguyen, Thanh-Hai, Thanh-Nghia Nguyen, and Ba-Viet Ngo. 2022. "A VGG-19 Model with Transfer Learning and Image Segmentation for Classification of Tomato Leaf Disease" AgriEngineering 4, no. 4: 871-887. https://doi.org/10.3390/agriengineering4040056