EffiMob-Net: A Deep Learning-Based Hybrid Model for Detection and Identification of Tomato Diseases Using Leaf Images

Abstract

As tomatoes are the most consumed vegetable in the world, production should be increased to fulfill the vast demand for this vegetable. Global warming, climate changes, and other significant factors, including pests, badly affect tomato plants and cause various diseases that ultimately affect the production of this vegetable. Several strategies and techniques have been adopted for detecting and averting such diseases to ensure the survival of tomato plants. Recently, the application of artificial intelligence (AI) has significantly contributed to agronomy in the detection of tomato plant diseases through leaf images. Deep learning (DL)-based techniques have been largely utilized for detecting tomato leaf diseases. This paper proposes a hybrid DL-based approach for detecting tomato plant diseases through leaf images. To accomplish the task, this study presents the fusion of two pretrained models, namely, EfficientNetB3 and MobileNet (referred to as the EffiMob-Net model) to detect tomato leaf diseases accurately. In addition, model overfitting was handled using various techniques, such as regularization, dropout, and batch normalization (BN). Hyperparameter tuning was performed to choose the optimal parameters for building the best-fitting model. The proposed hybrid EffiMob-Net model was tested on a plant village dataset containing tomato leaf disease and healthy images. This hybrid model was evaluated based on the best classifier with respect to accuracy metrics selected for detecting the diseases. The success rate of the proposed hybrid model for accurately detecting tomato leaf diseases reached 99.92%, demonstrating the model’s ability to extract features accurately. This finding shows the reliability of the proposed hybrid model as an automatic detector for tomato plant diseases that can significantly contribute to providing better solutions for detecting other crop diseases in the field of agriculture.

1. Introduction

Tomatoes are a fast-growing crop that matures in 90 to 150 days [1]. This worldwide ever-present product has rich nutritional values [2] and can be cultivated in nearly any reasonably parched soil [3]. In recent decades, the agricultural estate has increased tomato production by above 160% [4]. Tomatoes are the most consumed vegetable, accounting for about 15% of total vegetable consumption [5], and ranking as the sixth most abundant vegetable worldwide according to the Food and Agriculture Organization (FAO) annual production statistics [6]. The key production areas of tomatoes occur in India, the USA, Iran, China, Italy, Egypt, Mexico, and Turkey [7]. However, the plant is usually infected by diseases, which could be viral or fungal, resulting in a significant reduction in both the quality and quantity of crop production [3].

Due to the large demand for tomatoes globally, there is a need to develop techniques for enhancing crop yields while allowing for the early detection of plant diseases, including viral, bacterial, and fungal diseases [8], to increase the quality and production of tomatoes to meet economic goals [9]. Accurate and timely treatment is required to prevent diseases from spreading and causing in crop losses, and ensure ideal production. In a manual scenario, human expert-based detection is required to cope with these problems [10]. Moreover, screening symptoms manually is time consuming and costly due to insufficient human infrastructure capacity [11]. An automatic detecting system can assist in identifying the symptoms of a disease through the plant leaf in a cost-efficient manner. The application of artificial intelligence (AI), particularly machine learning (ML) and deep learning (DL), has significantly contributed to efforts to detect plant diseases.

Recently, the application of DL approaches has demonstrated outstanding performance and provided solutions to real problems in a wide range of computer vision and ML jobs, including image classification, detection, recognition, and medical imaging [12]. In the literature, several techniques have been developed based on the DL approach to enhance the persistence rate of field crops through the early detection of various diseases and succeeding disease management [5]. Currently, for plant diseases, the detection and classification rate have reached 100% in laboratory-based machine vision technology [13]. DL is broadly used in agriculture for plant disease detection and classification. Moreover, a DL-based convolutional neural network (CNN) is the most commonly used method for detecting, classifying, and recognizing tomato leaf diseases because of its significant success compared with other traditional methods [14]. CNN has the capability of extracting features from objects automatically. Therefore, CNN has been extensively utilized for tomato leaf disease identification, recognition, and classification.

Based on the widespread success of DL-based CNN architectures in agriculture, particularly, the detection of plant diseases, this study proposed a hybrid DL-based model that combines two pretrained models, namely, EfficientNet and MobileNet (referred to as EffiMob-Net) for detecting tomato leaf diseases. Taking advantage of the pretrained models’ architectures, the weights of both pretrained models were loaded to utilize them for feature extraction and then the outputs of both models were concatenated for the detection and classification of leaf images. The key contributions of this study are as follows:

• A deep hybrid model was proposed that combines the architectures of two pretrained models, EfficientNet and MobileNet, for extracting the significant features of tomato leaves. Their outputs were then concatenated for the detection and classification of tomato leaf diseases.
• In the proposed method, the softmax layers of both pretrained models were removed, and the output achieved from the dense layers of both models was combined. In addition, three FC layers of size 512, 256, and 128 channels were added after the concatenation process. The classification was performed using the softmax layer which was added at the end of the proposed model.
• The dataset was preprocessed and prepared for training the proposed hybrid model using various preprocessing steps.
• The proposed model was trained using the extracted features.
• The study ensured the prevention of the proposed model’s overfitting by using various techniques, such as regularization, dropout, and BN.
• The proposed hybrid model was evaluated, and the classification report with descriptions is presented.

2. Related Work

This section discusses the existing work related to the application of DL approaches to the detection and classification of tomato leaf diseases. The search criteria for investigating previous work in the same domain include keywords such as tomato leaf disease detection using DL and DL approaches for detecting and classifying tomato leaf. Several well-known search engines/databases such as Google Scholar, ScienceDirect, ResearchGate, and IEEE Explorer were explored to collect and discuss state-of-the-art methodologies used in this domain of research. The literature survey indicated that most previous related research is based on the pretrained DL models.

A study conducted by [15] utilized a plant village dataset to detect and classify tomato leaf diseases using the DL approach. For this task, several pretrained approaches such as AlexNet, GoogLeNet, SqueezeNet, Vgg16, and MobileNetv2 were applied. Vgg16 achieved higher results than the others, with an accuracy rate of 99.17%. An attempt was made by [16] to detect tomato leaf diseases using the DL method. In this regard, fuzzy-SVM, CNN, and region-based CNN (R-CNN) were applied to a dataset containing a total of 6 classes. The achieved results showed a higher performance of R-CNN, with an accuracy rate of 96.735%. Similarly, Ref. [17] utilized the mask R-CNN approach for the segmentation and identification of tomato leaf disease. The results showed a higher accuracy rate of 98%. A pretrained model and feature concatenation approach were used by [4] for tomato leaf disease classification. In this method, the features were extracted using pretrained models and concatenated, while the classification was performed using traditional ML methods. The study concluded that multinomial logistic regression (MLR) achieved the highest results, with 97% accuracy.

A multimodal hybrid DL-based approach using attention-based dilated CNN logistic regression (ADCLR) was proposed by [18] to identify tomato leaf diseases. In this approach, feature extraction was performed using attention-based dilated CNN. The processed features were combined and classified using logistic regression (LR). The classification results show a higher accuracy rate of 96.6%. A hybrid model CNN-SVM was developed by [19] to predict seven predominant diseases related to tomato leaves. The highest results were achieved with a 92.6% accuracy. Another hybrid SVM-LR model was proposed by [20] for detecting powdery mildew disease of tomato leaves. The results demonstrated that the proposed model reached 92.37% accuracy.

An optimized DL-based method was proposed by [21] to detect tomato leaf diseases. Various pretrained models were applied, and the performance of each model was tested using different optimizers. The study concluded that MobileNetv3 Large using the Adagrad optimizer outperformed other models, with an accuracy rate of 99.81%. An image-based forecast using CNN was proposed by [22], who detected the early blight disease (EBD) of tomato plants. The study reported a 98.10% accuracy rate for the model. Similarly, an optimized transfer learning approach was proposed by [23], in which two pretrained models were applied to the tomato early blight disease (TEBD) dataset. The results concluded that Vgg16 outperformed ResNet50, with an accuracy rate of 99%. A study by [24] detected nine diseases of tomato leaf using a DL approach. For this purpose, a CRNN model with GRU was implemented to classify and detect tomato leaf diseases. The model achieved 99.62% accuracy when detecting tomato leaf diseases. A classification of tomato leaves using DL methods by utilizing various optimizers and learning rates (LR) was performed by [25]. Two DL pretrained models were applied to a dataset containing tomato leaf diseases. The reported results showed that Xception with Adam optimizer and an LR of 0.0001 outperformed other combinations with Xception and the Resnet50 model. The highest accuracy achieved was 99%.

A comparative study between ML and DL methods was conducted by [13] to classify tomato leaf diseases. The results of both approaches were compared, and DL methods outperformed ML methods. Moreover, among the DL methods, ResNet34 achieved the highest accuracy rate at 97.7%. Another DL-based approach was proposed by [26] to detect tomato leaf diseases. The higher classification rates of the proposed model occurred for 5, 7, and 10 classes, which were 99.51%, 98.65%, and 97.11%, respectively. The authors of [11] proposed an image-based diagnostic system using several DL methods, which were applied to a dataset collected from a village plant database and privately collected images containing a total of 24 classes. The reported results showed a higher performance by the DenseNet121 model, which yielded a classification accuracy of 95.31%. The study by [27] classified tomato plant diseases using the Vgg16 model. The classification accuracy for multi-class classification reached 99% while binary classification (healthy and unhealthy) reached 100%, with no preprocessing of images.

A robust DL-based detector for tomato leaf and pest recognition was proposed by [28]. In this regard, 3 detectors referred to as DL meta-architecture—were combined into VggNet and ResNet. The study reported that faster R-CNN in combination with Vgg16 has a higher recognition capability. Another robust intelligent system for detecting tomato disease using the DL approach was proposed by [29]. To train the model, a dataset containing 9 diseases was utilized. The results showed that the proposed model accomplished a higher accuracy rate of 99.12% on the same dataset, compared to 71.43% on other images from a different dataset. In the study by [30], two pretrained models were trained for detecting tomato leaf diseases on a dataset acquired from a plant village database. The results indicated that AlexNet outperformed Vgg16 and accomplished 97.49% accuracy.

A study by [31] attempted to classify and visualize the symptoms of tomato leaf diseases using the DL method. The model accomplished higher accuracy, at 99.18%. A CNN approach was used by [9] to detect tomato leaf disease; several pretrained methods were trained using an open dataset acquired from plant health. The study reported better performance of the ResNet model and achieved a higher accuracy rate of 97.28%. Another CNN model was proposed by [32] to detect tomato leaf diseases. The model was trained and reported 99.84% accuracy.

3. Deep Learning Architectures

From a broad view, DL belongs to the family of ML techniques utilizing artificial neural networks (ANN) to solve real-world problems related to images (i.e., segmentation, detection, and classification of images) that are widely applied in the fields of computer vision and image processing and have shown the best performance with optimal results. DL has also recently been used in agriculture to detect plant diseases using image analysis and significantly contributed to farming with outstanding outcomes. This study presents a hybrid DL model that combines two different state-of-the-art DL models to detect tomato leaf diseases. In order to better understand the proposed hybrid model, this section highlights the core concepts of each individual model and its architectural design, followed by the proposed hybrid model.

3.1. EfficientNetB3

EfficientNetB3 belongs to the EfficientNet family [33], ranges from B0 to B7, and is regarded as one of the most computationally efficient DL models developed using ImageNet [34]. EfficientNet is a CNN architecture and scaling technique that uses a compound coefficient to consistently scale all depth, width, and resolution dimensions [33]. Furthermore, the scaling method evenly scales network width, depth, and resolution using a set of immovable scaling coefficients, in contrast to standard practice, which scales these variables arbitrarily [33]. In CNN, the kernel is a filter which is utilized to retrieve attributes from images [35], while convolution is utilized to construct a feature map. The model architecture of EfficientNetB3 consists of a convolution layer of kernel size (3 × 3) with BN and swish activation followed by 26 MBconvolution blocks. The MBconvolution blocks are varied with kernel sizes of (3 × 3) and (5 × 5), as shown in Figure 1. The last block of MBconvo is followed by a convolution layer. Global average pooling is utilized at the end of the convolution layers for dimensionality reduction of the feature maps. Fully connected (FC) and softmax are used at the end of the model architecture to generate the output.

3.2. MobileNet

MobileNet, a CNN-based model developed by [36], has a simplified architecture that builds lightweight deep convolutional neural networks using depth-wise separable convolutions. In the model architecture described by [36], MobileNet factorizes standard convolutions into a depth-wise convolution and a (1 × 1) pointwise convolution, as shown in Figure 2. A single convolution on every channel is performed using depth-wise filters, while the output of a depth-wise convolution is combined with the (1 × 1) pointwise convolution [37]. Due to factorization, the computation and model sizes significantly decrease, which eventually enhances the performance of the model. ReLu activation is used between the layers in order to flatten the nonlinear outputs of the preceding layer and provide it to the succeeding layer as input [12].

3.3. Proposed Hybrid Model

A hybrid model can be used to improve predictive performance by running two or more relevant but distinct models and combining the results into a single score [38]. The literature review revealed that tomato leaf diseases were mostly detected and classified using individual DL models such as EfficientNet, MobileNet, and others, or a hybrid of ML and DL models. This study proposes EffiMob-Net, a hybrid DL model for detecting tomato leaf diseases that is a combination of two individual pretrained DL models, EfficientNet and MobileNet (see Table S1 in Supplementary Materials). A total of 10 diseases related to tomato leaves are recognized and classified using the hybrid EffiMob-Net. According to [39], accurate classification can be achieved by fusing diverse models with different hypotheses concerning class labels, which may not be viable with separate models. Using this approach, we took advantage of the standard architectures of both DL models in which the formerly trained weights of both DL models were loaded for the feature extraction of leaf images and combined for detection purposes, as shown in Figure 3.

The model architecture of the EffiMob-Net is simple in that the softmax layers (output layer) are removed from both individual models, the output of each model is flattened and is passed to the fully connected (FC) layer of each model. The outputs of the dense layers (layers of neurons in which each neuron in the following layer receives information from each neuron in the preceding layer) of both models are then combined using the concatenation function, and three additional FC layers containing 1024, 512, and 128 channels are added after concatenating the models, as exhibited in Figure 3. Regularization is used to fine-tune the model in order to decrease the regulated loss function and avoid overfitting and underfitting [40]. The risk of model overfitting is handled using regularization operations (i.e., kernel regularizer and activity regularizer), which are added to the last three FC layers. Moreover, in order to avoid the model overfitting issue, BN and dropout are also used after the last FC layer. The detection of tomato leaf diseases is performed using the softmax layer, which is added at the end of the hybrid model. ReLu activation is used throughout the FC layers except for the softmax layer. Figure 3 shows the detailed architecture of the proposed deep hybrid EffiMob-Net model.

4. Dataset

The proposed hybrid EffiMob-Net model was trained using an openly available dataset gathered from multiple sources, mostly from a plant village database [41] containing a total of 11 classes. Among the 11 classes, one was healthy and the remaining 10 represented different diseases of tomato leaf. The dataset consisted of a total of 32,535 images acquired from a plant village dataset and some collected images distributed into two separate folders: training and validation sets. In this study, the whole validation set is utilized for testing purposes; therefore, the validation set is changed to the test set shown in Figure 3. Thus far, this is the largest publicly available dataset of tomato leaf diseases. The training set contained 25,851 images; 6684 images were part of the test set. The images in both sets were distributed to 11 classes as described in Figure 4 Figure 5shows the number of images per class in the training set. Figure 6 shows sample images in the training set. The dataset is suitable for building a DL model that can predict a particular disease of a tomato leaf and classify them accordingly.

Data Preprocessing

Data preprocessing is an indispensable procedure that converts data into a structure that can be easily and proficiently processed in ML and other data science tasks [42]. Removing garbage from data augments the quality of the data [43], which directly affects the performance of the trained models and ensures improved results [44]. In the first step, the images were resized to the required sizes for training the proposed model. As described in [45], CNN typically allows fixed-size images, which creates several challenges for data collection and model building. Such challenges were overcome by resizing the images to the required size of (224 × 224) when building the proposed model. TensorFlow in Python programming was used to resize images to the desired size. The images were also normalized in a pixel value of range 0 to 1 by dividing them by 255 and feeding them into the network. In the last step, the images in both sets were reshuffled to increase the predictability power of the proposed model.

5. Experimental Setup

The dataset used in this study was split into two separate sets: training and testing at a ratio of 80% to 20%, respectively. According to [46], experimental research indicates that using 20–30% of the data for testing and the remaining 70–80% of the data for training yields optimal results. In this study, 80:20 achieved optimal results and was thus chosen for data splitting. The training set was utilized to train the hybrid EffiMob-Net model on a Google Colab in a GPU environment using Python programming language. The testing set was used to validate the model performance. The experiment was executed using MacBook Pro for 20 iterations in 40 batches. The model was compiled using the Adamax optimizer with a learning rate of 0.001. The best classifier with respect to accuracy metrics was selected to show the results for detecting tomato leaf diseases. The 20% testing set was used to verify the performance of the hybrid EffiMob-Net model using training and validation accuracies and losses. Categorical cross-entropy was used as a loss function to measure the losses. The experiment was repeated several times, and the best-fitting model with respect to accuracy metrics was finalized. The finalized trained hybrid model was then saved to the local directory for future use. Figure 7 depicts the training and validation accuracies. Normally, the curve of training accuracy is greater; however, both curves come closer to each other as the epochs advance. An epoch represents one iteration of training a model with all training data. The best epoch in which both curves coincide is epoch 20, which was one of the main reasons for executing the model for 20 epochs. Likewise, the training and validation loss shown in Figure 8 demonstrates the validity of the proposed hybrid EffiMob-Net in that both curves come closer to each other, progress simultaneously as the epochs advance, then coincide at epoch 13 and progress together in the same manner. This indicates the lack of overfitting of the hybrid EffiMob-Net model, which was avoided by using regularization, dropout, and BN techniques. The performance of the model was measured using accuracy, precision, recall, and F1-scores from the following equations.

$$Accuracy=\frac{TP+TN}{TP+TN+FP+FN}$$

(1)

$$Precision=\frac{TP}{TP+FP}$$

(2)

$$Recall or Sensitivity=\frac{TP}{TP+FN}$$

(3)

$$\mathrm{F}1-score=\frac{\left(2*Precision*Recall\right)}{Precision+Recall}$$

(4)

6. Results and Discussion

After implementing and testing the hybrid EffiMob-Net model on the testing set, the performance of the model was measured, and the highest accuracy rate achieved was 99.92%, which is thus far the highest accuracy in the same domain. Moreover, the classification report based on Equations (1)–(4) was measured, and the outcomes are reported in

Table 1. Classification report of EffiMob-Net model.

Class	Accuracy	Precision	Recall	F1-Score
Bacterial spot	99.84%	99.29%	99.20%	99.23%
Early blight	99.84%	98.98%	99.29%	99.14%
Late blight	99.87%	99.51%	99.36%	99.44%
Leaf mold	99.84%	99.17%	99.28%	99.23%
Septoria leaf spots	99.86%	99.31%	99.39%	99.35%
Two spider mites	99.86%	98.99%	98.86%	98.93%
Target spot	99.86%	99.04%	98.91%	98.97%
Tomato yellow leaf curl virus	99.89%	99.27%	99.39%	99.33%
Tomato mosaic virus	99.87%	99.19%	99.30%	99.25%
Powdery mildew	99.87%	99.43%	99.43%	99.43%
Healthy	99.85%	98.25%	97.76%	98.01%

.

The results shown in the classification report table for all 11 classes are above 99% for all measures with the exception of a few values. For example, the precisions of early blight, two spider spots, and healthy are 98.98%, 98.99%, and 98.25%, respectively. Similarly, recalls of two spider spots, target spot, and healthy are 98.86%, 98.91%, and 97.76%, respectively. Likewise, F1-scores for two spider spots, target spot, and healthy are 98.93%, 98.97%, and 98.01%, respectively. The mentioned values with respect to classes surpassed 98% except for the F1-score of healthy, which was close to 98%, showing the reliability of the proposed hybrid EffiMob-Net model when used as a smart detecting system for identifying tomato leaf diseases.

The overall accuracy of 99.92% and the classification report in Table 1 demonstrate the high performance of the proposed hybrid EffiMob-Net with a classification error of only 0.08%, which is negligible. The idea of distinct feature extraction using two separate DL models and the fusion of these features for detecting and classifying tomato leaf diseases is superior to that achieved when using an individual model, as discussed in the related work section. The conventional methods in which the feature extraction is handcrafted require high expertise; otherwise, the model efficacy can be poor. Additionally, such methods require more effort and time-consuming tasks. Therefore, DL-based methods are more useful for automatically generating features and have shown a high success rate in the identification and classification of images. Similarly, the feature extraction using multiple DL methods and the fusion features resulting from different methods show increased model accuracy. This discussion and the facts presented in the tables and figures demonstrate the reliability of the proposed hybrid EffiMob-Net model, which can be used as a reliable detector for detecting and identifying tomato leaf diseases.

7. Conclusions

The necessary precautionary measures should be taken to prevent tomato plant diseases in order to increase the cultivation of tomato crops. This study proposed a hybrid DL-based model that accurately detects and classifies 10 different tomato plant diseases through leaf images. The model architecture was designed by the fusion of two DL models in order to extract the distinct features from tomato leaf images, which were then combined to achieve the accurate identification of each disease with respect to classes. Several techniques (e.g., regularization, dropout, and BN) were used to prevent the model from being overfitted. During implementation, the optimal parameters were set in the model based on hyperparameter tuning using a random grid search technique. The proposed hybrid EffiMob-Net model was tested on processed images of tomato leaf diseases with a split ratio of 80/20 for the training/testing datasets. The results achieved demonstrate the efficacy of the proposed hybrid EffiMob-Net in accurately extracting the distinct features from tomato leaf images, with an accuracy rate of 99.92%, and a classification error of only 0.08%. Moreover, the classification report on factors such as precision, recall, and F1-score demonstrates the high performance of the proposed hybrid model in detecting tomato leaf diseases. The model is efficient in its performance based on the results achieved and, thus, can be used as an automatic detector for identifying tomato leaf diseases early in the growing process in order to increase production. The proposed hybrid model can also be used to detect other plant diseases in the agriculture field based on leaf images.