Identification of Grape Diseases Based on Improved YOLOXS

Abstract

Here we proposed a grape disease identification model based on improved YOLOXS (GFCD-YOLOXS) to achieve real-time detection of grape diseases in field conditions. We build a dataset of 11,056 grape disease images in 15 categories, based on 2566 original grape disease images provided by the State Key Laboratory of Plant Pest Biology data center after pre-processing. To improve the YOLOXS algorithm, first, the FOCUS module was added to the backbone network to reduce the lack of information related to grape diseases in the convolution process so that the different depth features in the backbone network are fused. Then, the CBAM (Convolutional Block Attention Module) was introduced at the prediction end to make the model focus on the key features of grape diseases and mitigate the influence of the natural environment. Finally, the double residual edge was introduced at the prediction end to prevent degradation in the deep network and to make full use of the non-key features. Compared with the experimental results of relevant authoritative literature, GFCD-YOLOXS had the highest identification accuracy of 99.10%, indicating the superiority of the algorithm in this paper.

1. Introduction

Grapes are one of the four most produced fruits in the world, with an annual production of about 75 million tonnes; thus, they occupy an important position among the fruits of the world [1]. Various diseases can attack grapes during the growing process, which can reduces the quality and yield of the grapes and even lead to their deaths in severe cases. Controlling grapevine diseases is important for the grape industry due to the variety of diseases and their severe damage. The efficient identification of grape diseases is one of the key issues in grape disease control. Currently, the identification of grape diseases is mainly made using artificial intelligence methods, especially deep learning, of which there are two main categories: classification of grape diseases using picture classification, and detection of grape diseases using target detection.

In the study of the classification of grape disease pictures, ALKAN et al. [2] developed a grape disease identification system using convolutional neural networks and transfer learning techniques to achieve an accuracy of 92.5% in identifying ten grape diseases. Liu et al. [3] enhanced multidimensional feature extraction using the Inception network structure and applied the dense connectivity strategy to four cascaded Inception structures to achieve an accuracy of 97.22% in identifying six grape diseases. Shantkumari et al. [4] used CNNC and IKNN, respectively, and the accuracy of the two models was 96.60% and 98.07%, respectively, for identifying four grape diseases in the Plant Village dataset [5]. Math et al. [6] constructed their own CNN classification model (DCNN) using a dataset of four grape crop images from the Plant Village dataset, achieving an accuracy of 99.34%. Peng et al. [7] fused the features extracted from ResNet50 and ResNet10 to build a grape disease identification model, eventually achieving 99.08% accuracy in identifying four grape diseases in the Plant Village dataset. Yin [8] used transfer learning in combination with the MobileNetV3 network to achieve a final accuracy of 99.4% for six grape diseases. Yang et al. [9] achieved 96.05% accuracy in identifying six grape pests and diseases in complex environments based on the ShuffleNet V2 network combined with the MDF decision-making method. Lu et al. [10] combined ghost convolution with a transformer to propose a GeT grape pest identification model, which eventually achieved an accuracy of 98.14% for 11 grape pests and diseases. Kaur et al. [11] extracted features using a fully connected layer, removed irrelevant features from the feature extraction vector using the proposed variance technique and used a classification algorithm to classify the resulting features, achieving an accuracy of 98.7% for the detection of six grape diseases. Suo et al. [12] first used the GSLL algorithm to reduce image noise and then used CASM-AMFMNet to achieve an accuracy of 95.95% in identifying five grape diseases. Asad et al. [13] created a dataset of grape leaves affected by nutrient deficiencies from farmland in a controlled environment, before customizing a CNN model and validating it using an n-fold cross-validation method, ultimately achieving an accuracy of 97.07% for the identification of four grape diseases. Lin et al. [14] achieved 86.29% accuracy in recognizing seven grape diseases in AI Challenger 2018 through implementing feature fusion of different depths through the RFFB module and introducing CBAM to extract valid disease information. ZINONOS et al. [15] combined long-range radio technology with deep learning to transmit images over long distances, achieving an accuracy of 98.33% in identifying four grape diseases in the Plant Village dataset, even with a packet loss rate of 50%. Although much research has been performed on the classification of grape diseases, image classification can only determine the disease type of the whole picture. It cannot, however, directly locate and mark the location of the disease; thus, it has some limitations in real-time detection.

Fewer studies have previously been conducted on the target detection of grape diseases. Xie et al. [16] based their Faster R-CNN detection algorithm and introduced the Inception-v1, Inception-ResNet-v2, and SE modules to achieve a recognition rate of 81.1% for the four grape diseases. Zhu et al. [17] first enhanced the images of black rot in grapes in the Plant Village dataset via bilinear interpolation, before introducing the SPP module to the backbone network of YOLOv3 and replacing the loss function from IOU to GIOU to achieve an accuracy of 95.79% for the detection of black rot in grapes. Dwivedi et al. [18] achieved an accuracy of 99.93% in identifying four grape diseases in the Plant Village dataset based on Faster RCNN, introducing multi-task learning and a dual attention mechanism construct. However, most of these images are from the Plant Village dataset, which contains only four grape images, i.e., healthy leaves, black rot, brown spot, and Esca, which are mainly used to demonstrate object detection tasks in laboratory environments with relatively simple backgrounds that differ from those of grape diseases in natural environments; therefore, they may affect the generalization ability of the model. In addition, the detection speed of these algorithms is slow, which may have some drawbacks for application scenarios with high real-time detection requirements.

To solve the above problems, this paper, based on previous research and addressing the shortcomings of the identification method of improved YOLOV3 in terms of real-time and model size at earlier work [19], realizes real-time detection of grape diseases in natural environments using the more advanced YOLOXS [20] algorithm in the YOLO [21] series and combining it with the characteristics of grape diseases. The main work of this paper is as follows:

(1)

In response to the fact that most current grape disease detection is performed in a laboratory setting, this paper relies on the State Key Laboratory of Plant Pest Biology to establish a dataset of grape diseases in natural environments.

(2)

To solve the problem of slow detection of grape diseases, the advanced YOLOXS algorithm in the YOLO series is selected in this paper, and the FOCUS module, CBAM module [22], and double residual edges are added according to the characteristics of grape diseases.

(3)

In this paper, we experimentally compare the performance of the improved model before and after the improvement and compare the results with the current available related research. The experimental results show that the enhanced method used in this paper has greater practicality and application value by ensuring high accuracy while maintaining a faster detection speed.

2. Improved YOLOXS Model for Grape Disease Detection

2.1. YOLOXS Network

The YOLO (You Only Look Once) algorithm was proposed by Redmon et al. [21] in 2016 and developed into a series of algorithms; YOLOX was proposed by MEGVII Inc. of Beijing, China in 2021 and had superior performance. According to the characteristics of grape diseases, YOLOXS was selected as the benchmark model for grape disease identification in this paper.

YOLOXS consists of 5 parts: input, backbone network, neck, prediction, and output. On the input side, a 416 × 416 × 3 resolution image is input into the CSPNet [23] backbone network to obtain three features with different granularity. The neck part then uses FPN [24] and PAN [25] for feature fusion and extraction, and the prediction part uses a double decoupling head. Finally, the results are output at the output, whose network structure is shown in Figure 1.

2.2. Improved YOLOXS Network

The improved YOLOXS model (GFCD-YOLOXS) is shown in Figure 2, and the details are described below.

2.2.1. Addition of Three FOCUS Modules to the Backbone Network

FOCUS, first proposed in YOLOV5 [26], has very few parameters and can obtain down-sampled maps without information loss, as shown in the structure diagram in Figure 3.

In response to the possible loss of information when YOLOXS extracts features from grapes in natural environments, this paper adds three FOCUS modules to the backbone network of YOLOXS to connect feature maps of different depths, effectively using features of different depths and reducing the loss of disease features from grapes; the specific structure is shown in Figure 4.

2.2.2. Introduction of CBAM Module for Prediction Heads

The CBAM module was proposed by Woo et al. [22]. in 2018 and consists of channel attention and spatial attention modules. The weights of the feature map in the channel and spatial dimensions can be inferred to improve the extraction of key features of the feature map; the specific structure is shown in Figure 5.

This paper adds a CBAM module to the prediction head of YOLOXS, ensuring that the network can focus on the key information of grape diseases and improve the accuracy of grape disease identification. The network structure is shown in Figure 6.

2.2.3. Prediction Head Introduces Double Residual Edges

The residual edge is proposed by He et al. [27] in the ResNet network structure, which adds the input features directly to the output features to solve the problem of network degradation resulting from the depth structure of the network model, while ensuring the integrity of the original features, as shown in Figure 7.

This paper introduces the idea of residual edges to address problems resulting from the YOLOXS prediction head being located in the deeper part of the network, which can easily lead to network degradation. It is inspired by the fact that human beings need to associate non-key features with key features when recognizing objects, and adds double residual edges (DRES) to the YOLOXS prediction head to strengthen the information of non-key features. The improved prediction head is shown in Figure 8.

3. Experiments and Analysis

3.1. Experimental Environment

The programming language used is Python 3.8.13, the deep learning framework used is Pytorch 1.21.1, the development platform used is PyCharm 2021.3.2, the operating system used is Linux 4.15.0, the CPU model used is Intel(R) Xeon(R) Gold 5218 CPU @ 2.30 GHz, and the memory space used is 32 GB; the GPU Tesla T4 and cuda11.3 were also used for GPU acceleration.

3.2. Datasets and Pre-Processing

3.2.1. Initial Dataset

The dataset referred to in this paper contains 15 different diseases that may be encountered during the growth of grapes, namely: (1) healthy grapes, (2) healthy leaves, (3) powdery mildew, (4) Black rot, (5) Esca, (6) brown spot, (7) black spot, (8) downy mildew, (9) Sour rot, (10) deficiency, (11) anthracnose, (12) botrytis cinerea, (13) felt diseases, (14) leaf roll virus, (15) and phylloxera. These images come from the State Key Laboratory of Plant Pest Biology, which is the authoritative modern agricultural technology industrial system for grapevine pest prevention and control in China, possessing comprehensive and targeted research on grapevine pests and diseases in China and complete data on grapevine diseases and control strategies. Figure 9 shows representative images of some grape diseases in the dataset.

As shown in Figure 9, grape anthracnose manifests itself as small brown circular spots on the fruit surface; grape sour rot manifests itself as weak brown spots or streaks on the grape surface; grape downy mildew manifests itself as polygonal yellow–brown spots consisting of grape leaves; and grape leaf roll manifests itself as a recoiling of the growing leaves from the leaf margin downward.

Figure 10 shows the initial data volume of the dataset, with 15 categories and 2566 images. From this figure, it is found that the various types of image data in the original dataset are unbalanced, which may lead to problems such as overfitting and poor generalization of the trained model. Therefore, pre-processing of the original grape disease images is needed.

3.2.2. Data Preprocessing

In this paper, a data enhancement method [28] was used to expand 15 categories of grape images through rotating, scaling, cropping, and adding Gaussian noise to increase the number of training sets based on a balanced distribution of image data in each category; the effect of some of the images after processing is shown in Figure 11. Figure 11a–c correspond to the damage symptoms of the black spot of the grape, leaf roll of the grape, and felt disease of the grape, with the first image from the left in each category being the original image of the disease and the remaining four images being data enhanced. After the expansion, the number of images in each category was balanced, and the grape disease data set reached 11,056 images, as shown in Figure 12.

3.3. Evaluation Indicator

In this experiment, mAP (mean average precision), FPS (frames per second), and ModelSize are used as model performance evaluation metrics, which are defined as shown in Equations (1)–(3).

$$Precision=\frac{TP}{TP+FP}$$

(1)

$$R\mathrm{e}call=\frac{TP}{TP+FN}$$

(2)

$$AP={{\displaystyle \int }}_0^{1}Precision\left(Recall\right)d\left(Recall\right)$$

(3)

T and F denote the number of samples classified correctly and incorrectly, respectively, and P and N denote the number of samples predicted to be positive and negative, respectively. TP indicates the number of samples for which the sample is positive and which the classifier predicts to be positive. FP indicates the number of negative samples predicted to be positive using the classifier. FN indicates the number of samples that are positive and predicted to be negative using the classifier.

3.4. Model Training and Parameter Settings

In this paper, the input image has a resolution of 416 × 416, and the model is trained using transfer learning [29]. Training is divided into two phases: pre-training and formal training.

For pre-training, 300 Epoch thawing training was performed on the public dataset VOC [30] (visual object class) to obtain the initial weights for the grape disease identification model. Formal training divides the dataset of this paper into training, validation, and test sets in the ratio of 8:1:1 for training. Formal training starts with a freeze session of 50 Epochs, followed by a thaw session of 250 Epochs, for a total of 300 Epochs. The initial learning rates for freeze training and thaw training were 0.001 and 0.0001, respectively, and the learning rates were all adjusted between 0.01 and 0.0001 using cosine annealing decay. The batch sizes for freeze and thaw training were set to 16 and 8, respectively, and the optimizer used stochastic gradient descent (SGD). Mosaic data augmentation was used for the former 150 pre-training and formal training rounds, though not for the latter 150 rounds.

3.5. Improvement Experiments

3.5.1. Ablation Experiments

In this paper, eight sets of experiments were designed to verify the impact of the three aspects of GFCD-YOLOXS improvement and their combinations on the results of grape disease identification, as shown in

Table 1. Comparison of mAP in ablation experiment.

Model	FOCUS	CBAM	DRES	mAP	ModelSize (M)	FPS
YOLOXS				97.05%	34.3	64.58
YOLOXS_1	√			98.20%	37.0	50.21
YOLOXS_2		√		97.46%	34.4	53.57
YOLOXS_3			√	96.80%	34.3	64.32
YOLOXS_4	√	√		98.71%	37.1	49.83
YOLOXS_5	√		√	98.22%	37.0	50.14
YOLOXS_6		√	√	97.64%	34.4	53.32
GFCD-YOLOXS	√	√	√	99.10%	37.1	48.57

, where “√” indicates that the module was added in this experiment.

As can be seen from Table 1, the addition of the FOCUS module increased the mAP values for all disease identification methods by 1.15% compared to the original YOLOXS, as the use of the FOCUS network to connect features of different depths while retaining the complete features reduced the lack of information on features related to grape diseases; the introduction of the CBAM module also increased the mAP values for all disease identification methods compared to the original YOLOXS. The introduction of the CBAM module increased the mAP values for all disease recognition methods by 0.41%, as the introduction of the CBAM module allowed the network to focus on key features of grape diseases; the introduction of only double residual edges decreased the mAP values for all disease recognition by 0.25%, because the introduction of only residual edges caused the model to generate more noise, resulting in a decrease in the recognition rate when the CBAM module and DRES were introduced, compared to the introduction of only the CBAM module. This result occurred because the introduction of these two modules allows the model to focus on key features while connecting non-key features for recognition; when the three improved modules are used together, the highest mAP value of 99.10% is achieved for all grape disease recognition methods, which is because the improvement not only preserves the complete information but also allows the model to connect non-key features while focusing on key features. In addition, the improved ModelSize did not increase excessively, and the FPS values did not decrease.

To further demonstrate the effectiveness of these three improvements, we plotted the loss function and mAP plots for the validation set during formal training of the ablation experiment. The loss function plot is shown in Figure 13, with the loss values decreasing and leveling off as the training Epoch increases, and the GFCD-YOLOXS model having the lowest loss values. The mAP plot is shown in Figure 14. The mAP values gradually increased and leveled off with increasing training Epoch, with the highest mAP values for GFCD-YOLOXS. Combined with the analysis in Figure 13 and Figure 14, the GFCD-YOLOCS model improved in this paper has the most effective recognition effect.

3.5.2. A Comparison of Different Attention Mechanisms

In order to verify the superiority of introducing the CBAM module into the grape disease identification model, this paper replaces the CBAM with SE [31] and CA [32] (Coordinate Attention) modules, respectively, based on GFCD-YOLOXS, for comparative experiments. The results are shown in

Table 2. Comparison of different attention mechanism modules.

Disease Name	SE	CA	CBAM
Sour rot	95.16%	95.86%	95.26%
Anthracnose	99.91%	96.93%	99.90%
Black rot	100.00%	100.00%	100.00%
Black spot	100.00%	100.00%	100.00%
Botrytis cinerea	100.00%	100.00%	100.00%
Brown spot	100.00%	100.00%	100.00%
Deficiency	95.78%	97.47%	96.48%
Downy mildew	98.77%	99.85%	99.64%
Esca	100.00%	100.00%	100.00%
Felt diseases	98.40%	97.41%	98.21%
Healthy grape	99.28%	99.83%	99.36%
Healthy leaves	99.20%	99.06%	99.69%
Leaf roll	99.18%	99.04%	98.11%
Phylloxera	97.06%	99.86%	99.90%
Powdery mildew	99.97%	99.98%	99.99%
All diseases	98.85%	99.02%	99.10%

, where the introduction of the SE attention mechanism resulted in the highest AP values for the identification of grape anthracnose, grape felt, and grape leaf roll; the introduction of the CA attention mechanism resulted in the highest AP values for the identification of grape sour rot, grape deficiency, grape downy mildew, and healthy grapes; and the introduction of the CBAM attention mechanism resulted in the highest AP values for identifying healthy grape leaves, grape phylloxera, and grape powdery mildew. In a comprehensive comparison, the introduction of the CBAM module had the highest mAP value of 99.10% for all disease identification, which was 0.25% higher than the introduction of the SE module and 0.08% higher than the introduction of the CA module. This result indicates that introducing the CBAM module allows the model to more effectively focus on the key features of grape diseases and reduce the influence of noise, thus improving the accuracy of grape disease identification.

3.5.3. Comparison of Residual Edges

In order to verify the superiority of introducing double residual edges into the grape disease identification model, the GFCD-YOLOXS model is compared with the GFCD-YOLOXS model without introducing residual edges, while the GFCD-YOLOXS model is compared with the introduction of single residual edges in separate experiments. The results are shown in

Table 3. Comparison of different residual edges.

Disease Name	No RES	RES	DRES
Sour rot	93.99%	93.66%	95.26%
Anthracnose	99.73%	98.42%	99.90%
Black rot	100.00%	100.00%	100.00%
Black spot	100.00%	100.00%	100.00%
Botrytis cinerea	100.00%	99.98%	100.00%
Brown spot	100.00%	100.00%	100.00%
Deficiency	96.64%	96.42%	96.48%
Downy mildew	99.73%	99.88%	99.64%
Esca	100.00%	99.95%	100.00%
Felt diseases	98.24%	98.58%	98.21%
Healthy grape	99.48%	98.63%	99.36%
Healthy leaves	99.07%	99.36%	99.69%
Leaf roll	95.14%	99.18%	98.11%
Phylloxera	98.71%	99.34%	99.90%
Powdery mildew	99.98%	100.00%	99.99%
All diseases	98.71%	98.89%	99.10%

. The introduction of single residual edges improved the mAP values for all disease identification methods by 0.18% compared to the method without double residual edges, indicating that introducing single residual edges can prevent degradation of the network. The introduction of double residual edges increased the mAP values for all disease identification by 0.21% compared to single residual edges, suggesting that introducing double residual edges can further enhance non-critical features and improve the accuracy of grape disease identification.

3.6. Comparison of the Model in This Paper with Other Models

3.6.1. About This Dataset Compared to Other Models

In order to verify the feasibility of the GFCD-YOLOXS algorithm, this paper compares the model with the existing classical target detection models Faster-RCNN [33], SSD [34], RetinaNet [35], YOLOV5S, and Eff-B3-YOLOV3 [19]; the experimental results are shown in

Table 4. Results of different model checks for this paper’s dataset.

Model	mAP	ModelSize (M)	FPS
Faster RCNN	94.57%	108	17.20
SSD	90.54%	100	20.12
RetinaNet	94.60%	140	16.40
YOLOV5S	95.99%	28.6	60.3
Eff-B3-YOLOV3	98.60%	77.6	31.20
GFCD-YOLOXS	99.10%	37.1	48.57

. Table 4 shows that the GFCD-YOLOXS recognition method proposed in this paper has the highest accuracy of 99.10%. Moreover, its model size and detection speed are smaller and faster than Faster RCNN, SSD, RetinaNet, and the previously studied Eff-B3-YOLOV3 model, meaning that it can meet the requirements of real-time detection of plant protection equipment.

Furthermore, three pictures suffering from grape powdery mildew, grape leaf roll, and grape deficiency were selected to compare the detection effects of different models. The experimental results are shown in Figure 15. When using the Faster RCNN model, grape powdery mildew appears to be incorrectly identified, and grape leaf roll and grape deficiency appear to be missed in complex backgrounds; when using the SSD model, grape powdery mildew appears to be missed in boundary identification, grape leafroll appears to be missed in dim environment identification, and grape deficiency appears to be missed in the presence of occlusion; and when using the RetinaNet model, grape powdery mildew appears to be missed. The detection of small targets in grapevine leaf roll and grapevine deficiency improved when using the RetinaNet model; however, there were still missed detections. When using the YOLOV5S model, grapevine deficiency was missed for complex backgrounds and small targets. In Eff-B3-YOLOV3, the overall confidence of the identification results was low; in contrast, In contrast, the use of GFCD-YOLOXS reduces feature loss due to the inclusion of the FOCUS network to fuse features of different depths, thus reducing missed detection, and the inclusion of CBAM, double residual edges, allows the network to focus on key features while linking non-key features, thus improving the accuracy of model detection.

3.6.2. On Public Datasets Compared with Other Literature

In order to verify the advancements made using this model, experiments were conducted in this paper using data on grapes from plant trees, which were then compared with the latest literature. The comparison results are shown in

Table 5. On comparison of public datasets with other literature.

Literature	Model	Precision	F1	Accuracy
Math [6]	Custom Models	99.34%	99.44%	99.32%
Peng [7]	CNN + SVM	99.08%	99.26%	99.25%
Lin [14]	GrapeNet	86.29%	77.76%	79.05%
Ours	GFCD-YOLOXS	100%	100%	100%

, which shows that the GFCD-YOLOXS model has the highest recognition rate and better generalization and robustness compared with the methods proposed in the literature [6,7,14].

3.7. Example of Recognition Results

A typical example of the results of this model for grapevine disease identification is shown in Figure 16, which can successfully identify diseases suffered by the fruit stalks of grapes, such as Botrytis cinerea of grapes; Can successfully identify diseases suffered by grapes, such as Botrytis cinerea, acid rot, black spot, powdery mildew, and anthracnose; The possibility of successfully identifying diseases suffered by grape leaves, such as grape leaf roll and grape downy mildew; Multiple targets of the same species to be detected, such as grape leaf roll disease, healthy grapes, can be successfully identified.

4. Conclusions

This paper investigates the intelligent identification of grape diseases in their natural state based on an improved YOLOXS. A grape disease dataset in a natural environment was initially constructed. The YOLOXS algorithm was then improved through adding the FOCUS module to the backbone network, introducing CBAM, double residual edges in the prediction head. Finally, a grape disease identification model with a mAP of 99.10% (GFCR-YOLOXS) was obtained through training. The rich experimental and comparative results show that the model in this paper is superior and practical compared to the classical model and the models in the relevant recent authoritative literature, and has important reference values for the control of grape diseases.

In the future, we will investigate the following aspects: (1) maintenance and update of the dataset. The initial dataset for this work was small and did not include photos from complex environments, such as rain and snow, or the lack of breakdown of deficiency diseases. In the future, we will collect more experimental data, further refine the severity of grape diseases, and include non-invasive disease species of grapes to improve the comprehensiveness and reliability of the dataset. (2) Multimodal data fusion. In practical applications, multiple data sources may be involved, such as infrared images, hyperspectral images, etc. Therefore, the fusion of various data can improve the recognition accuracy and robustness of the model to further improve the diagnosis and treatment. (3) Identification of other crop diseases. We will study the disease characteristics of different crops and improve the existing models to apply them to disease identification in other crops. This approach will help farmers to detect and treat crop diseases promptly, and enhance the efficiency of agricultural production and the quality of the farm output. In conclusion, future research directions will further explore the characteristics of crop diseases and their detection techniques in depth, and continuously improve the models’ accuracy, robustness, and practicality to promote the sustainable development of agricultural production.