A Computational Procedure for the Recognition and Classification of Maize Leaf Diseases Out of Healthy Leaves Using Convolutional Neural Networks

Abstract

Plant leaf diseases can affect plant leaves to a certain extent that the plants can collapse and die completely. These diseases may drastically decrease the supply of vegetables and fruits to the market, and result in a low agricultural economy. In the literature, different laboratory methods of plant leaf disease detection have been used. These methods were time consuming and could not cover large areas for the detection of leaf diseases. This study infiltrates through the facilitated principles of the convolutional neural network (CNN) in order to model a network for image recognition and classification of these diseases. Neuroph was used to perform the training of a CNN network that recognised and classified images of the maize leaf diseases that were collected by use of a smart phone camera. A novel way of training and methodology was used to expedite a quick and easy implementation of the system in practice. The developed model was able to recognise three different types of maize leaf diseases out of healthy leaves. The northern corn leaf blight (Exserohilum), common rust (Puccinia sorghi) and gray leaf spot (Cercospora) diseases were chosen for this study as they affect most parts of Southern Africa’s maize fields.

1. Introduction

Gray leaf spot of maize is a maize disease caused by the Cercospora zeae-maydis fungus. It is now seen as one of the most significant yield-limiting diseases of maize (corn) worldwide, posing a serious threat to maize production in many areas of the eastern United States and, more recently, in large areas of the U.S. Corn Belt and Africa. Typically, its symptoms are observed on the lower leaves [1]. Figure 1 and Figure 2 show the stages of gray leaf spot of maize in terms of maturity.

Common rust maize disease, otherwise caused by the Puccinia sorghi pathogen is favoured, by cool temperatures (16–23 °C) and high relative humidity (100%) [2]. The functional leaf area and photosynthesis are reduced by the disease lesions. Spots are found on both upper and lower leaf surfaces [3]. Figure 3 and Figure 4 show early common rust lesions and more advanced disease development, respectively.

Northern corn leaf blight is a maize disease caused by a fungus called Exserohilum turcicum. Cool to moderate temperatures and high relative humidity favour the development of the northern corn leaf blight. The symptoms are recognised by the relatively large gray cigar-shaped lesions that can develop on leaves [4]. The changing stages of northern corn leaf blight due to a fungus concentration are as shown in Figure 5 and Figure 6.

In the literature, there are several algorithms of machine learning that were used for the recognition and detection of plant leaf diseases [5], utilizing Caffe, a deep learning framework developed by Berkeley Vision and Learning Centre for the recognition of plant leaf diseases by means of CNN. With feature extraction methods based on Open CV, the experimental results of the developed model achieved between 91% and 98% precision, and 93% on average for separate class tests [5].

By using deep learning methods, a public dataset of 54,306 images of diseased and healthy plant leaves was collected under controlled conditions and used to train a deep convolutional neural network to identify 14 crop species and 26 diseases. The feasibility of this approach was demonstrated by a trained model that achieved an accuracy of 99.35% on a held-out test set [6].

A study was carried out to apply the artificial neural network (ANN) analysis technique for discriminating and classifying fungal infections in oil palm trees [7]. Raw, first and second derivative spectra radiometer datasets were used at an early stage. These were acquired from 1,16 spectral signatures of foliar samples, in four disease levels (T1: healthy, T2: mildly-infected, T3: moderately infected, and T4: severely infected) [7].

A web based tool was proposed that helped farmers in the identification of fruit diseases by uploading fruit images to the system. Already trained datasets of the pomegranate fruit were used by the system. The images to be analysed were given by the user to undergo several processing steps in order to detect the severity of the diseases by comparing them with the trained dataset images. The experimental results of the proposed approach demonstrated 82% accuracy for the identification of the pomegranate disease [8].

Very little is known about the implementation of CNN for plant disease recognition built on a framework that utilises general user interface (GUI) to train the network by use of raw images with feature extractions embedded in the program’s library. The main objective of this study was to design a system that would recognise and classify the maize leaf diseases out of healthy leaves by means of facilitated CNN. The development and novelty of the proposed model lay in its simplicity; background images and healthy leaves were in accordance with other classes, empowering the model to distinguish between diseased leaves and sound ones or from the environment by utilizing deep CNN.

The rest of the paper is organised as follows: The Materials and Methods are presented in Section 2; achieved experimental results and related discussions are presented in Section 3; Section 4 discusses the results; and finally Section 5 draws up our conclusions of the study.

Related Work

The use of machine learning algorithms for the detection of plant leaf disease is dominant in the literature. Machine learning models for the prediction of these plant leaf diseases were found to differ in accuracy. Various techniques are at present being used for the recognition of plant leaf diseases by the application of computer vision. One of them is disease detection by colour feature extraction from images. CNNs are known to outsmart the classification and identification of images by use of a colour feature extraction. For such reasons, Sladojevic et al. performed the deep CNN training for plant disease recognition and classification [5]. The test results on the created model accomplished accuracy somewhere in the range of 91% and 98%, for separate class tests, 96.3% on average.

Utilizing an open dataset of 54,306 images of diseased and healthy plant leaves gathered under controlled conditions, Mohanty et al. trained a deep CNN to distinguish 14 crop species and 26 diseases. The trained model accomplished a precision of 99.35% on a held-out test set, showing the plausibility of this approach.

A study in comparison of support vector machine (SVM) and ANN, was performed by Pujai et al. [9]. Algorithms for colour extraction and texture features were developed, which were thus used to train SVM and ANN classifiers. The study presented a reduced feature set based approach for the recognition and classification of images of plant diseases. The results revealed that the SVM classifier was progressively reasonable for identification and classification of plant diseases. An SVM classifier was 92.17% than the ANN classifier that had an accuracy of 87.4%.

Recently, wheat disease detection through leaf image and data processing techniques is used extensively to assist farmers in monitoring the big plantation area. Researchers such as Dixit and Ema applied SVM for the recognition and classification of wheat diseases. In their study, they mentioned in detail the key issues and challenges in wheat leaf detection by means of SVM [10].

The purpose of this study is to model a facilitated GUI based CNN for the recognition and classification of maize leaf diseases. The use of the image recognition feature in the Neuroph Studio framework enabled the design of CNN with feature extraction features embedded in the program’s library. The overall system’s accuracy showed a progressive 99.9% for the classification of northern corn leaf blight (Exserohilum), 91% of gray leaf spot (Cercospora), 87% of common rust (Puccinia sorghi), and 93.5% of healthy leaves using a maize data set of 100 images for each disease class, and 100 images for healthy class.

2. Materials and Methods

2.1. The Concept of CNN

The regular NN (neural network) is not equipped for managing the images. If a regular NN were to be used with the images, then each pixel of the image would have to be connected to its neuron resulting in a network which would be computationally expensive. CNN handles the images in various ways, yet, at the same time, it pursues the general idea of regular NN. Figure 7 shows an architecture of the CNN.

Convolution and pooling (feature extraction): Convolution has a set of learnable filters which are a matrix (width (W), height (H), and depth (D)). The input image is considered as a matrix and the filter is imagined sliding through the input image matrix in order to get the convoluted image which is the filtered image of the actual input image. If a filter is applied on the input image, the result would be an output matrix smaller than the original image. Padding plays an important role if we need to get the same size outputted as the input size. Pooling is another important concept of CNN. Pooling is a form of non-linear down sampling function that in the case of CNNs is applied to convoluted images. Among several non-linear functions that can do the pooling, max pooling is the most common one.

Classification: The input images are classified by a fully connected layer just after the convolution and max pooling layers. Neurons in a fully connected layer have connections to all activations in the previous layer, as seen in the regular neural network. Figure 7 shows the maize plant diseases that were recognised and classified by a fully connected layer of the proposed CNN.

2.2. Materials

The Neuroph framework consists of the Java neural network library and integrated neural network and Java IDE based on the NetBeans Platform—Neuroph Studio. It is an integrated environment to create and deploy neural networks to Java programs. Neuroph supports common neural network architectures, and it is very flexible so one can easily extend it to suit the specific needs. The basic structure of the framework packages is shown in Figure 8.

2.3. Methodology

A CNN was built on the Windows PC platform. Neuroph Studio framework was used to build a CNN with 50 hidden layers for the recognition and classification of maize leaf diseases out of healthy leaves. The network was trained by using 100 colour images of each disease and healthy leaves with resolution settings configured as 10 × 20 × 3 (height × width × RGB). With the previously mentioned image resolution settings, the CNN was modelled to have 600 inputs through the input layer of the neural network classifier. Of 100 available images for each class to be recognised, 70% were used for training and 30% for testing of the CNN. Back propagation was used as the learning algorithm to train the CNN network.

A mathematical model of the proposed CNN was implemented as shown below: The order taken from the CNN was an order 3 tensor as its input. The input images that were tested for possible diseases were imaged with H rows, W columns, and 3 channels (RGB colour channels). An abstract description of our CNN model is illustrated by Equation (1).

$$x^1\to {\mathit{w}}^{\mathbf{1}}\to x^2\to \dots \to x^{L-1}\to {\mathit{w}}^{\mathit{L}\mathbf{-}\mathbf{1}}\to x^L\to {\mathit{w}}^{\mathit{L}}\to$$

(1)

Equation (1) shows how the proposed model of the CNN runs layer by layer. The input $x^1$ was an image of a maize leaf disease or a sound one with order 3 tensors. The $x^1$ was the input to the first layer’s input collectively known as tensor W¹. The output of the first layer was $x^2$ which also acted as the input to the second layer processing. The processing proceeded until all layers in the CNN had been finished, which gave an output of $x^L$. One additional layer, however, was added for backward error propagation for learning parameter values in the CNN. The last layer was a loss layer. A simple loss function we used is as shown in Equation (2).

$$z=\frac{1}{2}\parallel t-x^L{\parallel }^2$$

(2)

In Equation (2), $t$ denotes the corresponding target value for the input $x^L$. The loss function formula in Equation (2) was used to measure the discrepancy between CNN prediction $x^L$ and $t$. Alternatively, the output prediction of the CNN was given as shown in Equation (3).

$$ar{g}_{i}max{x}_i^L$$

(3)

The loss layer was not needed in the prediction, but was useful in learning of CNN parameters using a set of diseased and healthy maize leaf data sets as training sets. Our CNN model used the stochastic gradient descent (SGD) in order to learn the model parameters. Instead of outputting a prediction, we compared the prediction with target t corresponding to input. The loss Z was then a supervision signal, guiding how the parameters of the model were to be updated. The SGD way of modifying the parameters was as shown in Equation (4).

$$w^i\leftarrow w^i-\eta \frac{\partial z}{d{w}^i}$$

(4)

where $\eta$ represents the learning rate. The learning rate was chosen to be 0.01.

Equation (5) has a superscript “time” index (e.g., training epochs/iterations).

$${\left(w^i\right)}^{t+1}={\left(w^i\right)}^t-\eta \frac{\partial z}{\partial {\left(w^i\right)}^t}$$

(5)

The activation function used in the convolution layer was the rectified linear unit (ReLU) function. The ReLU can be regarded as a truncation performed individually for every element in the input. The ReLU is presented by Equation (6).

$$j_{i,j,d}=\max \{0,x_{i,j,d}^l\}$$

(6)

The l-th layer, had inputs that formed an order 3 tensor $x^l$ with $x^l\in R^{H^l\times W^l\times D^l}$. Thus, for this reason we needed a triplet index set $\left(i^l,j^l,d^l\right)$ to locate any specific element in $x^l$. Based on Equation (6), it is obvious that in Equation (7),

$$\frac{\partial y_{i,j,d}}{\partial y_{i,j,d}^l}=〚x_{i,j,d}^i>0〛$$

(7)

where $〚.〛$ was the indicator function, being 1 if its argument was true, and 0 otherwise.

Hence, we had

$${\left[\frac{\partial z}{\partial x^l}\right]}_{i,j,d}=\{\begin{array}{cc}{\left[\frac{dz}{dy}\right]}_{i,j,d}& if{x}_{i,j,d}^l>0\\ 0& otherwise\end{array}$$

(8)

$y$ is an alias of $x^{l+1}$. An alias means that a variable can be reshaped into another form. To be specific, the function max (0, x) is not differentiable at x = 0.

Focusing on the convolution layer, the normalised Kernel was used to convolve the input images of both diseased and healthy maize leaves. The convolution operation is explained in Equation (9).

$$\left(x,y\right)={\displaystyle \sum }_{m=-\frac{M}{2}}^{\frac{M}{2}}{\displaystyle \sum }_{n=-\frac{N}{2}}^{\frac{N}{2}}h\left(m,n\right)f\left(x-m,y-m\right)$$

(9)

$h\left(m,n\right)$ is a filtering mask of size $M\times N$. Each element in this filter mask represents the weights used in the linear combination. It is at this stage where the ReLU activation function was used with the convoluted input images of both diseased and healthy maize leaves in order to bring non linearity. The filter that we used for the convolution of the input images in this study, is as shown by Equation (10).

$$Kerne{l}_{normalised}=\left|\begin{array}{ccccc}-1& -1& -1& -1& -1\\ -1& -1& -1& -1& -1\\ -1& -1& 24& -1& -1\\ -1& -1& -1& -1& -1\\ -1& -1& -1& -1& -1\end{array}\right|$$

(10)

As previously explained in this study, pooling was used to down sample the convoluted non-linear image in order to form an input signal inputted to a fully connected classifier.

2.4. Data Collection and Testing of the CNN

The images to be analysed for possible existence of the maize diseases were captured using a Google Pixel 3 smart phone camera in a maize field and saved in a file located on Google Drive. To analyse the collected data from the maize field, Neuroph’s CNN was used at a computer station. Figure 9 shows a fully connected classifier of the proposed CNN. Figure 10 shows how the Google Drive’s file was accessed to select one of the field images that was analysed for possible existence of the maize leaf diseases. It can be seen in the result’s output window that the common rust disease had a high probability of 0.8 followed by the northern corn leaf blight with a probability of 0.5. By continuously accessing a file downloaded from Google Drive containing the field data, the collected field images were all analysed for possible existence of the diseases.

A select few of the test images from the maize test data set that were analysed by the model of the proposed CNN are shown in Figure 11, Figure 12, Figure 13 and Figure 14.

3. Results

Figure 15 shows the iterations during training of the CNN that were executed to minimise the error from 0.275 to a reduced error of 0.001. As can be seen in Figure 15, an approximated total of 150 iterations was executed to reduce the error to 0.01.

Figure 16 shows an assignment of weights to each of the 50 hidden layers that were used to build the CNN for the recognition and classification of maize diseases out of healthy leaves.

The results for separate class tests are shown in

Table 1. Accuracy results of the CNN in the classification and recognition of maize leaf diseases and healthy leaves.

Type of Maize Disease	Total Percentage of Training Images	Total Percentage of Testing Images	CNN Classifier Accuracy
Northern Corn Leaf Blight.	70%	30%	99.9%
Gray Leaf Spot.	70%	30%	91%
Common Rust.	70%	30%	87%
Healthy	70%	30%	93.5%

. The CNN was very accurate in the recognition and classification of northern corn leaf blight (Exserohilum) with an outstanding accuracy of 99.9%. Some of the test images contained the characteristics of both gray leaf spot (Cercospora) and common rust (Puccinia sorghi), a reason which justified the CNN’s under performance when it came to the classification of the two diseases.

The CNN’s overall accuracy was determined by Equation (11).

$$Overallaccuracy=\frac{\sum classifieraccuracies}{Average}\times 100,$$

(11)

The overall classifier accuracy of the CNN was 92.85%. The test results of the proposed CNN model achieved accuracies somewhere in the range of 87% and 99.9%, for separate class tests, and 92.85% on average.

Table 2. Table of convoluted results and histogram results for select few input test images through the CNN.

Input Image	Convolution Result	Histogram Result

shows the results of convoluted input images as well as histogram results of the input images through the proposed CNN.

4. Discussion

These results have further strengthened our hypothesis that the CNN was capable of recognizing and classifying the maize leaf diseases at an overall accuracy of 92.85%. The CNN was trained using approximately 150 iterations to reach a minimal error of 0.01. This proved that the model was quick to learn from the training data. In separate class tests, the model achieved accuracies of 99.9% for northern corn leaf blight, 91% for gray leaf spot, 87% for common rust and 93.5% of healthy maize leaves. These results were obtained by using a batch of 100 images for each disease class, and 100 healthy images to train the CNN. Different images for each batch were used to conclude the feasibility of the research and the obtained accuracy results. This study focused on three maize disease types that were caused by biotic stresses. However, the proposed CNN would also be used to recognise the diseases that could be the cause of the abiotic stresses if it were to be trained with the data collected from the abiotic stressed plants. Future research is proposed to determine the CNN’s performance that would be trained with the biotic stressed data and then tested against the abiotic stressed data.

In the literature, there are a lot of machine learning studies that explained the detection of plant leaf diseases. However, none so far has explored the detection of maize diseases in large and open maize fields. By the use of deep CNN built in the Neuroph studio and the Google Pixel 3 smart phone we managed to conduct a study to detect the three types of maize diseases that occurred in the large open maize fields. The maize field was divided into section areas of 10 × 10 square meters and the data was collected using a Google Pixel 3 smart phone from each section area. The base used store the collected data was Google Drive. During the training and testing of the CNN the data were retrieved from Google Drive and used to train or test the CNN. The proposed method eliminates the use computation methods and cameras for data acquisition. Our method is more accurate than other methods that utilise data acquisition computational methods as the data was collected by the user from any angle of the leaves.

Another advantage is that our proposed CNN was built in a GUI platform. This will enable people who are not familiar with high level programming languages, such as Python, MATLAB and C to mention a few, to build a CNN from scratch.

5. Conclusions

In this study, we have reviewed related works in the literature for the plant leaf disease classification algorithms of machine learning. Neuroph Studio framework was used as an IDE to build a more facilitated deep CNN whereby the convolution and pooling feature extractions were embedded in the Neuroph library. The proposed CNN was trained and tested using datasets from Plant Village’s online website. The CNN’s overall accuracy of 92.85% proved its feasibility. The CNN was also tested using the data collected for maize and the results approximated to the results we obtained during the usage of the testing data. It is recommended that the researchers who wish to use the proposed CNN in this study use the resolution settings of 10 × 20 × 3 (height × width × RGB). Future research is proposed to determine the CNN’s performance when the training and testing are executed with gray scale images.