Monitoring Wheat Powdery Mildew Based on Hyperspectral, Thermal Infrared, and RGB Image Data Fusion

Abstract

Powdery mildew severely affects wheat growth and yield; therefore, its effective monitoring is essential for the prevention and control of the disease and global food security. In the present study, a spectroradiometer and thermal infrared cameras were used to obtain hyperspectral signature and thermal infrared images data, and thermal infrared temperature parameters (TP) and texture features (TF) were extracted from the thermal infrared images and RGB images of wheat with powdery mildew, during the wheat flowering and filling periods. Based on the ten vegetation indices from the hyperspectral data (VI), TF and TP were integrated, and partial least square regression, random forest regression (RFR), and support vector machine regression (SVR) algorithms were used to construct a prediction model for a wheat powdery mildew disease index. According to the results, the prediction accuracy of RFR was higher than in other models, under both single data source modeling and multi-source data modeling; among the three data sources, VI was the most suitable for powdery mildew monitoring, followed by TP, and finally TF. The RFR model had stable performance in multi-source data fusion modeling (VI&TP&TF), and had the optimal estimation performance with 0.872 and 0.862 of R² for calibration and validation, respectively. The application of multi-source data collaborative modeling could improve the accuracy of remote sensing monitoring of wheat powdery mildew, and facilitate the achievement of high-precision remote sensing monitoring of crop disease status.

1. Introduction

In recent years, multiple crop diseases and insect pests have emerged, with considerable impacts on yield and productivity following local outbreaks. According to the United Nations Food and Agriculture Organization (FAO), 20%–40% of crops globally are damaged by disease and insect pests annually [1]. Powdery mildew is the major wheat disease; it causes considerable yield reductions or even no harvest, posing a major threat to wheat production and global food security. Conventional methods of monitoring wheat disease are time-consuming and laborious, and are associated with mechanical damage to crops. Therefore, it is essential to identify and develop approaches of carrying out rapid and damage-free wheat disease monitoring.

Plant disease and insect pest infestations lead to biomass reductions, leaf structure destruction, and chlorophyll and water content reductions. Shifts in chlorophyll, water, and other biochemical components in plant tissues would inevitably yield diverse absorption and reflectance characteristics on the plant reflectance spectrum curve, which provides a theoretical basis and facilitates the real-time monitoring of wheat diseases using remote sensing technologies [2]. In recent years, with continuous advancements in remote sensing technologies, numerous scholars have applied technologies to monitor wheat diseases. Generally, different crops, varieties, and diseases exhibit diverse spectral characteristics, which leads to varying reflectance sensitivities at different bands following disease infestation [3]. Consequently, the identification of crop diseases and crop disease incidence estimation can be achieved based on changes in spectral responses and reflectance characteristics [4,5,6]. Researchers have previously developed disease monitoring indices following the extraction of disease-sensitive bands for monitoring the infestation of crops by bacterial diseases, such as powdery mildew index (PMI) [7], double green vegetation index [8], and red edge vegetation stress index (RVSI) [9].

The modeling algorithms applied in remote sensing influence the accuracy of remote sensing technologies. Today, the algorithms applied in disease and pest monitoring with remote sensing technologies are mainly empirical models and machine learning algorithms [10,11]. Among them, the empirical models are relatively simple; however, the data are easily influenced by external conditions and have poor universality. In recent years, machine learning methods have emerged, with rapid development. Crop disease monitoring models established based on machine learning methods consider training error and generalization ability, and address the challenges associated with slight changes in reflection coefficient during crop disease detection [12,13]. Gu et al. [14] used hyperspectral imaging technologies to monitor tobacco infected by tomato spotted wilt virus and reported that the combination of a successive projections algorithm (SPA) and boosted regression tree was the optimal modeling approach. In addition, Liu et al. [15] established a wheat wilt monitoring model using an improved backward propagation neural network. Wheat is a crop planted in dense rows; if only a single spectral data type is applied in disease monitoring activities, the model is often insensitive to changes in canopy spectrum reflectance, and the reflectance data can be saturated, leading to significant model prediction errors [16,17].

Texture information obtained using imaging spectroscopy tools can reflect disease spot sizes and infestation levels of bacterial diseases [18], in addition to integrating plant morphology and canopy structure information in the spectral data, which enhances the accuracy of remote sensing tools in crop disease monitoring activities [19]. Many researchers have exploited the complementary advantages of spectrum and texture information, which has significantly improved crop growth parameters and the inversion effect of disease severity [20,21]. For example, Guo et al. [22] used vegetation index (VI) and texture features (TF) data obtained using an unmanned aerial vehicle (UAV) platform to establish a wheat stripe rust monitoring model based on partial least squares regression (PLSR). TF can provide plant morphology data that could be applied in the monitoring of crop growth based on remote sensing technologies, which can address the saturation and low accuracy shortcomings associated with single spectral information source-based monitoring, and in turn enhance the robustness of a model and model inversion performance.

Infrared thermal imaging technologies have high sensitivity and early warning capacity. Wheat plants are infected by powdery mildew fungus, and the early symptoms are mostly manifested by changes in internal physiological reactions. Thermal infrared images can reveal temperature changes infected regions that cannot be discerned by visible light images [23]. Mahlein et al. [24] used an infrared thermal instrument (IRT) to measure wheat canopy temperature (CT) and found that the temperature of diseased spikelets was significantly higher than that of healthy spikelets. Therefore, infrared thermal imaging technologies can be used to monitor crop stress during growth and physiological conditions. Many researchers have begun to combine IRT with other remote sensing data sources in plant disease monitoring activities. For example, Zarco-Tejada et al. [25] confirmed that the combination of VI, sun induced fluorescence (SIF), and crop water stress index (CWSI) could be used to effectively monitor diseased trees, and the identification accuracy rate exceeds 80%. In addition, Poblete et al. [26] combined the spectrum, SIF and CWSI, which could effectively distinguish diseased and non-diseased olive trees, whereas Zhang et al. [27] used UAV multi-spectral VI in combination with CT information to estimate disease severity in disease-stressed chickpea, with significantly enhanced detection accuracy. The results of the above studies indicate that thermal infrared data can reliably reflect abnormal conditions in the CT of stressed crops, and can facilitate disease identification and disease classification when combined with other remote sensing data sources.

In the wake of rapid advancements in modern electronic information science, numerous sensors are available for application in the detection of crop morphology and canopy structure, such as reflectance spectrometers, chlorophyll fluorescence meters, and IRT and RGB cameras, which detect crop morphology and growth status based on different factors and principles [28]. However, crop information associated with a single information source is often potentially biased and has certain limitations. Data from different sensor types can be deployed synergistically to enhance target detection and recognition capabilities [29]. Compared to a single sensor data source, multi-sensor data sources can enhance the reliability and robustness of real-time detection [30]. At present, few studies have reported on the monitoring of wheat powdery mildew disease based on a synergy of spectral data and thermal infrared temperature data; in particular, there is a dearth of studies on disease monitoring using approaches that synergize VI, TF and temperature parameters (TP).

To further explore the synergistic effects of multimodal data obtained from different sensors in disease monitoring, in the present study, multimodal data on the incidence of wheat powdery mildew was obtained using hyperspectral surface spectrometer and thermal infrared camera, and compared with ground disease investigations. Multimodal data were obtained using modern modeling and inversion algorithms, such as PLSR, support vector machine regression (SVR), and random forest regression (RFR). The results of the present study could provide a technical basis for the rapid and large-scale monitoring of wheat powdery mildew, and facilitate the prevention and precise control of wheat powdery mildew, in addition to the improvement of pesticide efficiency and food safety.

2. Materials and Methods

2.1. Experimental Design

Experiment 1 (EXP.1): The experiment was conducted in the 2020–2021 wheat growing season at the Science and Education Demonstration Park (34°51′ N, 113°35′ E) of Henan Agricultural University, Zhengzhou, China. The tested varieties were varieties susceptible to wheat powdery mildew: Aikang 58 and Yumai 49–198. The first crop was corn, and the stalks were crushed and returned to the field. The soil was loam, the 0~30-cm soil contained 0.99–1.18 g kg⁻¹ of total nitrogen (N), 0.023–0.034 g kg⁻¹ of available phosphorus, 0.114–0.116 g kg⁻¹ of available potassium, and 11.4–15.3 g kg⁻¹ of organic matter. In the experiments, relatively high water and N fertilizer amounts were used to create favorable conditions for powdery mildew. The amount of N applied was 270 kg·hm⁻², and the irrigation amount during the wintering period-jointing stage was 900 m³·hm⁻². Powdery mildew fungus was inoculated at the jointing stage, and the wheat was infected from the flowering stage, and the canopy spectrum data were obtained at the flowering and filling stages. Other field management approaches were similar to those applied locally.

Experiment 2 (EXP.2): Carried out simultaneously with experiment 1, experiment 2 was a variety comparison experiment in the field, involving Yanzhan 4110, Nongmai 18, Zhoumai 27, Jinfeng 205, Zhengmai 1342, Xumai 318, Bainong 207, and Xinmai 26. The amount of N applied was 225 kg·hm⁻², and the irrigation amount during the wintering period-jointing stage was 675 m³·hm⁻². The experimental area was close to fences and pig farms, and the terrain was low-lying. Due to the terrain, air humidity, rainfall, and diseases in previous years, the wheat growth environment was suitable for the occurrence and spread of wheat powdery mildew, without field inoculation. Disease emergence was natural and more severe. Other field management approaches were similar to those applied in EXP.1.

2.2. Ground Data Collection

2.2.1. Investigation of Powdery Mildew

During the wheat flowering and filling periods, wheat powdery mildew incidence was investigated manually, and 77 and 37 samples were collected in EXP.1 and EXP.2, respectively. About 0.2 m² of the experimental area was investigated at each point, and 20 representative wheat plants were selected to test for powdery mildew infection. The survey was conducted in strict accordance with the technical specifications for crop disease monitoring (Chinese Standard: NY/T 2738.2-2015) [31]. The ratio of the leaf area covered by the diseased mycelium layer on the diseased leaf to the total leaf area was expressed based on a grading method, with eight levels representing 1%, 5%, 10%, 20%, 40%, 60%, 80%, and 100% coverage. The grid method was used to calculate the ratio of the diseased spot area to the leaf area. The operation involved using grids to cover the leaves, recording the total number of grids with disease spots, to facilitate the calculation of the ratio of the diseased spot area to leaf area. The closest value between grades was selected as the actual level. For example, at onset with a severity of less than 1%, the coverage was considered 1%. The average severity of diseased leaves was calculated as follows (1):

$$\mathit{D}=\frac{{{\displaystyle \sum }}^{ }\left(\mathit{D}\mathit{i}\times \mathit{L}\mathit{i}\right)}{\mathit{L}}\times \mathbf{100}$$

(1)

where, $\mathit{D}$ is the average disease severity in leaves, and the unit is percentage (%); $\mathit{D}\mathit{i}$ is each severity value; $\mathit{L}\mathit{i}$ is the number of diseased leaves corresponding to each severity value, and the unit is slice; and $\mathit{L}$ is the total number of leaves under investigation, and the unit is slice.

On the basis of the severity of disease in leaves, the disease index (DI) is calculated to represent the average level of disease occurrence (Equation (2)).

$$\mathit{D}\mathit{I}=\mathit{F}\times \mathit{D}\times \mathbf{100}$$

(2)

where, $\mathit{D}\mathit{I}$ is the disease index; $\mathit{F}$ is the diseased leaf rate; $\mathit{D}$ is the average severity of disease in leaves.

2.2.2. Canopy Spectrum Data Measurement

From 10:00 to 14:00 (Beijing local time) with little wind and clear weather, a FieldSpec handheld spectrometer (FieldSpec Handheld 2, Analytical Spectral Devices, Boulder, CO, USA) was used to obtain wheat canopy spectrum data, and the probe was 1.0 m from the top of the wheat crop. The field of view of the spectrometer was 25°, in the 325–1075 nm band, the spectral sampling interval was 1.4 nm, and the spectral resolution was 3.0 nm. A 0.4-m × 0.4-m BaSO₄ calibration plate was used to calculate black and baseline reflectance. Ten spectral reflectance values were recorded at each sampling point as samples, and the average value was considered the spectral reflectance of the sampling area.

2.2.3. Thermal Infrared Image and RGB Image Acquisition

An FLIR T650sc thermal infrared camera (FLIR Systems, Inc., Wilsonville, OR, USA) was used to obtain the wheat canopy temperature (CT) and RGB images. The device has dual thermal infrared and visible light sensors, and the image resolution is 640 × 480 pixels. Synchronous with the spectral reflectance measurement, the lens was 1.0 m from the top of the wheat crop, and the thermal infrared and RGB images were obtained vertically (Figure 1).

2.3. Data Processing Methods

2.3.1. Spectral Vegetation Index (VI)

Before extracting the VIs, the bands with high noise before 400 nm and after 1000 nm were removed, and then the Savitzky-Golay function was used to smoothen the spectra in MATLAB 7.0 (The MathWorks Inc., Natick, MA, USA). VIs associated with the disease were pre-selected by consulting relevant literatures (

Table 1. Spectral vegetation indices.

Vegetation Index	Formula	References
Modified simple ration (MSR)	$\mathrm{MSR}=\left({\mathrm{R}}_{800}/{\mathrm{R}}_{670}-1\right)/{\left({\mathrm{R}}_{800}/{\mathrm{R}}_{670}+1\right)}^{0.5}$	[33]
Photochemical reflectance index (PRI)	$\mathrm{PRI}=\left({\mathrm{R}}_{531}-{\mathrm{R}}_{570}\right)/\left({\mathrm{R}}_{531}+{\mathrm{R}}_{570}\right)$	[34]
Physiological reflectance index (PHRI)	$\mathrm{PhRI}=\left({\mathrm{R}}_{550}-{\mathrm{R}}_{531}\right)/\left({\mathrm{R}}_{550}+{\mathrm{R}}_{531}\right)$	[35]
Transformed chlorophyll absorption in reflectance index (TCARI)	$\mathrm{TCARI}=3\left({\mathrm{R}}_{700}-{\mathrm{R}}_{650}\right)-0.2\left({\mathrm{R}}_{700}/{\mathrm{R}}_{500}\right)/\left({\mathrm{R}}_{700}/{\mathrm{R}}_{670}\right)$	[36]
Red-edge vegetation stress index (RVSI)	$\mathrm{RVSI}=\left(\left({\mathrm{R}}_{712}+{\mathrm{R}}_{752}\right)/2\right)-{\mathrm{R}}_{732}$	[37]
Structural independent pigment index (SIPI)	$\mathrm{SIPI}=\left({\mathrm{R}}_{800}-{\mathrm{R}}_{445}\right)/\left({\mathrm{R}}_{800}-{\mathrm{R}}_{680}\right)$	[38]
Visible atmospherically resistant index (VARI)	$\mathrm{VARI}=\left({\mathrm{R}}_{550}-{\mathrm{R}}_{670}\right)/\left({\mathrm{R}}_{550}+{\mathrm{R}}_{670}-{\mathrm{R}}_{480}\right)$	[39]
Renormalized difference vegetation index (RDVI)	$\mathrm{RDVI}=\left({\mathrm{R}}_{800}-{\mathrm{R}}_{670}\right)/{\left({\mathrm{R}}_{800}+{\mathrm{R}}_{670}\right)}^{0.5}$	[40]
Anthocyanin reflectance index (ARI)	$\mathrm{ARI}={\left({\mathrm{R}}_{550}\right)}^{-1}/{\left({\mathrm{R}}_{700}\right)}^{-1}$	[41]
Damage sensitive spectral index 2 (DSSI2)	$\mathrm{DSSI}2=\left({\mathrm{R}}_{747}-{\mathrm{R}}_{901}-{\mathrm{R}}_{537}-{\mathrm{R}}_{572}\right)/\left({\mathrm{R}}_{747}-{\mathrm{R}}_{901}+{\mathrm{R}}_{537}-{\mathrm{R}}_{572}\right)$	[42]
Greenness index (GI)	$\mathrm{GI}={\mathrm{R}}_{554}/{\mathrm{R}}_{677}$	[43]
Plant senescence reflectance index (PSRI)	$\mathrm{PSRI}=\left({\mathrm{R}}_{680}-{\mathrm{R}}_{500}\right)/{\mathrm{R}}_{750}$	[44]
Normalized pigment chlorophyll Index (NPCI)	$\mathrm{NPCI}=\left({\mathrm{R}}_{680}-{\mathrm{R}}_{430}\right)/\left({\mathrm{R}}_{680}-{\mathrm{R}}_{430}\right)$	[43]
Nitrogen reflectance index (NRI)	$\mathrm{NRI}=\left({\mathrm{R}}_{570}-{\mathrm{R}}_{670}\right)/\left({\mathrm{R}}_{570}-{\mathrm{R}}_{670}\right)$	[45]
Healthy index (HI)	$\mathrm{HI}=\left({\mathrm{R}}_{534}-{\mathrm{R}}_{698}\right)/\left({\mathrm{R}}_{534}+{\mathrm{R}}_{698}\right)-0.5{\mathrm{R}}_{704}$	[7]
Powdery mildew index (PMI)	$\mathrm{PMI}=\left({\mathrm{R}}_{520}-{\mathrm{R}}_{584}\right)/\left({\mathrm{R}}_{520}+{\mathrm{R}}_{584}\right)-{\mathrm{R}}_{724}$	[7]
Triangular vegetation index (TVI)	$\mathrm{TVI}=0.5\left[120\left({\mathrm{R}}_{750}-{\mathrm{R}}_{550}\right)-200\left({\mathrm{R}}_{670}-{\mathrm{R}}_{550}\right)\right]$	[46]
Green normalized difference vegetation index (GNDVI)	$\mathrm{GNDVI}=\left({\mathrm{R}}_{800}-{\mathrm{R}}_{550}\right)/\left({\mathrm{R}}_{800}+{\mathrm{R}}_{550}\right)$	[47]
Nir shoulder region index (NSRI)	$\mathrm{NSRI}={\mathrm{R}}_{890}/{\mathrm{R}}_{780}$	[48]
Soil-adjusted vegetation index (SAVI)	$\mathrm{SAVI}=1.5\left({\mathrm{R}}_{800}-{\mathrm{R}}_{670}\right)/\left({\mathrm{R}}_{800}+{\mathrm{R}}_{670}+0.5\right)$	[49]

). Considering the potential existence of the collinearity problem among VIs, SPA was used to optimize VIs and reduce their multicollinearity. SPA is a forward variable selection method that selects characteristic variables by calculating the sizes of the projection vector of the remaining variables and the selected variables, which can ensure that the linear relationship between the selected variables is minimized, so as to eliminate redundant information between variables and reduce multicollinearity, to achieve the purpose of selecting sensitive variables [32].

2.3.2. RGB Image Texture Features (TF)

The gray level co-occurrence matrix (GLCM) method, proposed by Haralick in 1973 [50] is one of the most widely used texture extraction methods. The method has the advantages of rotation invariance, multi-scale characteristics, and low computational complexity, and is widely used in image processing, pattern recognition, and remote sensing monitoring [51,52]. In ENVI (Harris, Bloomfield, CO, USA), the gray level image of the RGB image was subjected to 3 × 3 sliding filtering using GLCM. Eight texture feature maps in the directions of 0°, 45°, 90° and 135° were extracted (Figure 2,

Table 2. Texture feature calculation formula.

Texture	Equation	Description
Mean, MEA	$\mathrm{MEA}={\displaystyle {\displaystyle \sum }_{i,j=1}^G}\left(iP\left(i,j\right)\right)$	Reflects the average of the greyscale
Variance, VAR	$\mathrm{VAR}={\displaystyle {\displaystyle \sum }_{i=1}^G}{\displaystyle {\displaystyle \sum }_{J=1}^G}{\left(i-u\right)}^{2}P\left(i,j\right)$	Reflects the magnitude of grey scale variation
Homogeneity, HOM	$\mathrm{HOM}={\displaystyle {\displaystyle \sum }_{i=1}^G}{\displaystyle {\displaystyle \sum }_{J=1}^G}\frac{P\left(i,j\right)}{1+{\left(i-j\right)}^2}$	Reflects the roughness of image texture
Contrast, CON	$\mathrm{CON}={\displaystyle {\displaystyle \sum }_{i=1}^G}{\displaystyle {\displaystyle \sum }_{J=1}^G}{\left(i-j\right)}^{2}P\left(i,j\right)$	Reflects the local variations in the gray-level co-occurrence matrix
Dissimilarity, DIS	$\mathrm{DIS}={\displaystyle {\displaystyle \sum }_{i=1}^G}{\displaystyle {\displaystyle \sum }_{J=1}^G}P\left(i,j\right)\left|i-j\right|$	Same as contrast, used to detect similarity
Entropy, ENT	$\mathrm{ENT}=-{\displaystyle {\displaystyle \sum }_{i=1}^G}{\displaystyle {\displaystyle \sum }_{J=1}^G}P\left(i,j\right)logP\left(i,j\right)$	Reflects the degree of the gray distribution and the thickness of the texture
Second moment, SEM	$\mathrm{SEC}={\displaystyle {\displaystyle \sum }_{i=1}^G}{\displaystyle {\displaystyle \sum }_{J=1}^G}{P}^2\left(i,j\right)$	Reflects the homogeneity of an image’s distribution of greyscale
Correlation, COR	$\mathrm{COR}={\displaystyle {\displaystyle \sum }_{i=1}^G}{\displaystyle {\displaystyle \sum }_{J=1}^G}\frac{\left(i-ME{A}_j\right)\left(j-ME{A}_j\right)P\left(i,j\right)}{\sqrt{VA{R}_i}\sqrt{VA{R}_j}}$	Reflects the length of the extension of a certain grey value in a certain direction

Note: $i$ and $j$ indicate the row and column number of the images, respectively; $P\left(i,j\right)$ is the relative frequency of two neighboring pixels.

), and the average of four directions was taken as the final texture feature map. To ensure that the extracted texture features are all based on canopy vegetation, a K-means clustering algorithm is used for bare soil rejection. The soil and vegetation mask is shown in Figure 3.

2.3.3. Thermal Infrared Temperature Parameters (TP)

The thermal infrared image was annotated and combined with K-means clustering segmentation results (Figure 3) using FLIR Tools (FLIR Systems Inc., Wilsonville, OR, US), and the temperature parameters were extracted. Considering that CT changes with the daily change in atmospheric temperature, the canopy temperature difference (CTD), canopy temperature ratio (CTR), and normalized relative canopy temperature (NRCT) were extracted to eliminate the influence of atmospheric temperature on CT. The temperature parameter formula was as follows:

$$\mathit{C}\mathit{T}\mathit{D}=\mathit{C}{\mathit{T}}_{\mathit{i}}-\mathit{A}\mathit{T}$$

(3)

$$\mathit{C}\mathit{T}\mathit{R}=\frac{\mathit{C}{\mathit{T}}_{\mathit{i}}}{\mathit{A}\mathit{T}}$$

(4)

$$\mathit{N}\mathit{R}\mathit{C}\mathit{T}=\frac{\mathit{C}{\mathit{T}}_{\mathit{i}}-\mathit{C}{\mathit{T}}_{\mathit{m}\mathit{i}\mathit{n}}}{\mathit{C}{\mathit{T}}_{\mathit{m}\mathit{a}\mathit{x}}-\mathit{C}{\mathit{T}}_{\mathit{m}\mathit{i}\mathit{n}}}$$

(5)

where, AT is the atmospheric temperature, $\mathit{C}{\mathit{T}}_{\mathit{i}}$ is the CT of the $i$-th pixel in the image, $\mathit{C}{\mathit{T}}_{\mathit{m}\mathit{a}\mathit{x}}$ is the highest temperature measured in the entire experimental field, and $\mathit{C}{\mathit{T}}_{\mathit{m}\mathit{i}\mathit{n}}$ is the lowest temperature measured in the entire experimental field.

2.3.4. Estimation Model

(1)

PLSR

PLSR is a classic modeling method, which includes the characteristics of principal component analysis (PCA), canonical correlation analysis, and multiple linear regression analysis, and is often used for quantitative analysis in remote sensing [53]. PLSR transforms the original variables with high data redundancy into a few variables by selecting the optimal latent variables, to describe the linear model of the relationship between the predicted value and the true value.

(2)

SVR

The basic idea of SVR is to use training samples to establish a regression hyperplane, and to approximate the samples to the hyperplane to minimize the total deviation from the sample point to the plane [54]. The commonly used kernel functions of the SVR algorithm include the linear kernel function, radial basis function (RBF) kernel function, polynomial kernel function, and Sigmoid kernel function. Among them, the RBF kernel function can handle the complex nonlinear problem between the independent variable and the dependent variable.

(3)

RFR

RFR is a machine learning algorithm based on a classification regression tree [55]. RFR uses the bootstrap resampling method to extract multiple samples from the original sample, models each bootstrap sample into a decision tree, combines them into multiple decision trees for prediction, and then applies the majority voting method to determine the final classification result of the joint prediction model. The advantage of the method is that the training speed is relatively fast and it does not require cross-validation. In addition, the randomness of sampling and feature selection make the random forest averts overfitting [56]. It is widely used in classification and prediction in remote sensing-based monitoring activities.

2.3.5. Model Validation

With VI, TP and TF as independent variables, and DI as the dependent variable, a monitoring model for wheat powdery mildew disease index was established based on the three algorithms above. The workflow from feature extraction to model building and evaluation was demonstrated in Figure 3. To make the model evaluation results more objective, EXP.1 test data were used as the modeling set, and EXP.2 test data were used as the verification set. The accuracy of the wheat powdery mildew disease index monitoring model was evaluated based on three indicators: coefficient of determination (R²), root mean square error (RMSE), and relative error (RE). The closer R² is to 1, the lower the RMSE, and the lower the RE, the higher the accuracy of the monitoring model. The formula was as follows:

$${\mathit{R}}^{\mathbf{2}}=\frac{{{\displaystyle \sum }}_{\mathit{i}=\mathbf{1}}^{\mathit{n}}{\left({\mathit{x}}_{\mathit{i}}-\overline{\mathit{x}}\right)}^{\mathbf{2}}\times {\left({\mathit{y}}_{\mathit{i}}-\overline{\mathit{y}}\right)}^{\mathbf{2}}}{{{\displaystyle \sum }}_{\mathit{i}=\mathbf{1}}^{\mathit{n}}{\left({\mathit{x}}_{\mathit{i}}-\overline{\mathit{x}}\right)}^{\mathbf{2}}\times {{\displaystyle \sum }}_{\mathit{i}=\mathbf{1}}^{\mathit{n}}{\left({\mathit{y}}_{\mathit{i}}-\overline{\mathit{y}}\right)}^{\mathbf{2}}}$$

(6)

$$\mathit{R}\mathit{M}\mathit{S}\mathit{E}=\sqrt{\frac{{{\displaystyle \sum }}_{\mathit{i}=\mathbf{1}}^{\mathit{n}}{\left({\mathit{y}}_{\mathit{i}}-{\mathit{x}}_{\mathit{i}}\right)}^{\mathbf{2}}}{\mathit{n}}}$$

(7)

$$\mathit{R}\mathit{E} (\%)=\sqrt{\frac{\mathbf{1}}{\mathit{n}}\times {\displaystyle \sum }_{\mathit{i}=\mathbf{1}}^{\mathit{n}}{\left(\frac{{\mathit{y}}_{\mathit{i}}-{\mathit{x}}_{\mathit{i}}}{{\mathit{x}}_{\mathit{i}}}\right)}^{\mathbf{2}}}\times \mathbf{100}$$

(8)

where, ${\mathit{x}}_{\mathit{i}}$, $\overline{\mathit{x}}$, ${\mathit{y}}_{\mathit{i}}$, and $\overline{\mathit{y}}$ are the measured $\mathit{D}\mathit{I}$, average $\mathit{D}\mathit{I}$, predicted $\mathit{D}\mathit{I}$, and average $\mathit{D}\mathit{I}$, respectively; $\mathit{n}$ is the number of samples.

3. Results

3.1. Changes in Wheat Canopy Spectra under Different Powdery Mildew Severity Levels

With an increase in DI, the spectral reflectance of the visible light band from 400 to 780 nm increased gradually, and the discrimination of DI was better (Figure 4a). The spectral reflectance of the near-infrared bands region across 780 nm–1000 nm was less distinguishable when the disease was mild; when the disease was more than moderate, the spectral reflectance increased gradually; and when the disease was severe (such as DI = 80), the spectral reflectance rose sharply due to severe damage to the canopy structure, even higher than the spectral reflectance of healthy wheat. From the perspective of the correlation between disease severity and reflectance (Figure 4b), there was a positive correlation in the visible light band from 400 to 730 nm, and a negative correlation in the near-infrared region from 730 to 1000 nm, especially at 600–700 nm (r = 0.373–0.431, probability value, p < 0.01) and 780–960 nm (r=−0.355–−0.294, p < 0.01), which can be considered as disease-sensitive bands for the real-time monitoring of disease progression.

3.2. Selection of Vegetation Index

Based on the reported VIs related to plant disease, the correlations between 20 VIs and DI were analyzed. The VI with the highest correlation was NSRI (r = 0.743) (Figure 5a), followed by GI and NPCI. Because VI is a combination of bands, and there is a certain degree of information duplication between bands, there is considerable multicollinearity among the spectral parameters. Therefore, the SPA algorithm is used to optimize the VI. The minimum number of sensitive variables extracted was two, and the maximum number of sensitive variables was twenty. RMSE decreased with an increase in the number of variables. RMSE was the minimum (RMSE = 15.575) when the number of variables was 10; however, with an increase in the number of variables, the RMSE increased gradually (Figure 5b). After SPA screening, there were 10 VIs, namely NSRI, NPCI, PSRI, PRI, ARI, SIPI, PMI, MSR, RVSI, and GNDVI. According to the results, when a single VI was used to estimate the DI, the linear R² was low (R² < 0.56), and the error in monitoring powdery mildew disease was relatively large (Figure 6).

3.3. Selection of Texture Feature Parameters

Analysis of the correlation between TF parameters of canopy RGB images and DI showed that the correlation coefficients were all positive. Excluding the correlation coefficient between the correlation and DI, which was not significant, all the others were significant, among which entropy was the highest (r = 0.486, p < 0.01) (Figure 7a). The eight extracted TFs were all calculated from grayscale images, and considering the potential multicollinearity among TFs, the SPA algorithm was used to optimize the variables. The RMSE was the lowest (RMSE = 18.043) when the number of TF variables was five. Five TF parameters, mean, variance, homogeneity, entropy, and second moment, were selected as the input variables in the estimation model (Figure 7b).

3.4. Selection of Thermal Infrared Temperature Parameters

The TP parameters from thermal infrared images were extracted to analyze their correlation with DI, and the results showed that the correlation coefficient between CT and DI was 0.382, and was significant (p < 0.01) (Figure 8). Considering CT changes with daily atmospheric temperature changes, the CTD, CTR and NRCT were extracted to eliminate the influence of atmospheric temperature on CT. After eliminating the influence of atmospheric temperature, the significant level of correlation of TP was further improved, with a very significant level (p < 0.01) observed. Combining the principle of strong correlation and eliminating duplicate information, two TPs, CTD, and NRCT, were selected as input variables for the subsequent steps of modeling and analysis.

3.5. Comparison of Different Model Algorithms Based on Single Data Sources

With a single data source as an independent variable, three methods, including, PLSR, SVR and RFR, were used to invert the DI of wheat powdery mildew (

Table 3. Estimation performance of single data source model based on different algorithms.

Independent Variable Type	Number of Variables	Modeling Method	Calibration Set			Validation Set
			R2	RMSE	RE	R2	RMSE	RE
VI	10	PLSR	0.666	15.014	19.24%	0.650	16.425	19.86%
		SVR	0.670	14.757	18.69%	0.666	15.578	19.28%
		RFR	0.690	14.488	18.42%	0.680	14.298	18.16%
TF	5	PLSR	0.509	17.852	32.03%	0.486	18.367	32.05%
		SVR	0.517	18.616	30.70%	0.514	17.489	30.23%
		RFR	0.537	17.621	29.18%	0.537	17.799	27.95%
TP	2	PLSR	0.553	17.420	29.27%	0.546	17.673	29.66%
		SVR	0.567	17.347	28.96%	0.571	17.094	27.13%
		RFR	0.575	16.470	27.83%	0.577	16.791	27.79%

, Figure 9). Comprehensive comparison revealed that the RFR model performed optimally, followed by the SVR and PLSR models. Based on the performance results of the three data sources, regardless of which method was used to estimate wheat DI, the performance of the VI was the best, followed by TP and TF. Based on combinations modeling methods and independent variable data type, the RFR method with VI as the independent variable was the best combination, with R², RMSE, and RE values of 0.690, 14.488, and 18.42%, respectively, in the calibration set, and R², RMSE, and RE values of 0.680, 14.298, and 18.16%, respectively, in the validation set. The SVR method with VI as an independent variable was the second best combination, with R², RMSE and RE values of 0.670, 14.757, and 18.69%, respectively, in the calibration set, and R², RMSE, and RE values of 0.666, 15.578, and 18.16%, respectively, in the validation set.

3.6. Comparison of Different Model Algorithms Based on Multi-Source Data Combination

To fully exploit the information obtained from different data sources, TP, TF, and VI were combined to carry out a comparative analysis of three modeling methods (

Table 4. Estimation performance of multi-source collaboration models based on different algorithms.

Independent Variable Type	Number of Variables	Modeling Method	Calibration Set			Validation Set
			R2	RMSE	RE	R2	RMSE	RE
TP&TF	7	PLSR	0.624	16.265	21.67%	0.623	16.151	22.67%
		SVR	0.641	15.413	20.39%	0.637	15.900	20.91%
		RFR	0.646	15.328	20.29%	0.641	15.687	20.77%
VI&TF	15	PLSR	0.723	13.417	18.37%	0.728	13.236	18.15%
		SVR	0.744	13.211	17.33%	0.738	13.107	17.27%
		RFR	0.762	12.102	17.02%	0.761	12.203	17.71%
VI&TP	12	PLSR	0.763	13.385	16.08%	0.746	13.375	16.20%
		SVR	0.784	12.554	15.43%	0.776	12.221	15.59%
		RFR	0.791	12.531	15.51%	0.794	11.804	15.07%
VI&TP&TF	17	PLSR	0.840	11.606	14.07%	0.831	10.947	14.09%
		SVR	0.857	11.277	13.66%	0.854	10.200	13.07%
		RFR	0.872	10.108	12.54%	0.862	10.049	12.31%

, Figure 9). After fusing TF on the basis of VI data, the average R², RMSE, and RE values in the calibration set were 0.743, 12.91, and 17.57%, respectively, with the R² value representing an average increase of 10% when compared with the R² of single VI data source. The mean R², RMSE, and RE values in the validation set were 0.742, 12.849, and 17.71%, respectively, and the R² value represented a 11.6% increase when compared to the R² of the single VI data source. The addition of TP based on VI data enhanced the accuracy of the combined model. The average R² values in the validation and calibration sets were 0.779 and 0.772, respectively, the RMSE values were 12.823 and 12.467, respectively, and the RE values were 15.56% and 15.62%, respectively. The R² values were 15.4% and 16% higher, respectively, than the R² values based on the single VI data source. In addition, in the combined TP and TF modelling, the estimation performance was superior to those of the single data sources, both TP or TF; however, the model was inferior to VI based on both the calibration and validation sets. The results indicate that when using single data source-based VI as a benchmark, TF has a minor positive effect on model accuracy when performing multi-data collaboration modeling, whereas TP has a relatively high positive effect on model improvement.

Three modeling methods were used to synergize three data sources, VI, TF, and TP. The R² values of the calibration and validation sets were further improved compared with the R² value following combination of the two data sources. The average R² values of the calibration and validation sets in the model with the three data sources combined were 0.856 and 0.849, respectively, the RMSE values were 10.997 and 10.399, respectively, and the RE values were 13.42% and 13.16%, respectively. The R² values were 26.8% and 27.6% higher, respectively, than the R² values of the single VI data source model. Comparison of the modeling algorithm results showed that the RFR model had the highest R², the lowest RMSE and RE, and the greatest DI predictive capacity, followed by the SVR model and PLSR models (Figure 10). The R² values of the RFR fusion model of the three data sources were 0.872 and 0.862 in the calibration and validation sets, respectively, the RMSE values were 10.108 and 10.049, respectively, and the RE values were 12.54% and 12.31%, respectively. The above results indicate that collaborative modeling with multiple data sources is superior to single data source-based modeling, with the combined model exhibiting better fit, accuracy, and predictive ability (Figure 9).

4. Discussion

4.1. Combining VI and TF to Monitor Crop Diseases

Previous literature has confirmed the importance of reflectance spectrum data in crop disease monitoring and its application prospects. The visible light and near-infrared regions are the sensitive bands for spectral identification of different crop diseases and insect pests; furthermore, the spectral sensitivity bands of different crops and different diseases vary. The sensitive bands of wheat powdery mildew are located at 490–780 nm [57], and wheat powdery mildew monitoring is mainly based on the sensitive band [58,59], and different forms of disease VI can be established according to the reflection characteristics of the disease [7,8,9]. Disease emergence involves gradual development that alters internal tissue physiology and biochemistry, and, in turn, the external morphological structure, and then manifests externally as disease that can be detected by remote sensing. Due to the combined effects of internal and external factors, such as mesophyll cells, water, chlorophyll, and leaf yellowing and dryness, the ability to extract disease information from a single band is often limited. The VIs with good performance in the present study were NSRI, NPCI, and CVI. Among them, NSRI performed optimally, with a linear R² of only 0.552, which hardly meets the information requirements for accurate crop protection.

The onset of powdery mildew disease has a significant bottom-up characteristic. In the early and middle stages of the disease, the disease is mainly concentrated in the middle and lower levels of the plant. However, the canopy reflectance spectra data mainly originate from the upper level, which leads to lack of consistency between the collected canopy spectra data and disease characteristics, and increases the challenge of monitoring powdery mildew using canopy spectrum data only. Therefore, the use of multivariate analysis methods to identify and monitor disease has become a hotspot in quantitative remote sensing research.

In the present study, multiple VIs were used as independent variables, and three algorithms PLSR, SVR, and RFR were used to predict DI. The results showed that the RFR model had the highest monitoring accuracy; however, R² was still lower than 0.7. From the perspective of precise crop protection and disease prevention and control, the spectral data could not be used to monitor wheat powdery mildew reliably. Some scholars have attempted to incorporate fluorescence data in modeling when using hyperspectral data for disease monitoring, and achieved good monitoring results [60]. When a pathogen infects plants, the canopy structure changes following physiological and biochemical responses, and the TF can reflect the change in canopy structure caused by pathogen infestation to a certain degree [61,62]. Researchers have used hyperspectral VI in combination with TF to monitor wheat stripe rust, and reported that the estimation results of the two combined data sources were significantly better than that of the single data source [22]. In the present study, the VI and TF were modeled together, and model accuracy improved when compared with when the VI from a single data source was used. However, the highest accuracy of the combined model in the validation set was only 0.761, the optimization effect was limited, and it did not satisfy the requirements of accurate monitoring, which could be due to the gradual senescence of wheat leaves after the flowering period, and the increased background complexity of withered plants. Furthermore, multiple factors in some plots, such as disease, drought, senescence, and atmospheric temperature, which make it impossible to accurately distinguish whether the withered leaves and structural changes are attributed to disease stress, could have adversely influenced the modelling findings.

4.2. Combining VI and TP to Monitor Crop Disease

Thermal infrared imaging technologies have great application potential in remote sensing monitoring activities [63,64,65]. After crops are infected by fungi and pathogens, cell membrane permeability increases, water is lost, and plants exhibit dehydration and wilting. In addition, stomata are closed and heat loss on the leaf surface changes, which leads to leaf surface temperature response. At the onset of crop disease, changes in heat radiation energy caused by plant water loss, stomata closure, and increased respiration can be intuitively reflected in infrared heat maps; however, most studies have focused on disease classification and disease identification. Calderón et al. [66] demonstrated the capacity of using canopy temperature information and hyperspectral VI to identify olive trees with yellow dwarf disease; however, they did not estimate disease severity. In the previous monitoring research on wheat powdery mildew, no studies have reported the combination of VI and TP. The correlation analyses carried out in the present study showed that the thermal infrared temperature was sensitive to disease, and it was more effective to convert CT into CTD, CTR, and NRCT. Compared to canopy TF, temperature information had a greater role in disease monitoring. The RFR model performed best (R² = 0.577) and was slightly more accurate than the TF-RFR model; however, it was significantly less accurate than the VI-RFR model. VI as a single data source was more suitable for monitoring wheat powdery mildew, followed by canopy TP and canopy TF. To further improve the information limitations of single data sources, VI and canopy TP were modeled together (VI&TP). Model accuracies of different algorithms were higher than that of VI&TF on the whole, indicating that canopy temperature information has great application potential in disease monitoring.

Previous studies have also demonstrated that spectral information, texture information, and thermal infrared information have the ability to monitor crop diseases [22,24,26]; however, no study has reported their joint application in wheat powdery mildew monitoring. To that end, the present study conducted fusion modeling based on VI, TF, and TP (VI&TF&TP). According to the results, the combination of the three data sources had obvious advantages over single data sources or two combined data sources. Among them, the R² values of the three data source models based on the RFR algorithm was 0.862, which provides technical support and a reference method for the prevention and precise control of wheat powdery mildew.

4.3. Machine Learning Algorithms in Disease Monitoring

In the wake of rapid advancements in computer modeling science, machine learning technology has been applied extensively in crop disease monitoring, with the achievement of remarkable results [67,68]. Jiang et al. [69] demonstrated the high estimation capacity of the RFR model in the monitoring of mangrove disease and insect pests. In addition, Zhang et al. [70] demonstrated the good classification performance of the RFR model in the identification of wheat grains infected with Fusarium. In the present study, three modeling methods (PLSR, SVR, RFR) were used to monitor wheat powdery mildew DI. The RFR model performed best, regardless of whether it was based on single data source modeling or multi-source data modeling. This is mainly because the RFR algorithm has good anti-noise ability and does not easily exhibit over-fitting [71]. In the present study, SVR was used to integrate information from three data sources, and the average accuracy that the model achieved was 0.77. Considering the operation efficiency and prediction accuracy of the model, the method is effective for monitoring the disease. In contrast, the performance of the PLSR model was slightly worse, which might be because PLSR was better at addressing multicollinearity between parameters [72], and the parameters used in the present study were optimized by the SPA algorithm, which eliminated the influence of multicollinearity, resulting in an inability to maximize the performance of the PLSR model.

Although the overall performance of the RFR model in the present study was the best, the estimated value was lower than the actual value under more severe disease conditions, which was also observed in the other model algorithms. Generally, the greater the population density of wheat powdery mildew, the worse the air permeability, and the more severe the disease, which decreases the sensitivity of spectral and thermal imaging data to disease severity. Under the condition of multi-source data fusion, the saturation of the model was alleviated, which also demonstrates the effectiveness of multi-data source fusion.

When applying different model algorithms to monitor wheat powdery mildew in collaboration with VI, TF, and TP, the present study did not consider the contribution rates of different data source parameters to the model. How to use different algorithms to determine the weights of different data source parameters to further improve the accuracy of the model remains to be further studied. The occurrence and characteristics of powdery mildew are certainly associated with crop variety, growth period, and other diverse factors. Using targeted information extraction algorithms to clarify the effects and contribution levels of each influencing factor could facilitate the integration multiple effect factors to accurately monitor disease occurrence, and provide a theoretical basis for crop protection and precise operations.

5. Conclusions

Based on multi-source data fusion and machine learning, the present study explores the application potential of canopy spectral vegetation index, thermal infrared information, and texture feature information obtained using different sensors in wheat powdery mildew monitoring. In the case of wheat disease index prediction based on single data source, spectral information is better than thermal infrared information and texture features. Regardless of the modeling method, the results obtained following the fusion of data from multiple sources are more reliable than the data obtained from a single data source. When using the combination of vegetation index, thermal infrared information and texture features, higher prediction precision can be achieved. Regardless of whether single data source or multi-source data is used, the monitoring accuracy of the RFR model is higher than that of other algorithm models. Therefore, the combination of multi-source data fusion and the RFR model have broad application prospects in wheat powdery mildew monitoring, which could not only promote disease prevention and control but also reduce pesticide use and enhance the efficiency of disease prevention and control activities. However, the models identified should be tested under different crop types, growth stages, and environmental conditions, to further evaluate the robustness of the models.