Developing Novel Rice Yield Index Using UAV Remote Sensing Imagery Fusion Technology

Abstract

Efficient and quick yield prediction is of great significance for ensuring world food security and crop breeding research. The rapid development of unmanned aerial vehicle (UAV) technology makes it more timely and accurate to monitor crops by remote sensing. The objective of this study was to explore the method of developing a novel yield index (YI) with wide adaptability for yield prediction by fusing vegetation indices (VIs), color indices (CIs), and texture indices (TIs) from UAV-based imagery. Six field experiments with 24 varieties of rice and 21 fertilization methods were carried out in three experimental stations in 2019 and 2020. The multispectral and RGB images of the rice canopy collected by the UAV platform were used to rebuild six new VIs and TIs. The performance of VI-based YI (MAPE = 13.98%) developed by quadratic nonlinear regression at the maturity stage was better than other stages, and outperformed that of CI-based (MAPE = 22.21%) and TI-based (MAPE = 18.60%). Then six VIs, six CIs, and six TIs were fused to build YI by multiple linear regression and random forest models. Compared with heading stage (R² = 0.78, MAPE = 9.72%) and all stage (R² = 0.59, MAPE = 22.21%), the best performance of YI was developed by random forest with fusing VIs + CIs + TIs at maturity stage (R² = 0.84, MAPE = 7.86%). Our findings suggest that the novel YI proposed in this study has great potential in crop yield monitoring.

1. Introduction

Rice, one of the most important food crops, is the staple food for about half of the global population. Rice yield prediction is of great significance for ensuring world food security and breeding new varieties with high yield and good stress resistance. Through a brief review of previous studies, it is known that crop yield estimation methods mainly include yield sampling survey, agronomic meteorological yield prediction, yield simulation based on crop biomass, and yield estimation based on remote sensing. A field sampling survey is the most commonly used method. Survey samples of 1 m × 1 m size are taken from each field at the maturity stage, and the average yield of all samples represents the measured value of rice yield after rice threshing and moisture measurement. The yield sampling survey method is time-consuming and labor-consuming, and high-cost for researchers to conduct large-scale field measurements. Agro-meteorological yield prediction method needs long-term agro-meteorological monitoring data, and the lack of multi-period crop growth parameters leads to unstable accuracy and high cost. The yield prediction mechanism of yield simulation based on crop biomass is clear. But the model needs many input parameters, and it is difficult to determine the best parameters due to varieties and fertilization treatments. Therefore, it is urgent to develop a fast and reliable technology for yield prediction. Due to the multi-temporal image acquisition ability and low economic cost, the development and application of remote sensing in crop monitoring is the focus of many researchers at present. Although ground-based platform sensors [1,2] are easy to operate and obtain accurate crop information, a small scanning area each time leads to low efficiency. Due to the limitations of weather conditions and resolution, satellite-based platform sensors [3] are difficult to meet the requirements of modern precision agriculture management. The convenience of unmanned aerial vehicle (UAV) and various supporting sensors enable UAV-based sensors [4,5,6] to overcome these limitations.

Generally speaking, there are two main methods for crop growth monitoring. A common method is to use image classification based on high-resolution RGB images to obtain crop parameters, such as vegetation coverage [7], biomass [8], and plant height [9]. For example, in a potato growth monitoring study, RGB and hyperspectral imaging data of potato crop canopy were obtained by UAV to estimate crop biomass and predict crop yield [10]. In oilseed rape research, the UAV equipped with RGB and multispectral sensors was used to acquire a series of field images at the flowering stage, and vegetation indices (VIs) extracted from images were used to predict the number of oilseed rape flowers [11]. In soybean research, based on the data obtained by RGB sensors, the ability to estimate soybean yield is evaluated in the deep learning framework [12]. However, the accuracy of the model was found to be dependent on the soybean varieties. Another possibility is to use vegetation index (VI) or color index (CI) extracted from RGB images and multispectral images to monitor crop status at the field scale, such as crop yield [13], aboveground biomass [14,15], and crop lodging identification [16]. For example, Zhou et al. [17] used single-stage VIs and multi-temporal VIs derived from the multispectral images to predict rice grain yield with R² = 0.75 and RMSE = 947.69 kg·ha⁻¹. Naito et al. [18] found that a high correlation exists between several VIs and multiple yield traits (panicle number, grain weight, and shoot biomass), and simple ratio indices exhibited better performance in regards to the estimation of grain weight (R² = 0.80). Zhang et al. [19] proposed that the VI-based model accurately predicted the differences between treatments and grain yield (R² > 0.7). However, it is noted that the experimental materials of previous rice yield estimation studies were rice with a few varieties and several different nitrogen gradient treatments. Moreover, the accuracy of yield prediction models is often easily affected by crop varieties and environmental conditions, resulting in low generalizability. Therefore, it is imperative to construct a widely adaptable yield index (YI) for rice yield prediction. At present, there is no relevant report on the yield index.

Extracting VIs and color indices (CIs) from multispectral and RGB images is a reliable method for agricultural crop monitoring [20]. The sensitivity of VIs and CIs changes in different growth stages of crops, so it is necessary to combine multiple indices to complete the monitoring task. Rice with different yields usually shows differences in color and plant density, which leads to changes in texture characteristics. Some scholars have found that combining spectrum and texture to complete biomass estimation could improve the accuracy [21]. The method of fusing VIs, CIs, and texture from UAV images needs further research.

Furthermore, a few people have noticed that the commonly used VIs and CIs are calculated from specific bands based on the experience of previous studies. For example, the normalized difference vegetation index (NDVI) is calculated by normalization in the near-infrared and red band. It performs well in predicting chlorophyll and nitrogen content, but not in yield. Different fertilizer treatments can affect crop canopy structure, which ultimately leads to changes in yield and texture features. Some studies have used texture features to predict forest and wheat biomass, and these studies show that the use of texture features can improve the prediction accuracy of the model. In addition, the commonly used textures are calculated from a single band. Few researchers have studied how to construct a new index with textures in different bands, just like NDVI. Therefore, it is necessary to propose a method to select the best band combination of VIs and construct a new texture feature to improve the prediction accuracy.

This research was aimed at exploring the method of developing a novel YI for predicting rice yield by fusing UAV-based VIs, CIs, and TIs. The main objectives were:(1) to calculate new VIs, CIs, and TIs suitable for rice yield prediction, and to analyze the quantitative relationship between yield and VI, CI, and TI; (2) to develop the YI, and compare the performance using different regression methods (i.e., MLR, QNR, RF); (3) to evaluate the prediction performance of YI by the data of 2019 and 2020.

2. Materials and Methods

2.1. Experimental Design

The data used in this study were obtained by UAV-based sensors from six experiments with 24 varieties of rice and 21 fertilization treatments in 232 plots in two years (Figure 1). The information on rice varieties and fertilizer treatments is summarized in

Table 1. Information on the field experiments. Note: Exp.1 represents experiment 1, and so on for others; N represents nitrogen fertilizer; BF, PF, and TF represent base fertilizer, panicle fertilizer, and tillering fertilizer respectively.

Experiments	Rice Cultivars	Fertilization Treatments (N Treatments)	Sampling Date (Stages)
2019 Exp.1 (Fuyang) 24 plots	4 varieties: Ezao 18; Zhuliangyou 189; Zhuliangyou 819; Liangyou 287	(1) Blank control group; (2) N 150 kg·ha−1, BF:PF:TF = 5:3:2.	26 August (heading), 24 October (maturity)
2019 Exp.2 (Fuyang) 96 plots	2 varieties: Zhongzao 39; Zhongjiazao 17	(1) Blank control group; (2) ordinary N 180 kg·ha−1, BF:PF:TF = 5:3:2; (3) special N 180 kg·ha−1, BF:PF:TF = 8:1:1; (4) special N 150 kg·ha−1, BF:PF:TF = 8:1:1; (5) slow-release N 150 kg·ha−1, BF:PF:TF = 8:1:1; (6) slow-release N 150 kg·ha−1, single basal fertilization; (7)N 150 kg·ha−1 in 10 different brands of fertilizers, single basal fertilization	26 August (heading), 24 October (maturity)
2020 Exp.3 (Pingyao) 11 plots	1 variety: Yongyou 1540	(1) side deep slow-release N 160 kg·ha−1, BF:PF = 5:5; (2) side deep slow-release N 160 kg·ha−1, BF:PF = 8:3; (3) side deep slow-release N 160 kg·ha−1, BF:PF = 3:8; (4) N 160 kg·ha−1, single basal fertilization.	24 August (heading), 10 October (maturity)
2020 Exp.4 (Yuhang) 19 plots	19 varieties: Yongyou 1540; Yongyou 7850; Yongyou 7860; Yongyou 7872; Yongyou 6711; Chunyou 801; Chengyou 13; Zhejiang Jingyou 1578; Xiuyou 4913; Xiuyou 71,207; Jiaheyou 5; Xiushui 134; Jia 67; Zhongjia 8; Zhejiang Jing 99; Zhejiang Jing 100; Zhejiang Hujing 25; Chunjiang 157; Wankenjing 11,036	(1) N 207 kg·ha−1, BF:PF:TF = 3:3:4.	24 August (heading), 31 October (maturity)
2022 Exp.5 (Fuyang) 28 plots	4 varieties: Yongyou 1540; Yongyou 17; Zhongzheyou 8; Xiushui 134	(1) Blank control group; (2) N 160 kg·ha−1, BF:PF:TF = 4:7:5; (3) N 210 kg·ha−1, BF:PF:TF = 6:10:5; (4) N 220 kg·ha−1, BF:PF: TF = 4:7:11; (5) N 260 kg·ha−1, BF:PF:TF = 8:13:5; (6) N 270 kg·ha−1, BF:PF:TF = 6:10:11; (7) N 320 kg·ha−1, BF:PF:TF = 8:13:11.	17 August (heading), 30 October (maturity)
2022 Exp.6 (Fuyang) 54 plots	1 variety: Yongyou 12	(1) Blank control group;(2) N 0 kg·ha−1, with plastic film; (3) N 195 kg·ha−1, without plastic film; (4) N 165 kg·ha−1, with biodegradable membrane; (5) N 195 kg·ha−1, with biodegradable membrane.	17 August (heading), 30 October (maturity)

. Experiments 1 and 2 were conducted at Fuyang Experimental Station (30′04″61 N, 119′55″27 E) of China National Rice Research Institute in Hangzhou, China in 2019. A completely random block design with three replicates was applied in Experiment 1 (24 plots), which included four rice varieties and two fertilizer treatments. The rice varieties for Experiment 2 (96 plots) were Zhongzao 39 and Zhongjiazao 17. The first area of Experiment 2 was designed with three replicates including two varieties and six fertilizer treatments. The second area of Experiment 2 was designed with five replicates including two varieties and six fertilizer treatments. Four other experiments were carried out in four fields of three experimental stations (Fuyang, Yuhang, and Pingyao) in Hangzhou, China in 2020. Experiment 3 (11 plots) was designed with three replicates, which included one rice variety and four fertilizer treatments. One of the plots failed to collect effective remote sensing data due to rice lodging. 19 varieties of rice were treated with the same fertilizer treatment in Experiment 4 (19 plots) without replicate. Experiment 5 included 28 plots with four rice varieties and seven fertilizer treatments. Experiment 6 (54 plots) was conducted with 11 replicates including one variety and five fertilizer treatments. A plot was abandoned due to lodging.

2.2. Image Acquisition and Processing

A compact multispectral camera mounted on DJI Phantom 4 Multispectral (Da Jiang Innovations, Inc., Shenzhen, Guangdong, China) was used to obtain multispectral images and RGB images at the heading and maturity stages (Table 1). DJI Phantom 4 Multispectral is a small quadrotor UAV connected to the camera by a three-axis stabilizer cradle. The multispectral camera consists of six 2.08-megapixel camera sensors (1600 × 1300), including five monochrome sensors (Blue (B) 450 ± 16 nm, Green (G) 560 ± 16 nm, Red (R) 650 ± 16 nm, Red edge (RE) 730 ± 16 nm, Near-infrared (NIR) 840 nm) ± 26 nm and a true-color camera sensor (visible light; R, G, B).

To prevent image distortion caused by weather conditions, all UAV flights were conducted from 11:00 am to 2:00 pm on sunny days, during which there was full sunlight and low wind speed. The altitude of each flight was 25 m, and the average flight speed was 2.5 m/s. The camera lens was vertical to the ground to collect orthographic images of experimental plots. The forward and side overlap were both maintained at 75% to ensure good image mosaic results. Four calibration boards (reflectance values:10%, 30%, 50% and 80%) were placed horizontally before each flight. Then the UAV hovered 1.5 m above the calibration boards to photograph them with the multispectral camera. These images were used for radiometric correction. The UAV flew over each experimental field according to planned routes and automatically acquired multispectral and RGB images at fixed intervals. Except for the infrared bandwidth of 26 nm, the bandwidth of the other four bands was 16 nm. The ground sampling distance of multispectral and RGB images is 0.013 m/pixel. All rice was sown in May and harvested in October, and the yield of all the plots was manually measured.

The original multispectral and RGB images obtained by the UAV platform sensor were processed for orthophoto mosaic and radiometric correction before data extraction. The main steps of image processing described above were completed in DJI Terra software (Da Jiang Innovations, Inc., Shenzhen, Guangdong, China). All images and corresponding reflectance values of the four correction boards were imported to DJI Terra Software, which automatically completed radiometric correction. Individual aerial images were calibrated and aligned in DJI Terra software for generating an orthophoto of each experimental field. With calibrated internal parameters and onboard geolocation, the georeferencing RMSE (that of original and calibrated locations) of block adjustment was 1.1 cm. Then R, G, B, RE and NIR bands of each experimental plot were extracted in ENVI 5.3 software (Exelis Visual Information Solutions, Boulder, CO, USA).

2.3. Feature Selection

VIs and CIs can be used to sensitively monitor crop growth. The number of rice panicles is closely related to rice yield, and the texture feature can indicate the distribution of rice leaves and panicles. The use of fusion features including texture may help improve prediction accuracy. The correlation between VIs, CIs, TIs, and yield was analyzed to select the best combination of input variables. Six VIs and six CIs were selected with reference to the commonly used in a series of published optical indices. However, these VIs which were calculated from specific bands based on the experience of previous studies may not have the best performance in yield prediction. The original band combinations of each VI were not adopted directly. Each new index was reconstructed by any two or four bands from all five bands according to the formula. Then the optimal combinations were selected by analyzing the correlation between VI and yield. While the CIs was calculated by the re-normalized r, g, and b bands with reference to previous studies in

Table 2. Six VIs and six CIs selected in this study. R_λ1, R_λ2, R_λ3, and R_λ4 represent the reflectance of any bands selected from R, G, B, RE, and NIR.

Index Name	Formula	Reference
VI1	(Rλ1 − Rλ2)/(Rλ3 − Rλ4)	[22]
VI2	Rλ1 − Rλ2	[23]
VI3	(Rλ1 − Rλ2))/(Rλ1 + Rλ2)	[24]
VI4	Rλ1/Rλ2	[25]
VI4	1.5*(Rλ1 − Rλ2))/(Rλ1 + Rλ2 + 0.5)	[26]
VI5	1.16*(Rλ1 − Rλ2)/(Rλ1 + Rλ2 + 0.16)	[11]
CI1	2g − b − r	[27]
CI2	(g2 − r2)/(g2 + r2)	[28]
CI3	(g2 − br)/(g2 + br)	[28]
CI4	(r − g)/(r + g − b)	[29]
CI5	3g − 2.4r − b	[27]
CI6	(2g − b − r)/(2g + b + r)	[30]

. The calculation method is shown in the following formulas (Table 2).

r = R/(R + G + B)

(1)

g = G/(R + G + B)

(2)

b = B/(R + G + B)

(3)

where R, G, and B represent the average digital number value (0–255) of red, green, and blue bands in RGB images, and r, g and b represent the represents the normalized values of these three bands.

The frequency distribution of two gray pixels with distance (Δx, Δy) in the image can be expressed by gray level co-occurrence matrix (GLCM), which is an effective method for texture feature extraction. Four GLCM-based textures, including homogeneity (HOM), contrast (CON), Energy (ENE), and correlation (COR), were calculated from five band images by MATLAB 2018 (MathWorks, Inc., Natick, MA, USA). Furtherly, a new index based on texture, named texture index (TI), was calculated by any two textures selected from four GLCM-based textures. At present, this method is rarely reported. There are many combinations of four texture features from five bands. By analyzing the correlation with yield, a total of six TIs were calculated according to the top six texture combinations. The calculation formula is as follows:

$$TI(T_1,T_2)=\frac{T_1-T_2}{T_1+T_2}$$

(4)

where T₁ and T₂ present four GLCM-based textures (HOM, CON, ENE, COR) calculated from B, G, R, RE, NIR, and TI(T₁, T₂) presents the TI calculated by T₁ and T₂.

2.4. Data Sets Construction and Model Effect Evaluation

All data from 120 samples in 2019 and 112 samples in 2020 were mixed together to obtain a comprehensive dataset that ensured the robustness of the model. To evaluate the yield prediction performance in different growth stages, the mixed dataset was divided into two sub-data sets based on the heading stage (HS) and maturity stage (MS). 75% of the data from the two stages were randomly taken to form the HS and MS training datasets separately. Similarly, the remaining 25% of the data were used as test dataset to verify the model. The training data sets which were formed by two stages mixed data was called all stage (AS). The dataset composition and segmentation processes are shown in Figure 2, and the research flow chart for developing a novel rice yield index is shown in Figure 3.

The quality of data distribution has an important impact on the construction of a prediction model. Rice yield in the training dataset ranged from 3820.5 to 13,960.8 kg/ha (coefficient of variation, CV = 28.93%) (

Table 3. Statistical analysis of rice yield data. Note: SD represents standard deviation, and CV represents the coefficient of variation.

Datasets	Number of Samples	Range (kg/ha)	Average (kg/ha)	SD	CV (%)
Training dataset	174	3820.5–13,960.8	8546.6	2472.8	28.93
Test dataset	58	4558.6–14,117.4	8625.6	2524.2	29.26
All	232	3820.5–14,117.4	8566.4	2480.5	28.96

). For the test dataset, rice yield ranged from 4558.6 to 14,117.4 kg/ha (CV = 29.26%). The statistical analysis results of the dataset show that the test dataset was more variable than the training dataset. It can well verify the model performance. Due to contained conventional rice experimental plots and high-yield rice experimental plots, the two-year rice experiments offered a suitable dataset with large variability for evaluating the yield prediction model.

The training dataset was trained by three methods, including quadratic nonlinear regression (QNR), multiple linear regression (MLR), and random forest (RF). These three models include the nonlinear model, linear model, and machine learning model. The regression model formulas of quadratic nonlinear regression (QNR) and multiple linear regression (MLR) are as follows:

$$y=a{x}^2+bx+c$$

(5)

$$y={{\displaystyle \sum }}_i^n{a}_i{x}_i+a_0$$

(6)

where x and x_i represent the input variables (e.g., VI, CI and TI), y is the predicted value of rice yield, and a, b, a₀ and ai are the coefficients which were calculated through the training model in MATLAB 2018b and IBM SPSS Statistics 25 (IBM Corporation, Armonk, NY, USA).

Due to the introduction of two randomness, RF is not easy to fall into overfitting, and has good performance on the dataset. The number of decision trees (ntree) and the number of observations per tree leaf (mtry) are two important parameters in the RF model. When ntree is set to a large enough value, it mainly affects the running time of the model, not modeling accuracy. According to previous studies, ntree is usually set to 500. The setting of mtry greatly affects the accuracy of the RF model, so it needs to be adjusted according to the input variables. In this paper, a 10-fold cross-validation method was used to optimize the model. The average error of 10 iterations of modeling and validation is taken as the error of 10-fold cross-validation.

The coefficient of determination (R²), mean absolute error (MAE), and mean absolute percentage error (MAPE) were selected to evaluate the performance of the model. The calculation formulas are as follows:

$$R^2=\frac{{{\displaystyle \sum }}_i^n{(P_i-\overline{T})}^2}{{{\displaystyle \sum }}_i^n{(T_i-\overline{T})}^2}$$

(7)

$$MAE=\frac{1}{n}{{\displaystyle \sum }}_{i=1}^n\left|T_i-P_i\right|$$

(8)

$$MAPE=\frac{1}{n}{{\displaystyle \sum }}_{i=1}^n\frac{\left|T_i-P_i\right|}{\overline{T}}·100\%$$

(9)

where P_i and T_i represent the prediction value and truth value of sample rice yield respectively, $\overline{T}$ represents the average value of truth rice yield, and n is the number of samples in the dataset.

3. Results

3.1. Relationships between Yield and New VIs and CIs

The reflectivity of R, G, B, RE and NIR bands in the rice canopy of each plot was obtained by two-year rice experiments. By analyzing the correlation coefficient between rice yield and the indices calculated by any two bands from five bands, the optimal VIs was selected to enhance modeling (Figure 4 and Figure 5). The best band combinations of six VIs at HS are VI_{1(G-R, B-G)} (Pearson correlation coefficient, R = 0.5032, VI_{1(G-R, B-G)} represents the VI calculated by the combination of green, red and blue bands, and so on for other VIs), VI_{2(R, R-RE)} (R = 0.6390), VI_{3(R, G)} (R = 0.6882), VI_4(R,G) (R = 0.7036), VI_5(RE,R) (R = 0.4602) and VI_{6(NIR, B)} (R = 0.4903) (Figure 4). The best band combinations of six VIs at MS were VI_{1(RE-NIR, B-R)} (R = 0.7742), VI_{2(R, B)} (R = 0.5921), VI_{3(NIR, R)} (R = 0.6741), VI_{4(NIR, B)} (R = 0.6432), VI_{5(R, B)} (R = 0.8423), VI_{6(NIR, R)} (R = 0.6402) (Figure 5). Qian [31] studied the correlation between NDVI, ratio vegetation index (RVI) and rice yield at different stages (R_NDVI = 0.24_, R_RVI = 0.25 at HS, R_NDVI = 0.72_, R_RVI = 0.71 at MS). Rahman [32] showed the correlation between temperature condition index (TCI) and rice yield (R_TCI = 0.5 at MS). Compared with the results of previous studies, the method of reconstructing VIs does improve the correlation between VIs and yield. As well, the results show that the best combination of these indices is not the original common combination. Figure 6a,b shows the relationships between yield and VIs, CIs based on QNR. The best performance of VI in yield prediction at HS, MS and AS were VI₄ (R² = 0.62), VI₅ (R² = 0.68), VI₁ (R² = 0.54), respectively. while at these three stages the best performance of CI was CI₂ (R² = 0.48), CI₃ (R² = 0.59), CI₁ (R² = 0.43). The performance of VI was significantly better than that of CI, and the prediction effect at MS was the best, slightly worse at the HS, and the worst at AS. The relationship between the VI₅ and yield, as shown in Figure 6, supported this conclusion. When single VI₅ from HS or MS was used as the input variable, the coefficients of determination were R² = 0.58 (significant at the 0.05 level), R² = 0.68 (significant at the 0.01 level) respectively, while the accuracy of modeling at AS decreased (R² = 0.53, significant at the 0.05 level). As well, the same result could be observed in CI.

3.2. Relationships between Yield and New TIs

Figure 6c,d shows the correlation between yield and GLCM-based textures of six bands. The performance of texture at HS was better than that at MS, and texture from the red band outperformed all other bands, followed by the NIR band texture. CON_red (CON from the red band, and so on for other textures) performed best at MS (R² = 0.52) and AS (R² = 0.41), and ENE_NIR performed best at HS (R² = 0.43). Other textures such as CON_NIR, COR_red, HOM_NIR, and ENE_red also outperformed most others. Hlatshwayo et al. [14] showed that the coefficient of determination of raw band texture for aboveground was 0.51. The results show that the performance of the original GLCM-based texture used to predict yield needs to be improved. The performance of texture was better than that of CI, but worse than that of VI. It was considered that the key bands which could generally indicate the nutritional, and growth status of rice were red and NIR bands. Therefore, VI and TI including red and NIR bands had a good correlation with yield. In addition, the number of panicles was closely related to yield, and the distribution density of panicles affected texture [18]. So texture also had an acceptable performance in yield prediction (significant at 0.05 level).

Moreover, six newly constructed TIs, calculated by the top six combinations (

Table 4. Six top R² between TI and yield. Note: * means significant at the 0.05 level, ** means significant at the 0.01 level. TI(T₁, T₂) is calculated by the combination of two GLCM-based textures (T₁ and T₂)_.

Model	Input TI	HS	MS	AS
QNR	TI(CONNIR, ENEred)	0.58 *	0.56 *	0.46 *
	TI(CONred, ENENIR)	0.50 *	0.44 *	0.36 *
	TI(CONNIR, CONred)	0.47 *	0.42 *	0.30 *
	TI(CONNIR, CONred)	0.63 **	0.61 *	0.32 *
	TI(CONNIR, ENEred)	0.56 *	0.58 *	0.45 *
	TI(CONNIR, CORred)	0.64 **	0.60 **	0.48 *

), were evaluated with the QNR method. It is found that the performance of TIs was much better than the original texture. The top TIs for yield prediction were TI(CON_NIR, COR_red) (TI calculated by the combination of CON_NIR and COR_red, and so on for other TIs) at HS, TI(CON_NIR, CON_red) at MS, and TI(CON_NIR, COR_red) at AS.

3.3. YI Building

3.3.1. YI Building by QNR Model

The YI for yield estimation was constructed using the top VIs, CIs and TIs at two different growth stages. The results in

Table 5. The top three verification results of YI are based on VIs, CIs, and TIs in QNR. Note: * means significant at the 0.05 level, ** means significant at the 0.01 level.

Stages	Variable	R2	MAE (kg/ha)	MAPE (%)
	YIs based on VI
HS	YIVI4	0.62 **	1247.95	17.93
MS	YIVI5	0.67 **	1148.03	13.98
AS	YIVI1	0.54 *	1332.70	24.33
	YIs based on CI
HS	YICI2	0.48 *	1498.80	27.25
MS	YICI3	0.59 *	1446.21	22.21
AS	YICI1	0.43 *	1562.96	34.52
	YIs based on TI
HS	YI(CONNIR,ENEred)	0.61 **	1376.79	18.60
MS	YI(CONred,ENENIR)	0.52 *	1470.40	25.89
AS	YI(CONNIR,CONred)	0.46 *	1481.97	30.23

show that the YI based on VI, CI, and TI had obtained acceptable verification results, and the performance of YI based on VI was better than that based on CI and TI. The top YI based on VI at HS, MS and AS were YI_VI4 (R² = 0.62, MAPE = 17.93%, YI_VI4 represents the YI calculated by VI₄, and so on for other YI), YI_VI5 (R² = 0.67, MAPE = 13.98%) and YI_VI1 (R² = 0.54, MAPE = 24.33%), respectively (Table 5). The top YI based on CI at three stages were YI_CI2 (R² = 0.48, MAPE = 27.25%), YI_CI3(R² = 0.59, MAPE = 22.21%) and YI_CI1 (R² = 0.43, MAPE = 34.52%). YI(CON_NIR-ENE_red) (R² = 0.52, MAPE = 18.60%, YI(CON_NIR-ENE_red) represents the YI built by TI(CON_NIR-ENE_red), and so on for other YI), YI(CON_red-ENE_NIR) (R² = 0.64, MAPE = 25.89%) and YI(CON_NIR-CON_red) (R² = 0.46, MAPE = 30.23%) were the top YIs based on TI at three stages respectively. The performance of VI-based YI (MAPE = 13.98%) at MS was better than other stages, and outperform that of CI-based (MAPE = 22.21%) and TI-based (MAPE = 18.60%).

3.3.2. YI Building Based on Fusing VIs, CIs and TIs

The six VIs, six CIs, and six TIs mentioned above were fused to build YI by MLR and RF models. VIs and CIs, which are calculated by different bands, are spectral type data. TIs calculated by different GLCM-based texture is the data indicating arrangement regularity of rice canopy. Thus, input variables were divided into three kinds: (1) six VIs, (2) six VIs + six CIs, and (3) six VIs + six CIs + six TIs. The purpose of adding VIs, CIs, and TIs to the models step by step is to explore the influence of three indices on the accuracy of yield prediction. The YIs built by MLR and RF were validated using the test dataset (

Table 6. Verification results of test dataset on MLR and RF models. Note: * means significant at the 0.05 level, ** means significant at the 0.01 level.

Stages	Technique	R2	MAE (kg/ha)	MAPE (%)		R2	MAE (kg/ha)	MAPE (%)		R2	MAE (kg/ha)	MAPE (%)
		VIs				VIs + CIs				VIs + CIs + TIs
HS	MLR	0.69 **	1264.55	13.91		0.65 **	1374.55	15.12		0.61 **	1690.92	18.60
	RF	0.70 **	1245.46	13.70		0.79 **	851.82	9.37		0.78 **	883.64	9.72
MS	MLR	0.72 **	1145.46	12.60		0.71 **	1195.46	13.15		0.70 **	1245.46	13.70
	RF	0.73 **	1095.46	12.05		0.80 **	748.18	8.23		0.84 **	714.55	7.86
AS	MLR	0.54 *	2211.83	24.33		0.50 *	2372.74	26.70		0.52 *	2353.65	25.89
	RF	0.51 *	2393.65	26.33		0.65 **	1374.55	15.12		0.61 **	1690.92	18.60

). When the input variables were a fusion of six VIs, the performance of YI from MLR at three stages was slightly better than that from QNR. However, when YI was built by fusion of VIs + CIs and VIs + CIs + TIs, the MAPE of yield prediction results increased. It may be that too many inputs led to a complex model and worse prediction performance. The YI based on VIs achieved the highest accuracy (MAPE = 12.60%) at MS, and its performance was better than the MLR model based on VIs + CIs (MAPE = 13.15%) and VIs + CIs + TIs (MAPE = 13.7%).

According to previous studies, ntree was usually set to 500. The value of mtry was adjusted (1 to 20) to minimize the MAPE of the validation dataset for obtaining a general optimization RF model. When VIs + CIs + TIs was used as the input variable to build the RF model, the MAPE of the validation dataset at HS decreased and then increased, and the minimum MAPE was obtained at mtry = 7. The MAPE of the validation dataset at MS decreased and then kept steady, and the minimum MAPE was obtained at mtry = 12. The MAPE of the validation dataset at AS decreased, then increased, and finally kept steady, and the minimum MAPE was obtained at mtry = 6. When VIs and VIs + CIs were used as input variables to build the model, the value of mtry was also adjusted, and the results are shown in

Table 7. The optimal mtry value of RF model at three stages.

Stages	VIs	VIs + CIs	VIs + CIs + TIs
HS	3	5	7
MS	5	4	12
AS	4	6	8

.

Different from YI built by MLR, the verification of YI built by RF shows the decreasing trend of MAPE due to the increase of input variables. Compared with other combinations, there was a minimum MAPE at three stages based on the fusion of VIs + CIs + TIs. The best YI constructed by a combination of VIs + CIs + TIs from RF was superior to QNR and MLR, and the highest accuracy was obtained for yield prediction at MS (MAPE = 7.8%) (Figure 7). As an example, the visualization of yield prediction verification results, as predicted by the YI based on VIs, CIs, and TIs from the RF model using UAV imagery of experiment 5 at Fuyang station, is presented in three stages separately (Figure 8). The difference in yield prediction accuracy is obvious in the response to the YI constructed by different methods in different periods.

4. Discussion

4.1. Model Performance Using Cross Datasets

To have better generalization ability, the models were trained on the datasets of 2019 and 2020. The dataset used in the validation of the model contains experimental data of 24 varieties and 21 fertilization patterns at different growth stages. During the validation, we did not retrain the model for representing the actual challenge to yield prediction. Satisfactory cross dataset validation result (R² = 0.73, MAE = 1375.46 kg/ha and MAPE = 7.86% at MS) shows that our proposed model can overcome the problems caused by different varieties and fertilization patterns.

Although the yield prediction accuracy is not satisfactory at AS, it could be improved by the division of different growth stages. The performance of MS is better than HS (R² = 0.84, MAE = 1375.46 kg/ha and MAPE = 7.86%), which shows that MS is the best time to estimate yield. Panicles, which are an important part to determine the yield, were green and remained upright at HS. The upright panicles appeared as dots in the orthophoto image, and accounted for a small proportion of pixels in the whole image. Dense rice leaves and erect panicles made it difficult to observe and extract effective features from UAV images. In the subsequent growth period, climate and agricultural management would also have a great impact on the yield. These factors led to unsatisfactory yield prediction performance at HS. The panicles turned yellow and became curved at MS, which makes it easy to distinguish panicles from leaves in the orthophoto image of the rice canopy. Curved panicles can occupy a higher proportion of pixels in the entire image. Clear and accurate information on panicle size and distribution density at MS was helpful to improve the prediction accuracy. At all stages, rice canopy spectra and textures have been changing dynamically with rice growth, which will lead to poor prediction performance. However, some growth stages are short and the boundaries between them are not obvious, which brings challenges to the prediction of this method.

4.2. Performance of the Novel YI

In this study, YI built by RF based on six VIs + CIs + TIs obtained the best performance at MS (R² = 0.84, MAE = 1375.46 kg/ha, MAPE = 7.86%). Naito et al. [18] proved that the simple ratio index exhibited the best performance in regards to the estimation of grain weight (R² = 0.80). The top single VIs for yield prediction are YI_VI4 (R² = 0.62, HS) and YI_VI5 (R² = 0.67, MS) (Table 5). The best performances of YIs built by MLR and RF are R² = 0.69 (HS) and R² = 0.0.72 (MS), R² = 0.84 (HS) and R² = 0.79 (MS) (Table 6). Compared with single VI, the YI proposed by us can integrate more spectral and image characteristics of crops to improve the performance of yield prediction. Kang et al. [33] proposed an artificial neural network rice yield prediction model based on six VIs, which obtained a stable prediction accuracy (R² ≥ 0.71, RMSE ≤ 29.0 kg/1000 m²) under one variety and six nitrogen treatments. Three of the six indices in his study are in the same form as the indices we used. Wang et al. [34] showed that introducing fluorescence spectral information at the flowering stage into conventional VIs-based yield estimation models is helpful in improving rice yield estimation accuracy (R² = 0.869, MAPE = 3.98%, RMSE = 396.02 kg/ha) under four varieties and five nitrogen treatments. In these studies, the prediction model accuracy of rice yield is better than ours. However, their proposed model was only validated on rice research materials with a few varieties and several different nitrogen gradient treatments in a single year. The YI proposed in our study was built on RF by fusing the spectral and image information of 24 rice varieties under 21 fertilization modes in different regions in two years. When applied to the test dataset, the performance of YI can have good stability for yield prediction.

4.3. Advantages of the Novel YI

VI could help to enhance the interpretation of remote sensing data and has been widely used as a way of processing remote sensing information [35]. The top three VIs at HS are VI_{4(R, G)} (R = 0.7036), VI_{3(R, G)} (R = 0.6882) and VI_{2(R, R-RE)} (R = 0.6390) (Figure 4). The top three VIs at MS are VI₅₍R, B₎ (R = 0.8423), VI_{1(RE-NIR, B-R)} (R = 0.7742) and VI_{3(NIR, R)} (R = 0. 6741) (Figure 5). The new VIs calculated by canopy spectrum (especially red band, red edge band, and near-infrared band) can well monitor the rice yield with high correlation. Leaf color indicates the nutritional status of rice, and the number of flowers and ears is closely related to yield [5]. The CIs extracted from RGB images can sensitively perceive the color changes of leaves and ears during rice heading (CI₂, R² = 0.48 at HS) and ripening (CI₃, R² = 0.59 at MS). Four texture features of each gray image were creatively constructed as the normalized TIs. Compared with the top original texture feature (CON_red, R² = 0.52 at MS), TI (TI(CON_NIR, CON_red), R² = 0.61 at MS) can better represent the changes in rice yield which may be affected by distribution density and size of rice ears. Therefore, the YI built by fusing rice canopy spectrum, color, and texture has better robustness than the traditional remote sensing data processing methods in cross varieties, regions, and fertilizer treatments data.

4.4. Potential Applications of the Novel YI

It has been verified that the new YI developed by fusing VIs, CIs, and TIs on RF (MAPE = 7.86%) can further reduce MAPE by 1.89% and 6.12% than that on MLR (9.72%) and QNR (13.98%). It may help to better implement crop management, especially in crop insurance, harvest plan, storage demand, cash flow budget, and determination of input decisions such as nutrition, pesticide, and water. Rice yield is the most important parameter and ultimate target index in rice planting and breeding research. The new yield index is helpful to the breeding research of new rice varieties with high yield and good stress resistance. With the rapid development of breeding technology, hundreds of new breeding materials need to be processed key growth stage. Using yield index can realize digital technology with a high degree of automation, high precision, and high speed, and break through the bottleneck of traditional rice yield phenotype measurement.

5. Conclusions

This study developed a novel rice yield index using UAV remote sensing imagery fusion technology. Our findings suggest that the yield index (YI) built by RF based on fusing vegetation indices (VIs), color indices (CIs), and texture indices (TIs) extracted from unmanned aerial vehicle (UAV) imagery has great potential in rice yield monitoring. The relationships between yield and VIs, CIs, and TIs were calculated to select input variables for building YI. The results show that the performance of the newly constructed VIs and TIs was better than that of the original specific bands and single texture. VI_{4(R, G)} and VI_{5(R, B)} exhibited the highest correlation to yield at MS and HS, respectively. The performance of VI-based YI built by QNR at MS was better than other stages, and outperformed that of CI-based and TI-based. Compared with HS, the best performance of YI was developed by RF with combining VIs + CIs + TIs at MS (R² = 0.84, MAE = 714.55 kg/ha, MAPE = 7.86%). This research proved the feasibility of YI from UAV-based imagery in yield prediction. Further improvements are needed in modeling methods and optimal height of data collection for both efficiency and accuracy to provide accurate data for precision agricultural management and yield phenotyping measurement.