1. Introduction
Wheat is an important cereal crop, and its sustainable production is essential to ensure food security in the context of rapid global population growth [
1,
2,
3]. Nitrogen is one of the important nutrients required for crop growth and development and plays an indispensable role in crop growth. Lack of nitrogen fertilizer can limit crop photosynthesis, while the excessive application of nitrogen fertilizer can lead to problems such as resource waste, soil acidification, and environmental pollution [
4,
5]. As good indicators of nitrogen fertilizer application, leaf nitrogen content (LNC) and aboveground biomass (AGB) at the main growth stages (jointing, booting, and flowering) play an important role in evaluating the quality of nitrogen fertilizer application, assisting nitrogen fertilizer application, and reducing N loss [
6]. Therefore, the quantification of LNC and AGB is the key foundation for producing high-yield and high-quality crops.
Remote sensing technology, which allows for the rapid, non-destructive, real-time monitoring of crop growth, is now maturing. Satellite data with various temporal, spatial, and spectral resolutions are widely used in various scales of crop yield prediction studies [
7]. However, spectral data from satellite platforms are somewhat limited in terms of spatial resolution and temporal sampling, hampering the timely estimation of crop agronomic traits [
8]. Unmanned aerial vehicles (UAVs) can obtain high temporal- and spatial-resolution imagery and achieve large-scale crop monitoring, making them an attractive technology for crop growth assessment in smart agriculture in recent years [
9,
10]. Sampled data from UAV platforms have improved temporal, spatial, and spectral resolution compared to satellite platforms [
11]. Therefore, the high-throughput images acquired by the different sensors carried by UAVs offer great opportunities for crop phenotype monitoring [
12,
13].
Vegetation indices (VIs) extracted from UAV-based high-throughput imagery have proven to be a well-established method for monitoring crop agronomic traits [
7,
14]. Previous studies have demonstrated the potential of UAV RGB imagery and spectral imagery for monitoring biomass [
15,
16,
17] and nitrogen status [
18,
19]. The optimal VIs can maximize sensitivity to agronomic traits and reduce the impact of environmental factors and sensor types on spectral data [
20,
21]. Hunt et al. [
22] used small drones to acquire RGB images of farmland and their study found that the normalized green-red difference index (NGRDI) before canopy closure was sensitive to AGB. The red-blue ratio index (RBRI) extracted from UAV RGB images by Schirrmann et al. [
23] was strongly associated with biomass, with a coefficient of determination (R
2) ranging from 0.72 to 0.99. In addition, some studies have shown that spectral data from the red edge and near-infrared bands also have good applications in biomass estimation [
24,
25]. For example, commonly used VIs such as the green optimum soil adjusted vegetation index (GOASVI), the modified soil adjusted vegetation index (MSAVI), and the normalized difference vegetation index (NDVI) have been shown to give satisfactory results in estimating the agronomic traits (e.g., AGB and nitrogen status) of crops such as wheat, maize, and rice in many studies [
26,
27,
28,
29]. Therefore, RGB and hyperspectral images from UAVs contain a large amount of color and spectral information, which can be used to detect changes in crop growth, providing the technology to quantify LNC and AGB over large areas.
Traditional statistical analysis models, such as simple linear regression and multiple linear regression algorithms, are commonly used for the remote sensing inversion of crop agronomic traits. In recent years, the application of machine learning algorithms in crop growth monitoring has also become more and more extensive. Machine learning is a data-driven algorithm that can autonomously process complex linear relationships between data [
30]. Machine learning algorithms, which can solve strongly non-linear problems with remote sensing variables and agronomic traits [
31], are increasingly being applied. Previous studies have shown that machine learning algorithms such as artificial neural networks (ANNs), random forest (RF), and support vector regression (SVR) can be adequately applied to canopy spectral data [
18,
32], avoiding the inherent multicollinearity problem of multiple linear regression [
18]. Verrelst et al. [
33] applied four machine learning algorithms, including neural networks (NNs), kernel ridge regression (KRR), support vector regression (SVR) and Gaussian process regression (GPR), to estimate three traits of leaf chlorophyll content (LCC), leaf area index (LAI) and fractional vegetation cover (FVC), and compared model performance, and they found that the GPR model estimated the best results. Zheng et al. [
34] used UAV multispectral images to extract VIs and combined 13 regression algorithms (including simple linear regression, machine learning algorithms, and physical models) to construct and compare LNC estimation models for winter wheat. Their findings showed that simple linear regression algorithms and machine learning algorithms performed well for LNC assessment, but LNC inversion based on physical models was still challenging, and inversion accuracy was low.
Hyperspectral data contain a large amount of spectral data, but this also implies data redundancy issues. Choosing the appropriate variable-selection algorithm prior to modeling can reduce the model running time and improve the model estimation [
35]. However, much of the current research has focused on the application of different modeling approaches to estimate crop agronomic traits, with few studies using variable-selection algorithms for hyperspectral remote sensing to monitor traits such as LNC and AGB, even though the variable selection is also an important factor affecting model results. To address this issue, this study aims to improve the accuracy of LNC and AGB estimation for wheat based on UAV hyperspectral and RGB imagery by combining different variable-selection algorithms and modeling approaches to better assist N fertilizer application and increase N fertilizer utilization efficiency.
This study used UAV RGB images and hyperspectral images to dynamically monitor wheat AGB and LNC under different growing conditions. The specific objectives were to (1) investigate the relationship between color features and spectral indices from UAV imagery and LNC and AGB; (2) filter feature variables by using feature-selection algorithms to remove redundant variables, combine multiple regression models for LNC and AGB estimation, and explore the effects of different variables on the estimated models; and (3) evaluate the accuracy and robustness of the LNC and AGB estimation models developed from combined RGB and hyperspectral imagery using statistical analysis.
2. Materials and Methods
2.1. Study Site and Experiment Design
The two-year field trial was conducted in Yizheng City, Jiangsu Province (32°30′ N, 119°13′ E), which is a typical wheat cultivation area in the middle and lower reaches of the Yangtze River in eastern China (
Figure 1). The previous crop of the two-year field trial was rice. Winter wheat in Exp.1 was grown on clay loam soil with an average pH of 7.15, an organic matter content of 14.24 g/kg, an effective nitrogen of 65.23 mg/kg, an effective phosphorus of 43.43 mg/kg and an effective potassium of 112.37 mg/kg. The average pH of the soil in the experimental field was 7.24, the organic matter content was 14.12 g/kg, the effective nitrogen was 72.52 mg/kg, the effective phosphorus was 63.60 mg/kg and the effective potassium was 102.76 mg/kg in Exp.2. The field trials involved three wheat varieties (V1–V3: Yangmai 23, Zhenmai 9 and Ningmai 13). In addition, two planting densities (D1: 450 grains/m
2 and D2: 600 grains/m
2) and four N fertilizer application treatments (N1-N4: 0 kg/hm
2, 105 kg/hm
2, 210 kg/hm
2 and 315 kg/hm
2) were considered in the trial, which was replicated twice in total. The N fertilizer application strategy for the three trials was 5:1:4 in three applications: basal (1 d before transplanting), tiller (7 d after transplanting) and spike (at the beginning of spike differentiation). The field trial consisted of 48 sub-sample plots with an area of 18 m
2 (3 m × 6 m). The field trials in both years were mechanically sown in early November in rows 30 cm apart and harvested in early June. The phosphorus and potassium fertilizers (120 kg/hm
2 P
2O
5, 120 kg/hm
2 K
2O) were applied once before sowing. Other field management practices (e.g., weeding, pesticide application, etc.) followed local practices.
2.2. Field Data Acquisition
Ground sampling was carried out during the main growth period of wheat (
Table 1). Twenty wheat plants were randomly selected at each plot for destructive sampling to determine AGB at the jointing, booting and flowering stages. Plant samples were separated by organ and oven-dried at 105 °C for 0.5 h, and then they were oven-dried at 80 °C to a constant weight, after which each organ was weighed for dry matter. The sum of the dry matter weight of each organ was used to determine the AGB of wheat in different plots. The samples of leaves were ground and sieved to determine the leaf nitrogen concentration of each plot using the Kjeldahl method [
36].
2.3. UAV Image Acquisition
UAV missions were conducted prior to field data collection. A DJI Phantom 4 RTK (SZ DJI Technology Co.; Shenzhen, China) drone was selected as the RGB data collection platform to monitor wheat growth at the jointing, booting and flowering stages. The RGB camera equipped with the DJI Phantom 4 RTK drone used a 20-megapixel CMOS and a 24 mm focal length sensor, and the ground resolution of the RGB image was 2.47 cm. The ground station software (DJI GS PRO) was used to design the flight path of the UAV, which flew at an altitude of 25 m, with a forward and lateral overlap setting of 80% and a flight speed of 3 m/s. The images were acquired as JPEG RGB images at 5472 × 3648 pixels, with an effective pixel count of 20 million.
The DJI M600 Pro hexacopter was equipped with the Gaiasky-mini2-VN imager (Dualix Spectral Imaging Technology, Beijing, China) to obtain hyperspectral image data of the test field. The hyperspectral images captured by the Gaiasky-mini2-VN imager contain 176 channels and a wavelength range of 400–1000 nm, with a ground resolution of 8.5 mm. The UAV flight altitude was set to 100 m, the route coordinate points were manually planned, and hovering was used to capture the images. The content repetition rate between two adjacent images was 80%. The camera was calibrated prior to launch to adjust the exposure time. In addition, three gray-scale gradient calibration plates and five ground control points (GCPs) were placed on the ground to calibrate the images. The flight took place from 10:30 a.m. to 11:30 a.m. local time in clear weather with no strong winds.
2.4. UAV Image Processing
Before estimating wheat agronomic traits, the UAV images had to be processed to obtain relevant image data, including color indices and vegetation indices, as shown in
Figure 2.
The RGB images were calibrated and stitched using Pix4Dmapper (Pix4D SA, Lausanne, Switzerland) software, and the resulting RGB orthophotos were saved as tagged image format (TIF) files (
Figure 2). The UAV hyperspectral images were radiometrically calibrated using the SpecView software (Specim, Oulu, Finland), and reflectance was calibrated against captured reference grey cloth data. The method of radiometric calibration is shown in the following equation:
where
Rref is the reflectance value of the calibrated image,
DNr is the digital number (
DN) value of the raw image,
DNw is the reflectance of the white background plate and
DNd is the reflectance of the black background plate.
The method of reflectance calibration is shown in the following equation:
where
Rfixed is the spectral reflectance of the image after eliminating atmospheric, water vapor, etc.,
Rref is the reflectance of the calibrated image,
Rst is the spectral reflectance of the standard gray cloth, and
Rgray is the spectral reflectance of the reference gray cloth taken.
Hyperspectral images were geometrically calibrated and stitched using HiSpectralStitcher software (Dualix Spectral Imaging Technology) in conjunction with ground control point coordinates, and the resulting hyperspectral orthophotos were saved as TIF files (
Figure 2). Finally, the stitched RGB and hyperspectral images were subjected to plot-based orthophoto cropping using ENVI 5.3 (EXELIS, Boulder, CO, USA) software, and the correlation indices were extracted based on the cropped regions.
2.5. Image Feature Extraction
2.5.1. Color Index Extraction
Color indices can reflect crop growth conditions. The DN values of the red, green and blue (
R,
G and
B) channels of the RGB image are normalized to obtain r, g and b. The formula is as follows:
In this study, 16 color indices that are more commonly used for estimating agronomic traits in wheat were selected, and the calculation of each color index is shown in
Table 2.
2.5.2. Vegetation Index Extraction
Vegetation indices are linear or non-linear combinations between different remote-sensing spectral bands. In this paper, 18 more commonly used vegetation indices were selected as features of wheat canopy spectra; the specific names and calculation methods are shown in
Table 3.
2.6. Variable Selection
UAV hyperspectral images are characterized by high dimensionality and covariance. Therefore, this study applied three feature-selection algorithms, namely competitive adaptive re-weighted sampling (CARS), iteratively retains informative variables (IRIVs), and the random forest (RF) algorithm for the selection of RGB and hyperspectral indices to reduce redundant information and improve model performance.
CARS is a variable-selection algorithm that mimics Darwin’s “survival of the fittest” theory [
59,
60]. CARS assesses the relative importance of variables based on the stability index calculated by an exponentially decreasing function and then selects variables with high regression coefficients in the PLS model through an adaptive weighted sampling technique, combined with tenfold cross-validation, to choose the subset of the PLS model with the lowest root-mean-square error of cross-validation (RMSECV), which is used as the optimal combination of the characteristic variables [
61].
IRIVs is a new variable-selection algorithm that uses a binary matrix rearrangement filter (BMSF) to generate a large number of variable combinations [
62,
63], combined with PLSR, and uses the root-mean-square error of cross-validation (RMSECV) to assess the effectiveness of different random variable combination models [
62,
64]. IRIVs performs multiple iterations, retaining strong and weak information variables, removing confounding variables and uninformative variables, and finally determining the best combination of variable sets [
65].
The RF algorithm identifies covariance and nonlinear relationships between different variables and evaluates the importance of each variable with good generalization performance [
66,
67]. RF applied the bagging method, using random samples from the training set to build independent regression trees to estimate the importance of different variables [
68], with the following formula:
where
erroB1 represents the error of out of bag for variable
xi with one decision tree,
erroB2 represents the error of adding noise to variable
xi with one decision tree, and n represents the number of decision trees.
Detailed steps for implementing CARS, IRIVs and RF are given in references [
30,
60,
65]. The operations related to the three feature-selection algorithms were conducted in MATLAB 2019a (Matrix Laboratory, Math-Works, Natick, MA, USA).
2.7. Modelling and Validation of Agronomic Traits in Wheat
Considering the possible complex relationship between these UAV image feature variables and LNC and AGB, this study used one linear regression algorithm (multiple linear regression, MLR) and two integrated machine learning algorithms (random forest regression, RFR; gradient boosting decision tree, GBDT) to estimate wheat LNC and AGB (
Figure 3).
The GBDT algorithm uses all the samples in the training set to fit the regression tree [
69]. The regression tree of the RFR algorithm is parallel, whereas the tree of the GBDT is continuous. Each new tree in the GBDT is optimized by a loss function determined by the steepest gradient. Therefore, the last regression tree after several iterations and improvements is used to compute the target estimate. For detailed information on the GBDT algorithm, refer to Friedman [
70] and Wei et al. [
71].
MLR is a regression method that uses two or more independent variables to predict the dependent variable. This study used IBM SPSS Statistics 24 (Cary, NC., USA) for MLR model construction.
The RFR algorithm estimates wheat agronomic traits (LNC and AGB) by combining multiple regression trees [
7]. The RFR algorithm starts by randomly selecting subsamples from the training set (60% of the recorded target samples), then fits the regression trees with the sub-samples, and ultimately calculates the target modeled values by averaging the values of all the regression trees. Detailed information about RFR can be found in the study of Wang et al. [
72].
In this study, 2/3 of the dataset (
n = 216) is used for model calibration, and the remaining 1/3 of the dataset (
n = 72) is used for model validation. In the model evaluation, the coefficient of determination (
R2), root-mean-square error (
RMSE) and normalized root-mean-square error (
NRMSE) are used to evaluate the goodness of model accuracy.
where
is the estimated value of sample
i,
is the mean value of
,
is the estimated value of sample
i,
is the mean value of
,
is the measured value of sample
i, and
n is the number of samples.
4. Discussion
4.1. Comparison of Different Variable-Screening Algorithms
Three variable-screening algorithms, CARS, IRIVs and RF, were used to screen the input parameters for constructing wheat AGB and LNC estimation models, respectively. Screening variables is an important step in statistical analysis and data modeling to identify the eigenvalues that have the greatest impact on the target variable [
73]. Variable-screening algorithms should be selected based on target variables and data characteristics [
74]. For example, Wang’s results show that the RF variable-screening algorithm exhibits good performance in processing wheat SPAD data [
30]. The experimental results of Li et al. indicated that the estimation accuracy in the wheat yield prediction model could be improved by using the LASSO variable-selection algorithm [
75].
In this study, the variables screened using the RF algorithm were used to construct the AGB estimation model with the best accuracy, while for the LNC estimation model, it was the CARS algorithm that gave the best results. This may be due to the fact that AGB and LNC change differently at different growth stages. AGB increases all the time during the whole fertility of wheat, whereas LNC increases during the nutritive growth stage and starts decreasing at the reproductive growth stage. Therefore, it is necessary to select appropriate variable-selection algorithms based on data characteristics when selecting model input variables. After variable selection, the development of model regression algorithms can achieve the goal of reducing redundant variables and improving the accuracy and robustness of agricultural trait estimation models.
4.2. Impact of Different Combinations of Algorithms on Estimation Models
This study involved three wheat varieties, two planting densities, four nitrogen applications, three fertility periods, and ninety-six plots to estimate AGB and LNC, which resulted in a complex relationship between the two growth parameters of wheat (AGB and LNC) and the color and vegetation indices. Therefore, this study investigated the feasibility of a model development strategy combining multiple variable-selection algorithms and multiple regression algorithms to estimate wheat AGB and LNC. According to the results in
Table 5, the input parameters selected by the variable-screening algorithm for both the AGB dataset and the LNC dataset were mainly vegetation indices, while spectral indices were selected less, which is consistent with the results of the correlation analysis. This is due to the fact that hyperspectral cameras acquire a greater number of bands and fuller spectral information. Therefore, the type of sensor is also important for the estimation of crop growth parameters.
Nine AGB estimation models and nine LNC estimation models based on different combinations of algorithms were developed and compared, respectively (
Figure 7). As can be seen in
Figure 7, for the same variable-selection algorithm, there are differences in the accuracy of the estimated models obtained by combining different regression algorithms. Among the nine AGB estimation models, the estimation model developed by RF-RFR has the highest accuracy. Among the nine LNC estimation models, the CARS-RFR model had the best estimation accuracy. The results of this study are similar to previous studies. By comparing the effects of four machine learning regression algorithms on maize AGB estimation models, Han et al. found that the AGB estimation model constructed using the RFR algorithm had the highest prediction accuracy [
76]. Using UAV hyperspectral imagery and combining three machine learning algorithms to estimate rice LNC, Wang et al. found that the accuracy of LNC estimation models constructed using the three machine learning algorithms also differed [
77]. In addition, the redundant information input into the model is also reduced to some extent by the variable screening, which also improves the efficiency of the model operation and reduces the possibility of overfitting in the constructed model.
In addition, deep learning algorithms were not used in the evaluation of wheat LNC and AGB in this study, mainly due to two reasons [
78,
79]: firstly, deep learning is usually suitable for processing large-scale data and complex problems; secondly, deep learning algorithms typically require a large amount of data to construct models. If the dataset is relatively small, the deep learning model may overfit, while traditional machine learning algorithms are usually more robust for models constructed from small data samples. Therefore, deep learning algorithms were not applied to estimate wheat agronomic traits in this study.
4.3. Limitations and Future Research
This study compared the differences between different variable-screening algorithms and regression algorithms in constructing different growth indicators of wheat. A preliminary study proposed monitoring methods for wheat AGB and LNC, but there are still some limitations. It was found that as the reproductive process progressed, chlorophyll content decreased, differences in the appearance of wheat (mainly leaf color) diminished at different rates of N application, and as reproductive growth increased, spikelets appeared and the canopy structure changed. Methods for monitoring wheat growth based on a single type of image will be limited, and the accuracy of estimation will be reduced. Therefore, it is essential to monitor wheat growth using multi-source data-fusion techniques.
Hyperspectral data have attractive characteristics in crop monitoring, but hyperspectral data obtained based on proximal platforms are not suitable for large-scale crop growth monitoring. Satellite platforms have the advantages of being low-cost, continuous, and large-scale in acquiring spectral data [
80]. Compared to proximal platforms (e.g., handheld devices or drone platforms), satellite images have lower spatial resolution and are greatly affected by weather factors such as clouds and precipitation, making it difficult to provide real-time monitoring data like drones do [
7,
81]. However, with the gradual maturity of space-borne hyperspectral technology, the availability of hyperspectral data based on satellite platforms for large-scale estimation of crop agronomic traits has been greatly improved.
With the development of science and technology and the continuously upgrading iterations of a variety of sensors, the cost of use continues to decline. The types of sensors used in agriculture are also gradually increasing, providing more information for crop growth monitoring, such as thermal infrared images, elevation, and other data. In future research, the selection of model input parameters and the combination of regression algorithms can be further evaluated by combining multi-source data types from multiple platforms (such as proximal platforms and satellite platforms) to improve the accuracy of crop growth parameter monitoring.
This study only used data from two growing seasons in one region, and future studies should consider collecting data from more regions to enrich the diversity of model input parameters. Therefore, the conclusions drawn in this study can be further evaluated in future studies.