Strategies for the Efficient Estimation of Soil Moisture through Spectroscopy: Sensitive Wavelength Algorithm, Spectral Resampling and Signal-to-Noise Ratio Selection

Featured Application

In this study, a joint selection method of sensitive wavelength, optimal spectral resolution and signal-to-noise ratio of soil water content is proposed, which comprehensively considers the joint influence of spectral wavelength, spectral resolution and signal-to-noise ratio on the inversion accuracy. The method proposed in this paper can also be used to select the sensitive wavelength, optimal spectral resolution and signal-to-noise ratio of soil organic matter.

Abstract

It is found that the remote sensing parameters such as spectral range, spectral resolution and signal-to-noise ratio directly affect the estimation accuracy of soil moisture content. However, the lack of research on the relationship between the parameters and estimation accuracy restricts the prolongation of application. Therefore, this study took the demand for this application as the foothold for developing spectrometry. Firstly, a method based on sensitivity analysis of soil radiative transfer model-successive projection algorithm (SA-SPA) was proposed to select sensitive wavelengths. Then, the spectral resampling method was used to select the best spectral resolution in the corresponding sensitive wavelengths. Finally, the noise-free spectral data simulated by the soil radiative transfer model was added with Gaussian random noise to change the signal-to-noise ratio, so as to explore the influence of signal-to-noise ratio on the estimation accuracy. The research results show that the estimation accuracy obtained through the SA-SPA (RMSEP < 12.1 g kg⁻¹) is generally superior to that from full-spectrum data (RMSEP < 14 g kg⁻¹). At selected sensitive wavelengths, the best spectral resolution is 34 nm, and the applicable signal-to-noise ratio ranges from 150 to 350. This study provides technical support for the efficient estimation of soil moisture content and the development of spectrometry, which comprehensively considers the common influence of spectral range, spectral resolution and signal-to-noise ratio on the estimation accuracy of soil moisture content.

1. Introduction

Soil moisture content seriously affects the physical and chemical properties of soil [1]. The monitoring of soil moisture content plays a decisive role in crop yield estimation, drought monitoring, and evapotranspiration [2,3,4,5]. With the characteristics of high spatial resolution and rich spectral information, spectral remote sensing technology will gradually replace the traditional monitoring methods based on chemical analysis [6,7].

In recent years, spectral remote sensing technology has developed rapidly in China. Spectral remote sensing data plays an indispensable role in soil resources survey, environmental protection and other fields [8]. However, it is found that the parameters such as spectral range, signal-to-noise ratio (SNR) and spectral resolution directly affect the estimation accuracy, and the lack of research on the relationship between spectral remote sensing parameters and estimation accuracy restricts the prolongation of application [9]. Therefore, researchers have carried out the research on the relationship between these parameters and estimation accuracy, and achieved some scientific research results.

Considering the selection method of sensitive wavelength for soil moisture content, some scholars used the correlation analysis method, and other scholars used the variable optimization method (

Table 1. The selection of sensitive wavelengths for soil moisture content.

Method	Sensitive Wavelengths of Soil Moisture Content (nm)	Author
correlation analysis	880, 901–903, 924–929	Song et al., 2009 [10]
correlation analysis	400–410, 1400–1850 and 2050–2220	Yao et al., 2011 [11]
correlation analysis	712, 807, 1142, 1570, 1850, 2221	Zhang et al., 2017 [12]
CARS	443–449, 1408–1456, 1916–1943, 2209–2225	Yu et al., 2016 [13]
wavelet transform and CARS	400–500, 1320–1461, 1851–1961, 2125–2268	Cai et al., 2017 [14]
stepwise regression and grey correlation	1460, 1950 and 2200	Shang et al., 2020 [15]

) [10,11,12,13,14,15].

Only a few scholars have made preliminary research about the influence of spectral resolution on the accuracy of soil spectral analysis. Peng et al. resampled ASD Field Spec 3 spectra with bandwidths from 1 to 10 nm and reported that 9 nm was the best bandwidth for estimating soil organic carbon [16]. Liu et al. took black soil as the research object and pointed out that the best spectral resolution for the inversion of organic matter content in black soil is 50 nm [17]. Adeline et al. demonstrated that good and globally stable clay prediction results were obtained using laboratory spectroscopy with a spectral resolution from 3 to 60 nm [18]. The different methods used in model construction or the different selection of research objects may be the reasons for the differences in the above research results [19].

However, the current research has the following shortcomings: (i) the traditional wavelength selection method requires a large number of measurement data, and its performance is directly related to the quality and representativeness of “training data”; (ii) when determining the best spectral resolution, the spectral sampling interval is large, which is generally from ten nanometers to tens of nanometers; (iii) the influence of SNR on the estimation accuracy of soil moisture content has not been carried out; (iv) the estimation accuracy of soil moisture content is affected by modeling wavelength, spectral resolution and SNR. However, the current research does not take these three factors into account.

In order to solve the above problems, firstly, a method based on the sensitivity analysis of soil radiative transfer model-successive projection algorithm (SA-SPA) was proposed to select the sensitive wavelengths. Then, at the selected sensitive wavelengths, the spectral data were resampled with a 1 nm sampling interval to explore the impact of spectral resolution on the estimation accuracy, so as to determine the best spectral resolution. Finally, the noise-free spectral data simulated by soil radiative transfer model was added with Gaussian random noise to change the SNR. The influence of different SNR values on the estimation accuracy was studied, and the SNR range suitable for soil moisture content measurement was obtained.

2. Materials and Methods

2.1. Soil Sample Preparation and Spectral Measurement

In 2016, black soil samples were collected in Qiqihar (126°40′18.71′′ E, 47°37′18.28′′ N, Hei Long-jiang Province). The soil samples come from the same sampling site, so it is considered that the soil has the uniform soil particle property (i.e., mineral composition, organic matter, nutrients, etc.), ignoring the influence on the reflectance spectra of slight differences in organic matter, etc. The collected soil samples were further air-dried, and crushed to pass through a one-mm sieve. Prior to the rewetting experiment, all of the soil samples were oven-dried at 105 degrees Centigrade for 24 h. Finally, black soil samples were rewetted with 15 different soil moisture levels. For soil samples with different soil moisture content, the data set division method is the concentration gradient method. When establishing the soil moisture content inversion model, root mean square error of prediction (RMSEP), coefficients of determination (R²) and ratio of the performance to deviation (RPD) are used to evaluate the effectiveness and prediction ability of the model.

The hyperspectral reflectance data were acquired in a dark room using an ASD FieldSpec.3 Portable Spectrometer (Analytical Spectral Devices, Boulder, CO, USA) (Figure 1). The incident angle of the light source was 30°. In order to obtain the absolute reflectance, the reflectance was standardized using a white spectralon reference panel. The reflectance of each spectrum was narrowed to 400–2400 nm. The RLOWESS smoothing was to applied to the original reflectance spectra curve in order to eliminate the noise.

2.2. Sensitive Wavelengths Selectioon Method

This study first distinguished the sensitive wavelength intervals of soil moisture through the sensitivity analysis of the soil moisture to soil reflectance in radiative transfer model. Then sensitive wavelengths of soil moisture were ascertained using the SPA.

2.2.1. The Soil Radiative Transfer Model

Our previous studies have constructed an inversion model of soil moisture content, which can be referred to [20]. The soil radiative transfer model is the reverse form of the inversion model, which describes the change of spectral reflectance with soil moisture content.

According to [20], the relationship between transformed reflectance r and soil moisture content ${\theta }_{SM}$ can be expressed as:

$$r({\theta }_{SM})=\frac{r_1(1-{\theta }_{SM})+a({\theta }_{SM}-{\theta }_{SM1})}{1-{\theta }_{SM}}.$$

(1)

where, $r_1=\frac{{\left(1-R_1\right)}^2}{2R_1}$, R₁ is the reflectance of the soil when soil moisture content is ${\theta }_{SM1}$, a is the unknown parameter.

According to the KM theory, the relationship between infinite reflectance $R_{\infty }$ and soil moisture content ${\theta }_{SM}$ is derived as:

$$R_{\infty }({\theta }_{SM})=1+r({\theta }_{SM})-\sqrt{r^2({\theta }_{SM})+2r({\theta }_{SM})}$$

(2)

For wet soil, reflectance, which is related to soil moisture, mainly depends on diffuse scattering. The soil radiative transfer model can be expressed as:

$$\begin{array}{ll}R({\theta }_{SM})& =\frac{{(1-R_i)}^2\cdot R_{\infty }({\theta }_{SM})}{1-R_i\cdot R_{\infty }({\theta }_{SM})}\\ & =\frac{{(1-R_i)}^2\left[1+r({\theta }_{SM})-\sqrt{r^2({\theta }_{SM})+2r({\theta }_{SM})}\right]}{1-R_i\left[1+r({\theta }_{SM})-\sqrt{r^2({\theta }_{SM})+2r({\theta }_{SM})}\right]}\end{array}$$

(3)

With

$$\{\begin{array}{l}r({\theta }_{SM})=\frac{r_1(1-{\theta }_{SM})+a({\theta }_{SM}-{\theta }_{SM1})}{1-{\theta }_{SM}}\\ r_1=\frac{{\left(1-R_1\right)}^2}{2R_1}\end{array}$$

where, $R_i$ is Fresnel reflectance [21]:

$$R_i={\left(\frac{n_{water}-n_{air}}{n_{water}+n_{air}}\right)}^2$$

(4)

where n_water is refractive indices of water (≈1.33) and n_air is refractive indices of air (≈1).

2.2.2. Global Sensitivity Analysis Method—SOBOL

Sobol is a global sensitivity analysis method based on variance decomposition, which quantitatively evaluates the influence of each input parameter and the interaction between the parameters on the output variable by decomposing the variance of the output variable [22]. If $y=f(X_1,X_2,\cdots ,X_m)$ represents the model structure, $X_1,X_2,\cdots ,X_m$ represent the model parameters, and m represents the number of model parameters, the variance decomposition formula can be expressed as:

$$V(y)={\displaystyle \sum _{i=1}^m{V}_i}+{\displaystyle \sum _{i<j<m}^m{V}_{ij}}+{\displaystyle \sum _{i<j<k<m}^m{V}_{ijk}}\cdots +V_{1,2,\cdots ,m}$$

(5)

where: V(y) is the total variance of the model output y; V_i is the variance produced by the parameter X_i; V_ij is the variance produced by the interaction of parameters X_i and X_j; V_ijk is the variance produced by the interaction of parameters X_i, X_j and X_k; V_{1, 2, … , m} is the variance produced by the combined action of m parameters.

For parameter X_i, the first-order sensitivity index S_i can be used to express the direct contribution rate of parameter X_i to the total variance of the model simulation results.

$$S_i=\frac{V_i}{V(y)}$$

(6)

2.3. Spectral Resampling

For spectral remote sensing instruments, the spectral response function of its band generally presents a curve shape similar to Gaussian function. Therefore, the spectral response function can be simulated by Gaussian function [23]. The spectral reflectance after resampling can be expressed as:

$$R_i={\displaystyle {\sum }_{c-t}^{c+t}R(\lambda )}\frac{P_i(\lambda )}{{\displaystyle {\sum }_{c-t}^{c+t}{P}_i(\lambda )}}$$

(7)

$$P_i(\lambda )=\exp [-\frac{{(\lambda -c)}^2}{2{\sigma }^2}]$$

(8)

$$\sigma =\frac{FWHM}{2\sqrt{2\ln 2}}$$

(9)

where, c is the central wavelength of a certain band, λ is the wavelength, R(λ) is the reflectivity at the wavelength λ, and P_i(λ) is the spectral response function, t is the spectral resampling interval (SSI), and spectral resolution (SR) is the full width at half maxima (FWHM) of the spectral response (Figure 2).

2.4. Test Samples with Different SNR

The soil radiative transfer model was used to simulate the reflectance spectra of soil with different soil moisture content. Equation (3) includes an unknown parameter a, in the wavelength range of 400–2400 nm, obtained by using the least square algorithm based on the total set data of black soil. 33 soil reflectance spectra with soil moisture content of 0 to 320 g kg⁻¹ are obtained as test samples (Figure 3).

Because the test spectrum sample was simulated by soil radiative transfer model, it can be considered as noise-free. In this study, the awgn function in MATLAB software was used to add Gaussian random noise to the test spectrum samples, so as to change the SNR (Figure 4).

3. Results

3.1. Descriptive Statistics of Soil Moisture

This study used the concentration gradient method to divide 15 soil samples with different moisture content levels into a calibration set and a validation set. The statistical description of soil moisture contents for black soil is shown as

Table 2. Statistical description of soil moisture contents for black soil.

Dataset	Number	Maximum (g kg−1)	Minimum (g kg−1)	Average (g kg−1)	Standard Deviation (%)
Total set	15	240	0	142.1	66.2
Calibration set	10	240	0	146.0	70.1
Validation set	5	200	6	132.5	64.0

. The maximum, minimum, average, and standard deviation of the total set, the calibration set and the validation set are similar, which shows that the calibration set and the validation set are well divided to represent the total set.

3.2. Sensitive Wavelengths of Soil Moisture

The first-order sensitivity index of input parameters to soil reflectance in soil radiative transfer model was calculated by Equation (6), wavelength by wavelength, in the wavelength range of 450–2500 nm. The variation of the first-order sensitivity index with wavelength is shown in Figure 5a,b is an enlarged detail of Figure 5a. The sensitive wavelength ranges of soil moisture content are 853–1125 nm and 1900–1988 nm, and the influence of soil moisture in this range is significantly stronger than the other parameter.

Next, the SPA was used to further select sensitive wavelengths. The results are shown in

Table 3. Sensitive wavelengths of soil moisture.

Wavelength (nm)	RMSEP (g kg−1)
1010	12.1
1068	11.6
1915	10.4
1946	11.9
1988	12.1

. The RMSEP of soil moisture content at the selected five wavelengths (1010 nm, 1068 nm, 1915 nm, 1946 nm and 1988 nm) is between 10.4 g kg⁻¹ and 12.1 g kg⁻¹, indicating that the selected five wavelengths have excellent performance, especially 1915 nm. In addition, the estimation accuracy obtained through the SA-SPA (RMSEP < 12.1 g kg⁻¹) is generally superior to that from full spectrum data (RMSEP < 14 g kg⁻¹) [20].

In order to further verify the effectiveness of the SA-SPA method, this study compared and analyzed the performance of wavelengths selected by SA-SPA, ICO-SPA, MWSE-CARS-SPA and MWSE-GA-SPA. The results are shown in

Table 4. Comparison results between SA-SPA and 3 traditional empirical wavelength selection methods.

Method	Number	RMSEP (g kg−1)
SA-SPA	5	12.1
ICO-SPA	2	13.1
MWSE-CARS-SPA	8	12.7
MWSE-GA-SPA	8	12.7

. The comparison results show that there is no significant difference in the number of wavelengths selected by the four methods, and the number of wavelengths selected are 5, 2, 8 and 8 respectively. The RMSEP of SA-SPA method is 12.1 g kg⁻¹, that of ICO-SPA method is 13.1 g kg⁻¹, that of MWSE-CARS-SPA method is 12.7 g kg⁻¹, and that of MWSE-GA-SPA method is 12.7 g kg⁻¹. Therefore, the performance of the wavelength selected by SA-SPA method is better than the other three methods.

3.3. The Best Spectral Resolution of Soil Moisture

The influence of different spectral resolutions on the estimation accuracy of soil moisture content was explored. The reflectance spectra of soils with different moisture contents were resampled at sensitive wavelengths with a sampling interval of 1 nm. Next, based on the soil moisture content inversion model derived in [20], the soil moisture content was estimated by using the spectral data with spectral resolution set to 1 nm, 2 nm, 3 nm, 4 nm, 5 nm, …, 100 nm, respectively, so as to obtain the best spectral resolution for soil moisture content inversion. As shown in Figure 6, the RMSEP of soil moisture content first decreases and then increases with the increase of spectral resolution. The best spectral resolution for soil moisture content inversion is 34 nm.

3.4. Suitable SNR Range for Soil Moisture

The influence of different SNR on the estimation accuracy of soil moisture content was explored when the spectral resolution is 34 nm in the sensitive wavelengths of soil moisture content (1010 nm, 1068 nm, 1915 nm, 1946 nm and 1988 nm). In Figure 7a, the black spots show the change of the RMSEP with the SNR. It can be seen that the RMSEP decreases with the increase of the SNR. The relationship between the RMSEP and the SNR is fitted by using the MATLAB curve fitting toolbox.

In order to further determine the change of the decline rate of the RMSEP with the SNR, the derivative analysis of the fitted relationship was carried out. It can be seen from Figure 7a,b that when the SNR is less than 150, the RMSEP decreases rapidly with the SNR; when the SNR ranges from 150 to 350, the RMSEP decreases slowly with the SNR; when the SNR is greater than 350, the decline rate of the RMSEP with the SNR tends to 0, indicating that the SNR’s influence on the RMSEP is no longer significant. Therefore, in the sensitive wavelengths of soil moisture content (1010 nm, 1068 nm, 1915 nm, 1946 nm and 1988 nm), when the spectral resolution is 34 nm, the applicable SNR range is 150–350.

4. Discussion

The technology for selecting feature wavelengths can effectively eliminate the phenomenon of spectral information redundancy caused by the large amount of soil hyperspectral data, and it can improve the stability and prediction ability of the soil moisture content inversion model [24,25,26,27]. Since the wavelength selection algorithm proposed in this study is based on the soil radiative transfer model and verified by experimental sample data, the wavelengths selected in this study are effective for large-scale prediction of soil moisture content. In addition, compared with the traditional empirical wavelength selection method [28,29,30,31,32], the wavelength selection algorithm based on soil radiative transfer model improves the applicability and accuracy of wavelength selection results. However, this method also has limitations: the experimental samples with different soil moisture content in this study are black soil. When detecting other specific types of soil, other wavelength combinations may be more effective than the wavelength combinations introduced in this study.

Figure 6 shows that the RMSEP of soil moisture content decreases first and then increases with the increase of spectral resolution, the best spectral resolution is 34 nm. Therefore, it is not that the higher the spectral resolution, the higher the estimation accuracy of soil moisture content. This is because the spectral characteristics of soil moisture content of wide and sharp absorption peaks are not required. However, when using hyperspectral data for quantitative estimation of soil component content, due to the large correlation between adjacent bands, there are problems of information redundancy and overlap in hyperspectral data, which cannot achieve the expected estimation accuracy [18]. At this time, we need to resample hyperspectral data. It is particularly important to explore the best spectral resolution for quantitative estimation of soil component content.

As an important performance parameter of spectral remote sensing instruments, SNR reflects the detection ability of spectrometer for weak signals. Only spectral data with sufficiently high SNR can fully play its role in spectral quantitative remote sensing. Therefore, in the development of spectral instruments, it is necessary to trade off among structural parameters, cost and analysis accuracy in order to obtain the best spectral analysis scheme. The applicable SNR ranges from 150 to 350, which shows that it is not necessary to pursue high SNR when designing spectral instruments to monitor soil moisture content in the future.

5. Conclusions

In this study, the application requirements were taken as the foothold for developing spectrometry. Based on the soil moisture content inversion model previously constructed by the author and its reverse form, the sensitive wavelengths of soil moisture content, the best spectral resolution and the applicable SNR range were selected. This paper comprehensively considered the common influence of spectral range, spectral resolution and SNR on the estimation accuracy of soil moisture content. Firstly, the sensitive wavelengths of soil moisture were selected based on SA-SPA. Then the spectral resampling method was used to select the best spectral resolution in the corresponding sensitive wavelengths. Finally, under the selected sensitive wavelengths and spectral resolution, the SNR range suitable for soil moisture content inversion was explored. The research results show that the sensitive wavelengths of soil moisture content are 1010 nm, 1068 nm, 1915 nm, 1946 nm and 1988 nm. At the sensitive wavelengths, the best spectral resolution is 34 nm, and the applicable SNR ranges from 150 to 350. This study provides technical support for the efficient estimation of soil moisture content and the development of spectrometry.

This study solves the following problems:

• A joint selection method of sensitive wavelength, optimal spectral resolution and signal-to-noise ratio of soil moisture content is proposed, which comprehensively considers the joint influence of spectral wavelength, spectral resolution and signal-to-noise ratio on the inversion accuracy, and solves the problem that only a certain spectral remote sensing parameter affects the inversion accuracy and ignores its joint influence.
• The proposed sensitive wavelength selection method overcomes the problems that the traditional wavelength selection methods use a large amount of data, and the sensitive wavelength selection results are directly related to the data quality and representativeness.
• When exploring the influence of spectral resolution on inversion accuracy, the spectral sampling interval is set to 1 nm to avoid the problem of large spectral sampling interval and poor research on the influence of spectral resolution on inversion accuracy.

This study only preliminarily uses the laboratory measurement data to explore the effects of spectral band, spectral resolution and signal-to-noise ratio on the inversion accuracy of soil water content. The next research work is to deeply study the influence mechanism of spectral band, spectral resolution, signal-to-noise ratio and spatial resolution on inversion accuracy, and carry out remote sensing index parameter verification experiment combined with unmanned aerial vehicle.