Soil Moisture-Derived SWDI at 30 m Based on Multiple Satellite Datasets for Agricultural Drought Monitoring

Abstract

As a major agricultural hazard, drought frequently occurs due to a reduction in precipitation resulting in a continuously propagating soil moisture (SM) deficit. Assessment of the high spatial-resolution SM-derived drought index is crucial for monitoring agricultural drought. In this study, we generated a downscaled random forest SM dataset (RF-SM) and calculated the soil water deficit index (RF-SM-SWDI) at 30 m for agricultural drought monitoring. The results showed that the RF-SM dataset exhibited better consistency with in situ SM observations in the detection of extremes than did the SM products, including SMAP, SMOS, NCA-LDAS, and ESA CCI, for different land cover types in the U.S. and yielded a satisfactory performance, with the lowest root mean square error (RMSE, below 0.055 m³/m³) and the highest coefficient of determination (R², above 0.8) for most observation networks, based on the number of sites. A vegetation health index (VHI), derived from a Landsat 8 optical remote sensing dataset, was also generated for comparison. The results illustrated that the RF-SM-SWDI and VHI exhibited high correlations (R ≥ 0.5) at approximately 70% of the stations. Furthermore, we mapped spatiotemporal drought monitoring indices in California. The RF-SM-SWDI provided drought conditions with more detailed spatial information than did the short-term drought blend (STDB) released by the U.S. Drought Monitor, which demonstrated the expected response of seasonal drought trends, while differences from the VHI were observed mainly in forest areas. Therefore, downscaled SM and SWDI, with a spatial resolution of 30 m, are promising for monitoring agricultural field drought within different contexts, and additional reliable factors could be incorporated to better guide agricultural management practices.

1. Introduction

Driven by the imbalance between rainfall and evapotranspiration, drought is a natural hazard associated with a lack of water resources [1], threatening the ecological environment, economic development, and even human existence [2]. Under the threat of global warming, the frequency and severity of drought have increased since the second half of the 20th century [3,4,5]. With the widespread effects of drought, agricultural systems are facing significant and intense shocks in both developed and developing countries [6,7,8,9]. Therefore, accurate monitoring and assessment of agricultural drought are of primary importance for promoting food security and social stability [10,11,12].

In the 20th century, academics in the U.S. initiated research on drought monitoring based on rainfall observations [13,14,15]. The proposal of the “Palmer index” [16] was a major milestone in drought monitoring research (Liu et al., 2016). The index collectively refers to three indices, namely the Palmer drought severity index (PDSI) [17,18,19], the Palmer hydrological drought index (PHDI) [20,21,22] and the Palmer Z index (ZIND) [23,24,25], and it has been widely applied in drought monitoring and assessment across various countries and fields. In the late 20th century, notable progress was achieved in the development of satellite-based technology for the spatiotemporal monitoring of drought [26], which benefits from remote sensing, a mainstream Earth observation tool that provides abundant contextual data and key variables related to drought occurrence mechanisms [27,28,29]. For instance, the evaporative drought index (EDI) is commonly used for drought monitoring based on the quantification of evapotranspiration anomalies in time series [30]; the normalized difference vegetation index (NDVI) and enhanced vegetation index (EVI) can effectively indicate drought conditions through vegetation growth conditions or productivity obtained by multispectral sensors [31,32,33]; and the standardized soil moisture index (SSI) and soil water deficit index (SWDI) can be used to assess the water stress in soil layers to characterize the intensity, frequency, and distribution of drought over a large geographic area [34,35,36]. With the focus and research of international organizations and domestic and foreign scholars regarding agricultural drought in recent years, the relevant monitoring indicators have become increasingly abundant [11]. The vegetation health index (VHI) quantifies agricultural drought stress using the function of crop water demand, and its ability to monitor drought has been recognized by many studies [37,38,39].

The disruption of the surface water balance due to precipitation deficits is the dominant factor in the uncertainty about changes and the evolution of drought [40]. This process is often manifested as agricultural drought reflected by soil moisture (SM) drought, in a narrow sense [2,11]. Therefore, the acquisition of a reliable SM dataset is the premise of drought monitoring at the field scale. Since the beginning of the 21st century, active aperture radar tools, such as the Advanced Scatterometer (ASCAT) [41], and passive microwave radiometers, such as the Soil Moisture Active Passive (SMAP) [42] and Soil Moisture and Ocean Salinity (SMOS) sensors [43], have been successfully launched, providing the continuous monitoring of global SM levels. Based on these single remote sensing SM datasets, more combination products and data assimilation products, such as the European Space Agency Climate Change Initiative (ESA CCI), or the longer time series [44] Global Land Data Assimilation System (GLDAS) [45] and the European Center for Medium-Range Weather Forecasting (ECMWF) system [46], have also been produced to meet the diversified needs of SM applications for generating more reliable SM datasets [47,48]. Nevertheless, the spatial resolution of these SM products reaches the level of tens of kilometers, which could be most commonly used for monitoring extensive drought trends and their evolution at continental and even global scales [49,50].

Surface SM, as a major factor limiting crop growth [51,52], can be applied to two types of indices, according to the emphasis on agricultural drought monitoring: a univariate statistical index based on time-series data and a comprehensive index based on soil hydraulic parameters [53,54]. The univariate statistical indices used for drought monitoring are based on the long-term average or percentile of SM data to determine whether the current SM level is normal; these indices include the soil moisture anomaly (SMA) [55], soil moisture percentile (SMP) [56], soil moisture deficit index (SMDI) [57], and SSI. Zeri et al. (2022) calculated the SMA based on simulated SM data for the 1979–2018 period over the Brazilian semiarid region and reported that this indicator exhibited notable negative values during El Niño years, performing similarly to the standard precipitation index (SPI) in identifying drought events during the rainy and cropping seasons [58]. Wambua (2019) analyzed the spatiotemporal characterization of agricultural drought by using SMDI based on hydrometeorological data from 1970 to 2010 at eight stream flow gauge stations in the upper Tana River basin and found an increasing rate in risk and severity of drought at high altitudes and during the dry seasons [59]. Zhang et al. (2021) evaluated the duration, severity, and frequency characteristics of drought in China from 1979 to 2014 according to the SSI using a grid derived from GLDAS and ESA CCI products, and the results suggested that the two SM datasets exhibited significantly correlated (p < 0.05) results over 70% of the regions, realizing the precise monitoring of extreme drought events reported around the country [60]. However, these indices can only indicate SM anomalies over a long time series and impose strict requirements on the quality and volume of the data. More importantly, these studies did not consider the physical mechanism involved in the drought process, resulting in a failure to explain the state of agricultural drought from a crop water stress perspective [61].

Comprehensive indices, including the soil moisture index (SMI) and SWDI, can be used to monitor drought conditions within the framework of soil available water by incorporating soil water dynamic parameters. Kędzior and Zawadzki (2017) mapped the spatial distributions of the SMI based on SMOS data across the Vistula catchment in Poland from 2010 to 2014, demonstrating the ability of the SMI to assess the risk of agricultural drought [62]. The SWDI, which is based on the field water capacity and wilting point, fully captures the relationship between SM and the plant physiological state and overcomes the limitation of the length of SM data time series; thus, the SWDI is more promising for describing short-term agricultural drought conditions [63,64]. Mishra et al. (2017) computed the SWDI derived from SMAP data and subsequently compared it to the in situ atmospheric water deficit (AWD) index, revealing high agreement between these two indices for drought monitoring across the U.S [65]. Martínez-Fernández et al. (2016) calculated the SWDI derived from SMOS products from 2010 to 2014 in Spain based on different approaches to obtain soil water dynamic parameters (including a long time series of SM data, in situ data parameters, and pedotransfer function estimates), demonstrating their ability to reflect SM balance dynamics and track agricultural drought via a comparison between the crop moisture index (CMI) and AWD [35]. However, previous studies on the SWDI have commonly been based on kilometer-scale SM products, which homogenize heterogeneous soil and land cover types within the same grid and fail to provide a detailed reference for monitoring agricultural field drought [66]. Zhu et al. (2019) calculated the SWDI derived from the L-band radiometer SMAP to monitor agricultural drought in the Xiang River Basin of China [67]. Although a relatively accurate distribution and temporal evolution of drought in the study area were obtained, the complex variations affected by surface heterogeneity were difficult to depict on a finer scale due to the limitation of the 36 km coarse spatial resolution [68,69].

The spatial resolution of SM products in agricultural application is required to be in the tens to hundreds of meters [70]. Global microwave SM products display coarse grids of tens of kilometers, making it difficult for them to provide detailed spatial variations of SM at local or even field scales [66]. Fang et al. (2021) used the SWDI and SMDI based on 1 km downscaled SMAP SM data to monitor drought in Australia, and the resulting maps not only showed the occurrence of drought in detail in the Murray–Darling River Basin, but also tracked the deterministic factors of the drought conditions [71]. Therefore, it is more beneficial to accurately analyze the influences of geographic factors [72] on agricultural drought events based on a surface SM dataset with a high spatial resolution. There are three kinds of SM downscaling methods, including satellite-based methods [73,74], geoinformation-based methods [75,76], and machine learning-based methods [77], according to an overview of previous studies. The complex nonlinear relationship between land-surface variables and SM can be described comprehensively by using machine learning methods [78]. Furthermore, the downscaling model, developed based on the use of an excellent algorithm, makes it possible to obtain a surface SM dataset at the tens of meters scale, thus achieving fine agricultural drought monitoring.

In this study, we monitored agricultural drought based on surface SM data with a spatial resolution of 30 m. The main objectives were threefold: (1) produce the surface SM dataset based on the random forest algorithm (RF-SM) and evaluate its performance using observational records from in situ SM stations and satellite-based SM products; (2) calculate the RF-SM-based agricultural drought index (RF-SM-SWDI) and compare it to the VHI derived from the optical remote sensing satellite observations; and (3) analyze the distribution and temporal evolution of drought in the substudy area in 2016.

2. Materials and Methods

2.1. In Situ SM Dataset

The International Soil Moisture Network (ISMN), established by the Vienna University of Technology and managed by the International Center for Water Resources and Global Change (ICWRGC) of the Federal Institute of Hydrology (FIH) of Germany, is a centralized data-hosting facility created for the calibration of remote sensing products and the validation of land surface models [79]. The ISMN provides the harmonized fractional volumetric SM (m³/m³), while preserving the measurement methods and depths used by the networks [80]. To obtain reliable surface SM, data from all networks and sites must be filtered by measuring the depth (0–5 cm) and then applying quality flags (“G”, which denotes good). With the use of the ISMN dataset (https://ismn.geo.tuwien.ac.at (accessed on 15 June 2023)), we selected 242 stations from five networks (i.e., 81 stations from SCAN, 60 stations from SNOTEL, 46 stations from SOILSCAPE, 53 stations from USCRN, and 2 stations from iRON) distributed across the U.S., based on the availability of in situ data. The locations of these stations, which maintain long-term hourly records of SM observation, are shown in Figure 1. To develop the downscaling framework of surface SM, all stations were divided into two parts: 170 stations were used as training models, and 72 stations were used for testing the performance of the method.

Moreover, we selected a substudy area with a size of 30 km × 30 km, located in California, to analyze the spatial variability of drought distribution, thus demonstrating the advantages of fine-scale drought monitoring. The substudy area covers most of western Amador County, portions of eastern Sacramento County and southern EI Dorado County, and smaller portions of northern San Joaquin County and Calaveras County. The stations in the substudy area are covered by the SOILSCAPE network and are mainly distributed in shrubland areas.

2.2. Optical Remote Sensing Dataset

The reflectance dataset derived from Landsat 8 was downloaded and processed using the Google Earth Engine (GEE) platform. The Landsat 8, Collection 2, Level-2 dataset includes complete preprocessing and definite information on cloud states, and it is widely used in land surface analysis and application. In this study, we obtained and computed the data and imagery from the red band, near-infrared band, two shortwave-infrared bands, and the surface temperature band, with a spatial resolution of 30 m, from the year 2016.

2.3. Soil Properties and Meteorological Dataset

Originally released as a “proof of concept” by the International Soil Reference Information Center (ISRIC), SoilGrids V2.0 was designed and refined by Hengl et al. (2017) and Poggio et al. (2021) to form a mature global standard soil profile dataset at a 250 m grid resolution, namely [81,82]. In this study, we obtained the mean clay, sand, silt, and organic matter contents at a depth of 5 cm from SoilGrids V2.0 (https://soilgrids.org (accessed on 10 November 2023)) and rescaled the data to 30 m using the bilinear interpolation algorithm.

ERA5 is a reanalysis dataset, released by the European Center for Medium-Range Weather Forecasts (ECMWF), that uses a four-dimensional variational (4D-var) assimilation scheme to produce data for atmospheric variables at 139 pressure levels, with a horizontal resolution of 31 km at a 1 h time step [83]. In this study, we obtained the data and imagery of the atmospheric temperature at 2 m, the 24 h total precipitation, the relative humidity, and the wind at 10 m for the year 2016 from ERA5 (https://cds.climate.copernicus.eu (accessed on 7 November 2023)).

2.4. Soil Moisture Products

The SMAP is a satellite mission launched by the National Aeronautics Space Administration (NASA) in 2015 to retrieve global SM levels based on the brightness temperature [84]. There are different levels of SMAP products, and at Level 3, which serves as a daily composite, the surface SM within the top 5 cm is retrieved, based on the brightness temperature derived from a passive L-band radiometer [85]. In this study, we obtained SMAP passive Level-3 product data and imagery with a grid resolution of 36 km for 2016 from the National Snow and Ice Data Center (https://nsidc.org/data/smap (accessed on 11 November 2023)).

The SMOS is a satellite mission launched by the European Space Agency (ESA) in 2009 specifically dedicated to obtaining high-accuracy global surface SM and sea surface salinity measurements [86]. Based on the L-band Microwave Emission of the Biosphere (L-MEB) model, SMOS-INRA-CESBIO (SMOS-IC) simplifies complex parameters and completes computations using the heterogeneous pixels to minimize the dependence on auxiliary information [87]. In this study, we obtained SMOS-IC data and imagery with a grid resolution of 25 km for 2016 from the Centre Aval de Traitement des Données SMOS (https://www.catds.fr (accessed on 14 November 2023)).

The NCA-LDAS is a data assimilation system developed by the U.S. Global Change Research Program (USGCRP) using the NASA Land Information System (LIS) in the North American Land Data Assimilation System (NLDAS) configuration [88]. The product including the environment data records from 1979 to the present was generated by applying the uncoupled Noah land surface model at 0.125° × 0.125° across the continental U.S. [89]. In this study, we obtained data and imagery of the “SoilMoist0_10cm” layer of the daily NCA-LDAS Noah-3.3 LSM with a spatial resolution of 0.125° for 2016 from the NASA Land Data Assimilation System (https://ldas.gsfc.nasa.gov/nca-ldas (accessed on 2 December 2023)).

The ESA CCI SM dataset is a multidecadal SM dataset derived from global satellite observations released by the ESA for the Climate Change Initiative (CCI) program [44]. The product harmonizes a merged active dataset, a merged passive dataset, and a combined active–passive dataset by integrating various single-sensor-derived SM products based on the microwave backscatter or brightness temperature [90]. In this study, we obtained combined ESA CCI V.06.1 data and imagery with a spatial resolution of 0.25° for 2016 from the Center for Environment Data Analysis (https://data.ceda.ac.uk (accessed on 20 November 2023)).

2.5. Methods

2.5.1. Surface Soil Moisture Derived from a Downscaling Framework (RF-SM)

The downscaling framework for surface SM was established based on variables obtained from multiple satellite datasets, and the random forest (RF) algorithm was applied to deconstruct SM to a fine scale at a 30 m spatial resolution (Figure 2). The framework integrates four datasets into the model to accurately retrieve SM (

Table 1. The datasets used in the downscaling framework.

Datasets	Details	Description	Variables	Spatial Resolution	Temporal Resolution
In situ dataset	ISMN	In situ SM	SM	Point	Hourly
Optical remote sensing dataset	Landsat 8 surface reflectance	Red band	SR_b4	30 m	16-day
		NIR band	SR_b5
		SWIR band	SR_b6
		SWIR band	SR_b7
		ST band	ST_b10
Soil properties dataset	SoilGrids V.2.0	Clay content	Clay	250 m	Static
		Sand content	Sand
		Silt content	Silt
Meteorological dataset	ERA 5	Relative humidity	RH	0.25°	3 h
		Atmospheric temperature	T2m
		Wind	Wind
		Precipitation	Precipitation
Soil moisture products	SMAP	Satellite-derived	SM	36 km	Daily
	SMOS-IC	Satellite-derived	SM	25 km	Daily
	NCA-LDAS	Model-derived	SM	0.125°	Daily
	ESA CCI	Satellite-derived	SM	0.25°	Daily

). The first dataset includes reflectance data from the red, near-infrared (NIR), shortwave infrared (SWIR), and surface temperature (ST) bands from high spatial-resolution optical remote sensing, Landsat 8 OLI, and TIRS Level-2 images. Soil properties from SoilGrids V.2.0, including clay, sand, and silt, comprise the second dataset, which impact SM dynamics by influencing water infiltration. The third dataset indirectly reflects SM variation under the influence of land–air interactions and consists of meteorological data from ERA5, namely, atmospheric temperature (T2m), precipitation, relative humidity (RH), and wind. The last dataset comprises SMAP, SMOS, ESA CCI, and NCA-LDAS, four coarse-resolution SM products, and it plays a leading role in the model. In this study, an RF model was developed using in situ observations from the ISMN. The optimal parameters were obtained through the fivefold cross-validation method, based on the records from training stations and determined by accuracy control based on the records from testing stations. The RF-SM dataset was eventually generated for computing the SWDI.

2.5.2. Soil Water Deficit Index (SWDI)

Based on the effect of SM on agricultural drought, the SWDI accounts for water deficit accumulation and soil water storage [64] and has the potential to capture short-term SM variations and track drought dynamics [35]. In this study, we applied the SWDI based on surface SM, which has been proven to be a viable agricultural drought monitoring strategy [35,65]. The SWDI can be calculated as follows:

$$SWDI=\left({\displaystyle \frac{\theta -{\theta }_{FC}}{{\theta }_{AWC}}}\right)\times 10$$

(1)

$${\theta }_{AWC}={\theta }_{FC}-{\theta }_{WP}$$

(2)

where $\theta$ denotes SM; ${\theta }_{FC}$ denotes the field water capacity, which is the maximum water capacity that the soil can maintain to be effective for crops; ${\theta }_{AWC}$ denotes the available water capacity, which refers to the difference between ${\theta }_{FC}$ and ${\theta }_{WP}$ (wilting point) and can be calculated using Equation (2). In addition, the SWDI was proposed on the basis of the concept of the water deficit index [91], which can be multiplied by 10 to convert this fractional magnitude to a range of values with agricultural meaning [64].

The most critical issue in deriving the SWDI is the determination of the parameters ${\theta }_{FC}$ and ${\theta }_{WP}$. However, these data are typically unavailable in standard soil databases. To address this issue, there are generally three ways for researchers to obtain or estimate ${\theta }_{FC}$ and ${\theta }_{WP}$, i.e., using the SM time series [92,93], employing laboratory analysis [35], or through pedotransfer functions (PTFs) [94]. In fact, the first method requires SM data with a long enough time series to ensure the accuracy of these parameters, while the second method consumes large amounts of manpower and time. Therefore, we applied regression analysis-based PTFs to estimate ${\theta }_{FC}$ and ${\theta }_{WP}$ by using a soil property dataset. On the basis of previous research and prior achievements [65,95], the PTF regression equation developed by Saxton and Rawls (2006), using the soil physical properties (i.e., $clay$ and $sand$) and organic matter ($OM$) as independent variables, was applied in this study [96]. The predictive equations for these hydraulic parameters (i.e., ${\theta }_{FC}$ and ${\theta }_{WP}$) can be calculated as follows:

$${\theta }_{WP}={{\theta }_{WP}}^{*}+(0.14\times {{\theta }_{WP}}^{*}-0.02)$$

(3)

$${{\theta }_{WP}}^{*}=-0.024\times sand+0.487\times clay+0.006\times OM+0.005\left(sand\times OM\right)-0.013\times \left(clay\times OM\right)+0.068\times \left(sand\times clay\right)+0.031$$

(4)

$${\theta }_{FC}={{\theta }_{FC}}^{*}+\left[1.283\times {\left({{\theta }_{FC}}^{*}\right)}^2-0.374{{\times \theta }_{FC}}^{*}-0.015\right]$$

(5)

$${{\theta }_{FC}}^{*}=-0.251\times sand+0.195\times clay+0.011\times OM+0.006\left(sand\times OM\right)-0.027\times \left(clay\times OM\right)+0.452\times \left(sand\times clay\right)+0.299$$

(6)

2.5.3. Vegetation Health Index (VHI)

The VHI was first proposed by Kogan (1997), it and combines spectral and infrared channels to characterize the vegetation stress related to water and temperature during drought [97]. The VHI is derived from an integration of the vegetation condition index (VCI) and the temperature condition index (TCI) [98] and can be expressed by merging these, as defined in Equation (7), where $\alpha$ denotes the contribution of these two components to the total vegetation health.

$$VHI=\alpha \times VCI+(1-\alpha )\times TCI$$

(7)

The VCI represents the vegetation growth state based on the moisture conditions in the visible and near-infrared bands and can be calculated as follows:

$$VCI=100\times {\displaystyle \frac{({NDVI}_i-{NDVI}_{min})}{({NDVI}_{max}-{NDVI}_{min})}}$$

(8)

$${NDVI}_i={\displaystyle \frac{{NIR}_i-{RED}_i}{{NIR}_i+{RED}_i}}$$

(9)

where ${NDVI}_i$ denotes the normalized difference vegetation index of pixel $i$ and can be calculated by the reflectance in the red band (${RED}_i$) and near-infrared band (${NIR}_i$) (Equation (9). ${NDVI}_{min}$ and ${NDVI}_{max}$ denote the minimum and maximum values of ${NDVI}_i$ in 2016, respectively. The TCI represents the surface temperature conditions in the thermal infrared band and can be calculated as follows:

$$TCI=100\times {\displaystyle \frac{({LST}_{max}-{LST}_i)}{({LST}_{max}-{LST}_{min})}}$$

(10)

where ${LST}_i$ denotes the land surface temperature of pixel $i$, and ${LST}_{max}$ and ${LST}_{min}$ are the minimum and maximum values of ${LST}_i$ in 2016, respectively.

To ensure the consistency of the data, the Landsat 8, Collection 2, Level-2 dataset was used as the basic information to calculate the NDVI and LST in this study. In addition, the VCI and TCI were assigned equal contributions (i.e., α = 0.5) [99]

2.5.4. Evaluation Methods

(1)

Model Evaluation Methods

To evaluate the performance of the RF-SM dataset, the coefficient of determination (R²), root mean square error (RMSE), bias (Bias), and Kling–Gupta efficiency (KGE) [100] were adopted as accuracy metrics. Each statistical indicator can be expressed as:

$$R^2={\displaystyle \frac{{\left[{\sum }_{i=1}^N\left(S_i-\overline{S}\right)\left(S_i-\overline{O}\right)\right]}^2}{{\sum }_{i=1}^N{\left(S_i-\overline{S}\right)}^2{\sum }_{i=1}^N{{(O}_i-\overline{O})}^2}}$$

(11)

$$RMSE=\sqrt{{\displaystyle \frac{1}{N}}{\sum }_{i=1}^N{\left(S_i-O_i\right)}^2}$$

(12)

$$Bias={\displaystyle \frac{1}{N}}{\sum }_{i=1}^N\left(S_i-O_i\right)$$

(13)

where $N$ is the sample size, and $O_i$ and $S_i$ are the observed and simulated values of the $i$th sample, respectively. $\overline{O}$ and $\overline{S}$ are the average values of the observations and simulations, respectively, for the corresponding sample sets.

The KGE integrates the correlation ($r$), relative variability ($\alpha$, Equation (15)), and ratio of the mean ($\beta$, Equation (16)) to provide a comprehensive model evaluation indicator (Equation (14)). ${\sigma }_O$ and ${\sigma }_S$ denote the standard deviations of the observations and simulations, respectively, for the corresponding sample sets. ${\mu }_O$ and ${\mu }_S$ denote the mean values of the observations and simulations, respectively, for the corresponding sample sets. The KGE is a measure of the Euclidean distance from a given point to the optimal point. Thus, the value of all three components of the KGE is equal to 1 under ideal conditions and without simulation errors.

$$KGE=1-\sqrt{{(r-1)}^2+{(\alpha -1)}^2+{(\beta -1)}^2}$$

(14)

$$\alpha ={\displaystyle \raisebox{1ex}{${\sigma }_S$}\!\left/ \!\raisebox{-1ex}{${\sigma }_O$}\right.}$$

(15)

$$\beta ={\displaystyle \raisebox{1ex}{${\mu }_S$}\!\left/ \!\raisebox{-1ex}{${\mu }_O$}\right.}$$

(16)

(2)

Evaluation of the Drought Indices SWDI and VHI

As indicated in Equation (1), when the SWDI is 0 or greater, the SM content completely meets or even exceeds the field water capacity, and the soil exhibits no water deficit. A negative SWDI value indicates that the soil is under water stress, and once the value is below −10, the SM content dips below the lower limiting value of the available water capacity [101]. At this point, crops cannot absorb enough water from the soil to compensate for the transpiration loss, resulting in wilting of the leaves.

The VCI, TCI, and VHI are indices that represent the moisture, temperature, and vegetation health conditions, respectively, on a scale of 0 to 100. Normally, a lower value indicates higher stress and poorer vegetation health.

In this study, the following classification system was used for assessing agricultural drought conditions (

Table 2. SWDI and VHI classifications used in this study for different drought severities.

SWDI Value	VHI Value	Drought Severity
≤−10	0 to 10	Extreme drought
−10 to −5	10 to 20	Severe drought
−5 to −2	20 to 40	Moderate drought
−2 to 0	40 to 60	Mild drought
≥0	60 to 100	No drought

). The SWDI was adapted from Martínez-Fernández et al. (2015) [64], and the VHI was obtained from Kogan (2002) [102].

3. Results

3.1. Evaluation of the RF-SM Dataset

In this study, we used SM records from 242 observation stations for model development and evaluation, based on the downscaling framework, to obtain RF-SM data. To evaluate the performance of the RF-SM data, we further verified the different SM products from two aspects at the point scale. The SM data from in situ stations were compared with the RF-SM data and the estimations of these four SM products from the corresponding pixels.

Figure 3 shows scatterplots between the RF-SM predictions and the in situ SM data. The samples used to develop the model mostly exhibited low SM levels, resulting in concentrations along the 1:1 line. We trained the SM downscaling framework based on the RF algorithm, using available SM records from 170 in situ stations. Figure 3a shows that the model exhibited favorable performance, with a high KGE value (0.85) and a low Bias value (0.001 m³/m³). The model was then validated based on in situ SM from the remaining 72 observation stations. Figure 3b shows that the RF-SM dataset yielded relatively accurate results, with a KGE value of 0.72 and a Bias value of 0.028 m³/m³, compared to the SM records from the observations. In addition, the downscaling framework might be slightly overestimated at low SM levels and slightly underestimated at high SM levels within the constraints of the sample distribution.

Figure 4 shows the permutation importance of RF-SM. The SM products achieved higher importance scores for all the input variables, with NCA-LDAS scoring the highest. The variables derived from the soil properties and the Landsat 8 dataset displayed moderate importance scores, and the meteorological reanalysis dataset exhibited the lower importance scores, except in the case of RH. This result suggests that coarse-scale SM products are important features in RF-SM, and that fine-scale soil properties contribute more significantly to the model than does surface reflectance. It is difficult to achieve a satisfactory model prediction by relying on a single dataset, while the integration of multiple auxiliary variables will generate reliable SM results.

We then described the distributions of SM derived from RF-SM, SMAP, SMOS, NCA-LDAS, ESA CCI, and ground observations under six land cover types using boxplots. Figure 5 shows lower SM values in the barren and shrubland types and higher SM values in the cropland and forest types. Among the types with low SM values, the four products exhibited specific errors and even produced overestimated values that significantly deviate from the observations, while the median and extreme SM values derived from the RF-SM dataset agree better with those derived from the in situ dataset. Among the types with high SM values, the RF-SM distribution also remained stable and concentrated, with fewer extreme values and results closer to the observed median. Compared with the in situ observations, the RF-SM data achieved better consistency in terms of the data range and distribution, especially for the grassland and savanna types.

We also compared the statistics (R², RMSE, Bias, and KGE) of SM derived from the RF-SM dataset and the four SM products based on the in situ stations of five observation networks to evaluate the reliability of the dataset. Figure 6 shows that all the SM datasets exhibited the worst performance in the iRON network, with the lowest R² (from 0.01 to 0.34, $p$ < 0.01) and the highest Bias values (from −0.086 to 0.094 m³/m³). In the SOILSCAPE network, the RF-SM dataset and the four SM products all exhibited their best performance, with the highest KGE (from 0.14 to 0.86) and the lowest RMSE value (from 0.037 to 0.114 m³/m³). In addition, SM levels derived from RF-SM, SMAP, NCA-LDAS, and ESA CCI were higher than those derived from the observations (Bias > 0 in all networks), while SM derived from SMOS was lower than the actual value (negative Bias ranging from −0.086 to −0.015 m³/m³). Notably, the statistical metrics of the RF-SM dataset indicated better and more stable performance across the five networks, yielding an optimal performance, with favorable R² values ranging from 0.34 to 0.86 ($p$ < 0.01), RMSE values ranging from 0.037 to 0.065 m³/m³, Bias values ranging from 0.004 to 0.025 m³/m³, and KGE values ranging from 0.28 to 0.86. Therefore, we could conclude that the RF-SM dataset exhibits relatively accurate SM estimates at 242 stations across the U.S., and therefore, it could be considered as the basis dataset for follow-up drought monitoring.

To further analyze the temporal variations of RF-SM, the line charts of SM derived from in situ measurements, microwave products, and RF-SM were plotted during 2016 at three representative stations in the substudy area (Figure 7). RF-SM could well capture the temporal variations in SM and respond to precipitation events. Although there is high consistency across all SM data in terms of trends, RF-SM efficiently simulated SM in summer and autumn, while coarse resolution products other than SMOS displayed significant overestimates. In addition, in regards to the overestimation of SMOS during the growing season and its underestimation under rainfall deficiency, RF-SM also improved in the integrated downscaling model using auxiliary variables.

To discuss the performance of the spatial distribution evaluation in RF-SM, we mapped RF-SM in the substudy area during the four dates in 2016. Figure 8 showed the SM simulation capability for RF-SM at the field scale, clearly reflecting the spatial heterogeneity at 30 m × 30 m. Meanwhile, the spatial variation of SM was also affected by precipitation events, and the lowest SM value in the whole area on 20 June occurred in the summer drought period. The spatial features related to land cover types, such as higher SM levels in the center shrubland areas on 13 February and 11 November, and lower SM levels in the southwestern cropland areas on 20 June, which may be attributed to the varying degree of impacts of near-surface conditions, including precipitation and temperature, on ground vegetation, could be easily detected from the high-resolution SM maps.

3.2. Comparison between the RF-SM-SWDI and VHI

The SWDI derived from the RF-SM dataset (RF-SM-SWDI) and the VHI derived from the Landsat 8 data were calculated over the U.S. for 2016, and the two indices were subsequently compared for all 242 in situ stations in regards to the Pearson correlation coefficient (R). Figure 9 shows that the R values between the RF-SM-SWDI and VHI are greater than 0.5 at most stations in the western mountain areas and plateau basins, but below 0.2 at most stations in the Great Plains area and along the east coast of the U.S. Moreover, the R values were strongly influenced by the performance of the RF-SM dataset in the five ground observation networks (Figure 6), which exhibited obvious differences between the stations. The results from approximately 70% of the stations performed well (R ≥ 0.5), and those from more than 40% of the stations performed fairly well (R ≥ 0.7) in terms of the correlation between the RF-SM-SWDI and the VHI.

To analyze the temporal evolution of drought in 2016, we calculated the SWDI derived from the observations and the RF-SM data at the three representative in situ SM stations in the substudy area. Figure 10 shows the SWDI, based on in situ SM records (SM-SWDI) available throughout the year, and RF-SM-SWDI, VHI, and precipitation (P) anomalies extracted from the corresponding pixel values derived from the different datasets. The three stations all exhibited typical seasonal characteristics of wet and rainy springs and winters and dry and rainless summers and autumns. Drought occurred from May to November, and the drought severity increased (the SWDI remained below 0, and the VHI remained above 40), especially in September and October. Adequate rainfall in mid-October supplemented the SM deficit and notably relieved the drought. When comparing the different drought indices, the RF-SM-SWDI always agreed with the SM-SWDI, reflecting consistent drought conditions, and the method benefited from the highly accurate RF-SM data. In addition, the temporal variations of the VHI were also similar to those of the SWDI, with R values higher than 0.8 at all three stations.

3.3. Spatiotemporal Drought Monitoring at the Field Scale

To achieve spatiotemporal drought monitoring at the field scale, we mapped the RF-SM-SWDI and VHI data at a spatial resolution of 30 m in the substudy area in 2016. Figure 11 shows that drought occurred in June, and the data revealed specific distribution patterns. According to the RF-SM-SWDI map, the frequency and severity of drought were low in the eastern forest areas, while the western and central grassland areas experienced mild to extreme drought. A few shrubland areas in the west revealed different drought areas from those of the surrounding locations, which may be influenced by inaccurate SM and soil hydraulic parameters. According to the VHI map, vegetation growth and surface temperature stresses only occurred in June and were correlated with the distribution of land cover types. Notably, there was mild drought in the forest areas with dense vegetation in the east, while moderate or even extreme drought occurred in the grasslands and shrublands in the western and central substudy areas. Moreover, the effect of the vegetation cover on the VHI was also notable during an unusual drought in the northeastern cropland areas in April, which may be related to crops exhibiting the early stages of growth.

In addition, Figure 12 shows the spatial distribution of correlation coefficients between the RF-SM-SWDI and VHI. There was a high correlation between these two drought indices in the substudy area, especially in the grassland cover areas (R > 0.7). Poor correlations (R < 0.2) occurred in the eastern and northern areas, possibly due to the complex topographic conditions, i.e., mountains and river valleys, as well as the vegetation conditions affecting the accuracy of the drought indices. Therefore, although both the RF-SM-SWDI and VHI provide evident potential for monitoring agricultural field drought, their consistency and differences should be further analyzed.

4. Discussion

4.1. Utility of Surface SM Metrics for Monitoring Drought

SM, which is mainly derived from rainfall, accounts for only 0.005% of the global water distribution [103] and is an important variable in meteorology, hydrology, and ecosystems [104,105]. Surface SM usually refers to water in the surface layer of the unsaturated soil zone [69]. This variable not only controls latent and sensible heat exchange between the surface and atmosphere [106,107,108], but also directly influences the exchange of trace gases on land [109], thereby regulating ecosystem dynamics and the terrestrial carbon balance [110,111,112].

Based on the work of Sehgal et al. (2017), drought response analysis of an in situ SM dataset for different soil layers showed that the surface SM is sensitive to short-term variations in the water regime caused by evaporation [113,114]. There is a strong interconnection between surface and root zone SM levels in soil hydrological processes such as infiltration and transport, resulting in severe SM stress in deeper soil profiles when surface soil is persistently deficient in water [115,116]. Qiu et al. (2014) used ten years of SM observations derived from the SCAN network to examine the impact of the vertical measurement depth on mutual information using NDVI anomalies, which indicated that surface SM can effectively duplicate the SM information content at a depth of 60 cm and suitably reflect the variability in vegetation [51]. It was suggested that the shallow penetration depth of microwave-based surface SM retrievals is no longer a major limitation for agricultural drought monitoring [117]. Moreover, SM and precipitation achieve a positive feedback mechanism in which dry soils limit precipitation and lead to even drier soils via atmosphere–land coupling [118,119], which is important for predicting extreme precipitation events and analyzing drought evolution in areas notably controlled by surface SM [120,121,122]. Therefore, surface SM exhibits certain agronomic significance and high application potential in short-term agricultural drought monitoring [123].

Furthermore, unlike the coarse grid of SM products, RF-SM enables fine agricultural drought monitoring at a higher spatial resolution. In the downscaling model, the introduction of SM products is critical. SMAP and SMOS, as representative products of L-band, have been proven in previous studies to exhibit comprehensive performance in medium vegetation cover areas [124,125], while NCA-LDAS and ESA CCI display higher spatial resolution based on multi-product fusion, which makes SM disaggregation more accurate at a fine scale [69]. Reflectance and ST variables from Landsat 8 are scale references for downscaling models, in which red band, NIR band, and ST are important for retrieving SM and related surface parameters [73], while the two SWIR bands can promote the development of SM indices by responding to water absorption [126]. Soil properties are the internal factors driving the spatial heterogeneity of SM, affecting the dynamic of surface SM by controlling water infiltration [127] and characterizing the SM variation in more detail under wet conditions [128]. Meteorological stress variables affect surface SM through the process of the land–air interaction [129]. Although their contribution is weakened due to the limitation of the coarse grid, the consideration of meteorological information helps to explain the SM variation under more complex conditions from the perspective of the water cycle [130]. Therefore, in collaboration with multiple datasets, RF-SM provides a highly accurate surface SM dataset, with reasonable representation at 30 m spatial scale.

4.2. Performance of the RF-SM-SWDI

This study focused on applying a typical index (SWDI) reflecting the distribution and temporal evolution of agricultural drought, which requires SM and soil properties and makes use of 30 m downscaled SM data derived from the RF model to achieve high-resolution drought monitoring at the field scale. Integrating multiple datasets through machine learning algorithms, the RF-SM dataset retains the advantages of the existing datasets (satellite-based products: SMAP, SMOS, and ESA CCI) and model-based products (NCA-LDAS) [131] in regards to accuracy and disaggregates SM in the original coarse grid on a finer scale, further meeting the monitoring and management needs for a small geographic area [132].

4.2.1. Comparison between the RF-SM-SWDI and VHI

Both the SWDI and VHI are important indices for drought monitoring and severity assessment [68]. On the basis of existing theories, the SWDI adopts the field water capacity (${\theta }_{FC}$), which is considered to be the upper limit of the available water for crops [133], as the limit when the soil begins to be subjected to water stress by introducing information on soil properties. This suggests that once the SM content falls below ${\theta }_{FC}$, the optimal state for the growing environment will be disrupted, triggering a test of the ability of crops to take up water [61]. Moreover, the VHI severs as an indicator of the occurrence and severity of drought through the characterization of crop physiological conditions in the visible and near-infrared bands and the characterization of the canopy temperature in the thermal infrared bands [134], providing a reliable foundation for its application in agricultural drought monitoring [135].

Compared with the VHI, the RF-SM-SWDI showed substantial differences in the results for both the distribution and temporal variations. On the one hand, the correlation between the RF-SM-SWDI and VHI was poor in the forest area located in the eastern substudy area, which may have been affected by two factors. One is that the two drought indices are derived from different satellite sensors. The calculation of the RF-SM-SWDI and accuracy of the RF-SM data are closely related to the SM products derived from passive microwave remote sensing, and the ability of these sensors to penetrate clouds and fog is much greater than that of optical remote sensing products that generate the VHI [136]. Moreover, the error caused by the attenuation of microwave emissions from the vegetation to the soil [137] has been greatly improved in the RF-SM dataset [138], while the VHI mainly reflects canopy information [139], increasing the likelihood that vegetation health conditions are misrepresented. Another factor is that drought-resistant forests develop root systems that can absorb water in deeper soil layers to maintain vegetation health when SM is deficient [63]. Thus, the low correlation between the vegetation condition component of VHI (i.e., VCI) is particularly evident in forest areas (Figure 13a). On the other hand, the VHI was proposed with the underlying assumption that there is a negative correlation between the NDVI and LST; that is, the LST increases when drought occurs, while the NDVI decreases under stress [140]. There was a notable correction between the TCI and RF-SM-SWDI (R > 0.5) in most of the substudy area (Figure 13b), suggesting that the response of the temperature conditions to SM changes occurred faster than that of the vegetation conditions. Based on the temporal variation in the SWDI at the observation stations (Figure 10), a quick response to precipitation occurred, resulting in fluctuations in spring and winter that are consistent with the short-term heavy rainfall events. This instability may be responsible for the relatively obvious variation in the spatial distribution of the RF-SM-SWDI on different dates. Furthermore, the coupling relationship between the LST and surface SM has been confirmed in numerous studies [141,142,143], resulting in TCI achieving high correlation with SWDI, thus realizing the consistency between VHI and RF-SM-SWDI in regards to drought monitoring.

4.2.2. Spatiotemporal Consistency of Drought Monitoring

The occurrence of agricultural drought is often complicated by hydrological [2], meteorological [36], and physiological [11] factors that are difficult to monitor immediately and exhaustively in space and time. A universal multi-indicator weighted “blended approach” has been developed through coordinated monitoring efforts at the country level to address the limitations of a single index for a particular region and a particular drought event [144]. As one of the first entities to use hybrid indicators, the U.S. Drought Monitor (USDM) was founded in 1999 for assessing drought conditions across the U.S. [145]. With the development of gridded surface meteorological (gridMET) datasets [146], high-resolution (4-km) drought indices for the contiguous U.S. from 1979 to the present were produced by the University of California Merced and computed by the Climate Engine platform (https://app.climateengine.org/climateEngine (accessed on 20 December 2023)). The combination of these drought indices for short (ranging from several days to several weeks) and long (ranging from a few months to a few years) periods could provide targeted services to decision makers in different fields [147]. In this study, we used the short-term drought blend (STDB) variable from the gridMET Drought dataset, which combines PDSI (20%), 30-day SPI (20%), 90-day SPI (25%), and ZIND (35%), based on the professional judgment of drought experts for consistency analysis of drought monitoring by the RF-SM-SWDI.

Since the STDB dataset is updated every 5 days, representative drought monitoring maps at corresponding time intervals were selected for comparison (Figure 14). According to the comparison between the two indices at a grid scale of 4 km, the four days of drought severity in the substudy area exhibited a consistent trend of gradual decrease from west to east, which basically coincides with the RF-SM-SWDI results, particularly in regards to the drought maps of February and April. In addition, the extent of drought increased to the east, the drought severity increased from February to June, and the drought was relieved by precipitation in late fall, resulting in no or mild drought in the middle of November. The substudy area is located in the Mediterranean climate zone near the west coast of the continental U.S, characterized by cold and humid winters but hot and arid summers [148]. Encouragingly, the inevitability and severity of summer drought events [149] are reflected by the RF-SM-SWDI, VHI, and STDB. However, Figure 14 shows that the drought conditions in winter and spring (before May) are more serious than those in autumn (after October), which is not suitably reflected in the drought maps of the RF-SM-SWDI, especially in the western and southeastern areas of the substudy area, where the drought severity in mid-November was actually shown to increase. Moreover, a survey by the USDM of the counties primarily covered by the substudy area (Amador County and Sacramento County) also revealed extreme drought conditions during the first half of 2016 (https://droughtmonitor.unl.edu/DmData/TimeSeries.aspx (accessed on 20 December 2023)). This result was due to multiyear precipitation deficiencies and anomalously warm temperatures culminating in 2016, and these conditions were improved by the abundant precipitation received in the winter of 2016–2017 [150]. Therefore, the RF-SM-SWDI could yield a convincing spatial pattern, but there are still weaknesses in terms of the representation of the evolution of drought over time.

4.3. Issues of Drought Monitoring with High Spatial Resolution

Based on RF-SM-SWDI, this study achieved agricultural drought monitoring at a high spatial resolution of 30 m, which compensates for the lack of coarse grids containing composite SM and crop signals associated with terrestrial and atmospheric conditions. With the advances in remote sensing technology, the development of data fusion methods [78], and even hyper-resolution sensors [28], will provide more detailed information on the occurrence and evolution of drought. The RF-SM dataset not only fulfills the need to identify and assess the severity and extent of drought at the field scale, but also, more importantly, introduces various drought-related variables (e.g., spectral reflectance, LST, and precipitation) into the fusion process, providing reliable supporting data for computing relevant drought indices. Further, fine monitoring using a 30 m spatial resolution contributes to the differentiation of crop types and the management of water and land resources [151,152]. Therefore, the RF-SM-SWDI has the potential to become an effective reference tool for agricultural regionalization and policy formulation at the county level.

Due to the inevitable constraints, our study must still be improved upon for practical applications. First, it is difficult to track and monitor drought evolution with coordinated temporal resolution because the availability of RF-SM data is limited by the cloud state of the optical remote sensing dataset and the scanning mode of microwave remote sensors. The application of data fusion [153] and data assimilation [154] methods could help to generate both spatially and temporally continuous SM datasets in future research. Second, the RF-SM-SWDI yielded satisfactory spatial heterogeneity at small scales and exhibited differences in the impacts of microclimates on the severity of drought, but drought occurrence did not follow the defined scale [155]. It is necessary to comprehensively establish additional spatial units, such as climate divisions and crop districts, to achieve agricultural drought monitoring and early warning of drought conditions [156]. Finally, in addition to soil properties and land cover types, the analysis of topography, vegetation phenology, and agricultural management practices, such as irrigation, should be considered for accurate high-resolution agricultural drought monitoring and assessment [157].

5. Conclusions

In this study, two drought indices at a 30 m spatial resolution, the RF-SM-SWDI, which was calculated using an SM downscaling framework derived from multiple datasets, and the VHI, which was calculated using optical remote sensing dataset obtained by Landsat 8, were utilized to monitor agricultural drought conditions over a small geographic area in the U.S. in 2016. To conduct a comprehensive assessment, the RF-SM data was first evaluated based on in situ observations, and the RF-SM-SWDI was then compared with the VHI in terms of monitoring the drought distribution and evolution. In conclusion, the RF-SM-SWDI, with a high spatial resolution, could be a promising tool for drought monitoring, mainly due to the following:

(1)

The RF-SM dataset yielded better performance results than the four single SM products when compared with the observed SM at in situ stations, whether based on different land cover types or ISMN networks. When the utility of surface SM in agricultural drought monitoring is recognized, RF-SM data exhibit significant value in capturing drought conditions driven by SM because of their high spatial resolution.

(2)

The SWDI relies on SM estimation and corresponding soil hydraulic parameters, which effectively overcomes the limitations of drought research over certain time periods. The RF-SM-SWDI exhibited a favorable correlation with the VHI at approximately 70% of the stations. Further comparison of the substudy area showed that the coupling of the LST and SM produced a strong correlation between the TCI and SWDI, while the changes in vegetation conditions caused a significantly different spatial patterns for the VCI and SWDI, which ultimately affected the difference in the distribution and temporal variations of the RF-SM-SWDI and VHI related to the land cover types.

(3)

The RF-SM-SWDI provided data for drought conditions which included more detailed spatial information and demonstrated the seasonal evolution and patterns of the different land cover types. Compared with the results of the STDB, the RF-SM-SWDI recognized Mediterranean climate characteristics in the substudy area, accurately monitoring summer drought events and drought mitigation in winter, which made it possible to achieve real-time monitoring of short-term and flash droughts at the field scale.

(4)

Due to the limitations of the available datasets, the application of RF-SM-SWDI exhibits some difficulties in long-term agricultural drought monitoring on a larger geographical scale. Therefore, advanced data fusion and data assimilation technology, along with more drought-related surface information, will help to accurately analyze drought distribution and track drought evolution.

In the near future, we expect spatiotemporal SM datasets, with a high resolution, to be generated and applied in agricultural drought monitoring, resulting in meaningful indices. More importantly, accurate data regarding the extent and evolution of drought at multiple levels could be used in controlling the spread of drought, mitigating severe consequences, supporting agricultural policies and management practices, and ensuring crop yield and economic benefits.