Evaluation of Multi-Source Soil Moisture Datasets over Central and Eastern Agricultural Area of China Using In Situ Monitoring Network

Abstract

Multi-source soil moisture (SM) products provide a vigorous tool for the estimation of soil moisture on a large scale, but it is crucial to carry out the evaluation of those products before further application. In the present work, an evaluation framework on multi-source SM datasets over central and eastern agricultural areas of China was firstly proposed, based on a dense in situ SM monitoring network of 838 stations from 11 July 2012 to 31 December 2017. Each station adopted the most accurate gravimetric method for measuring the actual soil moisture. The effects of land use types and wet–dry conditions on the performances of multi-source SM products were further analyzed. Most satellite/reanalysis SM products could capture the spatial–temporal changes in soil moisture, especially for ERA5 products that matched the closest to the station-measured SM; by contrast, those satellite products showed poor spatial–temporal performances. Such phenomenon was also quantitatively demonstrated by the four statistical metrics correlation coefficient (CC), p-value, bias and root mean squared error (RMSE) between the satellite/reanalysis SM products and the ground-observed SM series. Further, most satellite/reanalysis SM products had poor performances in Forestland and Grassland areas, with a lower CC and a larger positive bias and RMSE. Such overestimation on soil moisture is possibly influenced by the inestimable parameter vegetation geometry and the vegetation water content in the radiative transfer models. The arid areas showed the worst CC between the station-observed SM data and different satellite/reanalysis SM products; meanwhile, the humid and semi-arid areas presented larger SM estimation errors than the other areas, especially for the satellite products. The fairly dry surface soil (arid area) and open water surface contamination (humid area) are suggested to hinder the reading of microwave-based retrieval systems. Additionally, the reanalysis SM products outperformed the satellite SM products in the evaluated areas, with better spatial–temporal performances, seasonality reflection and higher accuracy on SM estimation (higher CC, and lower bias and RMSE). This is because the reanalysis datasets assimilated various sources of datasets, especially the ground-observed data, with high quality. The evaluated results could provide guidance for fusing different satellite/reanalysis products, as a new feasible alternative to monitoring SM information in the future.

1. Introduction

Soil moisture (SM) is a key bridge in the interactions between the atmosphere, biosphere and hydrosphere, which influences the hydrologic cycle processes by controlling the partition of rainfall into surface runoff, infiltration and groundwater recharge, and the evaporation and transpiration rate from bare and vegetated areas [1,2]. Hence, a good knowledge of soil moisture is crucial for improving weather forecasts [3,4], drought and flood predictions [5,6,7,8], agricultural monitoring and water management [9], as well as climate change studies [10]. Soil moisture information can be acquired through in situ observation instruments, or by remote sensing techniques. Of them, the in situ soil moisture instruments represent point-based monitoring, which can hardly obtain soil moisture information at regional to global scales, particularly in some areas with uneven underlying surfaces of high spatial heterogeneity. By contrast, remote sensing techniques have shown huge advantages, which provide a vigorous tool for estimating soil moisture at high resolution both spatially and temporally and update data with short latency. Therefore, the feasibility study of remote sensing technology for monitoring soil moisture is a hot issue in current research.

Three primary remote sensing techniques have been developed for soil moisture estimation, including optical and thermal infrared remote sensing, and active and passive microwave remote sensing techniques. The microwave remote sensing technique is feasible in all weathers with a moderate penetration depth, which can remedy some defects of optical and thermal infrared remote sensing, and therefore regarded as a promising approach for estimating SM. Based on the large contrast between the dielectric constant of water and dry soil, microwave remote sensing can measure SM by establishing its relationships with the radar backscattering coefficient and radiometer brightness temperature, which is deduced by the emitted and reflected microwave radiation from the land surface to remote sensors (both passive and active). For previous passive microwave sensors operating at X- (e.g., 10.7 GHz) and C-band (e.g., 6.9 GHz), the Advanced Microwave Scanning Radiometer-Earth Observing System (AMSR-E) and its successor AMSR2 were often used for SM estimation [11]. Current passive spaceborne sensors operating at L-band (e.g., 1.4 GHz) include the Soil Moisture and Ocean Salinity (SMOS) satellite and the most recently launched Soil Moisture Active Passive (SMAP) satellite. In addition to these passive sensors, active microwave sensing with different synthetic-aperture radar (SAR) systems was also utilized for SM measurement. Each passive/active microwave sensor has its own strengths and weaknesses, and a combination of multi-source sensors could usually overcome the drawbacks of an individual sensor and attain a better performance. The Climate Change Initiative (CCI) program launched by the European Space Agency (ESA), as the representation of multiple combined sensors, has merged six passive remote sensors (e.g., AMSR-E, AMSR2, TMI, SSM/I, WindSat, SMMR) and two active sensors (e.g., ASCAT, ERS1/2). In particular, the subsequent ESA-CCI versions are continuously modified and extended by integrating newly launched satellite sensors (e.g., SMOS and SMAP at lower frequencies).

Numerous satellite SM products were retrieved based on those abovementioned spaceborne sensors, using different SM retrieval algorithms and transfer models. In this process, errors between satellite SM estimation and the true value of SM are inevitable and should be highlighted. Errors derive from not only the remote sensors themselves, such as the wavelength region of observation and sensor specifications (e.g., preset allowable errors, ascending/descending overpass), but also some assumptions and parameters in those transfer models. Moreover, SM product qualities vary greatly with different land properties (e.g., surface roughness), covering intensities of vegetation and snow/ice and topographical conditions. In addition, frequent radio frequency interference (RFI) from human activities in large areas also perturbs remote imaging and affects the effective inversion of SM products. Thus, evaluation on the data quality of satellite SM products is a critical requirement for further application.

The reanalysis dataset simulated by models can overcome some uncertain errors in satellite SM products, by assimilating various observation datasets from the ground measuring stations, spaceborne sensors, radar, aircraft, etc., into the global model. On this basis, previous studies on evaluation of satellite SM products were primarily carried out through comparison with either in situ ground measurements or those reanalysis products. Ground-based SM measurements are regarded as the most close to the true value of SM, which can be conducted by the gravimetric method manually, or by global/regional SM monitoring networks based on time domain reflectometry (TDR), gamma ray scanners and neutron probes [12]. Using the International Soil Moisture Network (ISMN) and the U.S. Climate Reference Network (USCRN), the suitability and reliability of AMSR2 and SMAP soil moisture products over the USA were validated by Wu et al. [13] and Stillman and Zeng [14], respectively. Yee et al. [15] also used the OzNet monitoring network to compare the performances of SMOS and AMSR2 soil moisture products at a catchment scale. Despite the high accuracy and temporal dynamic reflection of ground-based observations, in situ datasets also have limitations in the representation of sparse monitoring networks and uncertainties during upscaling processes from point to region scales. For this reason, quite a few researchers chose those modeled SM products that were reanalyzed based on both satellite and station datasets, for validation and calibration studies [16,17,18,19]. For example, the Modern-Era Retrospective Analysis for Research and Applications-Land (MERRA-Land) dataset has been used for evaluating the performances of SMOS, ESA-CCI and the Advanced Scatterometer (ASCAT) soil moisture products [18,20]. However, reanalysis SM datasets can only act as reference products and cannot fully represent the “ground truth”, the accuracy of which may hamper the interpretation of evaluation results [21,22]. Additionally, Pierdicca et al. [23] combined ISMN monitoring networks and two reanalysis SM datasets (NCEP/NCAR and ERA-Interim) to compare ASCAT and SMOS soil moisture products in Europe and North Africa. In sum, a dense SM monitoring network with better representation, or a reliable reanalysis product with higher accuracy, is the key issue in satellite SM product assessment.

Several evaluation studies on satellite SM products have also been conducted in China, which were mainly focused on Tibet Plateau areas due to their specific locations and meteorologic conditions [24,25]. However, few relevant studies have been executed in larger areas with manifold underlying surface conditions [26], particularly in the irrigated agricultural areas of China where surface SM has a significant impact on crop growth and yields. In fact, error characteristics of each SM product were proven to vary with different land covers and other environmental conditions [27,28]. Hence, evaluation on satellite SM products at a larger scale of China is critically needed, and their performances under different underlying land use types and meteorologic and hydrological conditions should be involved and comprehensively discussed.

This work explores the error characteristics of multi-source SM datasets in the irrigated agricultural areas of China before further application. To this end, an evaluation framework on the performances of AMSR2 and ESA-CCI soil moisture products was firstly proposed over the central and eastern agricultural areas of China, as well as three reanalysis SM datasets including the ERA5 (i.e., the successor of ERA-Interim), the Global Land Data Assimilation System (GLDAS) and Global Land Evaporation Amsterdam Model (GLEAM) soil moisture products. The accuracy estimation was based on a dense in situ SM monitoring network with a total of 838 in situ ground observation stations, which are evenly distributed in those agricultural areas. The spatial–temporal comparisons of performances between multi-source SM products (both satellite and reanalysis datasets) and in situ ground measurements were carried out, and assessments under different land use types and wet–dry conditions were further discussed. Note that unlike previous studies, the SM measurement in this study adopted the gravimetric method with dense monitoring stations, which is regarded as the most accurate method and can best characterize the true value of actual land SM.

2. Data and Methods

2.1. Study Area and In Situ Monitoring Network

Study area. The evaluation study was carried out in the central and eastern agricultural areas of China (92°40′E~131°31′E, 27°37′N~46°2′N), which cover an area of 2,194,503 km² and 15 major food-producing provinces. The cultivated area in the study area accounts for 54% of the total cultivated area in China, and the grain yield reaches 57% of the total national yields. Land uses are divided into Cropland (CRO), Forestland (FOR), Grassland (GRA), Mixed land (MIX) and Barren land (BAR), according to the Remote Sensing Monitoring Data on Land Use in China (2015).

In situ monitoring network. There are 838 soil moisture monitoring stations evenly distributed in the evaluation areas, constituting a dense, in situ ground monitoring network (Figure 1). Soil moisture measurement at each station used the gravimetric method [29], which is well acknowledged as the most accurate method. Four soil layers under 10, 20, 30 and 40 cm surface layers were sampled vertically by a cutting ring, and three parallel soil samples were taken from each soil sample and put into an aluminum box. The soil samples were dried at 105℃ for 8 h and weighed after cooling. The in-situ ground monitoring SM data started from 11 July 2012 to 31 November 2017, sampling at days 1, 11 and 21 per month.

2.2. Multi-Source Satellite and Reanalysis Soil Moisture Platforms

The specific descriptions on two satellite SM products (AMSR2 and ESA-CCI) and three reanalysis SM datasets (ERA5, GLEAM and GLDAS) were as follows, and a comparison between them can be seen in

Table 1. Summary of satellite and reanalysis soil moisture datasets.

Dataset		Retrieval/Assimilation Method	Period	Spatial Coverage	Temporal Resolution	Spatial Resolution	Depth	Latency
Satellite	AMSR2 (Ascending and Descending)	NASA-LPRM	2012–present	Global	diurnal	0.25°	~5 cm	Daily
	ESA-CCI (Active, Passive and Combined)	ESA-multisensory fusion	1978–present	Global	diurnal	0.25°	0–10 cm	Annually
Reanalysis	ERA5	ECMWF-Integrated Forecast System	1981–present	Global	1-hourly	0.1°	0–7 cm; 7–28 cm; 28–100 cm; 100–289 cm	5 days (Preliminary data); 3 months (Accurate data)
	GLEAM-V3.3a and -V3.3b	GLE-Amsterdam	2003–2018	Global	diurnal	0.25°	0–10 cm; 10–100 cm	Related to CERES radiation data
	GLDAS-V2.1 (Noah model)	NASA-GLDAS	2000–present	Global	3-hourly	0.25°	0–10 cm; 10–40 cm; 40–100 cm; 100–200 cm	1.5 months

.

2.2.1. Satellite Soil Moisture Products

(a)

AMSR2

AMSR2 as the successor of AMSR-E (stopped in October 2011) was on board the Global Change Observation Mission 1-Water (GCOM-W1) and launched in May 2012 by the Japan Aerospace Exploration Agency (JAXA). AMSR2 is a passive microwave single sensor with both C- and X-band microwave frequencies. Usually, the revisit time of AMSR2 is one to two days. Compared with the previous AMSR-E, AMSR2 enlarged the antenna diameter from 1.6 to 2.0 m and added C-band (i.e., 7.3 GHz C2-band) to lessen RFI and simultaneously improved the calibration accuracy through a modified thermal design [30,31]. Two algorithms including JAXA and the land parameter retrieval model (LPRM) were utilized for SM estimation (for more details on algorithms, please refer to Koike [32] and Van der Schalie et al. [33]. This evaluation work was mainly focused on the AMSR2 SM product (LPRM), a Level 3 (gridded) dataset consisting of descending (01:30 local time) and ascending (13:30 local time) overpasses every day.

(b)

ESA-CCI

ESA-CCI soil moisture projects were launched as a requirement of climate change research in 2010, producing continuously upgraded soil moisture products by various improvements (e.g., enhancing spatial–temporal resolution, merging new sensors and updating algorisms). Nine passive and active sensors such as ASCAT-A and -B, ERS-1/2 and -2, SMMR, SMOS, AMSR2, AMSR-E and WINDSAT were merged in this version, and three independent SM products were derived from those active, passive and combined (active and passive) sensors (denoted, respectively, by ESA-CCI-ACT; ESA-CCI-PAS; and ESA-CCI-COM). More specific descriptions of ESA-CCI soil moisture products can be found elsewhere [34,35,36].

2.2.2. Reanalysis Soil Moisture Products

(a)

ERA5

ERA5 is the most modern reanalysis product from the European Centre for Medium-Range Weather Forecasts (ECMWF) [37], which has replaced the ERA-Interim (ceased being produced) since 31 August 2019. ERA5 integrates large quantities of historical observations into the global land variable estimates, using the latest ECMWF modeling and data assimilation techniques. Currently released quality-assured ERA5 data cover the period from 1981 to 2–3 months before the present, while the preliminary data update the dataset which is available within only 5 days of real time.

(b)

GLDAS-Noah

NASA GLDAS aims to generate optimal fields of land surface states and fluxes, by merging satellite- and ground-based observation datasets, using advanced land surface modeling and data assimilation techniques. GLDAS drives multiple, offline (non-atmospheric coupling) land surface models; integrates a large quantity of ground-observed data; and operates globally with high resolutions. Herein, GLDAS-2.1 was highlighted in the three components of NASA GLDAS version 2. GLDAS-2.1 is forced with a combination of model and observation data from 2000 to present, with respect to GLDAS-2.0 with merely Princeton meteorological forcing input data.

(c)

GLEAM

The Global Land Evaporation Amsterdam Model (GLEAM) is a set of algorithms to estimate various components of land evaporation: transpiration, bare soil evaporation, interception loss, open water evaporation and sublimation. In addition, GLEAM also provides soil surface and root zone moisture, which has been widely recognized and applied [38,39]. GLEAM datasets have been constantly revised and updated since its development in 2011. The third version, the GLEAM-V3.3a dataset, is primarily based on the latest reanalysis of net radiation and air temperatures (from ERA5, ECMWF), satellite- and station-based precipitation, satellite-based vegetation optical depth (VOD) and snow water equivalent, whilst GLEAM-V3.3b is the global dataset mainly based on satellite data. For more detailed descriptions of GLEAM V3 datasets, please refer to Miralles et al. [40] and Martens et al. [41].

2.3. Methodology

Evaluation framework. Herein, an evaluation framework on multi-source SM datasets over large irrigated agricultural areas was firstly proposed, involving various datasets from both satellite (with active, passive and combined sensors) and reanalysis (with different data assimilation models and systems) sources; complex underlying meteorologic and hydrological conditions; and unit transformation, scale match and statistical methods (Figure 2). The first step was data collection and pre-processing, mainly including how to transform the station-observed SM data from gravimetric to volumetric units; and how to downscale the satellite/reanalysis SM data from grid scale to ground site scale. Once the units and the calculating scale between the observed and the satellite/reanalysis SM datasets were uniformed, the second step was to analyze the four statistical metrics (e.g., correlation coefficient (CC), p-value, bias and root mean squared error (RMSE)) among different datasets, which are used for the accuracy evaluation. On the basis of the results in the first two processes, the third step was to compare the spatial–temporal performance and seasonality reflection ability of different datasets, quantitatively assess their accuracy on SM estimation with respect to the ground observation sites and, finally, evaluate their performances under different land uses and wet–dry conditions. It should be noted that the Hausdorff distance method was adopted for the similarity estimation of two distributed trajectories, in order to judge which dataset can better reflect the intra-annual distribution of soil moisture.

Transforming gravimetric to volumetric SM. Most satellite SM products and reanalysis SM datasets provide volumetric soil moisture, while in situ ground observations measure gravimetric soil moisture. Thus, the first step before estimating on the performances of satellite/reanalysis SM products against the ground observations is to uniform the units of different datasets. For convenience, herein, all gravimetric SMs measured by ground gauges were transformed to volumetric SMs using the following formula:

$$SMv=SMg \times \rho$$

(1)

where $SMv$ denotes volumetric soil moisture (cm³/cm³); $SMg$ denotes gravimetric soil moisture (g/g); and $\rho$ denotes soil bulk density (g/cm³) and is highly related to the soil compactness. Note that $\rho$ can be obtained from the Harmonized World Soil Database (HWSD), of which the data source of China is the 1:1,000,000 soil data provided by the Institute of Soil Science, Chinese Academy of Sciences, for the second national land survey.

Downscaling transformation. To match satellite/reanalysis SM products with ground observation datasets at the same scale, the gridded datasets were downscaled from the regional scale to the point scale using the following method: selecting four grids that are nearest to each measured ground site, and interpolating the SM value of each grid to the ground site by inverse distance weighted (IDW) interpolation. For one ground observation site, with coordinates ($x_0$, $y_0$), the weight ${\mathsf{\lambda }}_i$ of the nearest four grids was calculated as the following formula:

$${\mathsf{\lambda }}_i=\frac{\frac{1}{d_i}}{{{\displaystyle \sum }}_1^4\frac{1}{d_i}}$$

(2)

where ${\mathsf{\lambda }}_i$ denotes the weight of grid i, and $d_i$ denotes the distance between the central point ($x_i$, $y_i$) and the ground observation site ($x_0$, $y_0$).

The interpolated SM values at the ground observation site were obtained by

$$\hat{Z}\left(x_0, y_0\right)={\displaystyle \sum }_1^4{\mathsf{\lambda }}_i\mathrm{Z}\left({\mathrm{x}}_{\mathrm{i}},{ \mathrm{y}}_{\mathrm{i}}\right)$$

(3)

where $\hat{Z}\left(x_0, y_0\right)$ denotes the interpolated SM value (m³/m³), and $\mathrm{Z}\left({\mathrm{x}}_{\mathrm{i}},{ \mathrm{y}}_{\mathrm{i}}\right)$ denotes the SM value of grid i.

Statistical metrics for evaluation. Four statistical metrics including the correlation coefficient (CC) and corresponding p-value, bias and root mean squared error (RMSE) were primarily computed and used for quantitative evaluation on the differentiation between satellite/reanalysis SM products against the ground observations. Of these, for m discrete points of two variables, $\varnothing sat$ and $\varnothing g$, the Pearson correlation coefficient and RMSE were given as

$$CC= \frac{{{\displaystyle \sum }}_{i=1}^m(\varnothing sa{t}_i-\overline{\varnothing sat})(\varnothing g_i-\overline{\varnothing g})}{\sqrt{{{\displaystyle \sum }}_{i=1}^m{(\varnothing sa{t}_i-\overline{\varnothing sat})}^2{{\displaystyle \sum }}_{i =1}^m{(\varnothing g_i-\overline{\varnothing g})}^2}}$$

(4)

$$RMSE=\sqrt{\frac{{{\displaystyle \sum }}_{i =1}^m{(\varnothing sa{t}_i-\varnothing g_i)}^2}{m}}$$

(5)

where $\varnothing sat$ denotes the satellite/reanalysis SM value (m³/m³), and $\varnothing g$ denotes the in situ ground-observed SM value (m³/m³).

Taylor diagrams. Taylor diagrams have recently been used for soil moisture dataset evaluation [15,42]. They were adopted herein to provide a comprehensive visualization for comparing temporal performances of different satellite/reanalysis SM datasets to the ground observations (as the reference dataset), using three statistical metrics, i.e., CC, the centered RMSE (cRMSE) and the standard deviation [43]. Note that the centered RMSE was calculated after subtracting the mean value for m discrete points of two variables, $\varnothing sat$ and $\varnothing g$, and thus cRMSE was given as

$$cRMSE=\sqrt{\frac{{{\displaystyle \sum }}_{i=1}^m{[(\varnothing sa{t}_i-\overline{\varnothing sat})-(\varnothing g_i-\overline{\varnothing g})]}^2}{m}}$$

(6)

where $\varnothing sat$ and $\varnothing g$ denote the same meaning as that previously mentioned, while $\overline{\varnothing sat}$ and $\overline{\varnothing g}$ denote the average of the satellite/reanalysis SM value (m³/m³) and the in situ ground-observed SM value (m³/m³), respectively.

Hausdorff distance for trajectory similarity. The Hausforff distance is often used to measure how far two trajectories are from each other and reflect the maximum mismatch of two distributed trajectories [44,45]. To evaluate the distributed trajectory similarity of annual soil moisture between different satellite/reanalysis SM datasets and the station-observed SM series, their Hausdorff distances ($D_H$) were calculated with the following formula:

$$D_H\left({\mathrm{L}}_{sat},{\mathrm{L}}_g\right)=\max \left(\mathrm{h}\left({\mathrm{L}}_{sat},{\mathrm{L}}_g\right), \mathrm{h}\left({\mathrm{L}}_g,{\mathrm{L}}_{sat}\right)\right)$$

(7)

where

•   $\mathrm{h}\left({\mathrm{L}}_{sat},{\mathrm{L}}_g\right)=\max \left(\mathrm{a}\in {\mathrm{L}}_{sat}\right)\min \left(b\in {\mathrm{L}}_g\right)\left|\right|a-b\left|\right|$;
•   $\mathrm{h}\left({\mathrm{L}}_g,{\mathrm{L}}_{sat}\right)=\max \left(\mathrm{a}\in {\mathrm{L}}_g\right)\min \left(b\in {\mathrm{L}}_{sat}\right)\left|\right|a-b\left|\right|$;
•   $\left|\right| \left|\right|$ denotes the distance norm between lines ${\mathrm{L}}_{sat}$ and ${\mathrm{L}}_g$;
•   ${\mathrm{L}}_{sat}$ denotes the annual SM changes in the satellite/reanalysis dataset;
•   ${\mathrm{L}}_g$ denotes the ground-observed annual SM changes.

Usually, the larger the $D_H$, the higher the mismatch or the lower the similarity of two trajectories, indicating the worst performance in the seasonality reflection of one dataset.

3. Results

3.1. Regional Comparison of Spatial–Temporal Performance among Multi-Source SM Products

3.1.1. Comparison of Temporal Performance among Nine Multi-Source Satellite SM Products

A Taylor diagram can provide a comprehensive visualization of how well two datasets relate to each other in terms of the CC, the centered RMSE and the standard deviation (Figure 3). Herein, Taylor diagrams were separately applied in four wet–dry areas, semi-humid, humid, semi-arid and arid areas, which mainly considered the influences of spatial heterogeneity (e.g., uneven hydrometeorological and underlying conditions). Further, the multi-year soil moisture changes of different satellite/reanalysis SM products and observed SM series in each area are shown in Figures S1 and S2. Most satellite/reanalysis SM products could effectively capture the temporal changes in soil moisture in the semi-humid, humid and semi-arid areas. Of these, the GLEAM-V3.3a and GLEAM-V3.3b, the ERA5, the ESA-CCI-COM and the GLDAS products outperformed the other four products, as presented by the shorter distances from the red point (the observed SM data). In particular, ERA5 showed the shortest distance in semi-humid and humid areas, which indicated the largest CC and the lowest RMSE compared to the observed ground SM data series. By contrast, the ESA-CCI-ACT and ESA-CC-PAS and the AMSR2-ASC and AMSR2-DES products showed either the lower CC or the larger RMSE, indicating their poor performances on capturing the temporal SM changes. In arid areas, however, most satellite/reanalysis SM products showed undesired performances in capturing temporal SM changes, as presented by their larger distances from the observed points. The centered RMSEs of those datasets were not large in arid areas, due to a relatively low soil moisture level (<0.30 m³/m³) (Figure S2), whilst the CCs of different satellite/reanalysis SM products with the observed datasets were much lower (all < 0.4). This may be related to the large surface roughness in arid areas of Western China, which affected the transmission and reception of satellite microwave signals [46].

3.1.2. Comparison of Spatial Performance among Nine Multi-Source SM Products

The spatial performance of different datasets lies in their abilities to depict the actual dry and wet degrees, and to capture the spatial variations in soil moisture. The actual wet–dry partition was ascertained based on the average annual rainfall data in those areas, which included four wet–dry areas as wet (>800 mm), semi-wet (400–800 mm), semi-arid (200–400 mm) and arid areas (<200 mm) (Figure 4a). By contrast, most satellite/reanalysis SM products can capture the northeast humid area, the southern humid area, the northwest arid area and the central semi-arid and semi-humid areas, especially for the ERA5 product. Other products such as GLDAS-Noah, GLEAM-V3.3a/V3.3b and ESA-CCI-COM/-PAS could also roughly capture the spatial changes in dry and wet areas, whilst ESA-CCI-ACT and AMSR2 products showed the worst spatial performance, as presented by the low capacity in depicting different wet–dry areas.

The spatial distribution of the measured SM on a multi-year average was obtained through an interpolation with 838 in situ monitoring stations (Figure 4b). Compared with the spatial distribution of station-measured soil moisture, most satellite/reanalysis SM products effectively captured the spatial variations in actual soil moisture. Of these, GLEAM-V3.3a/V3.3b and ERA5 products matched the closest to the interpolation results of the measured sites. Nevertheless, the performance of different SM products varied greatly in terms of the SM content estimation. The multi-year average surface SM content measured by observation stations was between 0 and 0.6 m³/m³, and that by GLEAM-V3.3a/V3.3b and ERA5 products was also within this range. The worst performance of SM estimation was found in the ESA-CCI-ACT and AMSR2 products with a much higher SM content of 0–0.9 m³/m³.

3.1.3. Seasonality of SM Datasets

Based on the temporal performances of different SM products, the seasonality of the SM distribution among reanalysis and satellite datasets was further compared and analyzed in the semi-humid areas (Figure 5). Herein, the semi-humid areas were chosen as a typical case due to the most densely distributed ground observation stations. It can be seen that soil moisture in the semi-humid areas decreased from the spring (Mar–Apr–May) season, whilst it increased during the summer (Jun–Jul–Aug) and post-monsoon (Sep–Oct–Nov–Dec) seasons. In general, the surface SM is greatly influenced by the local precipitation and evaporation. Soil moisture increased in February compared to that in January, due to the increasing precipitation. The evaporation was lowest in the winter season (with the lowest temperature); however, it increased with the increasing temperature in the spring season (> rainfall), which caused the decrease in surface soil moisture. Despite the large evaporation caused by the high temperature in summer, abundant precipitation in those rainy seasons (usually > evaporation) increased the surface soil moisture. During the post-monsoon season, soil moisture still presented an increasing trend due to the decreased evaporation with the decreasing temperature. The annual distribution of rainfall and evaporation in this area can be seen in Figure S3.

On the basis of the ground-observed annual SM changes, the Hausdorff distances and Pearson CC were adopted herein to evaluate the seasonality reflection performances of different datasets. Those reanalysis SM products (except GLDAS-Noah) could well reflect the seasonality of the surface SM distribution (Figure 5a): the CC of GLEAM-V3.3a and -V3.3b and ERA5 with the observed datasets reached 0.76, 0.73 and 0.73, respectively. Most reanalysis SM products considered the ground observation data, thereby presenting good correlations with the seasonality of soil moisture. GLDAS-Noah showed the worst performances in reflecting the seasonal distribution of SM (CC = 0.03), indicating that the simulated SM by the Noah model was not consistent with the actual ground data. The Hausdorff distances also proved that GLEAM-V3.3a and -V3.3b and ERA5 products ($D_H$ = 1.31, 1.3 and 1.43) outperformed GLDAS ($D_H$ = 2.09) in depicting the annual SM distributions. Overall, the satellite SM products were inferior to the reanalysis datasets in reflecting the seasonality of the surface SM distribution, as presented by the lower correlation coefficients with the observed SM series, with the maximum 0.67 of AMSR2-ASC and the minimum −0.09 of ESA-CCI-ACT. ESA-CCI-ACT also showed the largest Hausdorff distances ($D_H$ = 2.1) compared to the other satellite products ($D_H$: 1.31, 1.49, 1.53, 1.66). This might be because of the influences of seasonal vegetation changes and snow cover in winter on satellite measurements [20]. In addition, some soil parameters that are related to the seasonal changes (e.g., soil texture and soil porosity) might also influence the accuracy of retrieval algorisms. It should be noted that although the reanalysis products showed good seasonal performances, they still had limitations in reflecting the increasing trend of SM in the post-monsoon season.

3.2. Quantitative Comparison and Evaluation against Ground Observation Sites

The discrete distribution and spatial variation of the four statistical metrics were firstly compared and analyzed. After that, the statistical metrics CC, p-value, bias and RMSE between multi-source SM data series from satellite/reanalysis SM products and the observed SM series by 838 ground observation stations were further calculated and adopted for quantitative evaluation on the satellite/reanalysis dataset quality.

3.2.1. Spatial Variation of Main Statistical Indexes

(a)

Correlation Coefficient

Most satellite/reanalysis SM products (e.g., ERA5, GLEAM and ESA-CCI) showed good spatial performances on the correlation coefficient with station-observed SM data, as presented by the densely and mutually distributed red (indicating CC of 0.5–1) and blue (indicating CC of 0.3–0.5) dots (Figure 6). Of these, the ERA5 soil moisture product showed the most distinguished spatial performances on CC, which had the largest area of CC between 0.5 and 1, whilst the AMSR2-ASC and -DES products showed the worst spatial performances on CC, as presented by the densely distributed light-yellow dots (indicating CC of 0.1–0.3) in the evaluated area. In addition, the CC of most satellite/reanalysis SM products showed variations with the spatial land uses. It can be found that the CC of Cropland in the central areas was higher than that in the marginally distributed Forestland and Grassland areas.

(b)

Bias and RMSE

Most satellite/reanalysis SM products showed negative bias (as greenish dots in Figure 7) compared with the station-observed SM data in the evaluated areas, indicating that the SM estimation by most satellite/reanalysis SM products was lower than the ground-observed SM. On the contrary, the SM estimation by ESA-CCI-ACT, AMSR2-ASC and -DES SM products showed positive bias compared to the ground-observed SM, indicating their SM overestimation in the evaluated area. In particular, ESA-CCI-ACT overestimated soil moisture over 94.6% of the ground observation stations, and bias in 44.7% of them even reached over 0.2 m³/m³.

Similarly, the RMSE of most satellite/reanalysis SM products compared to the ground-observed SM series distributed evenly in the evaluated area, which showed good performances in the Cropland areas of the North and Northeast China plains (as presented by the dense gray and light-yellow dots in Figure 8). Nevertheless, the ESA-CCI-ACT SM products showed larger RMSEs than the other products, most of which were above 0.2 m³/m³. Despite the reasonable RMSE of AMSR2-ASC and -DES SM products in the cultivated areas of the North and Northeast China plains, they had poor performances in the marginally distributed Forestland and Grassland areas, as presented by the red dots (>0.3 m³/m³).

3.2.2. Discrete Distribution Comparison of Four Statistical Indexes

Firstly, the statistical proportion of 838 observation sites in CC, p-value, bias and RMSE based on the comparison between nine satellite/reanalysis products and the observed data series is shown in Figure 9. The CC between most satellite/reanalysis SM products and the station-observed SM data showed a high proportion in the ranges of 0.3–0.5 and 0.5–1, reflecting their good correlations. Correspondingly, the highest proportions of p-value were in the range of <0.01, followed by 0.01–0.05, and >0.05 (indicating not significant) accounted for the lowest proportion. Of these, ERA5 showed the best performance in CC, as presented by the highest proportion in the ranges of 0.3–0.5 and 0.5–1 of CC, which accounted for 91.2% of the total SM observation sites. Meanwhile, AMSR2 soil moisture products showed the worst performance in CC, as presented by the highest proportion in the ranges of <0.1 and 0.1–0.3 of CC, which accounted for 67.6% of the total SM observation sites. Further, AMSR2 had the highest proportion (44.5%) in p-value within the range of >0.05.

The absolute bias (|Bias|) and RMSE between most satellite/reanalysis SM products and the station-observed SM data were in a reasonable range, as presented by the high proportion in the ranges of 0–0.1 and 0.1–0.2. Of these, GLEAM showed the highest proportion, reaching 93.2% and 87.9% in the range of 0–0.2 of |Bias| and RMSE. ESA-CCI-PAS and ERA5 also showed much higher proportions in the ranges of 0–0.1 and 0.1–0.2 of |Bias| and RMSE. Meanwhile, ESA-CCI-ACT had the worst performances in |Bias| and RMSE, with the lowest proportions of 55.3% and 19.7% in the range of 0–0.2, respectively.

Secondly, the distribution statistics of CC, p-value, bias and RMSE between multi-source SM products and the ground SM observations are presented by the box diagrams (Figure 10). The average CC ($\overline{CC}$) between most satellite/reanalysis SM products and the station-observed SM data was in the range of 0.3–0.7, fluctuating around 0.5, and the corresponding p-values were below 0.05, as presented by the first quantile Q1 which was under the red short dot line (p = 0.05). Of these, ERA5 showed the largest CC with the observed data series ($\overline{CC}=0.51$) and the lowest p-value (far below the line of p = 0.05). By contrast, AMSR2 showed a lower CC (0.1–0.3, $\overline{CC}=0.23$) than the other products, and the corresponding p-values were mostly larger than 0.05, as shown by Q3 which was above the p = 0.05 line.

The bias and RMSE between most satellite/reanalysis SM products and the station-observed SM data were in the ranges of −0.1–0.1 m³/m³ (fluctuating around ±0.05 m³/m³) and 0.05–0.15 m³/m³ (fluctuating around 0.13 m³/m³), respectively, which showed that the error of most products compared to the measured soil moisture was small. In particular, ESA-CCI-PAS had the lowest bias ($\overline{Bias}=-0.03$ m³/m³), followed by GLEAM-V3.3a ($\overline{Bias}=-0.04$ m³/m³) and GLEAM-V3.3b ($\overline{Bias}=-0.05$ m³/m³). Further, the GLEAM datasets showed the lowest RMSE ($\overline{RMSE}=0.12$ m³/m³), and ERA5 and ESA-CCI-PAS ($\overline{RMSE}=0.14$ m³/m³) were next in this trend. Despite those products being within a rational range, ESA-CCI-ACT presented the largest errors both in bias and RMSE compared to the measured SM series, which reached 0.1–0.3 ($\overline{Bias}=0.18$) m³/m³ and 0.2–0.3 ($\overline{RMSE}=0.26$) m³/m³.

3.3. Performance Assessment of Nine Multi-Source Satellite SM Products under Different Land Covers and Wet–Dry Conditions

3.3.1. Performances under Different Land Uses

The 838 ground SM observation stations were divided into Cropland, Forestland, Grassland and Mixed land groups, by matching their locations with the grid in Remote Sensing Monitoring Data on Land Use in China (2015). The CC, p-value, bias and RMSE between multi-source SM products and ground SM observations under different land uses were statistically analyzed and presented by the box diagrams. As shown in Figure 11. a and c, the differences in CC and p-value between the nine multi-source SM products and those measured under different land uses were not obvious, but Forestland was the lowest among the four land uses. For the ESA-CCI-PAS product, CC in Forestland ($\overline{CC}=0.32$) was much lower than that in Grassland ($\overline{CC}=0.43$), which was down by 25.6%. A similar phenomenon was also found in AMSR2-ASC, AMSR2-DES, ESA-CCI-ACT and ESA-CCI-COM, down by 23.1%, 31.3%, 7% and 19.1%, respectively. Further, CC of Cropland was in between Grassland and Forestland for most multi-source SM products. The corresponding p-value presented a high correlation with CC under different land uses.

The bias between the satellite/reanalysis SM products and the station-observed SM data series showed variations with different land uses. For almost all the SM products, bias in Forestland and Grassland was larger than that in the other land uses, as presented by the higher median line in Figure 11b. This indicated that most satellite/reanalysis SM products usually estimated a higher SM value in Forestland and Grassland areas than that in Cropland or Mix land areas, especially for ESA-CCI-ACT and AMSR2 soil moisture products. For instance, the SM estimation by AMSR2-ASC in Forestland ($\overline{Bias}=0.13$ m³/m³) and Grassland areas ($\overline{Bias}=0.13$ m³/m³) was much larger than that in Cropland ($\overline{Bias}=0.03$ m³/m³) or Mix land areas ($\overline{Bias}=0.03$ m³/m³). Similarly, for ESA-CCI-ACT and -PAS and AMSR2 soil moisture products, the RMSEs in Forestland and Grassland areas were larger than those in Cropland or Mix land areas, as presented by the higher median line (Figure 11d). This indicated that SM estimation by those products in Forestland and Grassland areas was not as accurate as that in Cropland or Mix land areas. For example, the average RMSE of AMSR2-ASC in Forestland ($\overline{RMSE}=0.22$ m³/m³) and Grassland areas ($\overline{RMSE}=0.22$ m³/m³) was much higher than that in Cropland ($\overline{RMSE}=0.15$ m³/m³) or Mix land areas ($\overline{RMSE}=0.16$ m³/m³), whilst for the other SM products, RMSEs in Forestland and Grassland areas were lower than those in Cropland or Mix land areas, indicating a better SM estimation in Forestland and Grassland areas.

3.3.2. Performances in Different Wet–Dry Areas

The pairwise correlation between different multi-source SM datasets was separately calculated in the four dry and wet partitions (Figure 12), and the CC, p-value, bias and RMSE of different multi-source SM products compared to the station-observed SM data were further analyzed (Table 1). The overall CC between different SM datasets showed the best performances in the semi-arid areas, which was mostly in the range of 0.37–0.96. For the wet and semi-wet areas, CC presented coincident regularities among different SM datasets: most multi-source SM products showed good performances, except the ESA-CCI-PAS and AMSR2 products which had a lower CC with mostly less than 0.3, even negative values. The CC of multi-source SM datasets showed the worst performances in the arid areas, with a high proportion (accounting for ~31.1%) in the range of less than 0.3.

As shown in

Table 2. Performance evaluation of multi-source SM products in four different wet–dry areas. (Herein, ** indicates significant correlation at p < 0.001, * indicates significant correlation at p < 0.05 and red number indicates non-significant correlation.)

Wet–Dry Areas	Statistical Indexes	GLEAM −3.3a	GLEAM -v3.3b	ESA-CCI -ACT	ESA-CCI -COM	ESA-CCI -PAS	ERA5	AMSR2 -ASC	AMSR2 -DES	GLDAS -Noah
Semi-humid	CC	0.60 **	0.57 **	0.4 **	0.58 **	0.5 **	0.8 **	0.32 **	0.38 **	0.44 **
	Bias	−0.06	−0.07	0.15	−0.14	−0.05	−0.1	0.06	0.05	−0.15
	RMSE	0.07	0.08	0.19	0.15	0.08	0.1	0.1	0.08	0.16
Humid	CC	0.6 0**	0.58 **	0.29 **	0.36 **	0.51 **	0.84 **	0.39 **	0.31 **	0.33 **
	Bias	−0.08	−0.09	0.09	−0.11	0.06	−0.08	0.2	0.23	−0.11
	RMSE	0.09	0.09	0.15	0.12	0.13	0.08	0.24	0.27	0.12
Semi-arid	CC	0.62 **	0.54 **	0.52 **	0.59 **	0.55 **	0.72 **	0.45 **	0.39 **	0.37 **
	Bias	−0.01	−0.03	0.26	−0.05	0.01	0	0.14	0.12	−0.07
	RMSE	0.04	0.05	0.28	0.06	0.09	0.04	0.16	0.15	0.08
Arid	CC	0.37 **	0.29 *	0.2 *	0.21 *	0.23 *	0.29 **	0.14	0.22 *	0.08
	Bias	−0.06	−0.08	0.08	−0.03	−0.07	−0.13	−0.07	−0.09	−0.05
	RMSE	0.09	0.1	0.12	0.07	0.1	0.14	0.09	0.12	0.08

, most multi-source SM products had good performances in the CC (>0.5), p-value (p<0.01), bias (−0.1–0.1 m³/m³) and RMSE (0–0.15 m³/m³) over the semi-humid, humid and semi-arid areas. In particular, CC between the ERA5 and the station-observed SM data reached 0.8, 0.84 and 0.72 in humid, semi-humid and semi-arid areas. Nevertheless, several products (e.g., AMSR2-ASC and -DES, and ESA-CCI-ACT) had a larger RMSE (>0.2 m³/m³) in those areas. In addition, the multi-source SM products in the arid areas showed the worst CCs with the ground-observed SM series, which were mostly in the range of 0.2–0.37 and some even not significant.

4. Discussion

Soil moisture can be estimated by the traditional point-based ground measurements, or by the newly arising remote sensing techniques, as well as some land surface assimilation systems at the regional/global scale. Based on those methods, numerous ground SM observation datasets and satellite/reanalysis SM products have been developed. Each single dataset has its own advantages and disadvantages; thus, the fusion of multi-source SM datasets receives more focus in current studies [47,48]. However, the preliminary evaluation on the quality and error structures of different datasets is crucial for further data fusion. This work examined the performances of nine multi-source SM products over the central and eastern agricultural areas of China, using an in situ ground monitoring network. Herein, for the first time, the agricultural irrigated areas of China were specially chosen for the evaluation study on multi-source SM products, and the evaluated results could provide guidance for fusing different satellite/reanalysis products, as a new feasible alternative to monitoring SM information. Moreover, the effects of land use types and wet–dry conditions on SM estimation were specifically compared, and the performances between satellite and reanalysis SM products were further discussed.

4.1. Quality of the Ground-Measured Soil Moisture

The quality of the ground-measured soil moisture is important, which concerns the reliability of the overall evaluated results in this work. Herein, two aspects, i.e., the impact of topography conditions and the spatial heterogeneity, on the accuracy of the measured SM were mainly discussed. First, the impact of topographical changes on SM measurement can be negligible. The ground observation stations adopted the gravimetric method for measuring soil moisture, which is conducted by manual sampling, not by any automatic instruments, and samples are analyzed in the laboratory [29]; thus, such method will, at least, be influenced by the changing topographical conditions. Second, the spatial heterogeneity on the ground-measured SM involves the representation of ground observation stations. Previous studies have illustrated the uncertainty in evaluating the satellite/reanalysis SM datasets by using the point-based observation SM data as the reference, due to their different spatial resolutions [12,14,15]. A sparse monitoring network usually has unsatisfactory regional representations due to large spatial heterogeneity, and vice versa. Thus, a dense SM monitoring network with better representation is crucial to assess the satellite/reanalysis SM products. In this work, the study region covers an area of 2,194,503 km², which contains 838 ground observation sites, indicating a ~50 km spatial resolution (transformed to a 50 × 50 km grid). Such resolution is relatively high compared to the 25 km satellite/reanalysis datasets, which can reduce some uncertainty of the evaluation results.

4.2. Effects of Land Uses on Soil Moisture Estimation

Generally, vegetation cover is recognized as a critically important parameter in the soil moisture retrieval algorism for many satellites [49]. Many microwave SM products had been proven to have good performances in areas with sparse to moderate vegetation conditions and have unsatisfying performances over dense vegetation areas (e.g., tropical forest) where the forest canopy can intercept the microwave penetration [19,50]. Herein, this work has found the lowest CC and a much larger bias and RMSE in Forestland areas among the four land use types, particularly for the ESA-CCI-PAS, AMSR2, ESA-CCI-ACT and ESA-CCI-COM soil moisture products (Figure 11). Further, despite the high CC of Grassland areas, most SM products in those areas showed a larger bias and RMSE. This indicated that most satellite/reanalysis SM products had poor performances in Forestland and Grassland areas and usually estimated a higher SM value than that in Cropland or Mix land areas, as presented by the larger positive $\overline{Bias}$ and $\overline{RMSE}$. Such phenomenon that the errors of the SM retrieval increased with increasing vegetation intensity is consistent with other previous studies [12,20,51,52]. The causes of the overestimation on soil moisture over Forestland and Grassland areas are directly related to two vegetation parameters in the radiative transfer models: the vegetation geometry and the vegetation water content (VWC) [26]. The vegetation geometry parameter is usually hard to estimate, which was not involved in the retrieval algorisms for those satellite SM products. Adding that the VWC in those dense forest areas was estimated with high uncertainty, those factors contribute to the larger bias in Forestland and Grassland areas than that in Cropland and Mix land areas. It is noteworthy that unlike the satellite products, bias variations between the Forestland/Grassland areas and the other land use types were not obvious among those reanalysis SM products (Figure 11b,d). The possible reason is that reanalysis products controlled the data quality through assimilating various sources of datasets, especially the ground observation data, which can eliminate bias caused by uncertain underlying conditions (e.g., vegetation covers and land surface roughness) to some extent.

4.3. Effects of Wet–Dry Conditions on Soil Moisture Estimation

In terms of CC, the overall CC between different SM datasets showed good performances in the semi-arid (0.37–0.72), humid (0.29–0.84) and semi-humid areas (0.32–0.8), whilst that in the arid areas presented poor performances (0.08–0.37), and some were not even significant (Figure 12, Table 2). The bias and RMSE in humid and semi-arid areas were larger than those in semi-humid and arid areas, especially for several satellite SM products (e.g., AMSR2-ASC and -DES, and ESA-CCI-ACT), of which the largest bias and RMSE reached 0.26 and 0.28 m³/m³ in semi-arid areas (Table 1). The semi-arid/arid areas are known for systematic errors due to the fairly dry surface soil, which hampers the reading of microwave-based retrieval systems [53]. First, the semi-arid/arid areas add challenges in estimating the thickness of the effective temperature and the emitting layer [54]. Second, the surface SM in arid/semi-arid areas is relatively low, which may be influenced by the background noise of the instrument, adding difficulties in the SM retrieval by microwave observations [55]. This could explain the worst performances of CC in arid areas. Additionally, surface roughness can cause SM overestimation/underestimation in arid/semi-arid areas, due to the closeness/further distance of the remote sensors, changing the local incidence angle with respect to the sensor [56]. In fact, the evaluated arid/semi-arid areas were located in the northwest of China, with highly fractured mounds in those regions, which increased the difficulties in estimating SM accurately. As for the large bias and RMSE in humid areas for satellite SM products (especially for AMSR2), these may be influenced by the open water surfaces in those areas [57]. The evaluated humid areas located in the southeast of China have a developed network of rivers, lakes, wetlands and ponds, which might significantly contaminate the SM retrievals. Such overestimation on SM by microwave-based retrieval was also found in wet areas such as the south, east and southeast of China [26]. Similarly, those reanalysis products could overcome the uncertainty problems caused by surface roughness and small portions of the open water surface, by taking more ground observations into the assimilation system.

4.4. Performance Contrast between Satellite and Reanalysis Soil Moisture Products

Overall, the reanalysis SM products outperformed the satellite SM products in those evaluated areas. Most reanalysis SM products showed good spatial performances in capturing the wet–dry areas (with 1~3 well-matched areas) as well as reflecting the spatial distribution variations in ground-measured soil moisture (Figure 4). Moreover, the reanalysis SM products could effectively capture the temporal changes in soil moisture and reflect the seasonality of the surface SM distribution, as presented by the larger CC, lower centered RMSE and shorter Hausdorff distances compared to the observed ground SM data series (Figure 3 and Figure 5). Such spatial–temporal correlations could be further verified by the CC between most reanalysis SM products and the station-observed SM data, which showed a high proportion in the ranges of 0.3–0.5 and 0.5–1 (with $\overline{CC}$ in the range of 0.3–0.7, fluctuating around 0.5), reflecting their good correlations (Figure 9 and Figure 10). In particular, the CC between ERA5 and the station-observed SM data even reached 0.8, 0.84 and 0.72 in wet, semi-wet and semi-arid areas (Table 1). Moreover, as for the SM estimation, most reanalysis SM products were largely approximate to the measured SM content (within 0 to 0.6 m³/m³), except for GLDAS-Noah (usually underestimation). Take, for example, the SM estimations by ERA5 ($\overline{sm}$, ~0.28 m³/m³) and GLEAM-V3.3a/V3.3b ($\overline{sm}$, ~0.26 m³/m³), which were closest to the ground-observed SM content ($\overline{sm}$, ~0.28 m³/m³) (Figure 5). Such SM estimation accuracy could be ascertained by the bias and RMSE evaluation results: ERA5 and GLEAM soil moisture products showed a higher proportion in the range of 0–0.1 of |Bias| and RMSE, and lower $\overline{Bias}$ (e.g., GLEAM-V3.3a: −0.04 m³/m³) and $\overline{RMSE}$ (e.g., GLEAM-V3.3a: 0.12 m³/m³) compared to other products (Figure 9 and Figure 10). Previous studies have also reported the good performances of ERA5 and GLEAM SM products and indicated their prospective benefits for local hydrological applications such as drought monitoring and water management [39,57,58,59].

Those satellite SM products did not present satisfactory performances as reanalysis products. Despite the good performances of ESA-CCI-Combined products, most satellite SM products (especially for ESA-CCI-ACT, AMSR2-ASC and -DES) had low capacity in capturing different dry–wet areas, and in reflecting the spatial–temporal distribution changes of actual measured SM. This could be verified by the station-based CC results, which had the lowest proportion in the ranges of 0.3–0.5 and 0.5–1 and lower $\overline{CC}$ than other products, reflecting their poor correlations (Figure 9). The CC (mostly less than 0.45) of ESA-CCI-Act, AMSR2-ASC and -DES products also proved their poor correlations with the observed data series in different wet–dry areas (Figure 10). The ESA-CCI-Act and AMSR2-ASC and -DES products also significantly overestimated SM compared to the observed data series, particularly in the semi-humid/humid and semi-arid areas (Figure 3 and Figure 5). Such overestimation on SM could be further demonstrated by the bias and RMSE evaluation results (Figure 10). For example, ESA-CCI-Act had no proportion in the range of <0.1 of RMSE and presented the largest errors both in bias and RMSE, reaching within 0.1 to 0.3 ($\overline{Bias}=0.18$) m³/m³ and 0.2 to 0.3 ($\overline{RMSE}=0.26$) m³/m³. Cho et al. [60] also found that AMSR2-ASC and –DES had large errors in SM estimation over Korean peninsula areas. The quality of those satellite SM products can be influenced by various factors, which not only rest on the remote sensors themselves, and some assumptions and parameters in transfer models, but also the topographical and vegetation conditions [46]. In addition, effects of human activities (e.g., RFI) on satellite SM products have been demonstrated in many areas [57,61].

In sum, the reanalysis SM products were proven to outperform those satellite products in estimating the surficial SM of ~10 cm soil layers. In fact, there are still many other advantages of the reanalysis SM products. Generally, the current satellite remote sensing could only reach a limited penetration depth, while the reanalysis products could obtain more SM information at deeper depths by using data assimilation systems [38,41,57]. For instance, four layers of SM have been contained in the ERA5 products: 0–7 cm; 7–28 cm; 28–100 cm; and 100–289 cm. The stratified depths of soil moisture are crucial for knowing the vertical SM distribution and implying the hydrological processes [62,63]. Moreover, most satellite SM products could only provide diurnal data at ~25 km spatial resolution, while some reanalysis SM products (e.g., ERA5) have reached a much higher temporal resolution of 1 h and a spatial resolution of ~10 km [37]. Deeper stratified depths and higher spatial–temporal resolution render the reanalysis products a promising and reliable data source for improving weather forecasts, drought and flood predictions, agricultural monitoring and water management [59]. However, the data latency of reanalysis SM products is an inevitable issue, due to their quality control and assimilation processing coming from various sources (e.g., ground, ships, radio sounding, aircraft and satellites). Although the preliminary data of ERA5 update the dataset which is available within 5 days of real time, more accurate datasets will be produced in ~3 months. Thus, in terms of the demand in many hydrometeorological applications, the updated reanalysis SM products with shorter latency or the combination with the satellite products (diurnal) should be further considered in future study.

It should be noted that the gravimetric SM measuring method, although regarded as the most accurate method to best characterize the true value of land soil moisture, cannot dismiss the potential errors existing in the unit conversion process from gravimetric to volumetric SM. Furthermore, scale mismatch problems are inevitably uncertainty sources in evaluation on those satellite/reanalysis products. Consequently, the error reduction during the conversion process and the proper upscaling/downscaling techniques should be highlighted in future study. In addition, fusion of multi-source SM datasets could provide a promising approach in hydrological application and research.

5. Conclusions

This work firstly proposed an evaluation framework to investigate the widely used satellite SM products (AMSR2 and ESA-CCI) and the reliable reanalysis SM datasets (ERA5, GLEAM and GLDAS) over central and eastern agricultural areas of China and evaluated their performances under different land uses and wet–dry areas. First, the capacity in capturing spatial–temporal variations in different satellite/reanalysis SM products was compared, using the dense in situ SM monitoring network with a total of 838 measuring stations. Second, the SM estimation accuracy of each product with respect to the station-measured SM was quantitatively evaluated, and the error structures and spatial distribution were analyzed. Finally, the effects of different land use types and wet–dry conditions on the performances of multi-source SM products were further assessed. The major findings of this work are as follows:

• Most satellite/reanalysis SM products could capture the spatial–temporal changes in soil moisture. In particular, the ERA5 soil moisture products outperformed the other products, which presented the highest correlation with the station-measured SM series, whilst the ESA-CCI-ACT and -PAS and AMSR2 products showed the worst spatial–temporal performances, as presented by their poor correlations and large errors in soil moisture estimation.
• The reanalysis SM products could better reflect the seasonality of the surface SM distribution than those satellite products, with a higher CC and trajectory similarity to the seasonal changes of the ground SM data series; however, they still had shortages in detecting the increasing trend of SM during the post-monsoon season.
• CC, p-value, bias and RMSE between most satellite/reanalysis SM products and the station-observed SM data quantitatively demonstrated their good performances on estimating soil moisture. The accuracy of SM estimation by ERA5 and ESA-CCI-COM was highest, while that by AMSR2 and ESA-CCI-Act was the lowest among all those products.
• Most satellite/reanalysis SM products had poor performances in Forestland and Grassland areas and usually overestimated the SM value compared to that in Cropland or Mix land areas. Such phenomenon was much more obvious for those satellite SM products, due to the difficulty in effectively estimating the vegetation geometry and the VWC parameters in their retrieval algorisms.
• The arid areas showed the worst performances in the overall CC between the station-observed SM data and different satellite/reanalysis SM products, for the reason that the dry surface soil can hamper the reading of microwave-based retrieval systems; meanwhile, the humid and semi-arid areas presented larger SM estimation errors than the other areas, especially for AMSR2 and ESA-CCI-ACT products, which were greatly influenced by the open water surfaces in humid areas and surface roughness in arid/semi-arid areas, respectively.
• The reanalysis SM products outperformed the satellite SM products in those evaluated areas, which showed better spatial–temporal performances and higher accuracy on SM estimation. Further, for those reanalysis SM products, the estimation error under different land use types and wet–dry areas could be eliminated to some extent, possibly by assimilating various sources of datasets, especially the ground observation data with high quality.