Evaluation of Satellite-Based Soil Moisture Products over Four Different Continental In-Situ Measurements

Abstract

Global, near-real-time satellite-based soil moisture (SM) datasets have been developed over recent decades. However, there has been a lack of comparison among different passing times, retrieving algorithms, and sensors between SM products over various regions. In this study, we assessed seven types of SM products (AMSR_A, AMSR_D, ECV_A, ECV_C, ECV_P, SMOS_A, and SMOS_D) over four different continental in-situ networks in North America, the Tibetan Plateau, Western Europe, and Southeastern Australia. Bias, R, root mean square error (RMSE), unbiased root mean square difference (ubRMSD), anomalies, and anomalies R were calculated to explore the agreement between satellite-based SM and in-situ measurements. Taylor diagrams were drawn for an inter-comparison. The results showed that (1) ECV_C was superior both in characterizing the SM temporal variation tendency and absolute value, while ECV_A produced numerous abnormal values over all validation regions. ECV_P was able to basically express the SM variation tendency, except for a few overestimations and underestimations. (2) The ascending data (AMSR_A, SMOS_A) generally outperformed the corresponding descending data (AMSR_D, SMOS_D). (3) AMSR exceeded SMOS in terms of the coefficient of correlation. (4) The validation result of SMOS_D over the NAN and OZN networks was unsatisfactory, with a rather poor correlation for both original data and anomalies.

1. Introduction

Soil moisture (SM) plays an important role in storing and exchanging water on the land surface, and a significant feedback mechanism between SM and corresponding regional climate change exists [1,2]. In addition, it is of profound significance to analyze the effect of SM content variation on evapotranspiration [3], precipitation [4], heat extremes [5], and vegetation growth [6]. As a result, SM products with reliable quality and spatial-temporal sequences are a prerequisite to monitor the fluctuations in environmental, meteorological, ecological, and agricultural elements across the globe.

Since the 1980s, airborne and spaceborne remote-sensing technology have continually developed and have become increasingly popular, which has made obtaining near-real-time, global remote-sensing band information a reality [7,8]. Meanwhile, satellite-based SM-retrieving algorithms have also rapidly developed. Chang et al. showed that there is a positive correlation between L band data and SM in bare soil [9]. On the basis of the inversion of surface parameters via a microwave interferometer [10], Pellarin et al. [11] and Wigneron et al. [12] developed a model to split brightness temperature into a weighted average form of four land surface cover types (bare soil, forest, grassland, and water). Under the guidance of these researchers, Kerr et al. [13] put forward the Soil Moisture and Ocean Salinity soil moisture (SMOS_SM) inversion algorithm, which is based on using the radiative transfer model of different land cover types to obtain the integrated pixel brightness temperature and then adopting the iterative method to obtain the optimal combination of parameters. Additionally, by synthesizing the microwave brightness temperature from the C, X, and Ka bands, Njoku and Li [14] proposed an iterative algorithm to simultaneously calculate the SM and vegetation water content as well as land surface temperature, termed the Advanced Microwave Scanning Radiometer (AMSR-E, which was launched in 2002 by the National Space Development Agency of Japan) algorithm. Njoku and Chan [15] subsequently improved the original model and developed the Normalized Polarization Difference Algorithm (NPDA). Moreover, with the increasing variety of SM products and their own working modes of sensor-based merits and drawbacks, a multi-source merged SM retrieval algorithm appeared. The Essential Climate Variable (ECV) soil moisture (ECV_SM) algorithm, initiated by the European Space Agency (ESA), was synthesized using four passive and two active microwave products [16]. It provided a long time series (from 1 November 1978 to 31 December 2016), daily, and global SM dataset with a coarse resolution of 0.25°.

Various microwave remote-sensing-based SM products have exhibited unprecedented data accessibility and availability in large-scale, long time series SM and related hydrological element modeling and analyses. However, the spatial heterogeneity caused by natural condition differences increases the uncertainty of SM within a pixel that covers dozens to hundreds of square kilometers. In addition, radio frequency interference (RFI) can be an obstacle in microwave signal reception, which can result in an inaccurate SM value or even a gap area [17,18]. In addition, each SM inversion as well as merging model has its own conditional and regional applicability, and the SM data accuracy may vary as the corresponding land cover type or hydrothermal condition changes [19,20,21]. Thus, it is necessary to conduct assessments and evaluations of microwave remote sensing-retrieved SM products to clarify their quality in both variation tendency and value. Numerous studies focusing on evaluating satellite SM products using ground-based observations have been completed. Mei Sun Yee et al. [22] compared the performance of the AMSR2 and SMOS SM products using in-situ measurements in Southeastern Australia and concluded that the AMSR2 product performed better both in terms of absolute and temporal accuracy of SM. Jian Peng et al. assessed four microwave-based SM products and one reanalysis SM product over Southwest China [23]. The accuracy levels showed that both the reanalysis and ECV_SM products achieved satisfactory validation results. Dorigo et al. [24] assessed ECV_SM using ground-based measurements worldwide from 1979 to 2010 and found that ECV_SM had an increasing trend over time in general in terms of quality, except for a decreasing fitting trend during the period from 2007–2010.

Although many researchers have analyzed the degree of accuracy among different SM products, few have compared the applicability degrees of different SM products among diverse regions across the world. Consequently, in this study, ECV_SM (including active, passive, and combined products, individually named ECV_A, ECV_P, and ECV_C), SMOS_SM (including ascending and descending data, individually named SMOS_A and SMOS_D), and AMSRE_SM and AMSR2_SM (including ascending and descending data, individually named AMSR_A and AMSR_D) were separately evaluated over four dense in-situ networks in the south-central region of the United States of America, Northwestern Iberian Peninsula, the Tibetan Plateau, and Southeastern Australia. The objective of our study was to explore and differentiate the adaptability and accuracy of the different SM products over different regions globally.

2. Study Areas and Data Resources

2.1. Study Areas and In-Situ Measurements

To evaluate the applicability of global SM products over different continents, four different continental regions were used to separately assess the performance of satellite-based SM products: Oklahoma in the USA, Northwestern Spain, the Central Tibetan Plateau, and Southeastern Australia. In addition, these regions all had relatively stable and long-time-series observations of SM during recent years.

(1) Oklahoma Mesonet (OKM)

The Oklahoma Mesonet (OKM) network was built jointly by the Oklahoma State University and the University of Oklahoma [25]. As shown in Figure 1a, the OKM network is situated in the south-central United States of America. With more than one hundred stations in total, this network has been broadly used in multidisciplinary hydrologic management, such as drought monitoring, water balance exploration, and remote sensing SM assessment. It monitors daily SM values at 5 cm, 25 cm, 60 cm, and 75 cm depths. Furthermore, Oklahoma has a temperate continental climate with four clearly distinct seasons. The temperature increases from the northwest to the southeast, corresponding to the terrain (which is relatively higher in the northwest and lower in the southeast), while the precipitation decreases from east to west. With a mean temperature of 15.5° Centigrade, the relatively dry climate throughout the year results in slightly higher SM during winter compared to that during summer, as the evaporation during summer is more intense than that during winter. The land cover types, station numbers, and station percentages in the OKM area are shown in

Table 1. Land cover types, station percentages, and station numbers in the OKM area.

Land Cover Type	Station Number	Station Percentage
Cropland	45	45.45%
Grassland/Shrubland	29	29.29%
Sparse Deciduous Forest	8	8.08%
Herbaceous Vegetation	17	17.17%

. With many flowing rivers, this area is mainly covered by rainfed croplands, broadleaf deciduous forest, and herbaceous vegetation. Considering the depth limitation of satellite penetration, we used the 5 cm measured values to validate the quality of the satellite-based SM.

(2) REMEDHUS (REM)

The REMEDHUS (REM) is a dense observational network with twenty four SM stations within a 35 × 35 km area in northwest Spain [26]. Figure 1b shows the specific distribution of each monitoring site. It provides hourly 0–5 cm SM records as well as soil temperature via a type of sensor termed a Hydra Probe. In addition to evaluating satellite SM products across scales, REMEDHUS has also been engaged in the calibration of a hydrologic model [27]. Additionally, affected by North Atlantic Drift, northwest Spain belongs to a temperate maritime climate that includes considerable rainfall and a humid climate. Its annual precipitation is approximately 800 mm which largely falls during winter. Comparatively, summer is warm and dry [28]. Influenced by seasonal variation in precipitation and temperature, SM is high during winter and low during summer.

Table 2. Land cover types, station percentages, and station numbers in the REM area.

Land Cover Type	Station Number	Station Percentage
Cropland	13	65%
Grassland/Shrubland	1	5%
Sparse (<15%) Vegetation	6	30%

shows the land cover type, station percentage, and station number in the area. The Douro river flows through the plains supporting large tracts of farmland via irrigation. In addition, 65% of the stations are situated in cropland areas. There is also a minor distribution of shrubland and forest.

(3) Naqu Network (NAN)

As seen in Figure 1c, the Naqu Network (NAN) is situated in the Central Tibet Plateau which has an average altitude of 4000–5000 m and an alpine plateau climate. Basically, the year is divided into a cold, dry winter and a humid, cool monsoon season. Affected by the South Asian summer monsoon, its precipitation is intense during summer and approximately 400–500 mm annually.

Table 3. Land cover type, station percentage, and station number in the NAN area.

Land Cover Type	Station Number	Station Percentage
Cropland	6	10.53%
Grassland/Shrubland	43	75.33%
Herbaceous Vegetation	8	14.04%

shows the land cover types, station percentages, and station numbers in this area. The dominant land surface cover type of alpine grasslands is the distinguishing feature of the region [29] and is also the main land cover type of the SM network stations (with 43 stations). Antecedent rainfall and evaporation are dominant in the SM variation, which could change from residual to saturated contents [30]. Because of its climatic particularity and ecological fragility, the Tibetan Plateau is termed the third pole [31]. Thus, it is critical to systematically validate the accuracy of satellite SM products over the Tibetan Plateau when globally assessing the validity and accuracy of certain satellite SM products. The sensors monitored and recorded SM every 30 min at four depths (5 cm, 10 cm, 20 cm, and 40 cm). We used the 5 cm measurements to evaluate microwave SM products.

(4) OZNET (OZN)

OZNET (OZN) is situated in the Murrumbidgee River catchment in Southeastern Australia [22]. Figure 1d shows that most monitoring stations are in the central and southeast parts of the area with stable, low elevation. This area has a temperate marine climate. Although the climate is similar to that of Spain, the seasons are quite the opposite owing to the opposite temperature change rhythm between the Northern and Southern hemispheres. The mean daytime temperature is 32° Centigrade during January and 13° Centigrade during July, and the approximate 400 mm of annual precipitation is evenly distributed throughout the year. As is shown in

Table 4. Land cover types, station percentages, and station numbers in the OZN area.

Land Cover Type	Station Number	Station Percentage
Cropland	31	83.78%
Grassland/Shrubland	2	5.41%
Sparse (<15%) Vegetation	1	2.70%
Artificial Surfaces	3	8.11%

, the area in which the stations are located is composed of cultivated land (31 stations), native vegetation (three stations), and built-up area (three stations) [32]. Flood irrigation during November, artificial irrigation, and well-distributed precipitation constitute the main factors that increase the SM [22]. In addition to SM, several stations measure soil temperature and precipitation as well. OZN data has been broadly used to validate satellite SM products [33], Moderate Resolution Imaging Spectroradiometer (MODIS, which was a key part of Earth Observation System launched by National Aeronautics and Space Administration) evapotranspiration [34], and land surface models [35]. In addition, OZN monitors hourly SM at five different depths (0–5 (or 0–8), 0–30, 30–60, and 60–90 cm) [36]. Given that only surface SM can be detected via the microwave sensor, we applied topsoil layer measurements and averaged them to a daily value during the evaluation process.

2.2. Satellite-Based SM Products

(1) AMSR-E and AMSR2

The Advanced Microwave Scanning Radiometer-Earth Observing System (AMSR-E) was onboard the EOS-Aqua satellite, which was launched in 2002 by the National Space Development Agency of Japan (NASDA) [37]. EOS-Aqua was a solar synchronous satellite and scanned the earth at 1:30 p.m. local time (ascending) and 1:30 a.m. local time (descending). This microwave scanning radiometer instrument provided data over six bands (from 6.9 to 89 GHz) using a horizontal vertical dual polarization mode and twelve channels in total, from 19 June 2002 to 2 October 2011. In addition, both the 6.9-GHz (C band) and 10.7-GHz (X band) bands offered brightness temperature data. The brightness temperature measured by the 10.7-GHz X band was mainly employed in the land parameter retrieval model (LPRM) considering that the C band is susceptible to RFI (Radio Frequency Interference) [38]. With the storage format of a network, Common Data Form (NetCDF), the daily SM product had a spatial resolution of a 0.25° global grid. Moreover, the frozen area was masked out via land surface temperature data during the preprocessing stage so as to avoid inaccurate retrieval values [39].

As a follow on from AMSR-E, the Advanced Microwave Scanning Radiometer 2 (AMSR2) has the same overpass times and adds a new 7.3-GHz frequency band. AMSR2 was launched on 18 May 2012 on board the Global Change Observation Mission 1st-Water satellite (GCOM-W1, developed by Japan Aerospace Exploration Agency) [40,41]. Horizontal polarization channel brightness and vertical polarization channel brightness temperature as well as time records are the three main datasets provided by this instrument, and the Hierarchical Data Format (HDF5) is used as the storage format.

(2) Soil Moisture Ocean Salinity (SMOS)

The Soil Moisture Ocean Salinity (SMOS) satellite, successfully launched on 2 November 2009, by the European Space Agency (ESA), is currently the only one in the world that can simultaneously observe changes in SM and salinity [13,42]. The SMOS can obtain global coverage data every three days with ascending and descending orbits at 6 a.m. local solar time and 6 p.m. local solar time, respectively. Additionally, the synthetic aperture radar equipped on the SMOS satellite can penetrate to a 5-cm depth in the soil at a frequency of 1.4 GHz (band) with vegetation coverage of up to 5 kg/m² vegetation water content. The retrieved daily 0.25° pixel resolution SM product has been available since 15 January 2010. Furthermore, this instrument also provides a 3-day and 10-day arithmetic averaged SM product as well as a daily global SM product [43]. In this study, we used the level 3 (L3) daily SMOS SM product acquired from the Centre Aval de Traitement des Données SMOS (CATDS). Compared to the level 2 products, the SMOS L3 SM has both temporally and spatially improved its features via resampling and processing in version 3.0 which was released on 20 April 2016 [44]. Also, its theoretical uncertainties are no more than 0.04 m³ × m⁻³.

(3) ECV

Aimed at monitoring the long-term tendencies of linked global climate change elements, the ESA initiated the Climate Change Initiative (CCI) program, also known as the Global Monitoring of Essential Climate Variables (ECV) [45,46]. It is SM that acts as a key carrier in the process of water heat exchange on the underlying surface. Thus, the program acknowledged SM as one of the ECV and then merged six active and passive SM products to finally produce a unique, long time serial (from 1 November 1978 to 31 December 2016), daily 0.25° SM product [47]. Data from version 3.3 was utilized in our study. Because previous researchers have proved that active-based SM products behave better in arid and bare regions, while, in contrast, SM products with a passive basis have superior performance in relatively humid and thickly vegetated areas [19,48], the active, passive, and combined ECV (ECV_A, ECV_P and ECV_C) were analyzed and compared to explore the characteristics of each product, particularly the advantages of the combined ECV_C.

3. Assessment Strategy

Considering the penetration depth of the microwave, only the surface layer SM measurements (usually 5 cm or 8 cm) were used in the assessment. Additionally, abnormal values and trends could appear because of the sensitivity heterogeneity of in-situ SM sensors. To maintain the reality and reliability of the in-situ data, soil samples were periodically taken near stations to gravimetrically determine the SM content and the loggers recorded readings for calibration purposes [30]. In addition, the surface SM value is sensitive to surrounding environmental change (for example, artificial watering and short-term precipitation) and can considerably fluctuate over one day. All of the four ground-based networks monitored SM hourly and a null value occasionally appeared owing to sensor malfunction and ageing. Consequently, the arithmetic mean was calculated to access the daily SM value. Notably, to ensure the stability of the in-situ measurements, only SM data covering more than 12 h within one day was treated as valid data. Furthermore, the four SM networks that we selected have dense sites, thus there were usually several stations within an 0.25° (approximately 25 km) pixel. We retrieved the average values of these stations to decrease the heterogeneity between the point-scale and 0.25° pixel-scale SM data and to increase the representativeness of the in-situ measurements. The averaged data were utilized to represent the integral SM and as reference data to validate the satellite-based SM products.

The validation was processed based on SM ground networks to analyze the regional suitability. For the sake of evaluating the agreement between the satellite-based SM and in-situ measurements, four metrics were used to express the quality of the SM products. They were bias, correlation coefficient (R), root mean square error (RMSE) and unbiased root mean square difference (ubRMSD). Additionally, we developed a Taylor diagram as it can describe the performance of SM products via a single diagram with R, ubRMSD, and standard deviation (SD) together [49]. We utilized the Taylor diagram to enhance the contrast among the different metrics. Moreover, we calculated the anomalies to decrease the influence of seasonally resulted changes and then reanalyzed the correlation coefficient. The principle of the anomaly algorithm was a five-week sliding window [19]. If at least five days of data were available during the five-week period, the $\mathrm{ANO}\left(\mathrm{t}\right)$ could be obtained using Equation (6). All of the calculation equations were as follows:

$$Bias=\frac{{{\displaystyle \sum }}_{i=1}^n{S}_i-{{\displaystyle \sum }}_{i=1}^n{G}_i}{n},$$

(1)

$$R=\frac{{{\displaystyle \sum }}_{i=1}^n\left[\left(G_i-\overline{G}\right)\left(S_i-\overline{S}\right)\right]}{\sqrt{\left[{{\displaystyle \sum }}_{i=1}^n{\left(G_i-\overline{G}\right)}^2\right]}\sqrt{\left[{{\displaystyle \sum }}_{i=1}^n{\left(S_i-\overline{S}\right)}^2\right]}},$$

(2)

$$RMSE=\sqrt{\frac{{{\displaystyle \sum }}_{i=1}^n{\left(G_i-S_i\right)}^2}{n}},$$

(3)

$$ubRMSD=\sqrt{\frac{{{\displaystyle \sum }}_{i=1}^n{\left[\left(G_i-\overline{G}\right)-\left(S_i-\overline{S}\right)\right]}^2}{n}},$$

(4)

$$SD=\sqrt{\frac{{{\displaystyle \sum }}_{i=1}^n{\left(S_i-\overline{S}\right)}^2}{n}},$$

(5)

$$ANO\left(t\right)=S\left(t\right)-\overline{S\left(t-17:t+17\right)}$$

(6)

where $G_i$, $\overline{G}$, $S_i$, and $\overline{S}$ are the ground-based SM measurements at site i, averaged ground-based SM value of all sites, remote-sensed SM value at grid i, and averaged remote-sensed SM value of all grids, respectively. $S\left(t\right)$ is the SM value at day $t$, and $\overline{S\left(t-17:t+17\right)}$ is the average SM during the period (from $t-17$ to $t+17$).

4. Results

4.1. Regional Performance Comparison

4.1.1. OKM Area

There were 99 stations with relatively stable measurements within the OKM area. This area is mainly covered by rainfed croplands, broadleaved deciduous forest, and herbaceous vegetation.

Table 5. Daily bias, R, root mean square error (RMSE), and unbiased root mean square difference (ubRMSD) of seven SM products over the OKM area.

Metrics	Products
	AMSR_A	AMSR_D	SMOS_A	SMOS_D	ECV_A	ECV_C	ECV_P
Bias (m3 × m−3)	−0.001	0.047	−0.098	−0.093	0.128	−0.084	−0.048
R	0.589	0.627	0.578	0.607	0.643	0.679	0.654
RMSE (m3 × m−3)	0.123	0.136	0.145	0.140	0.231	0.098	0.114
ubRMSD (m3 × m−3)	0.106	0.108	0.091	0.088	0.185	0.042	0.079

shows the daily bias, R, RMSE and ubRMSD between the in-situ measurements and seven SM products over the OKM area. In addition, considering that the boxplots can effectively show the upper edge, upper four quantile, median, lower four quantile, lower edge, and outliers of an array, we used boxplots to intuitively reflect and compare the parameter values (Figure 2). As shown in Figure 2a, both the AMSR_D and ECV_A SM products overestimated the in-situ values, while underestimation occurred among all other products. Notably, AMSR_A had a tiny bias of a mere −0.001 m³ × m⁻³, as shown in Table 5, which is quite close to the in-situ measurements. In addition, the correlation coefficient varied from 0.578 (SMOS_A) to 0.679 (ECV_C). Moreover, when speaking to RMSE, ECV_A showed the maximum deviation between the observed value and the true value, while ECV_C showed the minimum. The situation was similar for the ubRMSD, which is an important basis for measuring the stability of the derived values. In general, AMSR_A and AMSR_D behaved similarly in the five indexes, and the superiority of AMSR_A was obvious in terms of bias. As shown in Figure 2, it is noteworthy that although AMSR_A, AMSR_D and SMOS_A, SMOS_D are of nearly the same quality, the ascending retrieved data always performed slightly better than the corresponding descending data. This phenomenon could have been a result of the difference in day/night land surface temperatures in the satellite-based SM disaggregation algorithms [50]. Furthermore, ECV_C outperformed all of the other products, except in terms of bias, and showed both excellent accuracy and stability in active-passive combined products.

We calculated the anomalies and then drew boxplots to analyze the correlation between seasonal fluctuation-eliminated satellite-based SM products and in-situ measurements (

Table 6. Daily bias, R, RMSE, and ubRMSD of the anomalies over the OKM area.

Metrics	Products
	AMSR_A	AMSR_D	SMOS_A	SMOS_D	ECV_A	ECV_C	ECV_P
Bias (m3 × m−3)	−0.00022	0.00035	−0.0004	−0.00052	−0.00087	−0.00011	0.00029
R	0.357	0.348	0.434	0.445	0.524	0.498	0.434
RMSE (m3 × m−3)	0.0551	0.0616	0.0765	0.0726	0.1358	0.0324	0.0546
ubRMSD (m3 × m−3)	0.0551	0.0616	0.0764	0.0725	0.1357	0.0324	0.0546

and Figure 3). On average, the ECV products, particularly ECV_A, had the best fitting trend with a high and stable R, as shown in Table 6 and Figure 3a, followed by SMOS. AMSR ranked last. However, SMOS and AMSR ranked oppositely in terms of correlation coefficient between the original SM and the anomalies. Generally, ECV_C showed an outstanding performance, with a relatively higher R, lower RMSE and ubRMSD, and smaller bias of anomalies than the other products. In addition, ECV_C showed stability and robustness in these parameters with quantiles closer to the median than those of the other products, as shown in both Figure 2 and Figure 3. Moreover, the R values of the anomalies were largely smaller than those of the original SM, which is coincident to the results of previous research focused on satellite-based SM evaluation and validation using in-situ measurements [51,52].

4.1.2. REM Area

Because the 20 stations were densely situated within a 35 × 35 km region, we calculated the arithmetic mean of certain stations that were distributed in the same 0.25°-resolution SM pixel. The main land cover types here included farmland and a sparse distribution of shrubland and forest.

Table 7. Daily bias, R, RMSE, and ubRMSD between the in-situ measurements and the seven SM products over the REM area.

Metrics	Products
	AMSR_A	AMSR_D	SMOS_A	SMOS_D	ECV_A	ECV_C	ECV_P
Bias (m3 × m−3)	0.091	0.139	−0.039	−0.026	0.197	0.075	0.131
R	0.788	0.803	0.745	0.663	0.721	0.826	0.822
RMSE (m3 × m−3)	0.141	0.176	0.089	0.097	0.286	0.088	0.152
ubRMSD (m3 × m−3)	0.104	0.105	0.061	0.074	0.206	0.037	0.074

shows the comparison statistics between the seven SM products and the in-situ measurements. The statistics were box-plotted (Figure 4) to quantitatively evaluate their accuracy as well as their fitting degree. As shown in Table 7, the SMOS products underestimated the in-situ measurements. However, their absolute values of bias were smaller than those of the overestimated products, which illustrates a distinct fitting degree of SMOS-retrieved SM. However, the correlation coefficient of the SMOS data was lower compared to that of the AMSR and ECV data. In terms of RMSE and ubRMSD, it was clear that ECV_C had the minimum values while the maximum values occurred in ECV_A, as shown in Figure 4c,d. Hence, the best performance was found with ECV_C data, while the worst was always in ECV_A data, illuminating the unfitness of active sensor-based SM products. Generally, SMOS outperformed AMSR with a smaller absolute value of bias and lower RMSE, ubRMSD. Additionally, SMOS had the minimum bias among all products, indicating good data accuracy. However, AMSR did well in terms of R, showing that it has a better fitting degree for the variation tendency of ground measurements compared to that of SMOS. On the whole, ECV_C exceeded not only ECV_A and ECV_P in terms of performance, but also AMSR and SMOS in all indexes, with a superior accuracy, higher fitting degree, and smaller fluctuation, except for bias, as shown in Figure 4.

According to Figure 4b and Figure 5b, both the R values derived from the original SM and anomalies had an identical ranking order, which is distinct from that of the OKM area. This phenomenon clarifies the difference in the same SM products in depicting surface SM fluctuation trends among diverse areas. Additionally, the ECV products agreed better than the AMSR and SMOS products, proving that they can competently represent SM temporal variation. Moreover, as shown in

Table 8. Daily bias, R, RMSE, and ubRMSD of the anomalies of the seven SM products over the REM area.

Metrics	Products
	AMSR_A	AMSR_D	SMOS_A	SMOS_D	ECV_A	ECV_C	ECV_P
Bias (m3 × m−3)	−0.00176	−0.00129	0.00004	0.00010	−0.00075	−0.00002	−0.00007
R	0.485	0.519	0.542	0.375	0.512	0.539	0.531
RMSE (m3 × m−3)	0.0657	0.0698	0.0521	0.0554	0.1353	0.0280	0.0505
ubRMSD (m3 × m−3)	0.0657	0.0698	0.0521	0.0554	0.1353	0.0280	0.0505

, SMOS_D is unqualified to describe the feature of SM over the REM region, because the average anomaly R was less than 0.40, far less than that of all the other products. Additionally, Table 8 shows that the anomalies had almost the same value and range in RMSE and ubRMSD.

4.1.3. NAN Area

A total 56 stations were included in the NAN area, and the characteristic land surface cover type in this area is alpine grasslands. As shown in

Table 9. Daily bias, R, RMSE, and ubRMSD between the in-situ measurements and the seven SM products over the NAN area.

Metrics	Products
	AMSR_A	AMSR_D	SMOS_A	SMOS_D	ECV_A	ECV_C	ECV_P
Bias (m3 × m−3)	0.074	0.102	−0.145	−0.114	0.477	−0.049	0.082
R	0.755	0.696	0.347	0.325	0.805	0.788	0.759
RMSE (m3 × m−3)	0.108	0.142	0.179	0.154	0.493	0.085	0.123
ubRMSD (m3 × m−3)	0.061	0.081	0.097	0.093	0.125	0.041	0.082

, AMSR_A, AMSR_D, ECV_A, and ECV_P had a positive bias, while the other three products had a negative bias. In particular, ECV_C, derived from the combination of ECV_A and ECV_P, showed a negative average bias, which was contrary to both ECV_A and ECV_P. In terms of correlation coefficient, SMOS was remarkably inferior to AMSR and ECV, with R_{SMOS_A} = 0.347 and R_{SMOS_D} = 0.325, in contrast to R_{ECV_A} = 0.805, as shown in Table 9. In addition, ECV_C had the minimum degree of discretization as reflected by the RMSE, followed by that of AMSRA_A, ECV_P, AMSR_D, and SMOS_D, and ECV_A had the maximum dispersion. Moreover, ECV_C had a prominent small value in terms of ubRMSD (ubRMSD_{ECV_C} = 0.041), illustrating a preferable coincidence to in-situ measurements. To summarize, the ascending SM outperformed the descending SM in AMSR, while the situation was the opposite for SMOS. In addition, because the average and total goodness of fit values were fairly low, as shown in Table 9 and Figure 6b, the daily changes in SMOS products could not effectively coincide with the in-situ measurements. In contrast, ECV_C showed excellent data quality in terms of accuracy, stability, and correlation coefficient, as seen in the box-plots of Figure 6a,b. Because ECV_A performed poorly in terms of bias, RMSE, and ubRMSD, and showed a comparatively obvious deviation from in-situ measurements, as shown in Figure 6, it is inappropriate to employ this product in the NAN area.

Figure 7 shows the value ranges of bias, R, RMSE, and ubRMSD among each SM product using bars with different lengths and marking lines.

Table 10. Daily bias, R, RMSE, and ubRMSD of the anomalies over the NAN area.

Metrics	Products
	AMSR_A	AMSR_D	SMOS_A	SMOS_D	ECV_A	ECV_C	ECV_P
Bias (m3 × m−3)	−0.01126	−0.02136	0.00422	0.07745	−0.00398	−0.00206	−0.00157
R	0.488	0.392	0.228	0.130	0.637	0.596	0.592
RMSE (m3 × m−3)	0.0311	0.0464	0.1148	0.1508	0.0775	0.0255	0.0361
ubRMSD (m3 × m−3)	0.0311	0.0464	0.1141	0.1495	0.0774	0.0254	0.0361

lists the average value of bias, R, RMSE, and ubRMSD to represent the general levels of these parameters. As shown in Figure 7, ECV_A, ECV_C, and ECV_P, which generally had anomalies R greater than 0.6, could comparably address the change characteristics of surface SM. In comparison, the ability of AMSR to fit in-situ records cannot compare to that of ECV, but it was much better than that of SMOS. In other words, the SMOS products, with R values ranging from −0.26 to 0.43, barely reflected the temporal evolution tendency in topsoil humidity. In terms of an inter-comparison between ascending and descending products, the performance of the ascending AMSR_A and SMOS_A were basically preferable to those of AMSR_D and SMOS_D, individually.

4.1.4. OZN Area

There were 37 ground stations distributed in the OZN area. Considering that several stations were located in the same 0.25° pixel, the arithmetic means were calculated to represent the SM value of the corresponding area. The main land surface types in this area are cultivated land, native vegetation, and built-up area. As shown in

Table 11. Daily bias, R, RMSE, and ubRMSD between the in-situ measurements and the seven SM products over the OZN area.

Metrics	Products
	AMSR_A	AMSR_D	SMOS_A	SMOS_D	ECV_A	ECV_C	ECV_P
Bias (m3 × m−3)	0.047	0.085	0.023	0.017	0.383	0.026	0.070
R	0.684	0.585	0.638	0.579	0.716	0.763	0.740
RMSE (m3 × m−3)	0.085	0.128	0.085	0.100	0.427	0.068	0.104
ubRMSD (m3 × m−3)	0.062	0.090	0.072	0.091	0.183	0.049	0.069

and Figure 8, all the products had positive biases, particularly ECV_A which seriously overestimated the in-situ measurements. Similarly, the RMSE and ubRMSD of ECV_A were evidently larger than those of the other products, as shown in Figure 8c,d. In general, the ECV series products had better correlation coefficient value ranges than those of the AMSR and SMOS products, as shown in Figure 8b. For AMSR, the ascending product was superior to the descending one in every parameter at all times. While, for SMOS, the ascending product displayed a closer variation tendency to the in-situ measurements, but the descending product performed better in terms of absolute values. On the whole, based on Table 11, ECV_C could, to some extent, comprehensively represent both the trend (R_{ECV_C} = 0.763) and the values (Bias_{ECV_C} = 0.026 m³ × m⁻³) of SM properly and accurately. The performance of passive sensor-combined ECV_P was not as good as that of ECV_C. The poor expression of active sensor-combined ECV_A indicates its unfitness to describe SM over the OZN area. The AMSR series products generally depicted SM slightly better than those of SMOS, in terms of both accuracy and stability.

To further quantitatively and intuitively assess the adequacy levels of these SM products in showing variation in accordance with ground-based records, Figure 9 shows the corresponding parameters of the anomalies’ ranges, respectively.

Table 12. Daily bias, R, RMSE, and ubRMSD of the anomalies of the seven SM products over the OZN area.

Metrics	Products
	AMSR_A	AMSR_D	SMOS_A	SMOS_D	ECV_A	ECV_C	ECV_P
Bias (m3 × m−3)	0.00023	−0.00017	−0.00012	−0.00013	0.00297	0.00047	0.00051
R	0.623	0.471	0.584	0.485	0.627	0.657	0.597
RMSE (m3 × m−3)	0.0466	0.0637	0.0670	0.0981	0.1150	0.0348	0.0522
ubRMSD (m3 × m−3)	0.0466	0.0636	0.0670	0.0980	0.1149	0.0347	0.0522

lists the specific values of these parameters. According to Figure 9, compared to the other SM products, ECV_C had smaller values in terms of bias, RMSE, and ubRMSD and a higher value for goodness of fit. As a result, the active and passive product-combined ECV_C showed its superiority in representing the temporal and spatial trends of SM. Remarkably, an unprecedented larger gap than any other sample area between the R of the ascending products and the descending products over the OZN area occurred, as shown in Figure 9. In general, AMSR_A, SMOS_A, ECV_A, ECV_C, and ECV_P were able to characterize the evolutionary trend of surface soil water content to a similar extent. In addition, as shown in Table 12, both ECV_C and ECV_P were able to simultaneously and accurately depict the in-situ measurements in anomalies, as they showed similar values for the four selected parameters.

4.1.5. Inter-Comparison

To conduct and integrally evaluate the fitness levels of different satellite- and sensor-based SM products among different regions across the globe, we drew Taylor diagrams (Figure 10) to analyze their performance. As shown in Figure 10, the ordinate axis represents the standard deviations of the SM data; the abscissa axis represents the gap between the standard deviations of the observational and SM products; and the polar axis represents the correlation coefficients between the in-situ observations and satellite-based SM products. The Taylor diagrams provide an efficient and intuitive means to compare results derived from different products. The subfigures (a), (b), (c), and (d) represent the OKM, REM, NAN, and OZN areas, respectively. The seven points inside each subfigure indicate the results of the seven satellite-based SM products. According to Figure 10, it is clear that the e point, representing ECV_A, always has the largest standard deviation and the highest correlation coefficient. In contrast, the f point (ECV_C) has the smallest standard deviation. The g point (ECV_P) is situated between e and f. Because points a, b (AMSR_A, AMSR_D) and points c, d (SMOS_A, SMOS_D) were derived from the same sensor, respectively, they have similar positions in Figure 10a,b. However, there is a difference in the NAN and OZN areas. This phenomenon may be attributed to the following: (1) The NAN network on the Tibetan Plateau has a special hydrothermal combination (large temperature difference between day and night), unique elevation (>5000 m), and an alpine/grassland-dominated land cover. (2) The OZN network is situated in the Southern hemisphere, which has an opposing seasonal variation and related temperature as well as precipitation variation compared to that of the Northern hemisphere. In summary, ECV_C, with its splendid performance in terms of data accuracy and variation fitting, was able to competently express the absolute values and variation in SM in the four selected experimental regions. In contrast, ECV_A could barely indicate any SM condition in the four areas because of its remarkable bias. ECV_P was average among the three ECV series products. Both the AMSR and SMOS products performed well in terms of correlation coefficient, but their standard deviations were larger than that of ECV_C. Moreover, AMSR behaved slightly better than SMOS in trend fitting. Furthermore, as the fitting coefficients between the SMOS products and ground measurements were less than 0.4, it is inappropriate to depict SM over the Tibetan Plateau using SMOS-retrieved data.

Given that there may have some heterogeneity between the original data and anomalies, Taylor diagrams of anomalies were drawn as well (Figure 11). As shown in Figure 11, the abscissa axis represents the standard deviations of the SM anomalies; the ordinate axis represents the gap between the standard deviations of the observational anomalies and the anomalies of the SM products; and the polar axis represents the correlation coefficients between the anomalies of the in-situ observation and those of the satellite-based SM products. The distribution of anomaly points was basically coincident with the original data points. Specifically, the ECV_A had a rather high value in terms of both standard deviation and correlation coefficient. In contrast, the ECV_C invariably had the smallest deviation and preferable correlation. However, a difference was that the anomalies of SMOS_A and SMOS_D had abnormally large standard deviations and weak correlations in the NAN area (Figure 11c). This phenomenon is not that obvious in the original data Taylor diagram, as shown in Figure 10c. Thus, we assumed that this further indicated the unfitness of SMOS SM products in determining the characteristics of topsoil humidity over the Tibetan Plateau.

4.2. Temporal Series Comparison

To explore the temporal fitness degree of the seven products, we drew scatter plot diagrams to observe the performance of different products over different regions. Figure 12, Figure 13, Figure 14 and Figure 15 show the temporal fitting trend between the in-situ measurements (black solid line) and satellite-based SM products (scatter points) over the OKM, REM, NAN, and OZN areas, respectively. For AMSR, the morning-pass and night-pass products overestimated the in-situ observations from November to April and underestimated the in-situ observations from May to October over the OKM and REM areas, as shown in Figure 12 and Figure 13. Curiously, AMSR presented a prevalent trend of overestimation over the NAN and OZN areas, as shown in Figure 14 and Figure 15. As for ECV, the ECV_A value was generally much larger than that of all the in-situ measurements over all areas. Meanwhile, ECV_C underestimated the ground monitoring value over the OKM and NAN areas, while it overestimated the ground monitoring value over the REM and OZN areas. The performance of ECV_P was similar to that of ECV_C, except in the NAN area where ECV_P showed an overestimation tendency. Moreover, the SMOS showed varied behavior over different regions. Overall, the SMOS data showed lower values than the in-situ values over the OKM area, but higher values than the in-situ values over the OZN area. Furthermore, it overvalued the REM in-situ records from December to March and undervalued the REM in-situ records during all other months. In addition, in the NAN region, as shown in Figure 14, the SMOS distribution range could barely express any relevance between the SMOS SM and in-situ measurements.

Figure 16, Figure 17, Figure 18 and Figure 19 show the time series variation in the averaged anomalies of the in-situ measurements and seven satellite-based SM products over each sample area. The black fold line represents the evolutionary tendency of the ground SM records, and the scatter represents satellite-based SM products at the corresponding positions. We found that the ECV_C was good at characterizing both the data values and the temporal evolution of the in-situ measurements with a relatively high accuracy across all regions. For example, as shown in Figure 17, the variation trend of the time series of ECV_C was highly consistent with that of the in-situ measurements. In addition, ECV_P could, to some extent, express the SM variation tendency, except for a few overestimations and underestimations. In contrast, a large number of abnormal peak and valley values appeared in the ECV_A data, proving that it is inadequate to feature SM values or evolution. For the AMSR datasets, the consistency with ground records was fairly good. However, as shown in Figure 16 and Figure 19, the bias of AMSR_D was, to a certain extent, larger than that of AMSR_A, particularly in the OKM and OZN areas. In terms of SMOS, as shown in Figure 18 and Figure 19, it was difficult to find any correlation with in-situ-measured SM over the NAN and OZN areas, indicating its inapplicability and irrelevance in these two regions.

5. Discussion

Satellite-based SM has already become an important resource for acquiring and monitoring global land surface near-real-time water storage and variation situations. Numerous SM products have appeared since the 1980s. However, considering the significant spatial heterogeneity of SM and the diverse band combinations as well as SM-retrieving algorithms, their fitness degree for different land cover types and climatic zones remains unclear. Although many validations have been conducted to evaluate the performance of certain SM products in characterizing the temporal evolution of surface soil water, there has been a lack of analyses among ascending, descending, active sensor-combined, and passive sensor-combined SM products over regions with distinct climates, land covers, and elevations as well as hemispheres. In this study, we evaluated the AMSR products (including ascending and descending data), ECV products (including active combined, passive combined, and active-passive combined products), SMOS products (including ascending and descending data) over four dense in-situ networks in south-central USA, Northwestern Iberian Peninsula, the Tibetan Plateau, and Southeastern Australia. Four metrics (bias, R, RMSE, and ubRMSD) and anomalies were calculated to attempt a comprehensive comparison.

To summarize, the active-passive combined ECV_C showed general applicability throughout the four selected areas. It was able to preferably seize the features of evolutionary trend and accurately fit factual SM values recorded with a higher R values and lower bias, RMSE, and ubRMSD than the other products. Additionally, the anomalies of ECV_C showed an outstanding correlation coefficient with the anomalies of in-situ records. This result is coincident with previous studies that compared the performance of ECV and SMOS products based on the OzNet monitoring network in Australia and in-situ measurements across the continental USA [36,53]. ECV_P showed a tolerable overestimation tendency in all networks, except for that of OKM. In addition, it could, on the whole, represent the temporal change characteristics of in-situ measurements. In contrast, unbearable abnormal values were prevalent in ECV_A, which suggested the inappropriateness of the active sensor-combined SM products to depict surface soil humidity. Moreover, AMSR slightly exceeded SMOS in terms of R. Furthermore, the ascending products (AMSR_A and SMOS_A) generally outperformed the corresponding descending products (AMSR_D and SMOS_D), which may be explained by the differences in day/night land surface temperatures in the satellite-based SM disaggregation algorithms. AMSR_A could basically represent both the tendency and values of soil humidity with stabilization. Comparatively, the bias of AMSR_D in the OKM and OZN areas was obviously greater than that of AMSR_A. In addition, Rajat Bindlish et al. illustrated that the AMSR products can act as a long-term dataset to describe SM and have a larger SM dynamic range than in-situ observations [54]. The larger dynamic range was similar to that shown in Figure 12a, Figure 13a and Figure 14a in our study. In terms of SMOS products, the original data had a poor correlation coefficient over the NAN region and the R values of the anomalies over the NAN area as well as the OZN area were also small. Moreover, SMOS showed an underestimation bias over all areas other than the OZN area. To a certain extent, this phenomenon reflects that the quality of the SMOS data has much room for improvement over the Tibetan Plateau and Australia. However, according to the validation of the NAN area carried out by Chen et al. [55], the SMOS product did not perform poorly in the NAN area from 21 June to 30 September 2015 and 1 May to 27 September 2016, which reflects a period difference, resulting in heterogeneity. Apart from the validation parameters employed in this research, an effective technique, termed the triple collocation method, has an advantage in estimating the RMSE. This method can solve for systematic differences in three mutually independent types of data [56]. In addition, sufficient in-situ measurements are a precondition for application of the triple collocation method. Using this technique, W. A. Dorigo et al. successfully featured the error characteristics of global active and passive microwave SM datasets [21]. Miralles et al. used the triple collocation technique to clarify the spatial sampling uncertainty of coarse resolution SM estimates derived from ground observations [57]. Given that the triple collocation method is a powerful tool for estimating the error in global soil moisture datasets, we plan to compare the features and performances of the triple collocation method and a traditional parameter evaluation method in SM validation in a future study.

Given the varied performances of different SM products over different regions during different periods, we tentatively attempt to explain this phenomenon from the following aspects: (1) Both ascending and descending satellite-based products instantaneously record the surface SM at different time points, while the ground networks monitor and record SM hour-by-hour. We averaged the hourly data to obtain a daily SM. Hence, a gap appeared between the instantaneous values and periodic values. Furthermore, an uncertain match degree between the instantaneous SM and periodic SM could lead to a fluctuation in the validation results. (2) Soil texture can profoundly affect the water storage capacity. For example, soil with a high sediment concentration typically has a poor water storage capacity, while low sediment concentration soil has a strong water-holding capacity. However, although SM-retrieving algorithms have considered this, soil texture can change considerably within a small region due to human factors such as fertilization, flood irrigation, and straw burning. This drawback limits the further improvement of accuracy and tendency between satellite-based SM and ground measurements. Additionally, a land cover difference resulting from spatial heterogeneity could increase the error in SM within a 0.25°-resolution pixel. Specifically, forest as well as dense vegetation would have a negative influence on microwave penetration and increase the SM error. Moreover, the extent of the impact varies among different continents because of complex natural condition combinations [58]. (3) Frequent radio frequency interference (RFI) could be an obstacle in microwave signal reception, which could result in inaccurate SM values. For example, RFI at the C band affected radiation reception in the USA. RFI at the L band has also been detected and has had an effect in China and Europe [17,18]. (4) Although there have been several in-situ measurements inside one-pixel grid, a mismatch remains between the point-scale and a 0.25° spatial resolution scale. This would decrease correlations in the validation procedure. (5) In addition, the uncertainty and error of SM products derived from a vegetation cover area is also a critical issue. For example, the L band signal is predominantly composed of SM, vegetation factors, and the land surface temperature [13]. There is a necessity to account for the effect of vegetation as it could influence the brightness temperature and vegetation optical thickness. During the SM retrieval process, dense canopies/forests can induce nonlinear effects; thus, ancillary data (i.e., NDVI, LAI and vegetation maps) was considered to separate the impacts [42].

6. Conclusions

Numerous satellite-based SM products have appeared and have been broadly utilized in surface hydrology analysis for decades. The diversity in satellite passing time, sensor working mode, band combination, and retrieving algorithms, and the complexity of land cover, soil texture, and hydrothermal condition have co-contributed to the uncertainty in satellite retrieved SM products. Consequently, we validated seven currently frequently-used SM products (AMSR_A, AMSR_D, SMOS_A, SMOS_D, ECV_A, ECV_C, ECV_P) over four varied continental regions.

The evaluation parameters applied during the evaluation process were bias, R, RMSE, ubRMSD, and anomalies. The results indicated that ECV_C is superior in determining both tendency and absolute data over the four test areas, which suggested the advantage of active and passive-combined SM products. In contrast, ECV_A, which showed numerous abnormal overestimations, was inappropriate for the depiction of topsoil moisture. The performance of ECV_P was slightly worse than that of ECV_C, but much better than that of ECV_A. In addition, AMSR performed better in trend fitting than SMOS did, and the ascending products performed better than the corresponding descending products on the whole. Moreover, weak correlations occurred in the evaluation of SMOS over the NAN and OZN areas, which showed that SMOS is unqualified to represent SM on the Tibetan Plateau and in Australia.

This study compared the capacity and suitability of seven satellite-based SM products by describing the data values and temporal evolution features across the globe over four dense ground networks. Because these networks are in North America, central Asia, Southwestern Australia, and Western Europe, networks distributed in South America, Africa, and other locations with unique features are needed in the future to conduct a more comprehensive assessment. In addition, it is necessary to further explore the mechanisms that result in different fitting degrees for the same SM product over different networks.