Global Evaluation of SMAP/Sentinel-1 Soil Moisture Products

Abstract

SMAP/Sentinel-1 soil moisture is the latest SMAP (Soil Moisture Active Passive) product derived from synergistic utilization of the radiometry observations of SMAP and radar backscattering data of Sentinel-1. This product is the first and only global soil moisture (SM) map at 1 km and 3 km spatial resolutions. In this paper, we evaluated the SMAP/Sentinel-1 SM product from different viewpoints to better understand its quality, advantages, and likely limitations. A comparative analysis of this product and in situ measurements, for the time period March 2015 to January 2022, from 35 dense and sparse SM networks and 561 stations distributed around the world was carried out. We examined the effects of land cover, vegetation fraction, water bodies, urban areas, soil characteristics, and seasonal climatic conditions on the performance of active–passive SMAP/Sentinel-1 in estimating the SM. We also compared the performance metrics of enhanced SMAP (9 km) and SMAP/Sentinel-1 products (3 km) to analyze the effects of the active–passive disaggregation algorithm on various features of the SMAP SM maps. Results showed satisfactory agreement between SMAP/Sentinel-1 and in situ SM measurements for most sites (r values between 0.19 and 0.95 and ub-RMSE between 0.03 and 0.17), especially for dense sites without representativeness errors. Thanks to the vegetation effect correction applied in the active–passive algorithm, the SMAP/Sentinel-1 product had the highest correlation with the reference data in grasslands and croplands. Results also showed that the accuracy of the SMAP/Sentinel-1 SM product in different networks is independent of the presence of water bodies, urban areas, and soil types.

1. Introduction

A Soil Moisture Active Passive (SMAP) satellite mission was launched by the National Aeronautics and Space Administration (NASA) on 31 January 2015, aboard a Delta II rocket [1]. Modeling of SMAP, a derivative of the formulation studies for the Hydrosphere State (Hydros) mission [2], was initiated in 2008 [3], followed by design studies, critical design review, system integration, and test processing in May 2013 [4,5]. The primary objectives of SMAP were to provide global Soil Moisture (SM) maps, Brightness Temperature (TB), topsoil layer moisture, and freeze–thaw state using L-band microwave spectrum observations [6]. SMAP is considered a follow-up of the Soil Moisture and Ocean Salinity (SMOS) [7,8]. However, the main difference between SMAP and SMOS and, of course, the earlier passive satellites, was the equipment of both L-band (1.26 GHz) radar and L-band radiometer instruments [9].

Microwave emission from the top ~5 cm of soil is measured with a SMAP radiometer with a spatial resolution of ~36 km. In addition, the L-band SAR, which is more sensitive to irregularities of vegetation [10], measures backscatter at higher spatial resolution (1–3 km) [11]. Subsequently, radar and radiometer measurements are effectively combined to produce global SM maps at two higher spatial resolutions: 3 km and 9 km [11,12,13,14]. Thus, the developed algorithms for the SMAP observations promised a unique means of measuring SM with unprecedented accuracy, resolution, and coverage. Unfortunately, due to an unrecoverable hardware failure, SMAP’s radar instrument was discontinued on 7 July 2015, after about 11 weeks of operation [11], and SMAP’s radiometer remains the only operating instrument onboard the satellite. Therefore, this satellite lost its ability to measure the surface SM with a spatial resolution of 3 and 9 km [14].

Using the Backus–Gilbert optimal interpolation technique [15], it became possible to enhance the antenna temperature measurements of SMAP radiometer and provide TB and SM products with a higher spatial resolution [16]. Using this technique, the optimal TB of arbitrary locations is calculated using a linear combination of radiometric measurements overlapped in along- and across-scan directions [17]. As such, the TB disaggregates to the new values with a native resolution equal to the 3 dB beamwidth of the radiometer, which is ~9 km for SMAP [5]. The 9 km TB is then converted to the SM using the Mineralogy-Based Soil Dielectric Model (MBSDM), also called the Mironov dielectric model [18,19]. Consequently, in addition to the 36 km products of SMAP SM, there are further improved SMAP SM products with a spatial resolution of 9 km [20].

To date, numerous researchers have studied the accuracy of SMAP SM products across a wide range of geographic regions and climatic and environmental conditions around the world. Several studies compared the potential of SMAP in determining SM values with the performance of earlier passive microwave satellites such as SMOS, AMSR-E, and ASCAT [21,22,23,24,25,26,27,28,29,30,31].

Moreover, given the data of the SMAP Calibration and Validation (Cal/Val) Core Validation Site (CVS), post-launch field campaigns (such as SMAPVEX-16, SMAPVEX19-22), prelaunch field campaigns, and sparse global networks, there is extensive literature quantitatively evaluating global SMAP products at 36 km and 9 km using in situ measurements, model-based SM estimates, and airborne SM observations [22,32,33,34,35,36,37,38]. Most of these studies demonstrated that SMAP outperforms other passive microwave sensors [8,26,37,39,40,41].

SMAP/Sentinel-1 SM is the latest SMAP product developed to map SM with spatial resolutions of 1 km and 3 km and near-global coverage (latitude from −60 to 60 and longitude from −180 to 180) [14]. In this product, 9 km enhanced TB of SMAP are resampled and downscaled to 3 km and 1 km EASE-Grid using Sentinel-1 radar backscattering data, aggregated onto 1 km and 3 km EASE-Grid 2.0 pixels [42]. The corresponding combination of SMAP data and Sentinel-1 measurements is based on the BT-Based Downscaling Algorithm (BTBDA) [43], followed by the Baseline Retrieval Algorithm and the Tau–Omega algorithm used to generate SM [13].

As SM’s first and only long-term global map with spatial resolutions of 1 km and 3 km, SMAP/Sentinel-1 data represent a turning point for measuring SM with RS systems because they have adequate spatial resolution, global coverage, and acceptable coverage repeat time for many applications. As one of the major sources of SM estimation with Earth Observations (EOs), validation of SMAP/Sentinel-1 products was needed to guide the community and provide information on the quality of products under different conditions around the world and at different times. Generally, since the footprint of passive microwave instruments spans over 1000 km², SMAP estimates of TB and SM are affected by numerous spatially and temporally variable factors, such as topographic, climatic, land factors, and the radiometric calibration of the passive microwave instruments [44]. Even over certain homogeneous regions, the SM retrieval algorithm and downscaling model applied to combine SMAP and Sentinel-1 data are still influenced by various sources of uncertainty. Satellite instrument noise, model parameterization errors, errors that occur when sampling elliptical footprints onto spatial grids, and imperfections in the retrieval model are some of the most important sources of uncertainty that eliminate the accuracy of SM products [45]. These problems cause the SMAP/Sentinel-1 SM to differ from the true state of SM. Therefore, the comparison of SMAP/Sentinel-1 estimation and the true value of SM illustrates its strength and weakness. So far, few attempts have been made to validate the SMAP/Sentinel-1 SM. Das et al. [46] studied the basic idea and methodology for fusing high-resolution Sentinel-1A/Sentinel-1B data and SMAP products on seven different CVSs. They evaluated the performance of SMAP/Sentinel-1 SM products using in situ measurements of the Walnut Gulch and TxSON sites in the United States, Kenaston in Canada, Monte Buey in Argentina, REMEDHUS and Valencia in Spain, and Yanco in Australia. In addition, Colliander, Reichle et al. (2021) assessed SMAP/Sentinel-1 estimates using the in situ measurements of various CVSs from March 2015 to March 2021. They detailed and investigated the results derived from Monte Buey, Yanco, Kenaston, Valencia, Walnut Gulch, TxSON, Tonzi Ranch in the United States, and HOBE in Denmark.

In line with the previous studies, the focus of this paper is on a global assessment of SMAP/Sentinel-1 SM products using in situ measurements across a wide range of geographic, environmental, and climatic conditions.

The main superiority of this study over the state of the art is the comprehensive evaluation of the performance of the new SMAP/Sentinel-1 SM products on a global scale using in situ measurements from a large number of stations. In this paper, we validate SMAP/Sentinel-1 SM observations using in situ measurements of 561 worldwide stations from 35 SM networks (13 in Europe, 7 in the Americas, 8 in Asia, 4 in Africa, 2 in Australia, and 1 in Oceania). From 31 March 2015 to 1 January 2022, SMAP/Sentinel-1 SM products and in situ measurements were used in this study to analyze the performance of SMAP from the beginning of the mission to the present.

As another innovation in validating the SMAP/Sentinel-1 products, we examine several factors that influence the radiometer and radar observations and the downscaling methods. We compare in situ measurements and SMAP/Sentinel-1 data separately for different seasons, land covers, vegetation densities, and climate zones. We also compare the accuracy of enhanced 9 km SMAP SM products and SMAP/Sentinel-1 data in dense networks to explore to what extent downscaling of SMAP using Sentinel-1 data preserves the accuracy of the source data. It can be said that this study focuses on the qualification of uncertainties in the SMAP/Sentinel-1 product and the comparison of this product with in situ or other suitable reference data across multiple regions and time periods. Consequently, our specific objectives are: (i) long-term evaluation of the SMAP/Sentinel-1 SM across the SM networks around the world, (ii) comprehensive investigation of the effect of various parameters such as land/vegetation cover, water bodies, climate conditions, etc., on the accuracy of the product, and (iii) comparing the SMAP/Sentinel-1 SM with the 9 km enhanced SMAP SM to explore the downscaling effects. The results of this evaluation, along with the official assessment reports conducted by the SMAP science team, enable researchers to determine what spatial resolution, temporal interval, and accuracy can be expected from SM products of a RS satellite in different vegetation conditions, soil textures, seasonal climates, land cover, etc.

2. Study Area and Data

2.1. Validations Sites

Figure 1 represents the central locations and conventional names of 35 SM sites used to validate the SMAP/Sentinel-1 SM product. The distribution and location of in situ stations of three sites, including CTP_SMTMN, HOBE, and a part of SOILSCAPE sites, are also shown in Figure 1. Measurements from these sites were collected by the International Soil Moisture Network (ISMN) platform and are available through https://ismn.geo.tuwien.ac.at/en/networks/ (accessed on 19 August 2022). ISMN is one of the critical existing platforms that has collected data from various SM sites developed by different organizations and countries [47,48]. From all 72 networks and ~2879 stations available in the ISMN, those that meet the following criteria are selected: (i) relatively long-time intervals (more than one year during SMAP/Sentinel-1 coverage from 31 March 2015 to present), (ii) located away from water bodies and urban areas, (iii) with an acceptable distribution and density of stations, and (iv) recording SM at a depth of 0–10 cm of the ground.

Table 1. The main characteristics of the validation sites and their respective stations, together with the number by which they are represented in Figure 1.

ID	Site Name	No. (1)	No. (2)	Data Availability	Sampling Depth (m)	SM Detector	Reference
1	AMMA-CATCH	7	7	01/01/2006–31/12/2018	0.05–0.05	CS616	[49]
2	SD_DEM	1	1	08/02/2002–12/11/2020	0.05–0.05	Decagon 5TE	[50]
3	TAHMO	70	21	17/06/2015–10/12/2021	0.05–0.05	GS1, TEROS10, TEROS12	[51]
4	COSMOS_1	8	5	03/02/2014–06/03/2020	00–0.05	Cosmic-ray Probe	[52]
5	OZNET	38	18	12/09/2001–27/08/2018	00–0.05	CS615 EnviroSCAN Stevens Hydra Probe CS616	[53]
6	COSMOS_2	11	2	28/11/2010–13/10/2019	00–0.17	Cosmic-ray Probe	[52]
7	PTSMN	20	20	30/10/2016–15/11/2018	0.07–0.13	AquaCheck	[54]
8	CTP_SMTMN	57	54	01/08/2010–19/09/2016	00–0.05	EC-TM, 5TM	[55]
9	KIHS_CMC	18	18	28/03/2018–10/12/2019	0.10–0.10	Soilmoisture Equipment Corp, Buriable Waveguide	[56]
10	KIHS_SMC	19	19	27/03/2018–05/12/2019	0.10–0.10	Soilmoisture Equipment Corp, Buriable Waveguide,	[56]
11	MAQU	27	21	13/05/2008–01/06/2019	0.05–0.05	ECH20 EC-TM	[57]
12	NAQU	11	9	15/06/2010–12/09/2019	0.05–0.05	5TM	[57]
13	NGARI	23	13	12/07/2010–10/09/2019	0.05–0.05	5TM	[57]
14	SMN/SDR	34	21	25/07/2018–31/12/2019	0.03–0.03	5TM	[58]
15	VDS	4	4	01/06/2017–13/02/2021	0.01–0.10	GS1 Port 2–5, TEROS12	[59]
16	BIEBRZA_S-1	30	18	23/04/2015–01/12/2018	0.05–0.05	GS-3	[60]
17	FR_Aqui	5	3	01/01/2012–01/01/2021	0.01–0.01	ThetaProbe ML2X	[61]
18	GROW	150	37	08/02/2017–08/10/2019	00–0.10	Flower Power	[62]
19	HOAL	33	32	11/07/2013–31/12/2020	0.05–0.05	SPADE Time Domain Transmissivity	[63]
20	HOBE	32	29	08/09/2009–13/03/2019	00–0.05	Decagon 5TE	[64]
21	IPE	2	1	03/04/2008–25/03/2020	00–0.06	Campbell Scientific, CS650,	[65]
22	MOL/RAO	2	1	01/01/2003–30/06/2020	0.08–0.08	TRIME-EZ	[66]
23	REMEDHUS	24	20	15/03/2005–01/01/2021	00–0.05	Stevens Hydra Probe	[48]
24	Ru_CFR	2	2	25/05/2015–31/12/2020	0.05–0.05	Hydraprobe II	[67]
25	SMOSMANIA	23	7	01/01/2007–01/01/2020	0.05–0.05	ThetaProbe ML2X	[68]
26	TERENO	5	4	31/12/2009–05/08/2021	0.05–0.05	Hydraprobe II Sdi-12	[69]
27	UMBRIA	17	1	09/10/2002–31/12/2017	0.05–0.15	EnviroSCAN	[70]
28	WEGENERNET	12	12	01/01/2007–03/11/2021	0.20–0.20	Hydraprobe II	[71]
29	LAB-net	4	2	18/07/2014–14/07/2020	0.07–0.07	Campbell Scientific, CS616	[72]
30	ARM	35	10	29/06/1993–02/10/2021	0.02–0.02	Hydraprobe II Sdi-12 E	[73]
31	COSMOS_3	109	9	28/04/2008–29/03/2020	0.00–0.04	Cosmic-ray Probe	[52]
32	FLUXNET/AMERIFLUX	8	4	01/01/2000–21/07/2020	0.00–00	ThetaProbe ML2X	[67]
33	RISMA	24	21	24/04/2013–25/03/2020	00–0.05	Hydraprobe II Sdi-12	[74]
34	SNOTEL	441	85	01/10/1980–16/11/2021	00–00	Hydraprobe Analog (2.5 Volt)	[75]
35	SOILSCAPE	171	30	08/03/2011–29/03/2017	0.04–0.04	EC5	[76]

provides the essential information for the selected sites shown in Figure 1.

In Table 1, No. (1) and No. (2) represent the total number of stations at each site and the number of stations used in this study, respectively. As shown in Figure 1 and Table 1, this study used the SM measurements from a total of 561 stations as reference data to investigate the accuracy of the SMAP/Sentinel-1 SM data. The corresponding stations belong to the 35 dense and sparse networks and are distributed worldwide. Dense networks have high spatial sampling densities with longer deployment durations. According to [11], the distribution of stations in a dense network is such that at least eight stations are within a real footprint of passive microwave sensors (approximately 36 km), five stations are within a 9 km spatial grid cell, and three stations are within a 3 km grid cell. Generally, there are 40 SM dense networks, of which eight sites are dense at 3 km pixel size. Based on the measurements of these dense networks, the accuracy of the SMAP/Sentinel-1 SM products was comprehensively evaluated by the SMAP Cal/Val team, reported in [46,77]. Compared to dense networks, sparse networks distributed globally have greater spatial coverage across various climate regimes and biomes [78]. Although sparse networks do not necessarily meet the condition of distributing stations within a 3 km grid pixel, they have some unique properties that make them efficient for validation studies of EOs [24]. Adequate global coverage, temporally coherent reports, accurate measurements, and easy access are some of the features of sparse networks that present them as essential sources for complementing core validation sites and evaluating various RS-based SM products and methods [24]. Moreover, the number of networks and their sampling stations around the world is large enough to present a comprehensive study of the performance of the SMAP/Sentinel-1 mission in estimating SM.

2.2. Data

2.2.1. SMAP/Sentinel-1 Soil Moisture

Level 2 of the SMAP/Sentinel-1 product is considered for the validation process. L2_ SM _SP or SPL2SMAP_S are the abbreviations for this product. SPL2SMAP_S is derived from the enhanced SMAP L-band (~1.4 GHz) radiometer half-orbit 9 km observations and Copernicus Sentinel-1A/1B C-band (~5.405 GHz) radar measurements [79,80]. SMAP/Sentinel-1 retrieves SM over a wide range of vegetation conditions with a temporal resolution of 12 days (based on Sentinel-1’s repeat cycle) and spatial coverage of 180°W to 180°E and approximately 60°N to 60°S [14] (

Table 2. Main characteristics of the RS and model-based datasets used in this study.

Dataset	Data Type and Description	Spatial Resolution	Revisit Time	Temporal Coverage	Reference
L2_ SM _SP (SPL2SMAP_S)	Remotely sensed SM map (L-band, C-band, active/passive)	3 km × 3 km	6–12 days	2015 to present	[14]
SPL3SMP_E	Remotely sensed SM map (L band, passive)	9 km × 9 km	1–2 days	2015 to present	[81]
CGLSLC100	Model-based land-cover product	100 m	3 years	2015 to present	[82]
HWSD	harmonized soil property dataset	30 arc-second	-	2008 to present	[83]
Sentinel-2A/B	Multispectral/multiresolution remotely sensed image	10 m to 60 m	~5 days	2015 to present	[84]

). For the validation sites, version 3 of SPL2SMAP_S was downloaded from 2015 to 2022. These products are available at https://search.earthdata.nasa.gov/ (accessed on 19 August 2022). The spatial extent of each SMAP/Sentinel-1 image provided on the global cylindrical EASE-Grid 2.0 is limited by the swath width of Sentinel-1A/1B. The image data are in HDF5 format. SPL2SMAP_S HDF5 files consist of three main groups: (i) SM retrieval data with a spatial resolution of 1 km, (ii) SM retrieval data with a spatial resolution of 3 km, and (iii) metadata fields containing all metadata describing the entire contents of each file. Only the 3 km SM and quality assessment plume closest to the SMAP a.m. data (from 6:00 a.m. descending half orbits) were downloaded to reduce the volume of downloaded data and achieve better agreement between reference measurements and observations from RS.

2.2.2. SMAP Enhanced Soil Moisture

In this study, we examine the performance of SMAP’s 9 km products in estimating SM and compare it with the accuracy of the 3 km SMAP/Sentinel-1 SM. Enhanced SMAP products are 9 km radiometer TB and SM derived from native 36 km SMAP measurements using Backus–Gilbert interpolation [85]. The TBs of the enhanced SMAP are corrected for the presence of water bodies (up to 0.1 fractions) before being utilized in the active–passive SM retrieval algorithm. The SMAP Enhanced L3 Radiometer Global Daily 9 km EASE-Grid Soil Moisture (SPL3SMP_E) is used in this study (Table 2). Since version 3 ended on 27 August 2020 and version 4 then replaced the older version, both version 3 and 4 of these products were used in this study. SPL3SMP_E covers the geographic range from 85.044°N to 85.044°S and 180°E to 180°W. Similar to SPL2SMAP_S, the SM and quality assessment flags were downloaded for all validation sites.

2.2.3. CGLS Land Cover

To assess the impact of land cover type on the accuracy of SMAP SM, we use the Copernicus Global Land Service (CGLS) land-cover product with a spatial resolution of 100 m (CGLSLC100) [86]. Version 3 of the CGLS-LC100 products covers the geographic range from 180°E to 180°W and 78.25°N to 60°S. The CGLSLC100 map includes 22 classes, including six different species of closed forest, six different species of open forest, shrubs herbaceous vegetation, wetland, moss and lichen, sparse vegetation, cropland, urban, snow and ice, permanent water bodies, and open sea. However, this paper examines the effects of eight different land-cover types in the accuracy of SMAP/Sentinel-1 SM: Build-up, Cropland, Forest, Shrubland, Grassland, Bareland, Water, and Wetland. To this end, we reclassified the CGLSLC100 map into the mentioned land-cover classes. Thus, the land cover of closed and open forests, including evergreen needle leaf, deciduous needle leaf, evergreen broad leaf, deciduous broad leaf, and mixed deciduous forest, has been reclassified as forest. In addition, herbaceous vegetation, mosses, and lichens are classified as grassland. Snow and ice, permanent water bodies, and open seas are also reclassified as water. Figure 2 shows the global land cover map of CGLSLC100 for the reference year 2015, downloaded from Google Earth Engine and reclassified to the eight mentioned land-cover classes.

2.2.4. Soil Texture and Vegetation Fraction Map

We used the Harmonized World Soil Database (HWSD) to evaluate the effects of soil properties on the accuracy of SMAP/Sentinel-1 SM. The HWSD is a 30 arc-second grid database consisting of 21,600 rows and 43,200 columns linked to harmonized soil property data [83]. The primary soil information included in the HWSD datasets is organic carbon, pH, water-holding capacity, soil depth, soil cation exchange capacity, clay content, total exchangeable nutrients, lime, and gypsum content, sodium exchange percentage, salinity, texture class, and granulometry. Version 1.0 of the HWSD, released in 2008, has been updated several times with new information and is used extensively around the world. Recently, the HWSD was adopted by the Global Soil Partnership (GSP) as the current authoritative soil database (Table 2).

The multispectral and multiresolution data from Sentinel-2 were used along with the land-cover and soil properties maps to calculate vegetation fraction and NDVI. Similar to Sentinel-1, Sentinel-2 was developed by ESA to provide a multiband dataset over land and coastal waters [84,87]. Depending on the spectral band, the spatial resolution of Sentinel-2 varies from 10 m to 60 m [88]. The red and NIR bands with a spatial resolution of 10 m are used in this study to estimate the monthly and seasonal NDVI.

3. Methodology

According to the quality assurance framework advocated by the Land Product Validation (LPV) subgroup of the Committee on Earth Observation Satellites (CEOS) (https://lpvs.gsfc.nasa.gov/ (accessed on 19 August 2022)), the validation process of SM datasets should be able to provide a reliable quantitative assessment of their systematic and random errors through analytical comparison with reference data [89]. In recent decades, several methods have been developed to validate coarse-scale passive microwave EOs through numerous in situ measurements [11,90,91], airborne campaigns [92,93,94,95], and model-based SM estimates [96,97]. Given data preprocessing, the evaluation strategies, and the applied reference data, the methods cause different uncertainties and ambiguities in validating SM products [98]. To standardize validation practices across EO communities, many efforts have been made to establish comprehensive protocols for validating satellite-based SM maps [89,98,99,100]. Refs [89,101] reviewed the compilation of the most common theoretical considerations for validating EOs and presented a comprehensive protocol for validation practices of satellite SM products. The SMAP/Sentinel-1 SM validation approach used in this work is largely consistent with the recommendations in [101], and was recently used in [77]. Figure 3 schematically shows a workflow of the validation method applied in this study to assess the performance of SMAP/Sentinel-1 active/passive SM data. This validation approach can be divided into the following four steps: (1) selection of reliable reference data, (2) preprocessing of data, (3) application of statistical indicators, and (4) presentation of validation results. All steps contain sub-steps, three of which are discussed in more detail in the following sections. The last step is presented in Section 4 and Section 5 as an outcome of this study.

3.1. Selecting Reliable Reference Data

The validation networks of SM usually provide the reference data at different soil depths from 0–5 cm to 2 m with different sampling intervals, from hourly to monthly collection periods [102]. Those best matching satellite estimates must be selected for the assessment process [103]. Regarding the penetration depth of the microwave portion of the electromagnetic spectrum, which is typically 3 cm [104], SMAP/Sentinel-1 products typically estimate the top few centimeters SM of soil [24,104]. Therefore, in the validation networks, out of all SM measurements at different depths (0–5 cm, 10 cm, 20 cm, 50 cm, and 100 cm), the measurement at a depth of 0–5 cm was used for the validation process. In the absence of 0–5 cm measurements, measurements at 0–10 cm depth were used, which correlated with surface SM [105,106]. Those stations that did not have any measurements at depths above 10 cm were masked out. According to Table 1, except for WEGENERNET, UMBRIA, MOL/RAO, KIHS_CMC, and KIHS_SMC, the rest of the sites measure surface SM. SM values of the depth above 10 cm at the mentioned sites were downloaded and applied in the validation process.

To minimize the time gap between the satellite and reference data, the in situ observations closest to the SMAP overflight time (approximately 6:00–7:00 a.m. local time) were used at each validation site. Moreover, of all the selected data, measurements on days with non-zero snow indicators (e.g., snow depth or snow water equivalent) and ground surface or air temperature below 4 °C were excluded from the entire procedure.

3.2. Data Preprocessing

Data preprocessing aims to spatially or temporally align the different data sets coming from various sensors, platforms, and techniques (e.g., coarse-resolution satellite products, modeled data, and field networks). In the evaluating satellite products, when coarsely gridded estimates must be accurately compared to point-based in situ measurements [107], data preprocessing is generally an essential step. Depending on the number and type of datasets used in a study, preprocessing and matching between measurements involve several sub-steps, all described in detail in [101]. In this study, the following two preprocessing steps were used:

3.2.1. Masking out Unreliable Pixels

To avoid unreliable data, SMAP/Sentinel-1 SM products were checked through a quality-control procedure to identify and discard unreliable observations, values outside the acceptable range, spikes, and sudden dips. According to [101] recommendation, we discarded all pixels that were at least 20% covered by tropical forests, urban areas, water bodies, and wetlands, as well as all areas with vegetation water content greater than 5 kg/m² and NDVI greater than 0.7. In addition, the data flags of the SMAP/Sentinel-1 products were used to identify and eliminate data that were not recommended. Data flags are typically placed on non-recommended targets for various reasons, including inappropriate locations for querying SM, unreliable measurements TB, and faulty algorithms for querying SM.

3.2.2. Calculating Reference SM within the Grid Pixels

All stationary in situ measurements within a particular grid cell were averaged to make the RS pixel-based data as representative as possible [108]. There is a sizeable spatial discrepancy between the point-based reference measurements and the grid-based satellite observations [109]. To this end, the weighted average of stations located within a SMAP/Sentinel-1 pixel was calculated at the geographic center of the pixel (Equation (1)).

$$SM{C}_{insitu}=\frac{{\displaystyle \sum _{i=1}^n{P}_i\times S{M}_i}}{{\displaystyle \sum _{i=1}^n{P}_i}}$$

(1)

where n is the number of stations located in a SMAP pixel, and the values SM_i and P_i are, respectively, the point measurement of the ith station and its weight, related to the distances to the center of the pixel (Equation (2)).

$$p_i=\frac{1}{\sqrt{{(x_i-x_o)}^2+{(y_i-y_o)}^2}}$$

(2)

where (x_i,y_i) and (x_o,y_o) are the geographic location of the corresponding station and the center of the SMAP pixel. As mentioned earlier, this study evaluates SMAP/Sentinel-1 SM 3 km products using in situ measurements of sparse networks with surrogate stations; therefore, there is no more than one point-based station in most SMAP/Sentinel-1 pixels.

3.3. Statistical Metrics for the Evaluation Process

Previous scientific literature has used various statistical indicators to compare RS data with field observations and to describe the performance of EOs collected by various sensors (e.g., multispectral [110,111], hyperspectral [112], LiDAR [113], thermal [114], and SAR [115]), and platforms (e.g., UAVs and satellites). The underlying concept is that the overall error of an observed/measured value (x) is separable into its systematic (α_x and β_x) and random (ε_x) components (Equation (3)).

$$X={\alpha }_x+{\beta }_{x}t+{\epsilon }_x$$

(3)

Here, ε_x is an additive mean random error, and α_x and β_x are additive first-order systematic errors and multiplicative (second-order) errors in the assumed grid cell with average SM (t). Equation (3) can be considered for both RS observations and in situ measurements. ε_x is assumed to be zero in almost all EOs validation studies for reference and EOs datasets. For the in situ measurements considered in the reference dataset, it is assumed that αx~0 and βx~1. In addition, for the evaluation of EOs, statistical moments are typically estimated in the temporal domain (i.e., temporal mean, variance, and covariance), assuming stationary first- and second-order systematic errors (i.e., means and variances are assumed to be constant over time). Under these assumptions, the performance of a SM product is directly related to its uncertainties, characterized by the actual measurements of the reference data.

Consequently, all validation metrics that assume the behaviors of EOs as a random variable can be computed to investigate the performance of a SM product. Mean Difference (MD), Root Mean Squared Difference (RMSD), and unbiased Root Mean Squared Error (ub-RMSE) are the most common relative metrics used to estimate the uncertainty of EOs. These comparison metrics are commonly used to evaluate coarse-scale SM products obtained from passive RS techniques and instruments [37,116,117]. The MD represents the uncertainty of the variable being evaluated and contains both systematic and random components. In some scientific literature, this parameter is also referred to as bias. Basically, MD measures the difference between the reference measurements from SM and the satellite retrievals from SM (Equation (4)).

$$\begin{gathered} \mathrm{MD}=(\overline{{\mathrm{SM}}_{\mathrm{S}}-{\mathrm{S}\mathrm{M}}_{\mathrm{R}}})=\frac{1}{\mathrm{N}}{\displaystyle \sum _{\mathrm{i}=1}^{\mathrm{N}}{(\mathrm{S}\mathrm{M}}_{\mathrm{S}.\mathrm{i}}-{\mathrm{S}\mathrm{M}}_{\mathrm{R}.\mathrm{i}})} \\ =\frac{1}{\mathrm{N}}{\displaystyle \sum _{\mathrm{i}=1}^{\mathrm{N}}{\alpha }_{{\mathrm{SM}}_{\mathrm{S}.\mathrm{i}}}+({\beta }_{{\mathrm{SM}}_{\mathrm{R}.\mathrm{i}}}-1)\mathrm{t}} \end{gathered}$$

(4)

where N is the number of data applied to calculate the statistics, which depends on the number of stations of sites, the number of coarse-scale pixels covering the site’s extent, and the time interval. Vinculums indicate the average of the values. SM_S and SM_R, respectively, refer to the satellite observation and in situ SM. With SM_R as the reference data, MD collapses to $\frac{1}{N}\sum {\alpha }_{SMs.i}+({\beta }_{S{M}_{R.i}}-1)t$, directly estimating the biases of the satellite retrieval. Note that, given the t phrase, MD is still influenced by the average SM conditions. In addition to MD, RMSD calculates the absolute difference in retrievals from SM relative to reference values using the following equation (Equation (5)):

$$\mathrm{RMSD}=\left|\left(\overline{{\mathrm{S}\mathrm{M}}_{\mathrm{S}}-{\mathrm{S}\mathrm{M}}_{\mathrm{R}}}\right)\right|=\sqrt{\frac{{\displaystyle \sum _{\mathrm{i}=1}^{\mathrm{N}}{{(\mathrm{SM}}_{\mathrm{S}.\mathrm{i}}-{\mathrm{S}\mathrm{M}}_{\mathrm{R}.\mathrm{i}})}^2}}{\mathrm{N}}}$$

(5)

RMSD is more applicable than the RMSE in validating coarse-scale SM products using station-based reference measurements [118]. The main idea behind estimating RMSD instead of RMSE is that the point-based reference value may differ from the actual SM value over a grid cell. Due to the possible errors in the reference values, the RMSD should be used to validate the raster pixels by stationary measurements. In addition, the ub–RMSE metric can provide a more reliable estimate of the RMSD. Unlike the RMSD, which is sensitive to additive and multiplicative biases, the ub–RMSE is only affected by multiplicative biases (Equation (6)).

$$\mathrm{u}\mathrm{b}-\mathrm{R}\mathrm{M}\mathrm{S}\mathrm{E}=\sqrt{{\mathrm{RMSD}}^2-{\mathrm{b}}_{\mathrm{SM}(\mathrm{R},\mathrm{S})}^2}=\sqrt{{\mathrm{R}\mathrm{M}\mathrm{S}\mathrm{D}}^2-{\mathrm{M}\mathrm{D}}^2}$$

(6)

where b_SM(R,S) is considered the first-order (additive) bias between reference data and EOs. Therefore, ub–RMSE is a bias-insensitive indicator and refers to the standard deviation of the error. In the sparse network, if a station’s measurement is considered the true SM value of the 3 km SMAP/Sentinel-1 grid cell, MD, RMSD, and ub–RMSE conservatively represent the true values of the corresponding indicators.

In addition to uncertainty-related parameters, parametric and non-parametric correlation methods intensely examine the dependency between two datasets. The statistical correlation value between two datasets is a significant indicator because it shows the detailed data’s signal-to-noise ratios (SNRs). The Pearson correlation coefficient (r) is one of the most common SNR-related relative metrics used to evaluate EOs. Although the correlation coefficient, representing the statistical dependence between the satellite observation and the reference value, can take values between −1 and 1, negative observations were not considered in the validation process. Therefore, the r values closer to 0 indicate a weak correlation between the satellite observation and the SM reference value due to noise and errors in the satellite measurements. The r values closer to 1 indicate a strong relationship between these two data sets.

4. Results

4.1. SMAP/Sentinel-1 Overall Accuracy

Table 3. Statistical details of the validation sites and SMAP/Sentinel-1 SM data applied for the evaluation process.

Site Name	SM			NDVI			No. of Pixels	No. of Pixels with More Than One Stations	No. of Data
	Min	Max	Mean	Min	Max	Mean			Overall	Spring	Summer	Fall	Winter
AMMA-CATCH	0.006	0.154	0.037	0.13	0.35	0.20	6	1 (2)	645	150	181	175	139
SD_DEM	0.014	0.147	0.038	0.10	0.27	0.15	1	_	105	22	28	28	27
TAHMO	0.014	0.418	0.236	0.20	0.46	0.34	21	_	2369	637	435	581	716
COSMOS	0.047	0.455	0.167	0.15	0.38	0.23	5	_	368	62	106	107	93
OZNET	0.001	0.558	0.154	0.18	0.56	0.28	18	_	2087	649	499	444	495
COSMOS_2	0.172	0.517	0.295	0.20	0.56	0.42	2	_	147	41	34	35	37
PTSMN	0.149	0.503	0.394	0.13	0.63	0.47	4	4 (3, 3, 6, 8)	1458	438	390	294	336
CTP_SMTMN	0.023	0.568	0.204	0.20	0.62	0.27	50	2 (4, 2)	172	0	0	172	0
KIHS_CMC	0.107	0.273	0.202	0.19	0.52	0.36	18	1 (18)	129	25	45	44	15
KIHS_SMC	0.081	0.201	0.121	0.19	0.52	0.38	19	1 (19)	74	19	24	23	8
MAQU	0.054	0.627	0.315	0.19	0.56	0.37	19	2 (2)	1941	773	709	374	85
NAQU	0.027	0.311	0.151	0.20	0.33	0.22	8	1 (2)	1144	426	516	153	49
NGARI	0.025	0.331	0.102	0.11	0.25	0.18	13	_	1103	431	555	82	35
SMN-SDR	0.059	0.363	0.158	0.15	0.44	0.26	20	1 (2)	1355	240	627	430	58
VDS	0.006	0.452	0.208	0.20	0.40	0.27	4	_	712	103	171	209	229
BIEBRZA_S-1	0.275	0.795	0.548	0.12	0.59	0.34	2	2 (9, 9)	1711	455	448	485	323
FR_Aqui	0.031	0.389	0.144	0.12	0.59	0.31	3	_	3283	853	849	869	712
GROW	0.001	0.448	0.254	0.07	0.55	0.24	4	4 (16, 15, 4, 2)	1308	312	348	363	285
HOAL	0.180	0.499	0.339	0.13	0.46	0.26	1	1 (32)	258	54	57	90	57
HOBE	0.017	0.758	0.207	0.08	0.56	0.34	20	6 (3, 3, 3, 2, 2, 2)	5448	1362	1464	1416	1206
IPE	0.155	0.319	0.233	0.20	0.68	0.40	1	_	214	48	57	68	41
MOL-RAO	0.044	0.303	0.154	0.18	0.53	0.31	1	_	576	158	137	150	131
REMEDHUS	0.001	0.750	0.128	0.15	0.40	0.25	20	_	15,813	3618	3956	4499	3740
Ru_CFR	0.221	0.755	0.565	0.12	0.63	0.34	1	1 (2)	2404	630	758	633	383
SMOSMANIA	0.029	0.475	0.182	0.15	0.63	0.30	7	_	5966	1268	1437	1814	1447
TERENO	0.011	0.843	0.410	0.20	0.50	0.33	4	_	5660	1362	1361	1578	1359
UMBRIA	0.155	0.319	0.233	0.20	0.68	0.40	1	_	216	48	57	68	43
WEGENERNET	0.180	0.576	0.389	0.14	0.62	0.37	8	3 (3, 2, 2)	9799	2449	2560	2528	2262
LAB-net	0.172	0.510	0.283	0.07	0.14	0.10	2		532	106	124	153	149
ARM	0.015	0.469	0.264	0.15	0.20	0.18	10	_	539	106	59	190	184
COSMOS	0.172	0.517	0.295	0.13	0.29	0.18	9	_	978	253	274	235	216
FLUXNET-AMERIFLUX	0.004	0.520	0.266	0.13	0.46	0.26	4	_	2196	552	519	551	574
RISMA	0.023	0.563	0.268	0.10	0.63	0.36	19	2 (2, 2)	2187	804	890	455	38
SNOTEL	0.001	0.364	0.128	0.09	0.62	0.30	85	_	12,917	3799	2978	3199	2941
SOILSCAPE	0.049	0.340	0.216	0.20	0.41	0.25	30	_	2479	358	641	755	725
Overall							440	31	88,293	22,611	23,294	23,250	19,138

provides additional information on the validation sites—including the minimum, maximum, and average NDVI and SM values, the number of SMAP/Sentinel-1 product pixels assessed at each site (column 8 of Table 3), the number of pixels with more than one station, the number of stations in the corresponding pixels (the number in the parenthesis in column 9), and the total number of data points used to assess the SMAP/Sentinel-1 products—separately for each site. The number of data points used to validate SMAP/Sentinel-1 is also listed separately for each season in this table.

Table 3 shows that validation sites are found in various vegetation and SM conditions. BIEBRZA_S-1, with an average SM of 0.54, and Ru_CFR, with an average SM of 0.56, are two of the wettest networks available in Europe. The lowest humidity is found in African networks, such as AMMA-CATCH, with an average of SM of 0.037, and SD_DEM, with an average of SM of 0.038. In situ stations were distributed among a total of 440 SMAP/Sentinel-1 pixels, 31 of which have multiple field stations. For this analysis, 88,293 data points were obtained from over three hundred thousand SMAP/Sentinel-1 files and images. This information is evenly distributed across all four seasons: 22,611 data in the spring, 23,294 data in the summer, 23,250 data in the fall, and 19,138 data in the winter.

Figure 4 shows four different estimated statistics as boxplots for all validation sites with a high-quality satellite. It should be noted that since the correlation coefficient between the in situ measurement and satellite data was negative in the HOAL site, the results of this site were not mentioned in Figure 4. In this figure, the results for the different continents are shown in different colors. Using the median, minimum, maximum, and 1/3 quantiles of the analyzed statistics for each site, these plots can provide an overview of the performance of SMAP/Sentinel-1 in estimating SM. For better schematically representation of the results, the global quantitative maps of average correlation coefficient, average RMSD, average MD, and average ub-RMSE of validation sites are shown in Figure 5.

Regarding correlation coefficients (r) (Figure 4a), the highest correlation between SMAP/Sentinel-1 observations and in situ measurements is founded on SOILSCAPE (average r value of 0.86) with 30 ground stations distributed across the United States. Some stations at this site and some stations at other sites in Africa and North America have correlation coefficients greater than 0.95. FR_Aqui follows SOILSCAPE in Europe with an average r of 0.85 and AMMA-CATCH with an average r of 0.84. On the other hand, the LAB-net with four stations has the lowest correlation coefficient (average r of 0.17). Following this network, the lowest correlation coefficients are found in KIHS_CMC (average r of 0.19), 18 SMAP/Sentinel-1 grid cell stations, and IPE (average r of 0.27).

Based on the RMSD values of the different sites (Figure 4b), it can be seen that in most networks, the normal difference between in situ measurements and SMAP/Sentinel-1 SM observations is greater than 0.1. The highest RMSD values are observed in BIEBRZA_S-1 and Ru_CFR sites with two stations, with an average RMSD of 0.3 and 0.33, respectively. The lowest RMSD values are founded on SD_DEM and COSMOS-1 in Asia and NGARI in Africa. Regarding MD values (Figure 4c), the worst results (corresponding to the largest MD values) were again obtained at the BIEBRZA_S-1 and Ru_CFR sites, confirming previous results. MD values at other sites ranged from −0.2 for LAB-net to 0.12 for KIHS_CMC. However, despite high RMSD and MD errors at some sites such as BIEBRZA_S-1, Lab-net, and Ru_CFR, the ub–RMSEs of these sites (Figure 4d) are acceptable and comparable to those of other networks. In general, ub–RMSE ranges from 0.03 for SD_DEM and NGARI to 0.16 for COSMOS-2. It can be seen that the ub–RMSE values of four sites, three of which are in Africa and one of which is in Asia, meet the nominal error (RMSE better than 0.04 m³/m³) of the SMAP mission. With an average r of 0.78 and a ub–RMSE of 0.05, Oceania shows the best results validating SMAP/Sentinel-1 SM products. Africa follows Oceania with an average r of 0.71 and an ub–RMSE of 0.045.

4.2. Comparison of SMAP/Sentinel-1 and Enhanced SMAP SM Products

Figure 6 shows a schematic summary of the number and percentage of available SMAP 9 km and SMAP 3 km images for the validation sites. The numbers in parentheses represent the actual temporal resolution of the SMAP/Sentinel-1 and enhanced SMAP products. The revisit time is calculated based on the number of data available at each site and their availability period (see Table 1), taking into account the SMAP/Sentinel-1 data gap, which is given in https://nsidc.org/data/SPL2SMAP_S/versions/3 (accessed on 19 August 2022).

In terms of data accessibility over the same period, the superior performance of SMAP 9 km products compared to SMAP/Sentinel-1 data is evident in Figure 6. In most parts of the world (e.g., Asia, Australia, Africa, and the Americas), the total number of SMAP/Sentinel-1 images is almost half of the total number of enhanced SMAP products. Moreover, the average actual revisit time of enhanced SMAP is 2 days, while SMAP/Sentinel-1 has an average revisit time of 6 days. Figure 7 shows the overall results for the performance of the enhanced SMAP and SMAP/Sentinel-1 products in terms of the four metrics: correlation coefficient (r), RMSD, bias, and ub–RMSE. Note that this figure includes results for validation sites where both SMAP/Sentinel-1 and enhanced SMAP products have grid cells within more than one station. The last columns of each graph in Figure 7 show the average values of the corresponding metrics.

Regarding correlation coefficient values (Figure 7a), the correlation coefficient ranges from 0.18 for the WEGENERNET site to 0.95 for the PTSMN site with enhanced SMAP products and from 0.189 for the KIHS_CMC site to 0.84 for the AMMA-CATCH site with SMAP/Sentinel-1 products. At almost all validation sites except WEGENERNET, higher r values were estimated for enhanced SMAP products at 9 km spatial resolution. The difference between the r values, obtained for SMAP/Sentinel-1 and enhanced SMAP data, is larger for some networks, such as KIHS_CMC and KIHS_SMC, and smaller for some sites, such as Ru_CRF, HOBE, and RISMA. Figure 7a shows that the enhanced SMAP product, with an average r of 0.83, outperforms the SMAP/Sentinel-1 product, which has an average r of 0.6, in estimating SM. Moreover, in almost all networks, the RMSD of the SMAP/Sentinel-1 products (with an average of 0.15) is larger than that of the enhanced SMAP products (with an average of 0.11) (Figure 7b). As noted above, the RMSD value is a function of representativeness error, SMAP reference error in estimating TB and the SM retrieving method, and uncertainties in the BTBDA downscaling algorithm and Sentinel-1 backscatter measurements. Therefore, Figure 7b implies that a portion of the error (average value of 0.04) is imposed on the SM observations after using the Sentinel-1 backscatter measurements and downscaling method. In all networks except BIEBRZA_S-1 and Ru_CFR, RMSD is between 0.06 and 0.17. The RMSD values of BIEBRZA_S-1 and Ru_CFR are abnormally high for SMAP 9 km and SMAP/Sentinel-1. Figure 7c shows the same result for these two sites, with substantial negative bias values for both products. This confirms that the very high values obtained in RMSD of BIEBRZA_S-1 and Ru_CFR are not due to satellite observation error. Instead, a large part of the values is a result of representativeness error. Given the ub–RMSEs of enhanced SMAP calculated for each site (Figure 7d), removing the biases from the RMSDs brings the error of almost all validation sites closer to the SMAP nominated error, 0.04 m³/m³. However, high ub–RMSEs in BIEBRZA_S-1 and Ru_CFR sites cause the average error to be 0.074. By deleting the information of these two networks, the average ub–RMSE is reduced to 0.065. On the other hand, ub–RMSEs for the SMAP/Sentinel-1 were generally presented as more than 0.04 m³/m³, as the average of ub–RMSEs of all sites is ~0.093.

4.3. Impacts of Vegetation Conditions, Land Cover, and Soil Texture on the Accuracy of SMAP/Sentinel-1 SM Products

To investigate the influence of land cover on the accuracy of SMAP/Sentinel-1 SM products, the statistical indicators were calculated for different land covers of the validation sites. For this purpose, the land-cover class that accounts for more than 70% of the study area is considered the validation site’s mainland cover. Regions equally covered by forest and grass are assigned to the “Forest + Grassland” class. In this way, the regions covered by forest and cropland are considered the “Forest + Cropland”. Figure 8 illustrates the effects of land cover on the accuracy of SMAP/Sentinel-1 SM products.

The SMAP/Sentinel-1 observations in Grassland have the highest correlation coefficient (average r of 0.701) with the in situ measurements. After Grassland, the correlation coefficient obtained the best results in the Cropland (average r of 0.651) and Forest + Cropland (average r of 0.613) classes. However, the lowest error (average RMSD and ub–RMSE values) are observed in Bareland regions (average RMSD = 0.06 and un-RMSE = 0.030). According to Figure 8, the error values are higher in Cropland (average RMSD = 0.115 and un-RMSE = 0.083) and Grassland (average RMSD = 0.113 and un-RMSE = 0.077) compared to the Bareland. On the other hand, SMAP/Sentinel-1 SM products showed poor results in the correlation coefficient, RMSD, and ub–RMSE in the Forest regions (R = 0.434, RMSD = 0.161, and ub–RMSE = 0.106). It can also be seen that the presence of forests in the grassland regions affects the accuracy of the SMAP/Sentinel-1 products (Grassland versus Forest + Grassland in Figure 8). These results show less sensitivity of the SMAP/Sentinel-1 to SM content variations beneath vegetation, especially in very moist vegetation and forest regions.

We also investigate whether water bodies and urban areas less than 0.25 km² that are not detectable in the MODIS land-cover products can affect the accuracy of the SMAP/Sentinel-1 SM products. The underlying concept is that the MODIS land-cover product with a spatial resolution of 500 m and 1 km is used as dynamic ancillary data to eliminate the influence of water bodies, building areas, woodlands, and croplands on both the active–passive algorithm [46] and the SM retrieval algorithm of TB [13]. The results of this assessment are shown in Figure 9. The plots (a) and (b) in Figure 9 show the relationship between the percentage of water/urban classes within the validation regions and the correlation coefficient obtained between the SMAP/Sentinel-1 SM observations and the in situ measurements. The same results for the RMSD are shown in the plots of (c) and (d) in Figure 9. Note that to better examine the impact of water bodies on SMAP/Sentinel-1 performance, we have considered the wetland as a water body in this part of the validation process.

According to Figure 9, the highest percentage of water bodies and urban areas in the validation sites were a maximum of 7% and 12%, respectively. Figure 9 shows that for those pixels of SMAP that include water bodies and urban areas, after MODIS land-cover correction, the accuracy of the SMAP/Sentinel-1 SM product is independent of the presence of water bodies and urban areas. It can be concluded that the corrections applied in the 3 km products of SM using the MODIS land-cover map with a spatial resolution of 500–1000 m were able to minimize the effects of water bodies on the dielectric constant extracted for L-band radiometry and the effects of the urban area on the backscatter coefficient retrieved by the Sentinel-1 radar instrument.

The representation in Figure 10 shows the overall accuracy of the SMAP/Sentinel-1 products at all validation sites as a function of soil type. Figure 10a shows the percentage of validation networks belonging to six different soil types. All soil types have medium or coarse surface textures. According to Figure 10a, 46% of the networks studied have sandy loam soil. After sandy loam, clay and loam soils have the most significant number of networks, with about 23% and 21% of the networks, respectively. Only one network had sandy soils.

Figure 10b,c shows the average correlation coefficient and ub-RMSE of each soil type. Given this figure, the SM measurements of 4% of the networks with sandy clay loam soil have the highest correlation with the SMAP/Sentinel-1 SM products. As can be seen in this figure, despite the slight similarity between the average correlation coefficients and ub-RMSE in the different soil types, it can be said that the influence of the soil type on the accuracy of the results can be ignored.

4.4. Seasonal Assessment of the SMAP/Sentinel-1 Performance

The statistical indicators were calculated for the four seasons for a more in-depth investigation into the dependence of the results shown in Figure 4 and Figure 7 on the seasonal climatic conditions. Figure 11 shows the histogram of r values of validation sites.

The results presented in Figure 11 show that SMAP/Sentinel-1 SM observations in winter have the lowest correlation with in situ measurements. In this season, the correlation coefficients of more than 50% of the validation networks were below 0.5 and even below the average r value of this season (i.e., 0.269). In contrast, in autumn, the correlation between in situ data and retrieval values from SMAP/Sentinel-1 SM is the highest. In this season, the correlation coefficients obtained for more than half of the validation networks range from 0.55 to 1. Thus, with an average value of 0.53, autumn has the best accuracy for estimating SM. Note that despite the significant differences in the histogram obtained in spring and autumn, the average correlation coefficients in these two seasons are not significantly different. Therefore, spring was identified as the second season with average correlation coefficients of 0.51.

5. Discussions

5.1. SMAP/Sentinel-1 Overall Accuracy

The ISMN SM networks provide in situ measurements of SM under a variety of conditions that can be used to validate SMAP/Sentinel-1 SM products. The analysis of SMAP/Sentinel-1 SM data in various validation sites shows that this moderate-resolution SM product can provide SM with acceptable accuracy (Figure 4). Consistent with the pattern of the performance metrics over all sites, dense validation sites, which have a larger number of stations distributed within the SMAP/Sentinel-1 cell, had a higher correlation coefficient and lowered ub–RMSE (Figure 4 and Table 3). According to Figure 4, despite the high correlation coefficient (R) between in situ measurements and SMAP/Sentinel-1 SM observations, there are high RMSD values at some sites such as BIEBRZA_S-1 and Ru_CFR (Figure 4b). However, the high observed bias (Figure 4c) at these two sites indicates that the high RMSD values are due to a possible undetectable and undesirable error, called representativeness error [119,120]. Representativeness error refers to the difference between the real SM value within a particular grid cell, and the average of point-based in situ measurements within the corresponding grid cell considered the reference values [101]. This error is unrelated to satellite performance and must be removed from the validation process. The large bias values between the enhanced SMAP SM observations and the in situ measurements from BIEBRZA_S-1 and Ru_CFR (Figure 7c) can show the representativeness error in these networks. According to Equations (3) and (4), this type of error in the validation process causes α of the reference data to be non-zero, which significantly affects the results, especially at higher SM values. As detailed in Table 3, BIEBRZA_S-1 and Ru_CFR are two of the wettest networks available, and both are located in Europe. Based on the assumptions considered for the validation process of EOs, representativeness error is unavoidable incredibly when coarse-scale SM product is evaluated using sparse or dense SM networks [121]. According to Figure 4c, from all validation sites, 18 sites have a negative bias, 13 sites have a positive bias, and 3 others have zero bias. The bias is not due to an overestimation or underestimation of satellite observations, but a possible representativeness error. Fortunately, this error is detectable using MD and bias and can be decreased from the overall error of the EOs. Therefore, the ub–RMSE, which separates the bias values (MD) from the RMSD, can better explain the performance of SMAP/Sentinel-1. By comparing RMSD and ub–RMSE indicators of the BIEBRZA_S-1 and Ru_CFR sites, it can be seen that the ub–RMSE decreased sharply after removing the MD from the RMSD values. The presence of good correlations (Figure 4a) in these two networks also confirms that the observed error is not related to satellite performance. Similar performance (SM retrievals without significant errors) have also been found and reported for previous SM products retrieved from various RS missions, such as SMOS, AMSR_2/E, and SMAP coarse-scale SM products [24]. Compared with coarse-scale SM products divided from previous passive microwave satellites, the SMAP/Sentinel-1 product with a higher spatial resolution can therefore be more beneficial for various purposes.

5.2. Comparison of SMAP/Sentinel-1 and Enhanced SMAP SM Products

Figure 6 and Figure 7 compare the performance of SMAP/Sentinel-1 and enhance SMAP from different points of view. The underlying concept is that for the derivation of SMAP/Sentinel-1 SM product, enhanced SMAP L-band radiometer observations from half orbit are used as a reference radiometer measurement and downscaled with Sentinel-1 radar data [19]. Comparison of SMAP-9 km and SMAP/Sentinel-1 3 km SM products allowed us to determine the magnitude of the error imposed on SMAP-3 km SM by the SMAP-9 km reference data, Sentinel-1 backscattering measurement, or BTBDA downscaling method. Moreover, it expressed the negative effects of integration of Sentinel-1 and SMAP data on the various characteristics of SMAP original products (e.g., its temporal resolution (Figure 6). As shown in Figure 6, the actual revisit time of the enhanced SMAP is twice as long as that of SMAP/Sentinel-1. Therefore, in terms of data availability over the same period, the SMAP 9 km products are superior to the SMAP/Sentinel-1 data. This result could be partly explained by the low revisit time of the Sentinel-1A/B satellites. In short, the studies conducted before the launch of the SMOS mission showed that an interval of 2–3 days is the optimal temporal resolution for a passive microwave mission SM [122]. In this way, as a follow of the SMOS mission, the nominal revisit time of SMAP is planned for 2–3 days [19,123]. However, in the case of SMAP/Sentinel-1, temporal resolution is limited by the Sentinel-1A/B satellites’ revisit time (~six days) [14]. Moreover, since the Sentinel-1B satellite is no longer available, the repeat coverage of radar data may worsen. In addition to the low revisit time of the Sentinel-1A/B satellites, the active–passive SMAP algorithm, which discards any suspect or erroneous backscatter coefficient, reduces the revisit time of the SMAP/Sentinel-1 product.

Given Figure 7a–d, the enhanced SMAP product has better performance when it comes to estimating SM than the SMAP/Sentinel-1 product. The SMAP/Sentinel-1 development team has attempted to overcome the major challenges of combining active and passive microwave measurements by considering (i) temporal mismatch between the two independent satellite platforms SMAP and Sentinel-1A/B, (ii) estimation of different beta and gamma parameters (the coefficient of the linear functional relationship between backscatter coefficient and TB), and (iii) different frequencies and penetration depths of the active and passive instruments. By comparing the results obtained from the evaluation of SMAP/Sentinel-1 products with the validation results for previous active–passive algorithms, it can be seen that they have been successful in achieving their goals. However, given the conditions and assumptions applied to the integration of the enhanced SMAP and Sentinel-1 data, and the inherent inaccuracies of the radar data due to the influence of target structure and vegetation condition, this conclusion is to be expected [124]. It must be kept in mind that, despite the differences in the performances between SMAP-enhanced and SMAP/Sentinel-1 data, these high-resolution SM products still meet the demand for overall accuracy in estimating SM.

5.3. Impacts of Various Geographical Parameters on the Accuracy of SMAP/Sentinel-1 SM Products

Based on Figure 8, the performance metrics over the non-forested sites were significantly better than the level achieved in the woodland areas. Minimal sensitivity to incident angle variations of SMAP radiometer for heavily vegetated areas [77], the sensitivity of Sentinel-1 radar to vegetation roughness and dispersion [125], masking of incident angle variations of active–passive microwaves by dense vegetation [46], lower penetration of electromagnetic waves in dense vegetation [126,127], and the effect of plant moisture on the dielectric coefficient might be the main reasons for this result. A similar result was obtained by [128], indicating that the accuracy of SMAP SM decreases with decreasing vegetation percentage, especially in the forest areas. On the other hand, Colliander, et al. [129] enhanced SM products with in situ measurements after comparing the observation of SMAP, demonstrates that spaceborne L-band radiometry is sensitive to SM under temperate forest canopies (RMSD range of 0.047–0.057 m³/m³ and r rang of 0.75–0.85). Since the results of this study show the same range of the correlation coefficient for SMAP-enhanced SM products at the forest validation sites (such as KIHS_CMC, SMOSMANIA, and TERENO, Figure 7), the substantial part of the error must be added to the products after combination with active radar data.

It must be noted that vegetation effect correction is considered in both the active–passive algorithm [46] and the SM retrieval algorithm of TB [13] using the MOD13A2 (V005) vegetation indices product [130]. However, due to the remarkable temporal changes in vegetation and surface roughness [12] and the presence of diverse vegetation with different spectral characteristics in a coarsely scaled SMAP pixel [131], the vegetation effect correction might be incomplete in some vegetated areas. The poor performance of the SMAP/Sentinel-1 products in the shrubland (R = 0.510, RMSD = 0.161, and ub–RMSE = 0.106, Figure 8) is evidence of the incomplete correction of the vegetation effect mentioned above. Shrubland regions of many parts of the world are classified as bare land because of their seasonal variation in vegetation conditions and low NDVI in particular seasons. As a result, active–passive algorithms sometimes miss vegetation corrections in these locations. In contrast, correcting for land cover using the MODIS MCD12Q1 (V051) land-cover map [19], temperature and precipitation effect using NASA Global Modeling and Assimilation Office (GMAO) earth system model (GEOS-5) [132], and correcting for soil type using the HWSD soil texture map [133] significantly eliminated the effects of varying soil texture and the proportion of water bodies and urban areas on the SM values obtained from active and passive microwave observations (Figure 9 and Figure 10). As shown in Figure 11, the correlation coefficient between SMAP/Sentinel-1 and in situ values SM was significantly lower in winter than in the other seasons, even in the seasons with the highest proportion of vegetation. This is somewhat contradictory to the results in Figure 8, where the vegetation fraction was observed as one of the leading causes of the error, while in winter, there is usually the least amount of vegetation in the region. Limited variations in SM values during particular seasons and the fewer data used to estimate the performance of products during a specific season are two of the most important reasons for reaching this conclusion. It must be noted that the number of data for winter was significantly less than other seasons, which can have a great impact on the final results related to these seasons.

5.4. Sources of Uncertainty in the Validation of SMAP/Sentinel-1 Product

Despite several preprocessing steps, careful selection of in situ reference measurements, dubious satellite data elimination, and highly accurate and appropriate statistics, there are still some sources of uncertainty in the SMAP/Sentinel-1 validation process. As claimed by [134], these errors are not caused by the poor design of the study methods. They are generally stochastic uncertainties resulting from the weak assumptions and conditions inevitably considered in space-based EO assessment methods. Gruber et al. [101] advised ways to reduce statistical uncertainty and develop a deep understanding of the quality of SM products. This study has tried to implement all the corresponding suggestions in data selection and processing (see Section 3.1 and Section 3.2). However, there are still some sources of error, perhaps the most serious of which are as follows.

5.4.1. Distribution of In Situ Stations

As mentioned before, observations of sparse networks that generally lead to a point within a SMAP footprint are unsuitable for validating SMAP coarse-scale pixels [135]. With less rigor, this rule also applies to evaluating extended 3 km products. The single-point station can later be used to validate the SMAP/Sentinel-1 3 km pixel once it has been verified as an accurate representation of SM over the SMAP cell extent [22]. However, this condition is not necessarily true for all stations of the sparse sites. In the current state of the art, the stations located at the corners of the grid cells of EO do not accurately represent the SM of the grid pixels.

Furthermore, when multiple stations are heterogeneously distributed within an EO grid, the average measurements of these stations cannot be expected as an accurate proxy for the actual SM of the grid cell. If either of these scenarios occurs in SM networks, the data collected from these stations will be inaccurate. To illustrate this issue, Figure 12 shows the site layout and SMAP/Sentinel-1 3 km grid for REMEDHUS in Spain, PTSMN in New Zealand, and CTP_SMTMN in China.

According to Figure 12a, the distribution of stations in the REMEDHUS network is that each SMAP/Sentinel-1 pixel has only one station (represented by black dots in Figure 12). Some of these stations are located practically in the center of the pixel, while others are located near the edges. In PTSMN (Figure 12b), four SMAP/Sentinel-1 pixels have more than one station. However, for two of them (the lower pixels), the distribution of stations within the pixel is not uniform. The CTP_SMTMN network (Figure 12c) includes all four possible types of distribution of stations within pixels: (1) pixels with one station in the center, (2) pixels with one station in the corner, (3) pixels with multiple stations homogeneously distributed, and (4) pixels with multiple grids with asymmetric distribution.

In conclusion, the distribution of stations in the sparse networks is one factor that leads to uncertainties in the evaluation results. It is worth considering that the effects of this source of error are amplified in the inhomogeneous SMAP/Sentinel-1 3 km cells, limiting the accuracy of the validation process. Note that in addition to the sites and stations listed in Figure 12, some other networks and stations have similar conditions to the listed ones.

5.4.2. Depth of Reference Measurements

Another source of uncertainty is the discrepancy between the sampling depth of the EO observations and the in situ measurements. In general, field measurements at some of the in situ reference stations used in this work were made at various depths from 5 cm to ~10 cm. However, for RS passive microwave radiometers, the adequate SM sampling depth in the L-band (~0–3 cm) is less than the sampling depth of the in situ measurements [22,136]. Because there is not a strong relationship between water volumes at the soil surface and deeper layers in some regions, depending on soil texture and vegetation types [106], the mismatch in penetration depths between different datasets creates a source of uncertainty that cannot be modeled [116]. Figure 13a,b show how the non-relationship between SM of different soil depths can cause an untraceable error in the validation process.

Figure 13a shows the temporary profile of the in situ estimate SM of the MccrackenMese station of the SCAN site in the United States at different depths of 5 cm, 10 cm, 20 cm, and 50 cm. The Hydraprobe Digital Sdi-12 (2.5 volts) detectors in the SCAN network measure SM with dynamic penetration depths ranging from 5 cm to 2.03 m [137]. Figure 13a shows no relationship or correlation between the SM measured at different soil depths at virtually all measurement times. Furthermore, there is no consistent relationship between the variations in SM behavior at different depths. As a result, the difference between the SM at different depths cannot be modeled as bias. Although all of the in situ SM measurements deeper than 10 cm were removed from the validation process, this effect may still result in low correlations between in situ measurements of SM and observations of RS when taken at different soil depths.

Figure 13b shows how important the discrepancy between in situ and EO measurement depths can be, especially when the different RS and field datasets are observed at different times of day. In this figure, the red columns show the difference in SM values recorded at 7 am and at 10 am for a depth of 0–5 cm. The same values for the depth of 10 cm are shown with the blue columns. The temporary profile in this figure belongs to the measurements from the Irrigation Technology Centre (ITC) station, located in Myanmar and part of the VDS site. In short, in situ reference measurements and EOs may have different observation times from 6 am to 10 am. This figure shows that SM does not have the same variations in the 3 h intervals, i.e., the time interval between receiving SMAP, Sentinel-1, and in situ data, at different ground depths. This problem proves that when evaluating RS-based SM products using in situ measurements reported at different times, the difference between the sampling depth of observations and measurements can cause a large error in the evaluation results.

5.4.3. In Situ SM Detectors

Another source of uncertainty is the measurement of SM with different detectors and probe sensors. The underlying concept is that different probe sensors have different sensitivity to environmental parameters, and consequently, different errors in their measurement, due to their various structure, design, and measurement techniques. In this way, using different probes, various SM measurements may be taken at a specific time and penetration of soil, but with different accuracy. This issue can be addressed via permanent calibration of probe sensors. However, since each station in many international SM stations is equipped with at least two different SM detectors, it is not possible to calibrate all of them based on their unique characteristics. To better understand this problem, Figure 13c shows the temporary profile (2016–2019) of SM measured by Deogon-5TE-B and Deogon-5TE-A at a depth of 5 cm over the stations of 3.01 of the HOBE site in Denmark. Figure 13c shows an absolute difference between the SM measurements of these two detectors located at the same soil depth. Since the measurements of one detector are randomly selected and used as reference data in the validation process, the corresponding difference may turn into an undetectable and unintentional error.

In addition to all mentioned possible error sources, the effects of temporary and undetectable waters, inaccurate and erroneous measurements in various SM detectors during their missions, the use of a variety of different SM detectors in all of the international SM networks, the presence of permanent vegetation in the SMAP-36 km reference footprints, and a variety of other non-modeling elements may have an impact on SMAP/Sentinel-1 validation results reported in this paper.

6. Conclusions

To better understand the benefits and limitations of the SMAP/Sentinel-1 SM products, we validated this product using in situ point-based measurements of global extended networks. This validation differs from previous studies primarily in terms of validation sites and the study period, which were evaluated over dense and sparse networks distributed around the world and over a long period of time. In addition, the effects of the disaggregation algorithm, land cover, water bodies, seasonal climate types, and soil texture on the SMAP/Sentinel-1 SM product were comprehensively evaluated. In situ measurements from 35 SM networks and 561 stations with constant geographical distribution were used as the reference data. Investigating and assessing the main sources of the error that affect the accuracy of the SMAP/Sentinel-1 data and the results of validation processes is one of the findings in this paper.

We conclude that there is an acceptable agreement (average correlation coefficient of 0.67 and ub-RMSE of 0.08) between SMAP/Sentinel-1 SM and in situ SM measurements at most sites in the world. For 10 validation sites, the average r values between the in situ measurements and SMAP/Sentinel-1 data were greater than 75%. The r values reach 90% in some stations and sites. We compared the accuracy of the enhanced 9 km SMAP SM and the SMAP/Sentinel-1 data with measurements of denser networks. The results showed that the enhanced SMAP product has better performance in estimating SM and a higher actual revisit time than the SMAP/Sentinel-1 product. In addition, the results showed that the accuracy of the SMAP/Sentinel-1 SM product is nearly independent of the presence of water bodies and urban areas, soil texture, and seasonal variation. However, the performance metrics in the non-forested areas were significantly better than those obtained in the forested areas. In addition, this article stated the non-uniformity of the depth of RS observation and in situ measurement, the time delay between these two datasets, and the variety of sensors in different stations as the major sources of error that affect the accuracy of the study. We hope that evaluating this novel SM product using global reference measurements will help researchers and will be a practical step towards improving the performance of satellite data in various studies.