*4.1. Statistical Analysis of SSM Time Series at the Network Scale*

Interestingly, SMAP and SMOS satellite products agree reasonably well among them, capturing the marked wet up and dry down variations along time. Nevertheless, a strong dependence of results from comparison to in situ on land use is found. Both, SMAP and SMOS products are overestimating the in situ measurements in vineyards (Figure 2a). Instead, satellite data underestimate the in situ SSM for irrigated crops (Figure 2b), while they almost match up with in situ observations for fallow/rainfed crops (Figure 2c), which are the most common land uses in the REMEDHUS area.

**Figure 2.** Daily evolution of the in situ SSM (black) and the three low-resolution (radiometer-only) SSM (SMAPL2\_E, red; SMAPL2, green; and SMOSL3, blue) at three REMEDHUS stations with different land use: (**a**) J3 (vineyard), (**b**) K13 (irrigated), and (**c**) O7 (rainfed/fallow).

The statistics derived from the temporal inter-comparison of low-resolution SSM products with in situ data using all the concurrent samples available for each dataset are summarized in Table 3. Comparing the different instruments, results show that the two SMAP products have the same or a slightly higher correlation (ΔR <sup>≤</sup> 0.12) and similar unbiased errors (ΔuRMSE <sup>≤</sup> 0.01 m3/m3) than the SMOS product in all the study cases. There are no significant differences between the metrics obtained for the SMAP SSM (9 km vs. 36 km). Regarding the different land uses, the worst results were obtained for K13, an irrigated station, with a R (and a bias) of 0.46 (and <sup>−</sup>0.142 m3/m3) for SMAPL2\_E, 0.48 (−0.143 m3/m3) for SMAPL2, and 0.46 (−0.183 m3/m3) for SMOSL3. Besides, the uRMSE of K13 is twice as high the objective accuracy of both space missions (SMOS and SMAP). This underperformance probably comes from the fact that irrigated land is not the most representative land use within the low-resolution SMAP/SMOS pixels, which is mostly covered by rainfed crops (see Table 4). On the contrary, the best results are obtained for the stations located over rainfed/fallow land cover (H13 and O07), with a R between 0.79 and 0.83 for both SMAP products (SMAPL2\_E and SMAPL2), and between 0.70 and 0.80 for SMOS. Their bias is low, between 0.027 and 0.035 m3/m3 for SMAP, and between 0.004 and 0.068 m3/m3 for SMOS, in absolute values. The uRMSE of these two stations are around 0.04–0.05 m3/m3, meeting or almost meeting required accuracy of both missions. In the case of vineyards (J03), intermediate results are obtained. The highest R is obtained for two SMAP products (0.85), but a high R is also obtained for SMOSL3 (0.73). The uRMSE is very similar (from 0.045 to 0.048 m3/m3). However, the bias of J03 is similar for SMOSL3 (0.057 m3/m3), but the SMAP ones are up to two or three times higher (0.106 and 0.103 m3/m3) than the aforementioned ones for rainfed/fallow. At these spatial scales, the number of available samples of SMAP and SMOS with in situ samples are of the same order (around 500 days), ensuring a robust statistical analysis.

**Table 3.** Statistics obtained from the comparison of *in situ* SSM against the concurrent low-resolution (radiometer-only) pixels SSM time series: of SMAPL2\_E (left), SMAPL2 (center) and SMOSL3 (right), from April 2015 to December 2017. The *in situ* stations used (and their respective land use) are: H13 (fallow), H9 (forest-pasture), J3 (vineyard), K13 (irrigated), N9 (rainfed) and O7 (rainfed/fallow).


**Table 4.** Percentage of rainfed and irrigated croplands (the two most common land covers over the REMEDHUS network) within the SMOS and SMAP pixels (36 km, 25 km, 9 km, 3 km and 1 km) enclosing the *in situ* stations J3 (vineyard), K13 (irrigated) and O7 (rainfed/fallow).


When analyzing the metrics derived from the validation of SMAP and SMOS at high resolution (see Table 5), the irrigated station K13 keeps showing the worst results as in the low-resolution case: a R (and a bias) of 0.45 (−0.142 m3/m3) for SMAP\_AP1, 0.51 (−0.129 m3/m3) for SMAP\_AP3, and 0.42 (−0.186 m3/m3) for SMOSL4. This indicates that irrigated areas are not even spatially representative at the scales of 3 km to 1 km, which denotes the small extent of these areas within the satellite footprint (see Table 4).

**Table 5.** Statistics obtained from the comparison of *in situ* SSM against the concurrent high-resolution pixels SSM time series: of SMAP\_AP1 at 1 km (left), SMAP\_AP3 at 3 km (center) and SMOSL4 at 1 km (right), from April 2015 to December 2017. The *in situ* stations used (and their respective land use) are: H13 (fallow), H9 (forest-pasture), J3 (vineyard), K13 (irrigated), N9 (rainfed) and O7 (rainfed/fallow).


Similarly, the best results are obtained for the stations H13 and O07, with R between 0.66 and 0.86 for SMAP\_AP1 and SMAP\_AP3, and between 0.71 and 0.80 for SMOSL4. The lowest bias is precisely observed in H13 and O07, ranging between 0.025 and 0.056 m3/m<sup>3</sup> for SMAP, and from 0.001 to 0.076 m3/m3 for SMOS, in absolute values. Again, the reason for that is the predominance of rainfed crops and fallow regions over REMEDHUS (see Table 4). Therefore, both satellites mostly see the land cover types leading to a cover-characteristic signal at low- as well as at high-resolution. As previously observed in Table 3, the metrics for vineyard are in a well acceptable range. On the one hand, taking into account both SMAP and SMOS, R varies between 0.70 and 0.83, and the uRMSE is always around 0.04–0.05 m3/m3. On the other hand, the bias of J03 is doubled or even tripled (0.081 and 0.114 m3/m3) with respect to the stations H13 and O07. All the SMAP and SMOS products are overestimating the in situ measurements of J03. One reason could be that grapevines are settled on very fine sand, which causes the water not to be retained and it quickly percolates into deeper layers. Additionally, the vineyard areas of REMEDHUS are not spatially representative at scales of 1 km and beyond.

Due to the missing synchronization of SMAP and Sentinel-1 acquisition orbits, the number of samples is much lower in the SMAP\_AP1 and SMAP\_AP3 (96 to 101 days) than in the SMOSL4 time series (443 to 513).

Similar statistical scores are obtained for the SMOS products when the same number of samples is used at high and low resolution, in line with the results obtained in a previous study [13]. When the same analysis is conducted for the SMAP products, only slightly worst performances are obtained for the SMAP\_AP1 product.

Table 6 shows the statistics obtained between the average SSM values of the stations located over a rainfed/fallow land use (F11, H13, J12, J14, K10, M9, and O7) and the average of the concurrent SMAP and SMOS products at high-resolution (see Figure 3). Lower correlations are obtained during summer season (0.62, 0.64 and 0.65, for SMAP\_AP1, SMAP\_AP3 and SMOSL4, respectively). This is consistent with the results of previous studies [13]. Slightly better results are obtained for SMAP in terms of R (and bias) 0.88 (0.014 m3/m3), against SMOS, 0.79 (0.04 m3/m3) calculated as an average of DJF, MAM, and SON.

**Table 6.** Statistics obtained from the comparison of *in situ* SSM against the high-resolution pixel SSM of SMAP\_AP1 at 1 km (left), SMAP\_AP3 at 3 km (center) and SMOSL4 at 1 km (right) from April 2015 to December 2017 for the different seasons of the year and also for the entire study period (ESP). Statistics are obtained after averaging all-time series of rainfed/fallow stations (F11, H13, J12, J14, K10, M9 and O7) and the pixels that contain these stations.


**Figure 3.** Daily evolution of in situ SSM (black) and the three high-resolution SSM products (SMAP\_AP1 at 1 km, red; SMAP\_AP3 at 3 km, green; and SMOSL4 at 1 km, blue) after averaging time series of rainfed/fallow stations (F11, H13, J12, J14, K10, M09, and O07) and the pixel time series that contain these stations.

#### *4.2. Analysis of the SSM Spatial Patterns*

The study carried out in the previous section shows a general agreement between the temporal dynamics of all the considered SSM products regardless of their spatial resolution (low resolution: ~40 km, 9 km vs. high resolution: 3 km, 1 km). However, as it can be seen from the maps shown in Figure 4, there are clearly visible differences in the spatial patterns attained by the downscaled SMAP and SMOS high-resolution products. In this section, we examine these differences and conduct specific analyses to test two hypothesis: (i) that they are due to differences in the multi-sensor synergies they are built upon (optical-microwave or active-passive) and (ii) that they are due to the rationale of the approach (e.g., whether the downscaling is conducted in brightness temperature- or in the soil moisture-space).

**Figure 4.** Temporally-averaged map of daily SMAP (**a**) and SMOS (**b**) products at 1 km over the Iberian Peninsula for the period December 2016 to February 2017.

4.2.1. Comparison of SSM Enhanced Resolution Products

Figure 5 shows the daily differences (map and histogram) between the SMAP and SMOS products at 1 km (SMAP\_AP1 minus SMOSL4) for the whole study period. The mean of these differences is minimal (of 0.03 m3/m3), less than their target accuracy, and their std is also low (of 0.09 m3/m3). The same behavior is observed when this study is performed on a year-to-year basis (not shown). In addition, we conducted the same analysis per season, and we obtained that daily differences ranged between 0.03 and 0.04 m3/m<sup>3</sup> in mean and from 0.05 to 0.07 m3/m3 in std (not shown). These results affirm that the differences cannot be explained by seasonal or yearly differences (e.g., dry or wet year). Yet the temporally-averaged map of daily SSM differences (Figure 5a) reveals that there is a geographic spatial pattern that persists over time when comparing the two high-resolution products, with higher differences located in the north, northwest and west of the Iberian Peninsula, in close correspondence to forested areas (see land cover maps on Figure 6).

**Figure 5.** (**a**) Temporally-averaged map of daily SSM differences between SMAP and SMOS at 1 km (SMAP\_AP1 minus SMOSL4) and (**b**) histogram of daily SSM differences maps, for the period April 2015 to December 2017.

**Figure 6.** (First row) The three most common land covers types over the Iberian Peninsula (**a**), agriculture; (**b**) forest; and (**c**), grassland) according to the CCI LC map. (Second row) Histograms of the daily SSM differences (SMAP\_AP1 minus SMOSL4) for the respective land covers.

The possible dependence of the differences between the two downscaled products on the land cover was further examined. Pixels from the temporally-average map of SSM differences (Figure 5a) were grouped for the most common land cover classes (agriculture, forest and grassland) and their

histograms were analyzed (see Figure 6). In general, pixels with SSM differences (SMAP minus SMOS) equal or above 0.1 m3/m<sup>3</sup> are located within forests (66.71% of the pixels) or agriculture (22.47% of the pixels). The largest SSM differences are observed for forest land cover, with a mean of 0.04 m3/m<sup>3</sup> and a std of 0.11 m3/m3.

Land cover only partially explains the SSM differences between the SMAP and SMOS products at 1 km. Results show that soil moisture values provided by SMAP over the Iberian Peninsula are systematically higher than the ones provided by SMOS (see Figure 5), although the absolute difference is minimal (mean difference of 0.03 m3/m3). The SSM values of SMOS exceed those of SMAP less often and with lower intensity, but this effect is mostly occurring in coastal areas. The same SSM difference pattern is found in the temporally-averaged map of daily TB differences (SMAP minus SMOS TB) shown on Figure 7. Since the spatial pattern is already present at TB level, we can conclude it was not introduced by neither SMAP nor SMOS downscaling methodologies. Figure 7b shows the histogram of the daily TB differences, with an absolute mean value of 2.92 K. In order to understand to what extent, the 2.92 K cold bias could affect SMAP or SMOS SM retrievals, we analyzed Davenport et al. in [42], who conducted a sensitivity analysis of soil moisture retrieval using the applied tau-omega microwave emission model. A bias of about 3◦ (K) TB approximately corresponds to 3–4 (vol.%) error in estimating volumetric soil water content, which would fit to the bias ranges reported in our study.

**Figure 7.** (**a**) Temporally averaged map of daily TB differences between SMAP (40◦ incidence angle) and SMOS (42.5◦ incidence angle) at 25 km and (**b**) histogram of temporally-averaged daily TB differences, for the period April 2015 to December 2017.

## 4.2.2. Downscaling Impact on SSM Differences

Figures 8 and 9 show the maps and histograms of daily SSM differences between SMAP\_AP1 and SMAPL2 (1 km vs. 36 km), and between SMOSL4 and SMOSL3 (1 km vs. 25 km), respectively. The mean (and std) obtained after calculating the differences along the complete study period are <sup>−</sup>0.01 m3/m3 (0.07 m3/m3) for SMAP, and of ~0 m3/m3 (0.03 m3/m3) for SMOS. Although mean differences are minimal over the whole domain for both sensors, the resulting average map of the SMAP SSM differences reveal some underlying spatial patterns. The highest positive differences obtained for SMAP are concentrated in the forested regions (see Figure 6) and the highest negative differences appear near the coast and in areas of complex topography. This is possibly due to the reduced sensitivity of the Sentinel 1 signal at C-band to soil moisture in presence of significant vegetation backscattering. According to [12], this leads to a decrease in TB after downscaling and therefore to an increase in estimated soil moisture. Also, the temporally-averaged map of SMOS SSM differences exhibits an underlying boxing effect that can be explained by the use of SMOS SSM at low-resolution as a reference to obtain the downscaling parameters of (2), as previously observed in [13,21]. However, this effect is nonetheless negligible and does not have a significant impact in the enhanced resolution product.

**Figure 8.** Temporally-averaged map (**a**) and histogram (**b**) of daily SMAP SSM differences (SMAP\_AP1 at 1 km minus SMAPL2 at 36 km), for the period April 2015 to December 2017.

**Figure 9.** Temporally averaged map (**a**) and histogram (**b**) of daily SMOS SSM differences (SMOSL4 at 1 km minus SMOSL3 at 25 km), for the period April 2015 to December 2017.

#### **5. Discussion**

Validation and comparison of SSM satellite products using in situ networks data is a difficult task. Many confounding factors can intervene and must be taken into account when interpreting the obtained results. Importantly for this work, it is essential to understand that the representativeness of each satellite dataset plays a crucial role when validated. In the case of the SMAP and SMOS products presented in this study, the data is stored in maps with spatial grid cells of 36, 25, 9, 3, or 1 km. The information contained in these cells represents an areal-averaged value, while the in situ measurements represent an isolated data point. Measurements of the most representative (in terms of land cover within satellite cells) SSM in situ stations have been averaged (as can be seen in Figure 3 and Table 6) to validate the large scale SSM estimates provided by the satellites [37,43]. It is found that validating satellite-based estimates works best with in situ measurements in locations where the dominant land cover of the satellite footprint prevails.

Besides the representativeness error due to the comparison of point-scale vs. areal-averaged measurements, the mismatch between satellite observations and in situ measurements can also be originated by the penetration depth of microwave frequencies at L-band, which is of 5 cm on average but also depends on the soil moisture content itself (with greater penetration on drier soils) [3]. In contrast, measurements of the REMEDHUS network probes are placed at a depth of 5 cm. This could be one of the reasons explaining the low correlations obtained under water-limited conditions (see Table 6 for JJA), when the soil surface dries out and therefore in situ sensors might measure slightly

wetter values than satellites. Moreover, the surface temperature used in SMAP and SMOS retrievals is derived from models. While the SMAP surface temperature is derived from the NASA GEOS-5 model, the SMOS surface temperature is obtained from the ECMWF model. An underestimation of the surface temperature leads to an overestimation of the soil emissivity and, as a result, to an underestimation of the SSM. This could explain the dry bias shown by the SMAP and the SMOS products with respect to the most representative in situ stations in REMEDHUS. Our results are in line with previous studies which have also reported this dry bias when comparing the SSM SMOS products (at high and low-resolution) against the in situ data provided by VAS (Spain), SMOSMANIA (France), and OzNet (Australia) networks [13,21,44]. In Figure 2 SMOS soil moisture estimates with a very low value close to zero can be found, mostly during the summer periods. These values have not been filtered in this study, as the product quality flags do not report they are measurement errors. However, we conducted specific tests and confirmed that they do not affect our overall conclusions (not shown).

Some differences between the SMAP and the SMOS products are intrinsic to the instrument they carry; while SMOS uses an interferometric radiometer with 69 receivers distributed on an Y-shaped antenna array and measurements at different incidence angles are obtained in each snapshot, SMAP uses a large rotating antenna and measurements are performed at a constant incidence angle of 40◦. On the other hand, the SSM retrieval algorithms have been tailored to the SMAP and SMOS instrument characteristics, and they involve the use of dedicated techniques to reduce or correct disturbing factors (e.g., surface roughness, soil temperature and vegetation canopy). In a global study conducted by Mariko et al. [45] the SMAP SSM was compared against the one provided by SMOS, Aquarius, Advanced Scatterometer (ASCAT) and Advanced Microwave Scanning Radiometer 2 (AMSR2). Overall, they found that SMAP and SMOS appeared to be the most similar among the five SSM products, in terms of uRMSE and R, excluding forested areas where some discrepancies were found, with SMOS being generally slightly wetter than SMAP. For the particular case of the Iberian Peninsula, which is mostly covered by crops and forested areas, we showed that SMAP is generally wetter than SMOS. Although differences are minor, we showed they are already present at the TB level (see Figure 7), and are also translated to the SMOS and SMAP derived products at enhanced spatial resolutions (see Figure 5). Mariko et al. [45], indicate that a highly potential cause of the mismatch between the SMAP and the SMOS products is the use of different ancillary data in the retrieval algorithm. Although both algorithms are based on the tau-omega model they use different land cover maps to select the albedo, roughness coefficient and the vegetation opacity; SMAP uses the International Geosphere Biosphere Program (IGBP) [46], and SMOS uses the ECOCLIMAP [47]. This could explain the differences observed between the original SMAP and the SMOS products (~40 km), but also between the satellite observations and the in situ measurements along the whole study.

Focusing on the enhanced resolution SSM products, they allow us to develop applications that will otherwise not be possible using exclusively SMAP and SMOS products in their original resolutions (~40 km). However, we showed that even the higher resolutions (3 and 1 km) of SSM maps may not be completely suitable for local or regional applications if the study area is small and the land cover is not representative of the SMAP or SMOS pixel to which it belongs (Figure 2a,b). In [48], Merlin et al. proposed a performance metric for soil moisture downscaling methods and it was applied to the 1 km disaggregation based on physical and theoretical scale change (DISPATCH) data in central Morocco. They showed that disaggregation applied to irrigated areas surrounded by drylands reduced the negative bias in SMOS observations at 1 km with respect to in situ data, but was not yet fully able to solve the sub-pixels variability in soil moisture. The scientific contribution of the downscaled SMAP and SMOS products is undeniable, adding value in a wide range of applications, such as the prevention management of insect pests [6], the prevention of forest fires [7,8] and the early detection of wild fires [9], but further improvements are needed to reduce the uncertainties when merging information from different sensors.

#### **6. Summary and Conclusions**

In this study, several space-borne SSM products (SMAP and SMOS), and their derived products at enhanced spatial resolution (SMAP/Sentinel-1 and SMOS/ERA5/MODIS) have been compared in space, and time.

For the temporal comparison, the in situ information of the REMEDHUS network has been used as a benchmark. In order to study the behavior of the remotely sensed data in different scenarios, we selected a variety of in situ SSM stations located in areas with different land uses (fallow, forest-pasture, vineyard, irrigated, and rainfed). We showed that, independently of the spatial resolution, all the SSM products were able to capture significant rainfall events (e.g., the rainfall event occurring in January 2016, see Figure 2), and seasonality pattern (summers with low and winters with high soil moisture values, see Figures 2 and 3). However, even the highest-resolution product used in this study (1 km) is not fine enough to capture local differences which are not dominant at the pixel scale, like the small irrigated areas where station K13 is located (see Tables 3 and 5). Consequently, when comparing the remotely sensed data with REMEDHUS measurements it is crucial to understand the representativeness of one in situ station within the satellite footprint. In situ SSM measurements are representative at the point scale and are highly sensitive to both the soil characteristics and the effects of precipitation. There are multiple strategies to upscale in situ soil moisture measurements for comparison with satellite-based estimates [49,50]. In this study, we have decided to average the SSM values of the most representative (in terms of prevailing land cover) in situ stations within the satellite footprint. One of the best results when comparing low-resolution as well as high-resolution SMAP/SMOS-based estimates against the in situ measurements, are obtained for the stations H13 and O07, which are located in regions with rainfed or fallow land use (the most common land uses in REMEDHUS). On the contrary, the worst results were obtained over stations J3 (vineyard) and K13 (irrigated), which represent only a minor land cover fraction within the footprints (see Table 4).

Statistically, the differences between the SMAP and SMOS products are considerably low, e.g., at low-resolution, for the station H13 the correlation (and the unbiased error) are 0.83 (0.044 m3/m3) for SMAP (SMAPL2\_E), 0.8 (0.052 m3/m3) for SMOS (SMOSL3); at high-resolution these statistics are 0.81 (0.04 m3/m3) for SMAP, and 0.8 (0.045 m3/m3) for SMOS. From Figure 3 and Table 6 it can be seen that both SMOS and SMAP have a slightly worst performance in terms of correlation (~0.6) during the summer season. In addition, SMOS shows an important bias (−0.067 m3/m3) in this period.

Concerning the spatial analysis, the high-resolution (downscaled) SMAP (passive/active) and SMOS (passive/optical) products have been compared across the Iberian Peninsula. Overall, SMAP is slightly wetter than SMOS, especially in the north, northwest, and west of the Iberian Peninsula. These differences are more pronounced over forested areas, which may be due to the fact that the microwave (radar) signal at C-band used in the SMAP product is not able to penetrate through dense (forested) vegetation [12,51]. Moreover, the differences between the two products can also be seen at the brightness temperature level and therefore are not introduced by the downscaling methodology.

This satellite inter-comparison study has provided and confirmed insights into the SMOS and SMAP multi-scale SSM products that are currently operational. These products are required in a wide spectrum of application and research studies, generally at the best radiometric accuracy and spatial resolution possible. Over the Iberian Peninsula, we showed that all products generally agree in their temporal dynamics, with lowest performances in summer, and SMAP-derived products being wetter than SMOS ones. Yet some differences in spatial patterns are observed in the high-resolution products, linked to the fine-scale information they use and the multi-sensor synergies employed, especially in forested areas. In future studies, the presented analysis can be extended to other regions of the world that have a sufficiently dense soil moisture network to establish reliable estimates at multi-scale resolutions. Also, the proposed spatio-temporal analyses can be widened to global scales with the use of sparse in situ networks.

*Remote Sens.* **2020**, *12*, 570

**Author Contributions:** Conceptualization, G.P., T.J., M.P. (Maria Piles), M.V., A.C., and D.E.; Methodology, G.P., T.J., M.P. (Maria Piles), and M.V.; Software, G.P. and M.P. (Maria Piles); Validation, G.P., T.J., and M.P.; Formal analysis, G.P., T.J., M.P., and M.V.; Investigation, G.P., T.J., M.P. (Maria Piles), and M.V.; Resources, T.J. and M.V.; Data curation, G.P.; Writing—original draft preparation, G.P., T.J., M.P. (Maria Piles), and M.V.; Writing—review and editing, G.P., T.J., M.P. (Maria Piles), M.V., A.C., M.P. (Miriam Pablos), and D.E.; Supervision, T.J., M.P. (Maria Piles), M.V., and A.C.; Funding acquisition, M.P. (Maria Piles) and M.V. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research was funded by the Spanish Ministry of Science, Innovation and Universities, through the coordinated project L-Band (MCIU/AEI/FEDER, UE): Sobre la continuidad de las misiones satelitales de banda L. Nuevos paradigmas en productos y aplicaciones, grant numbers ESP2017-89463-C3-2-R (UPC part) and ESP2017-89463-C3-1-R (ICM part), and the Unidad de Excelencia María de Maeztu MDM-2016-0600. M. Piles is supported by a Ramón y Cajal contract and the project RTI2018-096765-A-100 (MCIU/AEI/FEDER, UE).

**Acknowledgments:** The authors would like to thank the Water Resources Research group of the University of Salamanca for their support and helpful comments. Thomas J. and Dara E. also want to acknowledge MIT for supporting this research with the MIT-Germany Seed Fund "Global Water Cycle and Environmental Monitoring using Active and Passive Satellite-based Microwave Instruments".

**Conflicts of Interest:** The authors declare no conflict of interest.
