Spatial Downscaling of Satellite-Based Soil Moisture Products Using Machine Learning Techniques: A Review

Abstract

Soil moisture (SM) is a key variable driving hydrologic, climatic, and ecological processes. Although it is highly variable, both spatially and temporally, there is limited data availability to inform about SM conditions at adequate spatial and temporal scales over large regions. Satellite SM retrievals, especially L-band microwave remote sensing, has emerged as a feasible solution to offer spatially continuous global-scale SM information. However, the coarse spatial resolution of these L-band microwave SM retrievals poses uncertainties in many regional- and local-scale SM applications which require a high amount of spatial details. Numerous studies have been conducted to develop downscaling algorithms to enhance the spatial resolution of coarse-resolution satellite-derived SM datasets. Machine Learning (ML)-based downscaling models have gained prominence recently due to their ability to capture non-linear, complex relationships between SM and its driving factors, such as vegetation, surface temperature, topography, and climatic conditions. This review paper presents a comprehensive review of the ML-based approaches used in SM downscaling. The usage of classical, ensemble, neural nets, and deep learning methods to downscale SM products and the comparison of multiple algorithms are detailed in this paper. Insights into the significance of surface ancillary variables for model accuracy and the improvements made to ML-based SM downscaling approaches are also discussed. Overall, this paper provides useful insights for future studies on developing reliable, high-spatial-resolution SM datasets using ML-based algorithms.

1. Introduction

Soil moisture (SM) is a key variable in agricultural [1,2,3], hydrological [4,5,6], land–atmosphere energy interaction [7,8,9], and climatic [10,11,12] processes at global and regional scales. Therefore, SM information is required for a broad range of hydrologic, climatic, ecological, and agricultural applications for numerical modelling and decision making [4,13,14,15,16,17,18].

SM information is usually obtained through ground-based in situ measurements, land surface model (LSM)-based estimates, and satellite or airborne remote sensing observations [19]. Gravimetric methods (i.e., field sampling followed by weighing and drying in the laboratory), time domain reflectometers (TDRs), frequency domain reflectometers (FDRs), neutron probes, and cosmic-ray neutron probes can be named as widely used methods for in situ SM measurements [20]. In situ measurements can provide SM only at a point scale, but cannot inform SM changes over a large spatial domain. As SM is highly variable from one point to another, carefully arranged in situ SM networks over large spatial extents are required to provide areal estimates of SM variability, e.g., [21,22,23]. In general, in situ measurements are cost- and labour-intensive.

LSM-based data modelling and assimilation methods, on the other hand, can provide gridded SM estimates at multiple depth profiles and spatiotemporal domains, e.g., Global Land Data Assimilation System (GLDAS)-2.1 [24] and European Robotic Arm 5 (ERA5)-Land datasets [25]. But LSM-based SM estimates are often associated with uncertainties caused by input data quality, model algorithms, and parameters representing physical processes [26,27].

In the last two decades, satellite remote sensing has become an emerging technology which provides spatially continuous SM information. Optical/thermal sensors and X- (2.5 cm to 3.75 cm) and C-band (3.75 cm to 7.5 cm) sensors, such as Advanced Microwave Scanning Radiometer Earth Observing System (AMSR-E), Advanced Microwave Scanning Radiometer 2 (AMSR2), Advanced Scatterometer (ASCAT), and WindSat, have shown their capability to measure surface (skin) SM content on a global scale [28]. The remote sensing of SM gained increased significance after the launch of the European Space Agency’s (ESA) SM and Ocean Salinity (SMOS) mission in 2009, and the National Aeronautics and Space Administration’s (NASA) SM Active Passive (SMAP) mission in 2015. Compared to optical, thermal, X-, and C-band missions, these L-band (15 cm to 30 cm) microwave missions demonstrate the ability to penetrate sparse vegetation and clouds; hence, they are capable of providing near-surface (~top 5 cm) SM data in all weather conditions [28]. Both radiometric and radar observations can be used to retrieve SM with L-band sensors. However, radiometric data often provide a better accuracy compared to radar data [29,30]. When it comes to the target accuracy requirements, satellite SM retrievals are often validated against an accuracy threshold of 0.04 m³/m³ across the globe, excluding regions consisting of snow, ice, frozen ground, complex topography, open water, urban areas, and vegetation, where uncertainties in SM measurements are high. In achieving this accuracy threshold, researchers have tested general calibration functions, and the use of dedicated field calibration has been suggested to improve this accuracy. The Global Climate Observing System and individual products/missions like SMAP and SMOS follow this accuracy standard [31]. Many SM studies achieved this target accuracy over different regions when the derived downscaled SM was compared against in situ SM networks [32,33,34].

Although providing higher-accuracy SM retrievals, generally, L-band radiometric satellite SM products are available only at a coarse spatial resolution, i.e., ~40 km [35]. Therefore, a number of models have been developed and tested to enhance the spatial resolution of L-band SM products through downscaling approaches [19,36]. Figure 1 illustrates the basic concept of downscaling coarse-spatial-resolution satellite-derived SM products. Here, the coarse-spatial-resolution (x m in Figure 1) SM product does not capture the SM variability within that pixel, i.e., the pixel has only one SM value, represented by blue. After applying the downscaling algorithm with high-spatial-resolution ancillary data inputs, the spatial detail of SM is enhanced, revealing sub-pixel variations in SM within the original coarse-resolution pixel.

SM downscaling models can be categorised according to the approach they use. For example, [19,36] classified SM downscaling models as satellite-based, geo-information-based, and model-based approaches. Satellite-based methods include active–passive microwave data fusion methods, e.g., [29,37,38,39,40], and optical/thermal and microwave fusion methods [41,42,43]. The SMAP mission was supposed to deliver an active–passive fusion product with a 9 km spatial resolution by using high-accuracy, low-spatial-resolution (~40 km) passive L-band observations, and low-accuracy, high-spatial-resolution (1–3 km) active L-band observations [37,44,45], prior to its radar failure after 3 months of operation [46]. The triangular (or trapezoidal) feature space between the land surface temperature (LST) and vegetation index (VI) [47,48] has become the base for a number of downscaling models developed by using satellite-based optical/thermal datasets. Geo-information-based downscaling approaches use the relationship between SM, topography, and vegetation characteristics to estimate the SM at a high spatial resolution, e.g., [49,50,51]. Model-based methods involve approaches using statistical models, e.g., [52,53,54], LSMs, e.g., [55,56], and data assimilation techniques, e.g., [57,58].

Most downscaling techniques build a relationship between the high-resolution SM and the coarse-resolution SM using ancillary variables such as vegetation, land surface variables, soil characteristics, and climatic conditions [59,60,61,62,63,64]. This relationship often becomes non-linear and complex, in order to explain the processes underpinning SM variability in terms of key environmental variables. Machine Learning (ML) plays an important role in modelling these type of complex, non-linear relationships due to its ability to process vast amounts of data, identify dominant patterns, and make predictions or decisions based on these patterns [65]. One of the challenges in utilising optical/thermal-image-based ancillary variables is the data gaps due to the cloud contamination. Some studies have introduced ancillary variables derived using synthetic-aperture radar (SAR) as a solution to overcome this issue [66,67]. Even though ML techniques have become a popular approach for downscaling SM from satellite observations (mainly C/X/L-band brightness temperatures for passive sensors and C-band backscattering coefficients for active sensors), in recent years, e.g., [68,69], comprehensive reviews on ML-based SM downscaling methods are limited in the existing literature. In view of these considerations, this paper aims to provide guidance for future research employing ML techniques to downscale satellite-derived SM products. It reviews the different ML techniques used to downscale coarse-spatial-resolution SM products, the commonly used ancillary variable inputs, comparison studies, the accuracies achieved, and the improvements introduced over time.

The flow of the paper is as follows. First, an introduction to the ML methods used for downscaling SM is given. Then, the methodological framework of SM downscaling is discussed, outlining and summarising the outcomes of recent studies which have used ML techniques to downscale SM products. Studies using more than one ML technique are comparatively discussed for their performance in downscaling. The paper further explores important insights gained from each study regarding surface ancillary variables and the adaptations made to ML/deep learning (DL) methods to enhance the accuracy of the downscaled SM products. It also discusses the widely used validation methods for the model predictions and potential future research. The insights provided in this paper will be beneficial for the future studies aiming to develop high-spatial-resolution SM datasets at regional and global scales.

2. Review of ML Algorithms

ML is a subcategory of artificial intelligence used for developing algorithms or models which are capable of data-driven decision making. In traditional programming, the input and algorithms are provided to the computer to derive the output. On the contrary, ML produces a trained algorithm or model using the input data and corresponding response variables. This computer-based training is a learning procedure which can be categorised into unsupervised, semi-supervised, supervised, and reinforcement learning. Supervised learning requires the availability of labelled data [70]. ML methods could also be categorised according to the techniques that they utilise. For example, Couckuyt et al. [70] classified ML techniques into three classes, i.e., (i) classical methods, (ii) ensemble methods and, and (iii) neural nets and DL methods. Linear models, logistic regression, support vector machine (SVM), support vector regression (SVR), decision tree, principal component analysis (PCA), classification and regression tree (CART), and cubist are categorised as classical ML methods, whereas random forest (RF), gradient boosting decision tree (GBDT), extreme gradient boost (XGB), majority vote, bagging, and boosting are listed as ensemble methods. Neural nets and DL methods include artificial neural network (ANN), convolutional neural network (CNN), feedforward neural network (FNN), K-nearest neighbours (KNN), recurrent neural network (RNN), long short-term memory (LSTM), back propagation neural network (BPNN), residual network (RseNet), and deep belief network (DBN). An individual method or multiple methods have been jointly used to downscale coarse-spatial-resolution SM products in various studies. Each method poses its own advantages and disadvantages.

Classical ML methods such as regression trees are capable of addressing multiple regression problems. The terminal nodes of the trees are known as leaves containing the target variable [71]. CART is a technique based on decision trees. The tree-building process begins at the root node, where it is divided into child nodes and further subdivided into grandchild nodes (Figure 2). Each internal node signifies a feature or attribute, each branch signifies a decision rule, and each leaf node symbolises the final choice or ultimate decision [72].

The SVR focuses on fitting the maximum amount of instances for a particular class or margin, while controlling the margin violations between different classes. The training process of SVR minimises the number of points which violate these margins [73,74]. SVM is used to analyse discrete class labels, while SVR is used to analyse continuous class labels. SVM is mainly used for classification tasks using s-dimensional energy vectors as inputs. Further, SVM proficiently isolates distinct classes and furnishes accurate predictions for new classes. This technique encompasses different functions such as linear, polynomial, sigmoid, and Gaussian radial functions [75,76]. Cubist is an innovative rule-based regression method. The main algorithm comprises two main steps of creating criteria for segmenting the data into smaller subsets and employing regression models to forecast outcomes for these distinct subsets [77].

Ensemble methods, such as RF, employ multiple decision trees and are widely used in classification and regression processes. Decision trees are built using random subsets of the data and the predictions from each tree are combined to make the final prediction. The strength of the individual trees and the correlation between them are influential on the generalisation error of the model [78]. GBDT is a widely used ML algorithm due to its efficiency, accuracy, and interpretability. It combines gradient boosting with decision trees. In GBDT, decision trees are weak learners. The iterative addition of weak learners updates the model and gradually reduces the residuals to generate accurate predictions. The final prediction is achieved by summing up all the predictions [79,80]. XGB is a scalable ensemble technique and an enhanced version of gradient boost. This technique is used to derive state-of-the-art results by many scientists. It is popular for speed, scalability, and a high performance in regression and classification [81].

In general, a neural network is a collection of neurons arranged in layers to build a directed network. Each connection/vertex between two neurons is individually weighted and the minimum requirement of a neural network is having an input layer and an output layer. The layers between the input and output layer are called hidden layers. Neural networks can be categorised as simple or deep learning according to their number of hidden layers and how they are connected to the input and output layers, as can be seen in Figure 3. When the number of layers is more than three, it is called a deep learning neural network or deep learning model.

Neural networks such as ANN, KNN, CNN, and RNN can be listed as a few widely used neural network methods which are capable of handling more complex relationships between predictors and targets. ANN contains multiple layers of neurons, which are interconnected. The inclusion of hidden layers is pivotal, as they empower the network to learn and extract complex patterns and representations from input data. Neurons within these hidden layers perform computations using their inputs and activation functions, transforming information and conveying it to downstream layers. This iterative process of propagating information through hidden layers enables the network to capture intricate data relationships. As the final step, the generated predictions using these data relationships are directed to the output layers [82,83]. KNN is also an intuitive and powerful supervised learning algorithm, which can be used in downscaling coarse-resolution SM products. KNN makes predictions based on the proximity of data points in the feature space. The predicted value is assigned based on the majority vote of its nearest neighbours. The advantages of KNN are that it has been proven to be resilient against noisy data and reduce the impact of outliers in predictions [84,85,86].

The CNN operates as a feed-forward network, which is capable of the automatic extraction of features using convolution structures. A CNN comprises various layers such as a convolution layer, pooling layer, and fully connected layer. The convolution layer applies the convolution operation to the input data. The reduction in the spatial dimension of feature maps is performed by the pooling layer. The final step of mapping the predictions to the output layer is performed by the fully connected layer [87,88]. The LSTM is a solution created for error back-flow problems, which mostly occur in learning long-term dependencies via gradient estimations [89]. Further, LSTM is an RNN, which includes four main components of a forget gate, input gate, output gate, and cell state [90,91].

Both ML and DL are included under artificial intelligence. One of the main differences between them is the amount of human intervention. DL is trained to accommodate little or no human intervention. DBN and ResNet are two DL neural network models used for SM downscaling. DBN is a class of deep neural network made of multiple layers of variables, which are known as hidden units. There is no connection defined between the units within each layer rather than the connection between different layers. DBN layers are capable of detecting features after training the model using a set of examples without supervision [92]. The residual network or ResNet is a model easier to train than a deep neural network. It learns residual functions with respect to the input. Residual learning contains layers/blocks, which are connected by shortcuts. The output of the previous layer/block is connected to the output of the next layers/blocks. However, when establishing this connection using shortcuts, some layers/blocks are skipped [93].

As explained here, ML accommodates multiple algorithms, which can be used to learn the hidden relationships between different data. Despite its excellence, there are undesirable ML behaviours, such as overfitting and underfitting (discussed in detail in Section 4.1), to be considered carefully in analysis.

3. ML-Based Downscaling Techniques for Microwave-Based SM Products

This section reviews studies in which ML models were tested to downscale satellite-based coarse-spatial-resolution SM products. There are methods for estimating high-spatial-resolution SM using different datasets and models [94,95,96] without downscaling a coarse-spatial-resolution satellite SM product. For example, Yu et al. [97] worked toward downscaling a continental-scale Australian Water Resource Assessment Landscape (AWRA-L) SM product (5 km) into local scale (500 m) using an RF-model-based approach. The RF model was generated using multi-source geo spatial predictors such as a digital elevation model (DEM), soil properties from the Soil and Landscape Grid of Australia, and MODIS data products. The 5 km AWRA-L data were downscaled into monthly 500 m resolution products, while maintaining the dynamics in the continental scale. Such methods, which do not involve downscaling coarse-resolution-satellite SM products, are not the focus of this paper.

3.1. Methodological Framework of ML-Based Downscaling Approach

In ML-based downscaling models, typically, a relationship is established between SM with driving forces such as surface temperature, vegetation indices, surface albedo, soil texture, meteorological factors, and topographic variables [33,66,98,99,100,101,102] in order to capture the soil hydrologic properties and processes. Therefore, data related to these driving forces are often used as ancillary variables in downscaling SM data. Subsequently, the chosen driving forces are input into the model at a high spatial resolution to output high-spatial-resolution SM. The driving forces of SM, such as precipitation, significantly influence both the spatial and temporal variability of SM [61,98,103]. High surface temperatures can lead to increased evaporation rates, leading SM to evaporate from the near-surface layers. Healthy vegetation, on the other hand, can remove SM through transpiration, a process in which plants absorb water from the soil and release it into the atmosphere through their leaves. Vegetation cover can also positively correlate with SM retention due to the shade by reducing evaporation. Surface albedo refers to the amount of solar radiation reflected by the Earth’s surface [104]. Surfaces with a higher albedo reflect more sunlight and heat, potentially reducing surface temperatures and, consequently, slowing evaporation rates. Lower-albedo surfaces absorb more heat, leading to increased evaporation and potentially drying out the soil. Soil texture (e.g., sand, silt, and clay content), influences SM retention [98]. Soils with a high clay content tend to hold more moisture compared to sandy soils, which drain water more rapidly due to their high porosity. Topography, such as slope and aspect, affects water movement and accumulation [105]. Steep slopes can lead to water runoff, reducing SM in areas where water drains quickly. Depressions or valleys may accumulate water, resulting in a higher SM content. Therefore, topographic indices such as topographic wetness index (TWI) and SAGA wetness index (SWI) are closely related to SM retention [106,107].

3.2. ML-Based Downscaling Methods

Table 1. Summary of the documented references in this paper that have used individual ML/DL methods or comparative analyses of multiple methods for downscaling SM products.

Category	ML/DL Methods	References
Classical methods	SVR	[108]
	Self-regularised Regressive models	[62]
	Regression/Regression Tree	[109,110,111,112]
Ensemble methods	RF	[33,63,66,67,97,98,113,114,115,116]
	GBDT	[34]
Neural nets and deep learning methods	ANN	[100,101,102]
	Kernel weighted KNN	[117,118]
	CNN	[119]
	Bayesian deep image prior algorithm combined with a fully convolutional neural network and a forward model	[99]
	LSTM	[120]
	Deep neural network with a generalized linear model (GLM)	[32]
Comparisons of methods	Kernel-weighted KNN, RF	[121]
	RF, ANN	[59]
	CART, KNN, Bayesian (BAYE), RF	[122]
	ANN, BAYE, CART, KNN, RF, SVM	[123]
	Multivariate linear regression, SVR, RF	[124]
	Multiple linear regression, RF, SVR	[60]
	GBDT, RF, ANN, RseNet, LSTM, CNN	[103]
	RF, CART, GBDT, and XGB	[125]
	regression tree, ANN, and Gaussian process regression (GPR) models	[126]
	ANN, SVM, Relevance Vector Machine, and linear regression	[127]
	DBN, neighbourhood constraint-based improved DBN, ResNet	[128]
	DBN, RF	[129]
	Regression, ANN	[130]
	CNN (SM-residual Dense Net), RF	[131]
	RF, boosted regression trees, Cubist	[64]
	RF, LSTM	[61]
	SVR, FNN	[65]
	CNN, ANN, XGB, LSTM and ResNet	[132]
	SVR, KNN	[133]
	ANN kriging	[134]

summarises the categories of ML-based methods used by researchers to downscale SM data into a fine spatial resolution. Some researchers have employed individual ML methods to downscale coarse-spatial-resolution SM products, while others have performed comparative analyses between multiple ML methods. This section reviews the downscaling models developed and tested by various researchers by categorising them into three main ML/DL categories, as mentioned in Section 2.

3.2.1. Classical-ML-Model-Based Downscaling Approaches

Classical ML offers accurate and computationally cheap techniques for SM downscaling. Hernández-Sánchez et al. [62] used self-regularised regressive models in downscaling an SMAP 36 km SM product into 1 km by using high-spatial-resolution auxiliary information. Regularisation in ML is normally used to reduce the overfitting of the model. The study area was an agricultural field located in Texas, and this analysis was performed during a growing season of corn. It is challenging to monitor the SM in a growing season, as it contains considerable temporal fluctuations. The downscaled SM indicated that the adopted methodology was capable enough to capture the SM dynamics during the growing season.

The LST is a commonly used ancillary parameter in downscaling SM. LST differences can be a result of SM changes on land. Soil with a high moisture content will indicate smaller diurnal temperature differences compared to dry soils, due to the increased volumetric heat capacity. This resistance to changes in temperature is known as thermal inertia. Wet soils show a high thermal inertia compared to dry soils due to the presence of moisture. Accordingly, Fang et al. [109] and Fang et al. [110] used the thermal inertia theory to downscale AMSR-E and SMAP SM data in Little Washita water shed, Oklahoma, and Walnut Gulch experimental watershed, respectively. The relationship between the SM and the diurnal temperature differences was developed by using a regression model modulated by the normalised difference vegetation index (NDVI). Senanayake et al. [111] also used a similar approach to downscale SMAP–Enhanced 9 km data (SMAP-E) and SMOS 25 km gridded data. The regression tree was classified using season, vegetation density, and soil texture. The MODIS LST was used as an ancillary dataset in the downscaling process. The comparison of downscaled SMAP-E data and SMOS data with in situ data indicated unbiased root-mean-square error (ubRMSE) values of 0.07 and 0.05 cm³/cm³, respectively. Figure 4a–c shows the spatial details of the SM, as captured by the SMAP-E product, SMOS passive 25 km product, and downscaled 1 km SM product obtained by using the regression tree algorithm over two southeast Australian catchments, Krui and Merriwa River, by Senanayake et al. [111]. It can be clearly seen how the sub-pixel SM variability of the satellite-derived SM products was captured by the 1 km downscaled product. Further, Figure 4d shows a comparison between the time series of downscaled SM at a 1 km pixel and observations from an in situ SM station within that pixel, obtained from the Scaling and Assimilation of SM and Streamflow (SASMAS) network in 2015 [135]. The figure illustrates that the downscaled products captured the temporal fluctuations quite well over that pixel.

Senanayake et al. [112] compared soil-thermal-inertia-based downscaling models built at two scales, using point-scale in situ datasets and GLDAS estimates on a 25 km grid. Even though the datasets used to train the models were at two different scales, both models showed a similar accuracy of ubRMSE = 0.07–0.1 cm³/cm³ [115].

Methods like simple linear regression and least squares regression do not apply weights based on different locations. Locational weighting is advantageous in situations where the quality of the data points or the influence that they can make are varied. SVR is one of the classical methods that use locational weighting. Kim et al. [108] used SVR for SM downscaling and achieved an accuracy of correlation coefficient (r) = 0.68 and an average RMSE of 0.08 m³/m⁻³.

3.2.2. Ensemble-Method-Based Downscaling Approaches

RF can be credited as one of the most utilised ensemble ML techniques for downscaling SM in the literature [136]. Chen et al. [33] used the RF algorithm for downscaling SM using SMAP data, MODIS products (NDVI, enhanced vegetation index (EVI), LST, and land cover type), DEM, and longitude and latitude data over the Tibetan Plateau. The downscaled SM was compared against in situ SM observations, the China meteorological administration land data assimilation system (CLDAS), and GLDAS SM data. The comparison of the downscaled SM data with in situ data indicated that the differences between the mean values were very small. The downscaling method worked more efficiently for grasslands in the context of accuracy. However, the results indicated an overestimation over SM-rich regions relative to CLDAS. Abbaszadeh et al. [98] also used the RF algorithm to estimate SM at a high spatial resolution by creating an ensemble learning approach using several RF models to downscale SMAP L3_SM_P SM from 36 km to 1 km. Here, different RF models were created considering the categorised soil texture groups. The model was calibrated by using five inputs, i.e., NDVI, LST, precipitation, elevation, and SMAP SM data, and trained to target in situ SM data. The significant advantage of using multiple RF models is the enhanced model performance through minimised bias. For example, the overestimation of SM over arid regions with bare soils and underestimation over cold vegetation areas were minimised in this study. Most importantly, the downscaled SM was consistent with the in situ observation networks. NDVI, LST, and precipitation showed a similar influence on the downscaling accuracy of the model. The downscaled SM showed inconsistencies with the original SMAP SM in densely vegetated areas due to signal attenuation. Ghafari et al. [116] developed an RF model to downscale an SMAP 36 km SM product to 1 km, using airborne radar backscatter and radiometric SM retrievals, NDVI data, soil properties, topographic data, and ground-based SM observations. The downscaled data showed r = 0.97 and RMSE = 0.048 m³/m⁻³ when compared against airborne SM retrievals. In addition, Bai et al. [67], Wakigari and Leconte [114], and Sishah et al. [66] used RF for downscaling SMAP data. Li et al. [115] downscaled a 25 km resolution microwave-based surface SM dataset developed by fusing SMAP SM data with European Space Agency Climate Change Initiative (ESA-CCI) data using an RF algorithm.

GBDT is also an ensemble method used in SM downscaling. This technique is robust to outliers and unbalanced data [103]. Wei et al. [34] used GBDT to downscale SMAP 36 km data into 1 km. The validation of the derived SM was performed by using point-, daily-, and network-scaled ground measurements. The best accuracy obtained in this study showed a coefficient of correlation, r = 0.904 and ubRMSE of 0.044 m³/m³.

Data gaps due to cloud cover are one of the limitations of using visible- and infrared-image-based ancillary variables. Karami et al. [137] overcame this issue by combining Sentinel-1 radar backscatter with geophysical auxiliary variables in an RF model. Even though there was still a gap of 6 to 12 days to address due to the temporal resolution of the Sentinel-1 data, spatially, the results showed a satisfactory performance.

3.2.3. Neural Nets and DL-Method-Based Downscaling Approaches

ANN is one of the most efficient methods for building the non-linear relationships between various ancillary datasets and SM. Alemohammad et al. [102] used ANN to downscale SMAP SM observations into 2.25 km. First, SMAP 36 km data, SMAP vertical polarization brightness temperature at 36 km, MODIS monthly NDVI 9 km data, and MODIS-based LST data were used to produce 9 km SM. Then, the derived 9 km SM estimates combined with enhanced vertical polarization brightness temperature data were downscaled into 2.25 km by assuming similar relationships between the scales of 36 km to 9 km and 9 km to 2.25 km. Alemohammad et al. [101] also adopted the same method by using ANN for downscaling SM data. Additionally, they used TWI as ancillary data in the first step of downscaling and validated the results using the observations from the international SM network (ISMN) [23,138]. Here, a MODIS monthly mean NDVI product was used as an ancillary dataset by assuming moderate vegetation cover in the study area. Therefore, the downscaling algorithm indicated a limited performance on bare soil and sparsely and densely vegetated areas. Lv et al. [100] used ANN to build a relationship between passive microwave (SMAP and AMSR-2), active microwave (ASCAT), MODIS (LST, VI, EVI, and NDVI), and topographic data (DEM, slope) as inputs and CLDAS SM estimates as targets. They also applied a triple collocation (TC) to ANN, Feng-Yun 3C, and Goddard Earth Observing system-5 SM data to obtain the error variance and r-value between each product and the actual SM data. The ANN SM product derived in this study earned the lowest error variance and the highest r-value compared to the Feng-Yun 3C and Goddard Earth Observing system-5.

Even though ANN is effective in modelling non-linear relationships where data are limited, CNN is more powerful in the context of automating the detection of features without human supervision [139]. Xu et al. [119] used CNN to address a prevailing problem about the loss of SM information among pixels in downscaling. They created a residual SM layer as an output to improve the accuracy. The SMAP 36 and 9 km SM products were downscaled into 3 km and 1 km resolutions in this study. The downscaled 3 km and 1 km SM products showed r-values of 67.28% and 4.97% and ubRMSEs of 65.9% and 5.18%, respectively.

Spatial weighting can be applied based on locational significance. Other than the locational weighting and weighted layers, kernel-weighted KNN assigns less weight to neighbours distant from the query point. Guevara and Vargas [117] and Warner et al. [118] downscaled ESA-CCI data into 1 km and 100 m using kernel-weighted KNN. Guevara and Vargas [117] obtained an accuracy of r = 0.46 and bias = 0.057 m³/m³. Warner et al. [118] considered seasonality (spring, summer, fall, winter, and annual) in downscaling. They emphasized the importance of calibrating raw ESA-CCI and downscaled SM using ground SM observations/field-observed values to accurately observe seasonal patterns. The downscaled SM had a low error of 27%.

In the BAYE context, SM is a maximum a posteriori problem, where the unknown quantity equals the mode of the posterior distribution [99]. Fang et al. [99] used a Bayesian deep image prior algorithm combined with a fully CNN and a forward model for downscaling an SMAP-enhanced 9 km SM product. The fully CNN was an inverse model, while the forward model was responsible for down sampling. The output of the fully CNN was 1 km SM and the output of the forward model that was validated with actual SMAP-enhanced data was 9 km SM. The MODIS-derived NDVI and LST products were used as the ancillary data to train the model. The downscaled data provided more spatial details and were very close to in situ measurements, with a higher r-value of 0.88.

Wide and deep learning consolidate a wide linear model with a deep neural network. Xu et al. [32] analysed a wide and DL method which combined a deep neural network with a linear regression model to downscale an SMAP L3_SM_P SM 36 km product to 1 km. Horizontally and vertically polarised brightness temperature, surface reflectance, LST, climate, land cover, and topographic and soil attributes were used as the ancillary data in this work. The verification of the results performed by using the precipitation and in situ SM data indicated a good temporal consistency and an accuracy with an average r-value of 0.715 and an average ubRMSE of 0.041 m³/m^3.

3.2.4. SM Downscaling Studies Which Used Comparative Analysis between Different ML Techniques

With the evolution of both ML and the remote sensing of SM, researchers have focused on learning the best-performing model for downscaling SM under different conditions and environments. Comparative analyses of various ML models in downscaling SM revealed important insights about the behaviour of different ML/DL approaches in downscaling SM. Further, such studies provided insights on the efficient use of ancillary information in training these ML/DL algorithms.

Some previous studies found the superiority of tree-based algorithms such as RF over other ML/DL methods. Nadeem et al. [59] used RF and ANN algorithms to downscale SMAP SM products by using ancillary data acquired from both MODIS Terra and Aqua satellites. The land surface variables from these two satellites were used for these ML algorithms to downscale SMAP SM data into a 1 km resolution. In validating the downscaled results with in situ observations at different scales, RF which was trained with Terra satellite data generated the best-fitting regression model with an ubRMSE of 0.034 m³/m³ and r-value of 0.54. Grassland, farmland, and woodland land cover types were investigated in this study to learn the vegetation impact on downscaling SM. RF models which were trained with ancillary data from the MODIS Aqua and Terra satellites indicated a high correlation with the in situ SM in all types of vegetation cover compared to ANN models. Compared to ANN, the capability of the RF to model the complex and non-linear relationships was suggested as one of the reasons for this outperformance. Among the ancillary data used in this study for the RF model trained with MODIS Terra products, LST was found to be the most important variable, followed by NDVI, normalised shortwave infrared difference bare SM index, and wetness indices.

Both RF and GBDT are ensemble learning methods, where they combine individual models of a high bias or variance to produce better models. The only difference is the way that these individual models are combined inside RF and GBDT. Data with less or no noise are important in deriving an accurate output using GBDT. Liu et al. [122] employed four ML algorithms, i.e., CART, KNN, BAYE, and RF, to downscale an ESA-CCI SM product from 25 km to a 1 km spatial resolution. LST, NDVI, surface reflections, and DEM were used as ancillary data in their study. The results demonstrated that the RF-downscaled SM estimates had a better agreement with both the ESA-CCI SM and in situ SM measurements. Similarly, Liu et al. [123] developed high-resolution SM maps at a 1 km resolution based on ANN, BAYE, CART, KNN, RF, and SVM using the Essential Climate Variable SM dataset by the ESA. The MODIS-based NDVI, LST, surface albedo, day/night LST, daily LST fluctuation, DEM, and longitude and latitude were used in the construction of the regression model. The RF outperformed the other models considered in this study. The BAYE and KNN also demonstrated good results. Here, DEM, daytime LST, and NDVI were found to be the major contributors to the SM regression models. Mohite et al. [124] downscaled SMAP SM data using MODIS NDVI, normalized difference water index, albedo, LST, and Shuttle Radar Topography Mission (SRTM) elevation data. Regression-based ML models, such as multi variate linear regression, SVR, and RF, were used in this study to test two SM downscaling scenarios, i.e., monthly based and seasonally based. The results revealed the capacity of both monthly and seasonal data to retrieve downscaled SM, and the RF results predominantly outperformed the other methods. Zhang et al. [60] used RF to downscale AMSR-E data over North China. The model was trained by using five ancillary variables of LST, NDVI, albedo, precipitation, and DEM. The efficiency of the RF-based model was confirmed when the results of RF, multiple linear regression, and SVR were compared against each other.

Shangguan et al. [103] tested six ML algorithms, namely GBDT, RF, ANN, RseNET, LSTM, and CNN, to downscale ESA-CCI SM data over the Qinghai–Tibet plateau. The GBDT showed the best performance, followed by RF, in comparing the r-value and ubRMSE. The performance also depended on the spatial location. For instance, GBDT indicated the lowest uncertainty in the Qinghai–Tibet plateau. The RF and ANN exhibited the lowest uncertainty in eastern and northwestern regions of the Qinghai–Tibet plateau. The areas with a high SM were mainly located in the east and southeast, while the dry regions were in the northwest. The downscaled result was further improved by a hybrid downscaling method that integrated several approaches based on Bayesian three-cornered hat merging (MATCH) [93]. MATCH was used to integrate various ML/DL algorithms. The results of the MATCH SM were robust with mean r and ubRMSE values of 0.55 and 0.047 m³/m³ when validated against the in situ measurements. The best r and ubRMSE values obtained by different individual ML methods were 0.5 and 0.052 m³/m³. The sensitivity of the downscaled SM to the temporal variations was slightly lowered due to the weighted merging process in the MATCH and the non-consideration of precipitation. All the ML methods performed well in barren and shrubland areas, while GBDT and RF had the lowest uncertainty for all the other land cover types, such as forest, grassland, and savannahs. The spatial distribution of the downscaled SM using the MATCH method was deteriorated by certain degree of overestimation over dense vegetation. This effect was observed in the original SMAP SM data in previous research [98].

Liu et al. [125] tested regression-tree-based ML algorithms (RF, CART, GBDT, and XGB) to downscale SMAP SM in southwestern France. The vegetation density and variation in topography were influential on the accuracy of the downscaling. The best results were obtained by GBDT was over grassland (r-value = 0.77 and ubRMSE = 0.04 m³/m³). This accuracy was higher than the original coarse-resolution SMAP SM data over grassland and shrubland data. RF can also be used in combination with conventional SM detection algorithms such as disaggregation based on physical and theoretical change (DisPATCH) [43]. DisPATCH is an evaporation-based semi-physical model and it is inapplicable when there is a weak link between surface SM and evapotranspiration (ET). Li et al. [115] downscaled a fused product of SMAP and ESA-CCI surface SM using DisPATCH and used RF to estimate the gaps caused by DisPATCH when the relationship between surface SM and ET was not strong. This combined approach could produce highly accurate downscaled SM values considering seasonality.

Senanayake et al. [126] tested CART, ANN, and a GPR model to downscale SMAP 36 km SM products over a southeastern Australian catchment. The downscaling algorithms were primarily based on the soil thermal inertia relationship between diurnal LST differences and daily mean SM contents. Clay content and NDVI data were also employed in developing these models. The downscaled SM from the regression tree, ANN, and GPR models showed RMSEs of 0.03, 0.09, and 0.07 cm³/cm³, respectively, when compared against high-spatial-resolution airborne SM retrievals. Figure 5 shows a comparison of the downscaled results obtained from CART, ANN, and GPR with the 1 km airborne SM product derived from the Soil Moisture Active Passive Experiments-5 (SMAPEx-5) observations over the Yanco study area of the Murrumbidgee River catchment, Australia, on 13 September 2015. The figure shows that the spatial patterns of SMAPEx-5 SM and downscaled SM from CART had a better resemblance. All three models were able to capture the diagonal dry patch in the northwest to southeast direction and in the northeast corner of the area.

Srivastava et al. [127] employed ML algorithms, including ANN, SVM, Relevance Vector Machine, and linear regression, to find an efficient technique for downscaling the SMOS SM product. The validation of the downscaled SM showed that all techniques can effectively be used to downscale SMOS SM retrievals. However, ANN outperformed the other methods tested here, despite its long computational time. This study particularly calculated two algorithms for the growing and non-growing season using ANN and found that the compound use of these two algorithms is an efficient way of downscaling SMOS SM data. Linear regression was observed as an alternative to complex ML techniques such as ANN due to its easy implementation.

Some researchers compared the performances of different DL methods in SM downscaling. For example, Zhao et al. [128] tested SMAP 36 km SM downscaling using three DL methods, i.e., DBN, neighbourhood constraint-based improved DBN, and a ResNet model consisting of several residual dense blocks. This study was conducted in the Tibetan Plateau and the number of reference sites to compare the downscaled SM were limited. Therefore, to make a comprehensive assessment of the results derived from these three methods, this study adopted the RF and BPNN methods. The comparison between these methods was conducted by using the three-cornered hat method [140]. This approach was used by He et al. [93] as well. The three-cornered hat method can be used to compare different datasets and generate the relative uncertainty among them. For example, when there is a lack of reference data for validation, SM products generated by different approaches can be used to generate relative uncertainty. The three DL models provided enhanced spatial patterns and details of SM compared to other ML/DL methods such as BPNN and RF. In addition, the ResNet model can significantly improve the spatial texture details and capture the temporal changes in SM. Cai et al. [129] also used DBN to downscale SMAP L4 and AMSR SM products. The results were compared against the results of RF-based downscaled SM over the same area. The DBN outperformed RF with a high r-value and lower RMSE.

Climatic conditions and land use/cover types naturally exert considerable influences on the SM estimation. Imanpour et al. [130] evaluated two downscaling methods of regression and ANN over various land cover types and climate conditions. The best downscaling accuracy belonged to the areas consisting of bare soil and flat regions, which emphasised the results of Abbaszadeh et al. [98]. The highest spatial variations in the downscaled SM were recorded in mountainous areas. The performance of both methods was higher under more homogeneous climatic conditions. In particular, the ANN was trained successfully under a low degree of environmental variabilities and the regression was not limited by the conditions such as an insufficiency of ground measurements in this research. Mining landscapes are also complex environments where some researchers have analysed SM variability. Sang et al. [131] investigated a CNN SM residual dense net and RF in downscaling SMAP/Sentinel-1 Level 2 radiometer/radar SM data into a 10 m spatial resolution. The SM residual dense net performed well in completeness and accuracy compared to RF.

4. Significance of Ancillary Variables, Modifications Made to ML Techniques in Downscaling SM Products, and Validation Methods Used for Downscaled SM

4.1. ML-Based Insights: Key Geophysical and Remote-Sensing-Based Land Surface Variables in SM Downscaling

Meteorological forcing, vegetation cover, topography, ambient temperature, soil temperature, and soil depth are widely used ancillary variables in downscaling SM products based on their influence on SM variability. Meteorological forcing affects SM both directly and indirectly. Its direct effects include precipitation, which directly adds moisture to the soil [141,142,143], and temperature, which influences ET rates [144,145]. Indirectly, meteorological factors like wind speed and humidity can affect ET rates, impacting the SM balance [146,147,148]. Additionally, factors such as solar radiation and cloud cover can also influence SM by affecting the amount of energy available for ET [149,150]. Therefore, these variables are often employed in SM downscaling models. Topography affects SM distribution by influencing runoff and infiltration rates, water retention, and lateral flows. The influence of topography is often investigated using elevation, slope, aspect, curvature, and the TWI [117,151,152]. Soil properties such as texture (sand, silt, and clay content), structure, soil organic matter, porosity, and bulk density influence SM distribution, primarily by affecting water holding capacity, and, thus, they are used in downscaling models. Vegetation cover (including the type of vegetation, vegetation density, root depth, canopy cover, and LAI) can influence SM distribution based on their impact on ET, water uptake, and the interception of rainfall [153]. While all of these factors more or less contribute towards surface SM, the change in each factor can sometimes influence the importance of other factors.

Some studies have provided important insights in describing the non-linear relationship between SM and associated land surface variables using ML algorithms. Im et al. [64] employed three ML algorithms, namely RF, boosted regression trees, and cubist to downscale AMSR-E SM data over two regions in South Korea and Australia. MODIS 1 km surface albedo, LST, NDVI, EVI, leaf area index (LAI), and ET were used as the predictor variables in this study. The results revealed that RF outperformed other ML models, with r= 0.71 in the South Korean study site and r = 0.84 in the Australian study site, in downscaling AMSR-E SM data. The SM downscaling over the South Korean site was mostly influenced by ET and LST, whereas ET, surface albedo, and LST were influential variables in the Australian site. One of the problems in this downscaling method was its inability to capture moisture content at some elevations, (500–1900 m) for site A and (100–1000 m) for site B. According to Im et al. [64], this was caused due to the effects of forests and the insufficient temporal resolution of the MODIS products. Wei et al. [34] successfully reproduced the dynamic range of the original SMAP SM in downscaled SM products using GBDT, except for areas with high NDVI (>0.7) values, i.e., areas with dense vegetation. They tested 26 SM-related indices derived by using MODIS data and DEMs and found the distance drought index, modified perpendicular drought index, modified shortwave infrared perpendicular water stress index, NDVI, and shortwave angle slope index as effective indices in downscaling SM. However, the relationship between these indices and SM can be time-varying and unstable. Zhang et al. [61] applied ML algorithms, including RF and LSTM, based on the non-linear relationship between SM and environmental variables, such as LST, EVI, surface albedo, cumulative precipitation, DEM, and soil texture, to downscale the SMAP SM products from a coarse (36 km) to a fine spatial resolution (1 km). The downscaled results showed spatiotemporal trends while correlating well with the ground observations. The DEM, precipitation, EVI, and surface albedo were effective in explaining the variability in observations. DEMs become critical in the SM downscaling of these areas, especially if there are large topographic differences in a region. Zhang et al. [61] also mentioned the importance of using precipitation and NDVI in downscaling SM. Further, Lv et al. [100] showed an improved accuracy of the SM retrieval model by using LST, NDVI, normalised shortwave infrared difference bare SM index, DEM, and slope based on a neural network analysis. In addition, Sang et al. [131] depicted an improved downscaling performance of CNN (SM-residual Dense Net) using NDVI, DEM, and slope as ancillary data.

LST is one of the important geophysical variables used as a predictor in downscaling SM. Sun and Cui [65] focused on the possibility of using land surface evaporative efficiency (LEE) as an alternative to LST in downscaling SM, since LST data are often cloud-contaminated. The study used seven experiments with various model combinations, using ancillary data such as LST, LEE, fractional vegetation coverage, albedo, latitude, and longitude. SVR was used to downscale the SMAP SM data using the ancillary variables. FNN was used to investigate whether there were any changes in the results. Both these techniques demonstrated similar results of an increased accuracy, when the ancillary data were composed of group of variables that integrated LST, LEE, and geographical parameters. Similar to this study, Zappa et al. [63] conducted a comparative analysis between various model combinations of surface SM data and ancillary variables using RF, and selected the best model which combined surface SM data with soil texture, topography, and vegetation. According to Zappa et al. [63], topography and vegetation play key roles among the ancillary data in downscaling SM. LST was not included in this model, since data with the required resolution were not available.

Sentinel-1 is an open-access C-band SAR satellite operated by the ESA. Sentinel-1 datasets have been used to derive predictors of SM downscaling, as it produces the desired high-spatial-resolution (down to 5 m) inputs in all weather, day and night. Sishah et al. [66] made efforts to understand the importance of utilising Sentinel-1 soil surface roughness (SSR) parameters in downscaling SM. The surface height and the effective correlation length were derived using Sentinel-1 satellite data as SSR parameters. They utilised an RF approach to downscale an SMAP 36 km SM product. This study observed the increased quality of the RF regression after introducing the SSR parameters. Among all the ancillary parameters used, the daytime LST was the most influential variable and the LAI was the least important. Bai et al. [67] indicated Sentinel-1 Sigma VV as an important ancillary variable in their RF-based model, especially with their study area being covered by sparse vegetation. Sparse vegetation can make the C-band SAR penetrate the soil surface easily, minimising the issues such as vegetation attenuation. The Sigma VV was followed by LST, LAI, NDVI, slope, and elevation in the order of significance of the ancillary variables. Wakigari and Leconte [114] conducted two experiments on SM downscaling to find the impact of Sentinel-1 predictors on improving the predictive accuracy of the RF model they used. Sigma VV, Sigma VH, contrast, entropy, correlation, and variance were the high-spatial-resolution predictors of Sentinel-1, which were used along with other surface variables such as MODIS albedo, LST, NDVI, SRTM-derived elevation, slope, aspect, and SMAP brightness temperature data. The results indicated brightness temperature as the most significant predictor, and the Sentinel-1-based predictors only had a minor impact on the predictive accuracy of the downscaling model. The SMAP 36 km and 9 km brightness temperature were used in this study as the coarse-resolution response variables.

Even though ML and DL are powerful tools in addressing complex, non-linear relationships between predictor and response variables, they inherit a number of drawbacks and flaws. Vulnerability to spoofing, the requirement for a large dataset for model training, the limited opacity of the process, overfitting, underfitting, and the dependency on data quality are some of them. Overfitting occurs when an ML model learns the training data too closely, aligning well with the specific data points. As a result, it fails to generalize and provide accurate predictions on new, unseen data. In underfitting, the model is too simple to identify the patterns in a dataset. It typically has a high bias towards a specific output value, treating the variations in the input data as noise. As a result, it produces similar outputs, regardless of the input [154,155]. Data noise, insufficient training data, model complexity, and having too many features can cause overfitting. Insufficient training samples, data bias, and model simplicity can lead to underfitting [156,157]. Therefore, ML and DL algorithms heavily depend on the quality of the data used for training.

The accuracy of downscaling depends not only on the performance of the ML/DL techniques, but also on the accuracy of the variables fed into the AI model. As mentioned earlier, the selection of variables in SM downscaling depends on their heterogeneity and their influence on SM variability in a certain area. For example, if there are different vegetation types in an area, factors such as vegetation indices should be incorporated into the model to capture the varied influence of these types on SM. Ancillary variables used to express strong atmospheric evaporative demands, such as soil temperature, are more suitable for SM downscaling in arid and semi-arid areas. However, collecting sufficient training data for some temporally static variables, such as soil texture and topography, might be difficult for time series analysis. Most of these ancillary variables are derived from optical/thermal images, which are often subjected to cloud coverage. Data gaps caused by clouds and aerosols can make optical/thermal-data-based downscaling models less robust.

4.2. Improvements Made to ML-Based SM Downscaling

Some studies have combined ML with other techniques such as weighting and geomorphometry, while others have improved the quality of validation to enhance the accuracy of downscaling SM products.

Weighting has been performed considering the spatial pattern of the SM distribution over the study region. Xu et al. [119] used CNN and a weight layer to improve their SM downscaling strategy, while minimising the loss of information between the pixels. The information between the pixels was an important focus in this study, because pixel-to-pixel downscaling strategies (i.e., SM downscaling based on on-site conditions) completely ignore the information between the pixels (i.e., the spatial patterns over the region). Here, SMAP 36 km and 9 km SM data were targeted to be downscaled into 3 km and 1 km by using MODIS Terra surface reflectance and LST data. The accuracy was improved by adding a weight layer as an input and residual SM as an output. The 3 km downscaling enhanced the resolution of the SM information and preserved the accuracy at the same time. Kim et al. [108] introduced a new SM downscaling technique which combined SVR with locational weighted geophysical variables. They compared the results of this new method with in situ measurements and a polynomial downscaling method based on regression. In comparison to the results of the polynomial regression, locational weighting made the seasonal differences between the original SM and the SVR-downscaled SM consistent.

Geophysical variables used as ancillary data in downscaling SM can be computationally costly and sometimes contaminated by clouds. Therefore, Guevara and Vargas [117] investigated combining geomorphometry and kernel-weighted KNN to improve the resolution of SM by adopting a methodology independent of ecological data (vegetation) and climate information (precipitation and temperature). The prediction variables of the model were fifteen hydrologically meaningful terrain parameters, including primary parameters such as slope and aspect and secondary parameters including cross-sectional curvature, longitudinal curvature, analytical hill shading, convergence index, closed depressions, catchment area, TWI, length slope factor, channel network base level, vertical distance to channel network, and valley depth index. The study downscaled 27 km resolution ESA-CCI data to a 1 km scale, which showed a good compatibility with field-measured SM records with r = 0.46 and bias = 0.057 m³/m³. Warner et al. [118] also used a geomorphometric kernel-weighted KNN for downscaling ESA-CCI SM into 100 m using meteorological data, spatial coordinates, and terrain attributes. The error percentage of the downscaled daily SM data was 27%, but with an underestimation of the surface SM in wetland areas. The advantage of using the geomorphometry combined with ML in downscaling is the ability to compare the spatiotemporal relationships of the downscaled SM with vegetation and land cover data without using them as ancillary data. Llamas et al. [121] also postulated a terrain-parameters-based downscaling model using kernel-weighted KNN and RF. The target downscaling resolution for the ESA-CCI data was 1 km. Llamas et al. [121] suggested terrain parameters as the most suitable predictors in the regional-scale SM subjected to the influence of seasonality. For instance, geomorphometry can highly influence the SM in autumn and spring. In the comparison of the kernel KNN and RF, the RF outperformed in areas with sparse sampling data. The kernel KNN built many simple models for downscaling when there was a high density of sampling points and indicated a better correlation with in situ measurements.

The accurate fusion between different data types generally reveals more information about a phenomenon rather than using a single data type. Ming et al. [120] used TC to merge satellite- and model-based SM data and then to downscale SM using LSTM. TC is used to measure the simultaneous errors between three data sets [158]. The merging between the SM products in this study was conducted by combining the errors obtained by TC and least squares framework in every pixel. The pixels which violated the assumptions in the TC analysis were merged using simple arithmetic average. However, it was evident that the TC-based merging was more efficient than the mathematical averaging, especially when there were large differences between the parent products. Shangguan et al. [132] used an ML-based downscaling and merging process to generate a long-term high-accuracy SM dataset with a good accuracy, with r = 0.52 and ubRMSE = 0.047 m³/m³, by using CNN, ANN, XGB, LSTM, and ResNet. This study further highlights the importance of using short-wave-infrared-band-derived SM indices due to their sensitivity to SM.

The combination of ML with a geo-statistical model has been tested by some researchers in downscaling SM to capture the complex relationships between SM and ancillary data while maintaining the spatial structure of downscaled SM data. For example, Jin et al. [133] proposed an ML-based geo-statistical model to combine several kinds of auxiliary information in downscaling SM. The proposed method integrated SVR and area-to-area kriging (SVATARK) to deal with the non-linearity of the ancillary variables. This method was compared against five other algorithms, i.e., SVR kriging, SVR with interpolation, KNN, geologically weighted regression kriging, and DisPATCH. SVATARK obtained a high accuracy in producing 1 km SM data from 25 km using the ancillary variables such as land cover, LST, NDVI, blue sky albedo, and terrain factors. Similarly, Karamouz et al. [134] used ANN kriging for real-time SM estimations, while fulfilling the product accuracy requirement of SMAP. The real improvement here was the capability of this estimator to predict SM for a certain month of a year based on the model trained for another year.

One of the biggest challenges experienced by the researchers was the scarceness of ground truth measurements to validate the downscaled SM. Zappa et al. [63] found a solution for this issue using the parrot flower power low-cost sensor. The parrot flower power measures the SM and incoming solar radiation at the same time. Zappa et al. [63] trained an RF model with coarse-resolution SM data and ancillary data against the in situ SM measurements obtained from this low-cost sensor. Here, ASCAT, SMAP, or spatially averaged SM from the parrot flower power were used as coarse-spatial-resolution SM product, and topography, soil texture, and vegetation cover data were used as ancillary variables. The vegetation cover was derived using the fraction of absorbed green radiation measured by the parrot flower power. They gained a model accuracy of r = 0.8 and RMSE = 0.056 m³/m³. The model and validation accuracy significantly improved using a sufficient number of sensors to capture the spatial heterogeneity of the surface SM. They further emphasised the improvement of the model accuracy by using averaged SM from the parrot flower power as the coarse-scale SM rather than using ASCAT or SMAP SM.

Various types of ML/DL methods have been tested for downscaling SM. Shangguan et al. [103] tabulated the advantages and disadvantages of different ML/DL methods in downscaling SM. Accordingly, RF has its advantages in insensitivity to outliers, a great performance on tabulated data, and relatively high accuracy in downscaling. Failing to consider spatiotemporal information is one of its drawbacks. GBDT is competent in handling the non-linear relationships between ancillary variables and SM, yet occasionally fails to consider the overall spatiotemporal information over the region throughout time. It is largely a pixel-to-pixel downscaling strategy, where the algorithm is implemented at a given pixel for that snapshot of the time window. LSTM has the advantages of a full capacity to work with time-series data and capture long-term trends. However, it sometimes fails to consider geo-spatial relationships. Well-established spatial integrity through residual connection and a better representation of the overall spatial patterns in original SM can be shown as the advantages of ResNet in downscaling SM. The main technique-related strengths and weaknesses of widely used ML/DL techniques common to all applications are listed in

Table 2. Strengths and weaknesses of some widely used ML/DL techniques.

ML/DL Technique	Strengths	Weaknesses
Linear Regression [159,160]	Easy and simple implementation. Fast training capacity. Less complexity. Overfitting can be avoided by dimensionality reduction techniques.	Applicable for linear relationships. Sensitivity to outliers. Prone to multi collinearity.
Logistic Regression [161,162,163]	Easy and simple implementation. Easy updating. Fast training capacity.	Prone to model overfitting. Difficulty of capturing non-linear relationships. Necessity of large number of training samples.
SVM [164,165,166]	Works well with structured and semi-structured data. Scales well for high dimensional data. Capacity of generalization.	Long duration of training for large datasets. Complex to understand and interpret the final output.
Decision Tree [167,168,169]	Easy implementation. Ability to handle both numerical and categorical data. Preforms well with large datasets.	Trees are probe to non-robustness. Prone to overfitting.
PCA [170,171]	Remove correlated features. Reduce the possibility of over fitting.	Must perform data standardization before PCA. Prone to loss of information.
RF [78,172,173]	Avoid overfitting in decision trees and improve the accuracy. Can be used for both classification and regression. Data normalisation is not necessary as it uses a rule-based approach.	More complex than decision trees. Considerable time of training.
XGB [173,174,175]	Flexibility of the technique. Handles missing data.	Often ignores the overfitting. Increased complexity in classification. Long computational time. Sensitive to outliers.
ANN [163,166,171,176]	Less formal statistical training is required before developing. Detect complex and non-linear relationships between variables. Ability to detect all possible interactions between predictor variables. Multiple training algorithms can be used.	Limited opacity in identifying possible casual relationships—“black box”. Can be more difficult to use. Greater computational resources. Prone to overfitting.
CNN [166,177,178]	Computationally efficient. Parameter sharing.	Difficulty in classifying tilted or rotated images. Requires large number of training samples.
KNN [166,168,173]	Intuitive and simple. Responds quickly to real-time changes in the input data. Easy to implement for multi-class problems. Can be used for both classification and regression.	Sensitive to outliers. Not capable of dealing with missing value problems. Biased if the data are imbalanced. Necessity to choose optimal number of neighbours to be considered. Works well with smaller number of input variables.
RNN [179,180,181]	Processes inputs of any length. Excellent capacity in time series prediction.	Gradient vanishing and exploding issues. Complexity in training.
LSTM [180,181,182]	Avoids long-term dependency problem. Processes inputs of any length. Excellent capacity in time series prediction.	Complexity in training. Prone to overfitting.

.

Recently, with the increased capacities of data processing, storage, and high-performance computing, some studies have published global daily high-resolution SM datasets, which are sometimes publicly available. The resolution of these datasets varies from 30 m to 1 km at both regional and global scales [96,183,184,185,186]. However, the gridded resolution (1 km/30 m) of these data products can be different from their actual resolution based on observed data, e.g., over-sampling. Therefore, users should consider this aspect when utilising downscaled products for SM analysis.

4.3. Validation Methods Used for Downscaled SM

The validation of downscaled SM products is important to assess their accuracy and reliability. This process typically involves comparing the downscaled SM products with observed SM measurements from in situ stations [187] or other independent sources such as airborne data [112,116,126] and different model outputs such as GLDAS [24], ERA5-Land datasets [25], and CLDAS [33,100]. However, the validation of downscaled SM products is often challenging due to the difficulties in obtaining a sufficient number of representative in situ measurements within a downscaled SM pixel, both spatially and temporally. For example, [187] explains that a minimum of eight SMAP calibration and validation sites are required to validate the SMAP 36 km grid with a 70% confidence for a 0.03 m³/m³ SM uncertainty and 0.07 m³/m³ variability. Similarly, a minimum of five sites are needed for a 9 km grid with the same confidence levels and uncertainties, and a minimum of three sites are required for a 3 km grid with a 70% confidence for a 0.03 m³/m³ SM uncertainty and 0.07 m³/m³ variability. Many global-scale in situ SM observation networks have spatially sparse datasets. Catchment- or regional-scale SM networks such as SASMAS [188], the Yanco [189] dataset, and observations from spatially dense field data collections over shorter periods have been used to validate downscaled SM products [21,22]. Airborne SM datasets are rare; therefore, they are impractical in areas where such campaigns are not conducted. Airborne SM campaigns such as SMAPEx and NAFE were carried out for the validation of satellite SM data, developing retrieval and downscaling algorithms using L-band sensors [21,22,190,191,192]. Thus, many downscaling studies were carried out in the areas where such airborne campaigns were conducted due to the ability to validate their results. Figure 6 shows the validation between 1 km NAFE’05 airborne SM retrievals and 1 km downscaled SM derived from in situ data and GLDAS estimate-based models on 21 November 2005 over the Krui and Merriwa River catchments in southeastern Australia [112], as an example for the use of SM retrievals from an airborne campaign to validate downscaled SM products. Statistical metrics such as ubRMSE, RMSE, r (correlation coefficient), coefficient of determination (R²), and mean absolute error (MAE) are generally used to evaluate the accuracy of downscaled SM products [193,194,195].

4.4. Future Research

The applicable ML models vary according to geographical areas and conditions. Potential future research could explore the transferability of models across diverse areas or regions to gain a deeper understanding of their robustness and adaptability under varying climate and environmental conditions. By examining how these models perform in different geographical settings, researchers can evaluate their reliability and accuracy, ultimately enhancing their utility in a wide range of scenarios. This comprehensive approach could lead to more resilient and versatile models, capable of addressing the complexities of climate and environmental changes globally. Furthermore, the accuracy of different ML/DL models over a certain area could vary. Therefore, comparison studies over the same study area and datasets could potentially improve the understanding of their performances compared to each other. A reliable validation strategy for downscaled SM products is still lacking. There is a need to develop and assess effective performance metrics specifically designed for evaluating downscaled SM at the point scale. Additionally, the evaluation of spatial patterns of downscaled SM could be enhanced through the incorporation of upscaled in situ SM data [19].

5. Conclusions

This paper examined the ML techniques used for downscaling coarse-resolution satellite SM products. From fundamental algorithms such as decision trees to sophisticated neural networks, Bayesian models, and deep learning models, a wide spectrum of ML/DL techniques were discussed in the context of downscaling SM. Furthermore, the methodological framework for ML-based downscaling approaches was described, elucidating the intricate relationships between SM and various predictor variables, including meteorological data, land surface parameters, vegetation indices, and topographic attributes.

The usage of individual or multiple ML techniques for downscaling SM products was detailed in this paper. Comparative analyses of different ML/DL techniques were discussed, focusing on studies where multiple ML techniques were used for downscaling SM. Important insights into the ancillary variables significant for improving downscaling model accuracy were also presented, including the use of SAR-based ancillary variables to minimise issues related to cloud contamination in optical/thermal datasets. This review included insightful case studies that have improved SM downscaling accuracy either by combining ML algorithms with other techniques or by enhancing validation accuracy to fit applications at local and regional scales.

Estimating and predicting high-spatial-resolution SM globally is challenging due to its complexity, rendering a single universal method ineffective. The performance of ML and DL methods, as well as the input ancillary variables, varies depending on geographic location. Ancillary variables should be carefully selected to capture their variable influence on SM distribution. While ML is robust in addressing the complex, non-linear relationships between SM and ancillary variables, it has its own strengths and weaknesses in handling data.

In summary, this paper underscores the significance of ML techniques in enhancing the accuracy of downscaling coarse-resolution satellite SM products, offering valuable insights and methodologies for advancing research and applications in this field.