Stability Analysis of Unmixing-Based Spatiotemporal Fusion Model: A Case of Land Surface Temperature Product Downscaling

Abstract

The unmixing-based spatiotemporal fusion model is one of the effective ways to solve limitations in temporal and spatial resolution tradeoffs in a single satellite sensor. By using fusion data from different satellite platforms, high resolution in both temporal and spatial domains can be produced. However, due to the ill-posed characteristic of the unmixing function, the model performance may vary due to the different model setups. The key factors affecting the model stability most and how to set up the unmixing strategy for data downscaling remain unknown. In this study, we use the multisource land surface temperature as the case and focus on the three major factors to analyze the stability of the unmixing-based fusion model: (1) the definition of the homogeneous change regions (HCRs), (2) the unmixing levels, and (3) the number of HCRs. The spatiotemporal data fusion model U-STFM was used as the baseline model. The results show: (1) The clustering-based algorithm is more suitable for detecting HCRs for unmixing. Compared with the multi-resolution segmentation algorithm and k-means algorithm, the ISODATA clustering algorithm can more accurately describe LST’s temporal and spatial changes on HCRs. (2) For the U-STFM model, applying the unmixing processing at the change ratio level can significantly reduce the additive and multiplicative noise of the prediction. (3) There is a tradeoff effect between the number of HCRs and the solvability of the linear unmixing function. The larger the number of HCRs (less than the available MODIS pixels), the more stable the model is. (4) For the fusion of the daily 30 m scale LST product, compared with STARFM and ESTARFM, the modified U-STFM (iso_USTFM) achieved higher prediction accuracy and a lower error ($R{ }^2$: 0.87 and RMSE:1.09 k). With the findings of this study, daily fine-scale LST products can be predicted based on the unmixing-based spatial–temporal model with lower uncertainty and stable prediction.

1. Introduction

As a key physical parameter of land surface, land surface temperature (LST) provides essential information on spatial–temporal changes in surface energy and water balance at local or global scales [1,2,3]. LST is now a key input of the physical model for evapotranspiration estimation, urban heat island analysis, vegetation monitoring, agricultural monitoring, etc. [4,5,6,7]. However, limited by a single satellite platform and sensor, it is difficult to obtain surface temperature with both high spatial and temporal resolution, limiting applications such as fine-scale wildfire monitoring and local-scale farm management [8,9,10].

To address this problem, in the last decade, many downscaling models have been introduced in both computer vision and remote sensing fields [11]. Generally, all these downscaling models can be classified as super-resolution models and spatiotemporal fusion models. Compared to the super-resolution models, which can be defined as the point spread function rebuilding problem, the spatiotemporal fusion methods mainly focus on how to take advantage of different remote sensing platforms and sensors by a fusing process [12,13]. Under the proper fusion hypothesis with suitable auxiliary data, the spatial–temporal fusion model can overcome the larger downscaling ratio and benefit from tempol continuity features in time series remote sensing observations [14,15]. From the perspective of key ideas, spatial–temporal fusion models can be commonly divided into three categories: weight function-based methods, learning-based methods, and unmixing-based methods.

The weight function-based methods assume that the temporal variation of surface reflectance is spatially consistent nearby, so the reflectance residual between the low- and high-resolution image can be formed with a linear weighted function, then selecting multiple nearby pixels under the measurement of spatial, spectral, and temporal similarity to solve this weighted function [16]. Under this idea, the spatial and temporal adaptive reflectance fusion mode (STARFM) [17] is the first weight function-based method to be developed and introduces the concepts of moving windows to search the candidate pixels to solve the weighted function. In this model, the search processing is limited by the window size. Sufficient pixels can easily be selected in the homogeneous region. However, for the regions with complex landscapes or changes, this method may result in large errors in the fusion results. Aiming at the shortcomings of the STARFM method, the spatial temporal adaptive algorithm for mapping reflectance change (STAARCH) uses tasseled cap transformations to detect reflectance changes, thereby improving the ability to detect land cover changes from low-resolution imagery [18]. To improve the temporal fusion capability, the enhanced STARFM (ESTARFM) [19] was formed by involving conversion coefficients, linear spectral unmixing, and spectral correlation coefficients in the temporal domain. With the extra-high-resolution images, ESTARFM shows higher prediction accuracy in heterogeneous landscapes with more spectral details. Similar results can be found with the robust adaptive spatial and temporal fusion model (RASTFM) [20], which consists of a nonlocal linear regression (NL-LR)-based weighted average module. Overall, the weight function-based methods usually have few parameters with high computational efficiency. However, for the parameters with rapid change, such as land surface temperature, these models still have limitations.

The second category is the learning-based model, which predicts high spatial and temporal resolution images using machine learning algorithms to solve the nonlinear problem between high- and low-resolution images [13]. In the context of this idea, the sparse representation-based spatiotemporal reflectance fusion model (SPSTFM) uses dictionary learning to establish the relationship between low and high spatial resolution images corresponding to reflectance changes and predicts target high-resolution images by time weighting [21]. For a stable dictionary, error-bound regularized semi-coupled dictionary learning (EBSCDL) [22] introduced a regularized error-bound method and constructed an optimized semi-coupled dictionary to capture the divergence between low spatial resolution images and high-resolution images. However, the sparse representation-based fusion method requires a properly designed dictionary. In algorithm implementation, dictionary learning, sparse coding, and image reconstruction are separated, which increases the instability and complexity of the algorithm. In this regard, the fusion method based on deep convolutional neural networks (STFDCNN) combines CNNs and nonlinear mapping models to design double CNNs, which realizes the automatic extraction of image features and improves prediction accuracy [23]. To reduce the impact of a single algorithm failure, a modified ensemble LST model was used, combining three regressors including a random forest, a ridge, and a support vector machine [24]. Generally, based on large training samples, learning-based methods can capture more surface spatial details and are suitable for heterogeneous regions, etc. However, this generalization of the model across different locations and times can be a challenge.

Unmixing-based methods are based on linear spectral mixing theory, where endmembers are extracted from high spatial resolution images to decompose low spatial resolution images to obtain the abundances of each endmember [25]. The typical steps of an unmixing-based model include endmember selection, abundance calculation, unmixing processing, and fusion [26]. The multisensory multiresolution technique (MMT) is the first unmixing-based method to fuse the different resolution images, assuming a similar class contains a similar reflectance [27]. The unmixing processing is handled by a moving window and then assigns the result with high spatial resolution classes. However, the assumptions in the algorithm may not hold for locations where the landcover types vary with time. In this regard, Zurita-Milla et al. introduced constraints in the linear unmixing process to deal with the negative and outlier problems of the unmixing results [28]. The spatial–temporal data fusion approach (STDFA) is based on the assumption that the time-varying attributes of each land cover class are constant, using a modified linear mixed model that accounts for spatial variation [29]. Based on the STDFA algorithm, Zhang et al. combined the multiscale segmentation algorithm and ISODATA algorithm to generate classification maps, used the moving window method to unmix low spatial resolution images, and introduced temporal weights to fuse the temporal images [30]. To improve the accuracy of model prediction in land cover changes, the unmixing-based spatial–temporal reflectance fusion model (U-STFM) assumes that the reflectance change ratio remains the same across different sensors under the given location during the same period [31]. In the spatial domain, regions share the same ratio as the homogeneous change regions (HCRs), which can be obtained using the multiresolution segmentation algorithm. Then, the image on the target date can be predicted by using the high spatial resolution data before and after with the change ratio. The unmixing process is based on a linear unmixing function with low computational complexity [26]. Based on the multiple predictions with the time series data, U-STFM improves the fusion accuracy for predicting features with rapid change such as ocean surface chlorophyll ab concertation [32].

LST data with strong volatility of temporal changes and low spectral features makes the parameters of the weight function class method more sensitive [33]. Because LST is the continuous variable, generally from 250 k to 350 k, it is hard to build sufficient training for the learning-based model to capture the relationship with continuous mapping. In contrast, the unmixing-based methods, such as the U-STFM model, are more suitable for this problem since the linear unmixing process is applied to the time-changing signal, which can capture the rapid changes in LST when these changes are recorded in the low-spatial resolution data. Moreover, it is temporally based, which means each location can have unique weights, which affords the model more flexibility for continuous mapping.

However, due to the complexity of temperature changes and the ill-condition problem of unmixing processing, the stability of the unmixing model can be affected by the following three aspects [34]: Firstly, the definition of the HCRs or endmembers. HCRs are mainly obtained from two types of models: clustering and segmentation. Segmentation processing focuses on the local similarity based on Tobler’s first law, while clustering concentrates on the similarity in feature space, so the pixels nearby may be classified into different HCRs; secondly, the level of unmix processing. In U-STFM, the least squares algorithm is commonly used to solve linear mixing problems, which can be applied at the single date level, the differential level, and the change ratio level. Different levels require a different number of applications of the unmixing process, which affects the stability of the results; thirdly, the number of HCRs or endmembers. Generally, the higher the number of HCRs, the greater the detail in which the surface changes can be described. However, more HCRs may lead the unmixing process to the ill-posed problem then increase the uncertainty of the prediction.

Overall, for the unmixing-based fusion model, the aforementioned factors are essential for the model’s stability. The quantitate analysis is needed to answer the question of how these factors affect the prediction and how to find a suitable strategy to minimize the prediction error. In this study, we focus on these three factors for quantitative analysis of the effect of the different HCR definitions and unmixing levels for LST spatial temporal fusion. Part of the Guangdong–Hong Kong–Macao Greater Bay Area (GBA) was selected as the study area, and Landsat7 LST products and MODIS LST products were selected to build the high-spatial and -temporal products. The main unmixing model we used in this study was the U-STFM model. In Section 2, we describe our study area and datasets; Section 3 describes the experimental design and the theoretical basis of U-STFM. The performance of the model was compared with the commonly used STARFM and ESTARFM to verify the effectiveness and stability of the LST prediction. The results and discussion are delivered in Section 4 and Section 5. The main conclusion is in Section 6.

2. Study Area and Dataset

2.1. Study Area

The study area was selected within the part of the Guangdong–Hong Kong–Macao Greater Bay Area (GBA)(113°49′13″E–114°16′10″E and 22°37′17″N–22°59′48″N) with an area of about 1843 km² (Figure 1). The topography of the region is complex, with various types of landforms. With rapid economic development, Dongguan and Shenzhen, as the core cities of GBA, have undergone significant changes in land use and urban expansion. A large area of forest and wasteland has been transformed into a city area. The spatial distribution of land surface temperatures has changed rapidly.

2.2. Dataset

The Landsat7 ETM+ LST products and MODIS LST products (MOD11A1.006) were downloaded from USGS Earth Explorer (https://earthexplorer.usgs.gov (accessed on 12 April 2022)), and the “ST_B6” band and “LST_Day_1km” band were used, respectively. The spatial resolution of ETM + LST is 60 m, and the revisit frequency is 16 days; the spatial resolution of MODIS LST products is 1000 m, and the revisit frequency is 1 day. Landsat7 data were resampled (nearest neighbor method) to 30 m using arcgis10.8. Due to the failure of ETM + SLC after 31 May 2003, and the perennial cloudy and rainy conditions in the study area, this study selected data with a cloud cover threshold of less than 1% from September 2000 to May 2003 and collected eight pairs of valid Landsat7 LST and MODIS LST images pairs. Details are shown in

Table 1. Data list of Landsat 7 LST and MODIS LST products used in the study area.

Date	Landsat 7 LST and MODIS LST Data Names	Spatial Resolution (m)
14 September 2000	LE71220442000258SGS00	30
	MOD11A1.A2000258.h28v06.061	1000
1 November 2000	LE71220442000306SGS00	30
	MOD11A1.A2000306.h28v06.061	1000
17 September 2001	LE71220442001260SGS00	30
	MOD11A1.A2001260.h28v06.061	1000
20 November 2001	LE71220442001324SGS00	30
	MOD11A1.A2001324.h28v06.061	1000
22 December 2001	LE71220442001356BKT00	30
	MOD11A1.A2001356.h28v06.061	1000
7 January 2002	LE71220442002007SGS00	30
	MOD11A1.A2002007.h28v06.061	1000
7 November 2002	LE71220442002311EDC00	30
	MOD11A1.A2002311.h28v06.061	1000
10 January 2003	LE71220442003010EDC00	30
	MOD11A1.A2003010.h28v06.061	1000

.

3. Methodology

3.1. Experimental Design

Three aspects of the stability of the unmixing model were evaluated: (1) the definition of the HCRs or endmembers, (2) the level of unmix processing, and (3) the number of HCRs or endmembers. The U-STFM method is used as the baseline model and then we compared the performance of the improved model with the STARFM and ESTARFM models.

Experiment I: For the definition of HCRs, the ISODATA [35], K-means [36], and multiresolution segmentation algorithms [37] were used to obtain the HCRs, respectively. The selected prediction date was 20 November 2001, and a total of 12 experimental date pairs were arranged and combined.

Experiment II: For analyzing the impact of the level of unmix processing, we unmixed the single date level (the unmixing process was applied to the image from each date, unmixing three times), two-date differential level (the unmixing process was applied to the differential images among three dates, unmixing twice), and three-date change ratio level (the unmixing process was applied to the change ratio of the three dates, unmixing once). For 6 available target dates (Table 1), 56 pairs of image groups in total were used to analyze how the different unmixing levels affect the stability of the model.

Experiment III: For testing the effect of the different number of HCRs, the number of HCRs was set from 10 to 259 by using the ISODATA algorithm; as in experiment two, 56 groups of image pairs were also used to evaluate the performance.

Experiment IV: The performance of the model with the best setup strategy from experiment I–IV was tested by comparing with the three classical models, STARFM, ESTARFM, and the original USTFM. The overall experimental design is shown in Figure 2.

3.2. The Theoretical Basis of U-STFM

U-STFM was selected as the representative algorithm among the unmixing-based methods. The reason for choosing this model is that USTFM is a linear unmixing-based built based on the temporal change ratio, which suits the rapid land cover changes.

The U-STFM model needs to input the MODIS LST data of the target date ($t_p$), as well as the MODIS LST data and Landsat LST data before and after the target date ($t_0$, $t_e$) to predict the Landsat7 LST data of the target date. The model is based on a linear spectral mixing model, which assumes that a low spatial resolution pixel can be represented by the linear combination of the spectral values of HCRs weighted by the fractional coverage within that pixel. The linear mixing processing is described in Equation (1):

$$M_t\left(i,j\right)={\displaystyle \sum }_{x=1}^n{f}_t\left(x\right)\overline{M_t^{SR}\left(x\right)}$$

(1)

where $M_t\left(i,j\right)$ represents the reflectance of MODIS pixel $\left(i,j\right)$ on date t; $\overline{M_t^{SR}\left(x\right)}$ is the average reflectance on the $x$ th HCRs, where SR stands for “super-resolution endmember”; $f_t\left(x\right)$ is the coverage of the $x$ th HCRs on MODIS $\left(i,j\right)$ pixels.

Assuming that the ratio of LST change in Landsat and MODIS sensors remains consistent over the same observation period ($t_0$, $t_p$, $t_e$):

$${\alpha }_P^L\left(i,j\right)={\alpha }_P^{MSR}\left(i,j\right)$$

(2)

$${\alpha }_P^L\left(i,j\right)=\frac{\Delta L_{pe}\left(i,j\right)}{\Delta L_{0p}\left(i,j\right)}=\frac{L_e\left(i,j\right)-L_p\left(i,j\right)}{L_p\left(i,j\right)-L_0\left(i,j\right)}$$

(3)

$${\alpha }_P^{MSR}\left(i,j\right)=\frac{\Delta M_{pe}^{SR}\left(i,j\right)}{\Delta M_{0p}^{SR}\left(i,j\right)}=\frac{M_e^{SR}\left(i,j\right)-M_p^{SR}\left(i,j\right)}{M_p^{SR}\left(i,j\right)-M_0^{SR}\left(i,j\right)}$$

(4)

where ${\alpha }_p^L\left(i,j\right)$ is the time-series change trend ratio of a Landsat pixel $\left(i,j\right)$; ${\alpha }_p^{MSR}\left(i,j\right)$ is the change trend ratio of HCRs in MODIS pixel $\left(i,j\right)$; $\Delta L_{pe}\left(i,j\right)$ and $\Delta L_{0p}\left(i,j\right)$ represent the difference in Landsat pixel reflectance over the two periods [$t_p$, $t_e$] and [$t_0$, $t_p$]; $\Delta M_{pe}^{SR}\left(i,j\right)$ and $\Delta M_{0p}^{SR}\left(i,j\right)$ represent the difference in HCR pixel reflectance over the two periods [$t_p$, $t_e$] and [$t_0$, $t_p$], respectively.

Therefore, combining Equations (2)–(4), we can speculate the Landsat pixel reflectance for the target date ($t_p$):

$$L_p\left(i,j\right)=\frac{L_e\left(i,j\right)-{\alpha }_P^{MSR}\left(i,j\right)L_0\left(i,j\right)}{1+{\alpha }_P^{MSR}\left(i,j\right)}$$

(5)

Equation (5) shows that the Landsat image of the target date is related to the Landsat images of the two periods before and after and the rate of change of HCRs. The linear unmixing theory (Equation (1)) is applied to the MODIS time series to obtain ${\alpha }_p^{MSR}$, and finally, the Landsat image at the target date $t_p$ is predicted.

3.3. Evaluation

In this study, qualitative and quantitative evaluation metrics are used to evaluate the effect of the model in predicting the land surface temperature. For each prediction, the Landsat LST production with 30 m spatial resolution was used as the ground truth. There are 6 reliable dates in total across 8 dates. For each date, the different three-date combination groups are evaluated, for example, there are 12 available three-date groups for 20 November 2001. The qualitative evaluation of model fusion is evaluated by comparing and analyzing the visualization effects of predicted and real LST images. The peak signal-to-noise ratio (PSNR), correlation coefficient (CC), root mean square error (RMSE) and mean absolute error (MAE) are used for quantitative evaluation. PSNR is a full-reference image quality evaluation index. The effective range of CC value is between the interval (−1, 1); closer to 1 indicates a better fusion result. Higher PSNR means better image fusion, while smaller RMSE and MAE values mean higher-quality fused images. All the above quantitative evaluation indicators are implemented by calling the evaluation indicator function corresponding to the scikit-learn module. The definitions of PSNR, CC, RMSE, and MAE are as follows:

$$\mathrm{RMSE}=\sqrt[2]{\frac{{\sum }_{i=1}^M{\sum }_{j=1}^N{\left(L\left(i,j\right)-P\left(i,j\right)\right)}^2}{M\times N}}$$

(6)

$$\mathrm{PSNR}=10\times lo{g}_{10}\left(\frac{MA{X}_I^2}{MSE}\right)$$

(7)

$$\mathrm{CC}=\frac{{\sum }_{i=1}^M{\sum }_{j=i}^N\left(L\left(i,j\right)-{\mu }_L\right)\left(P\left(i,j\right)-{\mu }_P\right)}{\sqrt{{\sum }_{i=1}^M{\sum }_{j=i}^N{\left(L\left(i,j\right)-{\mu }_L\right)}^2{\left(P\left(i,j\right)-{\mu }_P\right)}^2}}$$

(8)

$$RMSE=\sqrt{MSE}$$

(9)

$$\mathrm{MAE}=\frac{{\sum }_{i=1}^M{\sum }_{j=1}^N\left|L\left(i,j\right)-P\left(i,j\right)\right|}{M\times N}$$

(10)

where L$\left(i,j\right)$ and P$\left(i,j\right)$ represent the actual observed Landsat pixel $\left(i,j\right)$ and the predicted image pixel $\left(i,j\right)$, respectively; M and N represent the height and width of the image, respectively; $MA{X}_I$ represents the maximum value of the image color; ${\mu }_L$ and ${\mu }_P$ represent the average value of the observed image and the predicted image, respectively.

4. Results and Discussion

4.1. Analyzing the Effect of the Definition of the HCRs on Model Stability

The HCRs or endmembers definition is the first step of the unmixing model, so accurate HCRs are crucial to improve the performance of the unmixing process. In this study, the multiresolution segmentation algorithm, K-means algorithm, and ISODATA algorithm are selected to obtain HCRs.

The multiresolution algorithm is a bottom-up region merging technique, which divides different regions spatially by identifying the differences in spectra, shapes, and other features of remote sensing images. The K-means algorithm is a clustering algorithm based on distance, which uses the distance between data objects as the similarity criterion and divides the data into different clusters by iterative update; the ISODATA algorithm is a dynamic cluster analysis algorithm based on the K-means algorithm with the “merge” and “split” operations, and the cluster centers and cluster numbers can be updated based on the characteristic of data distribution [38].

The quantitative evaluation results are shown in Figure 3. It can be seen that the ISODATA algorithm (the median PNSR: 46.93, CC: 0.94, RMSE: 1.80 K, MAE: 1.08 K) is more accurate in dividing HCRs and more suitable for capturing the LST variation in time series than the K-means (the median PNSR: 46.73, CC: 0.94, RMSE: 1.84 K, MAE: 1.19 K) and the segmentation algorithm (the median PNSR: 45.76, CC: 0.95, RMSE: 2.07 K, MAE: 1.18 K) with high PNSR and low RMSE and MAE. The lower CCs of ISODATA are caused by the outliers of the prediction. Overall, the segmentation-based methods consider the local similarity and provide the most accurate spatial division of homogeneous change regions. However, it easily over-segmented the region and lead to an unstable solution of the linear unmixing system (Equation (1)). The clustering-based algorithm can reduce this problem by providing more accurate division and reducing the number of HCRs at the same time, which is more suitable for the linear unmixing system.

Figure 4 shows the distribution of the absolute error of predictions on 20 November 2001. For each HCR definition, most of their absolute errors are distributed within 5 K. The ISODATA-based model (Figure 4c) is generally less than 2 K, showing better results compared to the K-means and multiresolution segmentation. Overall, compared to the segmentation method (the original chosen in U-STFM), the clustering algorithms have better performance with lower RMSE and MAE variation. The reason is that the clustering-based HCRs is more related to the different land surface type, which is more likely to share the same LST change ratio. Compared to the K-means, the ISODATA algorithm can more accurately obtain LST with lower variation, so it is selected for experiment IV.

4.2. Analyze the Effect of the Different Unmixing Levels on the Stability of the Model

The linear unmixing equation is the core component of the unmixing-based models. The unmixing matrix composed of HCR values, abundance, and MODIS pixels can easily become an ill-conditioned matrix when the number of HCRs is larger than the number of MODIS pixels. At the same time, the least squares optimization is more sensitive to outliers. As mentioned before, there are three different levels for applying the unmixing processing (Figure 5): (1) the single-date level: unmixing each datum on $t_0$, $t_p$, $t_e$, then calculating the change ratios; (2) the differential level: unmixing the difference between $t_e$ − $t_p$ and $t_p-$ $t_0$ then calculating the change ratios; and (3) the change ratio level: calculating the change ratio first then solving the unmixing problem directly on the change ratio. In this section, we will explore the impact of the different levels on the model performance.

The results are shown in Figure 6. As we can see, the fusion accuracy at the single-date level is significantly lower than the other two levels, and the performance of the model in the differential level and the change ratio level is much more similar. Based on the RMSE and MAE, the model on the change ratio level has lower variation (the median PNSR: 42.00; CC: 0.94, RMSE: 3.18 K, and MAE: 2.18 K). The reason behind this is that most prediction error comes from the unmixing process. So, the less unmixing, the lower the error. On the other hand, the differential image between different dates can reduce the additive systematic noise of sensors, so the performance of the model at the differential level is much better than that at the single date level. The change ratio level can further reduce the multiplicative noise of the system and then obtain better results.

Figure 7 shows the spatial distribution of the absolute error of the final prediction on 20 November 2001. We can see from Figure 7 that the prediction errors of the model in the change ratio level and the differential level are mostly distributed below 5 K. The errors of the model in the differential level have more regions within 2–5 K. However, the error of the model at the single-date level is relatively large, most of which is distributed between 5 K and 20 K. based on this result, the model in the change ratio level is recommended for experiment IV.

4.3. Analysis of the Effect of the Number of HCRs on Model Stability

Theoretically, the finer the HCRs, the more details of LST are obtained. However, when the number of HCRs is larger than the available number of MODIS pixels, it leads to instability in solving the linear unmixing equations. The number of HCRs also affects the accuracy of solving equations by the least squares optimization. Therefore, how to set up the appropriate number of HCRs is crucial to the performance of the model. In this section, we use the ISODATA algorithm to divide HCRs and apply the model at the change ratio level. The number of HCRs was set from 10 to 295 to evaluate the model performance.

As shown in Figure 8, we can see that there is a strong correlation between MAE and the number of HCRs. The MAE shows a decreasing trend as the number of HCRs increases. The PSNR and RMSE showed an increasing and decreasing trend when the number of HCRs categories was below 104, and then both gradually stabilized. The CC values were above 0.92, and the values were relatively high when the number of HCRs categories was within 100, and then had a decreasing trend. Figure 9 is a visualization of the absolute error of the LST results for selecting models based on 10, 104, and 207 HCRs. Their absolute errors are generally distributed within 5 K, with the largest distribution of absolute errors in the range of 2–5 K for the models based on HCR category 10. The overall distribution of the model based on the HCR category of 104 and the model of the HCR category of 207 is relatively similar, and the former is more concentrated. When the number of HCR categories in the model is 104, it has a median PNSR of 43.11 K, a median CC of 0.93, a median RMSE of 2.81 K, and a median MAE of 2.17 K. Overall, the model fusion effect is better when the HCRs category is 104.

4.4. Compare Analyzed USTFM with STARFM and ESTARFM

Based on the above experimental results, the best setup for the USTFM model uses the ISODATA algorithm for HCR detection, the number of HCRs was set above 100 (the final number is 104), and the unmixing processing was set at the change ratio level. To distinguish from the original model, we denoted these setups as iso_USTFM and compared this model with the three classical models, STARFM, ESTARFM, and USTFM (the best segment size: 59).

Figure 10 is a visualization of the absolute error of the LST results for selecting models based on STARFM (Figure 10a), ESTARFM (Figure 10b), USTFM (Figure 10c), and iso_USTFM (Figure 10d). We can see that the prediction error of the model in iso_USTFM is mostly distributed below 1.5 K. The error of the model in ESTARFM and USTFM are mostly distributed below 2.5 K. However, the error of the model in STARFM is relatively large, most of which is distributed below 5 k. STARFM has the largest error in predicting LST, and the LST distribution shows many low- and high-temperature extremes that do not match reality. A comparison of the absolute error distribution of model fusion results for land cover categories are shown in Figure 11. We can see that the performance of the iso_USTFM is better than STARFM, ESTARFM, and USTFM on barren, cropland, forest, grassland, and impermeable surfaces, with lower absolute error, but has higher errors for water and shrubs (Figure 11b).

Figure 12 shows the scatterplots of the LST predicted by the four models. According to the data distribution, all groups have data points close to the 1:1 diagonal, STARFM has a large number of outliers, and iso_USTFM has the highest correlation. Based on the metrics, the determination coefficient $R^2$ of the iso_USTFM model is 0.875 with the RMSE being 1.089 K, which is better than STARFM ($R^2$: 0.764; RMSE: 1.729), ESTARFM ($R^2$: 0.821; RMSE: 1.334) and USTFM ($R^2$:0.841; RMSE: 1.275).

5. Discussion

5.1. The Error Source of the Unmixing-Based Model

Generally, the unmixing-based model has two key components, one is the unmixing strategy and the other is the weighting function.

For the unmixing strategy, when the actuarial mixing process is unknown, the linear unmixing strategy is commonly used. However, the linear system may not fit reality. The thermal infrared signal mixed from the fine to coarse pixels may be dominant by the hottest object inside this area. Simply mixing them by the linear function may cost systematic error. The other linear system problem is the instability in solving the linear unmixing equations. When the number of unknown variables (the HCRs units) is larger than the number of equations (the available MODIS pixels), many unknown variables covered only by one or two equations will lead to no reliable and stable solution. This can explain the finding in Figure 3, where the segmentation-based method tends to over-segment the area.

The weighting function used by the U-STFM model is the area coverage ratio of each HCRs in each MODIS pixel. This strategy may suit the reflectance data fusion but may cost the problem in thermal infrared signals in the area containing hot or cold spots. In these areas, the hot or cold spots dominate the thermal infrared signal, so the weighting function fails in these areas. This can explain the finding in Figure 10(c3) where the hot spot region shows the biggest error.

The outliers (Figure 11) are mainly shown in cropland, forest, and impervious area. One of the reasons behind this is the land cover changes rapidly in these areas compared to the stable ones such as barren, shrub, and water. The USTFM model was originally designed for land cover change situations, as we can see the USTFM performed better compared to the STARFM and ESTARFM. However, for the stable categories, such as water and shrub, USTFM lost this benefit.

5.2. The Difficulty of LST Data Fusion

One of the big differences between the LST and the surface reflectance data is that the thermal infrared signal in LST data has the radiation effect of the hot spots. These hot spots will dominate the signal in the coarse pixel during the mixing process. So, the weighting function based on area coverage ratio (USTFM), the nearby stable pixels (STARFM), or tempol information (ESTARFM) may not be suitable for these areas. More accurate weighting functions and unmixing strategies need to be developed.

The other key factor for LST fusion is the tempol consistency. Most unmixing-based spatial–temporal fusion models require consistency between fine and coarse sensors. This consistency can be held under careful calibration for the surface reflectance data. But for the LST data, the model error involved by the different models may break this consistency, leading to unreliable prediction.

5.3. The Model Generalization

The most unmixing-based fusion model is the unsupervised model. This means that the unmixing process and the corresponding solution for HCRs only can be applied in the fixed region and dataset. In the GBA area, the land cover and land use units are piecemeal with rapid change, So the finding in this study may not be general in other places with different land cover conditions. The learning-based supervised nonlinear model can be developed to add generalizability to the model across different locations.

6. Conclusions

In this study, we identified three key factors for the unmixing-based spatial–temporal fusion model: (1) the definition of the HCRs, (2) the unmixing levels, and (3) the number of HCRs. Based on the step-by-step control experiment, we evaluated these three key factors quantitively to answer the question of how these factors affect the stability of the model. The U-STFM model was used as the baseline model. Landsat7 and MODIS LST products were used to generate daily 30 m high spatiotemporal resolution LST products. The best setups of U-STFM for LST prediction were compared with the classical STARFM and ESTARFM model. Based on the findings in our study area, the main conclusions are as follows:

• The clustering-based algorithm is more suitable for detecting the HCRs and endmembers for unmixing. Compared with the multi-scale segmentation algorithm and K-means algorithm, the ISODATA clustering algorithm can more accurately describe LST’s temporal and spatial changes to HCRs.
• The fewer times the unmixing processing is used, the more stable the model predictions are. For the U-STFM model, applying the unmixing processing at the change ratio level can significantly reduce the additive and multiplicative noise of the prediction.
• The larger the number of HCRs (less than the available MODIS pixels), the more stable the model is. There is a tradeoff effect between the number of HCRs and the solvability of the linear unmixing function. The larger number of HCRs means that more details of the LST change can be captured, but the perdition of the change ratios in small HCRs that are only covered by a few MODIS pixels will be unstable.
• For the fusion of the daily 30 m scale LST product, the suitable setups for U-STFM use ISODATA as the detector for HCRs, controlling the HCRs above 100 and applying the unmixing model at the change ratio level. Compared with STARFM and ESTARFM, modified U-STFM (iso_USTFM) achieved higher prediction accuracy and lower error.

The findings in this study can effectively help current unmixing-based spatiotemporal fusion models avoid the wrong setups and have the potential to achieve lower uncertainty and stable prediction. However, as shown in this study, the linear unmixing model based on the least squares algorithm is influenced by the outliers, which may not suit reality. Therefore, the nonlinear-based unmixing methods may further improve the model stability and robustness for daily fine-scale LST prediction.