Evaluation of TsHARP Utility for Thermal Sharpening of Sentinel-3 Satellite Images Using Sentinel-2 Visual Imagery

Abstract

A spatially distributed land surface temperature is important for many studies. The recent launch of the Sentinel satellite programs paves the way for an abundance of opportunities for both large area and long-term investigations. However, the spatial resolution of Sentinel-3 thermal images is not suitable for monitoring small fragmented fields. Thermal sharpening is one of the primary methods used to obtain thermal images at finer spatial resolution at a daily revisit time. In the current study, the utility of the TsHARP method to sharpen the low resolution of Sentinel-3 thermal data was examined using Sentinel-2 visible-near infrared imagery. Compared to Landsat 8 fine thermal images, the sharpening resulted in mean absolute errors of ~1 °C, with errors increasing as the difference between the native and the target resolutions increases. Part of the error is attributed to the discrepancy between the thermal images acquired by the two platforms. Further research is due to test additional sites and conditions, and potentially additional sharpening methods, applied to the Sentinel platforms.

1. Introduction

The land surface temperature (LST) at both high spatial and temporal resolutions is a key variable in many environmental studies, including vegetation monitoring, moisture estimation [1,2], drought assessment, evapotranspiration mapping [3,4,5], land-cover classification [6], and surface energy balance quantification [7]. However, until recently, satellite-derived LST has been provided at either a high temporal resolution (1 day) but low spatial resolution (1 km), e.g., National Oceanic and Atmospheric Administration–Advanced Very High Resolution Radiometer (NOAA-AVHRR) and Moderate Resolution Imaging Spectrometer (MODIS)-Terra/Aqua, or at a higher spatial resolution (60–120 m) but a low revisit time of 16 days, e.g., Landsat Thematic Mapper (TM) - Enhanced Thematic Mapper Plus (ETM+)/ Operational Land Imager (OLI).

To bridge this gap, attempts have been made to downscale frequently acquired low spatial resolution thermal images in order to assess LST at a high spatial resolution, and further use the data for the above-mentioned applications. The downscaling methods can be roughly classified into two categories: temperature unmixing and thermal sharpening [8,9,10,11]. Temperature unmixing is based on obtaining a coarse resolution temperature for different land-cover types and applying a split-window algorithm to retrieve the fine resolution results [12,13,14]. Thermal sharpening methods are based on a correlation between thermal images and auxiliary data (e.g., Normalized Difference Vegetation Index (NDVI), emissivity, albedo, digital elevation model, and normalized multi-band drought index (NMDI)). A variety of sharpening methods have been proposed, among which are DisTrad [7], STARFM [15], TsHARP [16,17], Artificial Neural Networks [18], the Data Mining Approach (DMS) [19], and NL-DisTrad [20].

DisTrad utilizes the quadratic relationship between NDVI and LST [7]. Bindhu et al. [20] applied a non-linear method, based on the DisTrad algorithm, to disaggregate the land surface temperature of MODIS with a 960-m spatial resolution to Landsat ETM+ at 60-m resolution over India. Agam et al. [17] improved the DisTrad algorithm by correlating the LST data with fractional vegetation cover (fc) instead of NDVI. The algorithm, termed TsHARP, was developed and tested over homogeneous fields in central Iowa, USA, and was proposed to disaggregate LST values over agricultural fields for monitoring irrigation processes and evaluating evapotranspiration [16,17,20,21]. It was demonstrated that sharpening thermal data at 960 m to 250 m, mimicking the sharpening of MODIS thermal to visible-near infrared (VNIR) data, yields root-mean-square-errors (RMSE) of between 0.67 and 1.35 °C. Sharpening thermal Landsat data from 120 and 60 m to a VNIR resolution of 20 m resulted in an RMSE of 1.8–2.4 °C. TsHARP was also tested over different land cover types, moisture conditions, and vegetation types [22,23,24,25]. Bisquert et al. [22] sharpened MODIS images with TsHARP with mean errors of 1.9 °C between disaggregated and reference Landsat thermal images at a 120-m spatial resolution. TsHARP’s utility was able to disaggregate MODIS temperature from 1000 m to 90 m with an error of 2.7 °C [26].

Among all the sharpening methods, TsHARP is the most widely used due to its simplicity and effectiveness. Bisquert et al. [27] examined three downscaling methods with MODIS and Landsat images in the heterogeneous vegetated area of central Spain and reported that TsHARP outperformed the others. Improving TsHARP was proposed by using a local modification of utility [24] and by combining it with spatial interpolation [23]. Jeganathan et al. [24] tested five different versions of TsHARP over a heterogeneous agricultural landscape in India. The methods were applied to aggregated data from the Advanced Spaceborne Thermal Emission and Reflection Radiometer (ASTER) and used for MODIS thermal maps. The local aggregation model yielded higher accuracy. Chen et al. [23] combined TsHARP with thin plate spline interpolation, which improved the sharpening accuracy by about 10% for grasslands and 5% for rural areas and croplands. Despite these encouraging results, the fact that TsHARP is based on the LST–NDVI relationship makes it inapplicable when this relationship is weak or reversed [28,29].

The recent initiation of the Copernicus Earth observation program of the European Commission (EC) and the European Space Agency (ESA), and the launch of the Sentinel constellation of satellites provides new opportunities to further examine the potential of TsHARP. Sentinel-2A and Sentinel-2B were launched on 23 June, 2015 and 7 March, 2017, respectively. The MultiSpectral Instrument (MSI) onboard these satellites has VNIR bands at around 6.65 and 8.35 µm, which enables the calculation of different vegetation indices for vegetation characterization at a 10-m resolution. This constellation has a revisit time of five days. Sentinel-3A and Sentinel-3B were launched on 16 February, 2016 and 25 April, 2018, respectively. The Sea and Land Surface Temperature Radiometer (SLSTR) on board these satellites has two thermal bands, at 10.8 and 12 µm, which acquires LST at a ~1-km resolution at nadir. The Sentinel-3 constellation has a revisit time of less than two days [30]. The remarkable advantages of sharpening the Sentinel-3 LST to the VNIR resolution of the Sentinel-2 are that the Sentinel satellites are synchronized in overpass time and ground coverage, and the images are in the same projection and processing chain. Moreover, archived data are available free-of-charge by the Copernicus Open Access Hub through several platforms (https://scihub.copernicus.eu). Lastly, the planned life cycle of the satellites is 15–20 years [31]. A first attempt to sharpen Sentinel-3 LST using Sentinel-2 NVIR data has recently been published [32]. A methodology to derive land-surface energy fluxes using the DMS approach was tested in agricultural and forest sites located in temperate zones. The results of the study indicated an improvement in derived energy fluxes from sharpened LST in comparison with low-resolution fluxes. However, the fluxes were still less accurate than those derived from high-resolution LST. The overarching goal of the current project was to examine the utility of TsHARP to sharpen the Sentinel-3 thermal data using Sentinel-2 VNIR imagery.

2. Materials and Methods

2.1. TsHARP Method

The TsHARP algorithm is based on the linear correlation between LST and fractional vegetation cover (fc). A detailed description of the TsHARP procedure is given in Agam et al. [17]. An empirical linear regression model is fitted to the LST and f_c data at the native coarse resolution of the LST image after aggregating the fine (target) resolution f_c image to match the coarse resolution of the LST image (LST_coarse and f_{c coarse}, Equation (1)).

$${\mathrm{LST}}_{\mathrm{coarse}}=\mathrm{a}+{\mathrm{bf}}_{\mathrm{c}\_\mathrm{coarse}}$$

(1)

with f_c computed as follows (Equation (2)).

$${\mathrm{f}}_{\mathrm{c}}=1-{\left(\frac{{\mathrm{NDVI}}_{\max }-{\mathrm{NDVI}}_{\mathrm{i}}}{{\mathrm{NDVI}}_{\max }-{\mathrm{NDVI}}_{\min }}\right)}^{0.625}$$

(2)

where NDVI_i is the index value in a specific pixel while NDVI_max and NDVI_min are the maximum and minimum values from the scene, respectively [33].

Assuming that the correlation between LST and f_c is scale-independent, the resulting regression coefficients (a and b) obtained at the low resolution are applied to the fine spatial resolution vegetation index together with the fine resolution f_c data (f_{c fine}) to predict the regression-based LST at the target fine resolution (LST_fine, Equation (3)). Lastly, the residual error (i.e., the difference between the native, LST_native, and the estimated, LST_coarse, temperatures) for the corresponding coarse resolution pixel is added to the predicted temperature to increase the prediction accuracy (Equation (4)).

$${\mathrm{LST}}_{\mathrm{fine}}=\mathrm{a}+{\mathrm{bf}}_{\mathrm{c}\mathrm{fine}}$$

(3)

$${\mathrm{LST}}_{\mathrm{downscale}}={\mathrm{LST}}_{\mathrm{fine}}+\left({\mathrm{LST}}_{\mathrm{initial}}-{\mathrm{LST}}_{\mathrm{coarse}}\right)$$

(4)

2.2. Methodological Workflow

Testing the accuracy of TsHARP for sharpening Sentinel-3 thermal images using Sentinel-2 VNIR images was assessed by comparing the sharpened thermal images to Landsat-8 thermal images that were acquired at a fine resolution. This was applied in three stages (Figure 1).

Stage 1–sensor comparison and inter-calibration–before using Landsat-8 to validate the sharpening product, the comparability between the Sentinel-3 and the Landsat-8 thermal images had to be evaluated. To this end, six pairs of Sentinel-3 vs. Landsat-8 thermal images, acquired within a maximum of a 15-minute time difference, were compared. To minimize potential additional sources of error due to different atmospheric corrections between the two sensors [34], the comparison, as well as the sharpening, was applied on top-of-atmosphere brightness temperature (BT) images, which has been previously done (e.g., [19,25,35]). The six BT image-pairs were compared at the coarse resolution of 960 m (see more in Section 2.4 hereafter). For the following analyses, two of the six pairs of images, for which a coinciding same-day Sentinel-2 overpass was available (

Table 1. List of study sites and Landsat 8, Sentinel-3, and Sentinel-2 images used in this study.

Site	Acquisition Date	Location	Coordinates	Landsat 8		Sentinel-3	Sentinel-2
				Path/Row	Overpass Time (Local Time)
1	13-Jul-17	Campo Grande, Brazil	20.13°S, 53.32°W	224/74	9:34	9:34	9:51
2	20-Jun-17	Podebrady, Czech Republic	50.14°N, 15.15°E	191/25	11:50	11:50	12:00
3	16-Jul-17	Omaha, Iowa - Nebraska, USA	41.45°N, 96.1°W	28/31	12:05	12:00	-
4	16-Jul-17	Sacramento, CA, USA	38.54°N, 121.39°W	44/33	11:45	11:44	-
5	9-Jul-17	Sacramento, CA, USA	37.28°N, 120.35°W	43/34	11:39	11:25	-
6	17-Oct-17	Madhya Pradesh District, India	23.6°N, 79.33°E	114/44	11:08	11:08	-

), were chosen. For these two pairs, the linear regression coefficients resulting from regressing the Sentinel-3 BT values against the Landsat-8 BT values were used to calibrate the images. The calibrated Landsat-8 BT images were used in Stage 3 for validation (see below).

Stages 2 and 3 were, as mentioned, based on an analysis of two sites only, for which images from all three satellites were available. In Stage 2–thermal sharpening–the TsHARP algorithm was applied over the two sites, and, in Stage 3–validation–the accuracy of TsHARP was evaluated by a visual and statistical comparison.

2.3. Study Sites

Six areas representing different climatic conditions and land-cover types were selected (Table 1 and Figure 2).

- Site 1–Campo Grande, Brazil

A relatively homogeneous landscape characterized by extensive savanna formations crossed by forests and stream valleys. The altitude of the site ranges from 298 to 764 m. The area includes irrigated and rainfed wheat, corn, barley, rice, and soybeans fields, orchards, natural sand banks, savannahs, open pastures, and bushes. The climate is classified as subtropical and tropical with an annual mean temperature of 24 °C and annual mean precipitation of 1471 mm [36].

- Site 2–Podebrady, Czech Republic

Lying in a temperate climate zone with annual precipitation of about 660 mm, this site is characterized by mild and humid summers with an average annual temperature of 8.8 °C. The altitude varies from 146 to 1,385 m. This site contains mixed evergreen and broadleaves forests, urban areas, croplands, pastures, orchards, wetlands, and water bodies. Most agricultural fields are planted with winter wheat, barley, sugar beet, rapeseed, and maize.

- Site 3–Iowa-Nebraska border near Omaha, Nebraska, USA

A relatively flat plain with an average altitude of 310 m above mean sea level (a.m.s.l) dominated by croplands, mainly corn and soybean fields (93%), with a minority of sorghum, oats, and wheat. Small areas of forest, grassland, riparian vegetation, and urban regions occur within the scene. The area is characterized by a humid continental climate, with an annual precipitation rate of 700 mm and an annual temperature of 10.5 °C.

- Sites 4 and 5 are located near Sacramento, California, USA

These areas are predominantly agricultural landscapes with mainly irrigated rice fields. Other common crops are grapes, citrus, almonds, and lettuce. A small fraction of the area comprises grasslands, hay fields, and cattle ranches. The climate in the region is Mediterranean with average annual high and low temperatures of 24.2 °C and 9.2 °C, respectively, and average precipitation of 485 mm that falls predominantly during the winter season (November–March). The altitude reaches a maximum of about 2,230 m. Site five has a 15-minute gap between Landsat 8 and Sentinel-3 imagery, while images were extracted at the same time at Site 4.

- Site 6–Madhya Pradesh District, India

This is a fertile land region with the main crop production of wheat, pulses, rice, oilseeds, sorghum, soybean, and sugarcane. The average field size is 0.5 x 0.8 km, which is the smallest among all sites, with the most diverse land cover. Other land covers found in the scene are ponds, grasslands, savannas, hills, urban areas, and bushes. The climate is humid subtropical, with hot and dry summers and cold and wet winters. The annual rainfall of the region is 1218 mm with an average annual high and low temperatures of 33 °C and 18 °C, respectively. The altitude of the region varies from 312–970 m.

All scenes correspond to entire tiles of Landsat 8. The outlines in Panels b and e in Figure 2 delineate the analyzed areas for Site 2 and Site 5 since Sentinel-3 did not cover the whole Landsat scene (Site 2) and to avoid clouds (Sites 5).

Sites 1 and 2, for which Sentinel-3, Sentinel-2, and Landsat 8 overlapped (Figure 3), were used to perform the inter-calibration of NDVI maps (Stage 1, Figure 1), as well as to apply TsHARP (Stage 2, Figure 1) and to validate the sharpening results (Stage 3, Figure 1).

2.4. Data

Thermal images from Landsat 8 OLI and Sentinel-3 Sea and Land Surface Temperature Radiometer (SLSTR) were collected in June–October 2017. Landsat 8 OLI images containing VNIR and shortwave infrared (SWIR) with 30-m spatial resolutions were downloaded from the official USGS website (http://earthexplorer.gov.by). Landsat 8’s native thermal resolution is 100 m, but only the 30-m images disaggregated by the Landsat Science Team were available publicly. The Sentinel-3 SLSTR products were provided by the Scientific Sentinel Hub (http://scihub.copernicus.eu/). They consist of two thermal bands at a 1,000-m spatial resolution in addition to the VNIR bands at a 500-m spatial resolution.

The Sentinel-2A data are composed of 13 spectral bands in the VNIR region resolutions ranging from 10 to 60 m. The Sentinel-2 tile has a small fixed size of 100 x 100 km, so the selection of satellite images was limited to the overlapping areas of the Landsat and Sentinel overpasses.

2.5. Data Processing

Landsat 8 images were calibrated to top-of-atmosphere reflectance and brightness temperature using the Landsat Ecosystem Disturbance Adaptive Processing System [37]. Then, the reflectance maps were atmospherically and geometrically corrected using the ATCOR module included in the ERDAS IMAGINE software [38]. The atmospherically corrected red and near-infrared bands were used to calculate NDVI. Water and artificial pixels were masked out using supervised classification since they do not conform to the NDVI–BT relationship. Landsat 8 BT images were aggregated to 960 m, which is the closest multiple of 30 m to the 1,000-m resolution of the Sentinel-3 BT images. These images were used to sharpen Landsat-to-Landsat to eliminate errors stemming from the differences between sensors. In addition, Landsat 8 BT images were aggregated to 60, 120, and 240 m. All three were used for validating the sharpening into these three resolutions to test the accuracy dependency on the target resolution. Aggregation was conducted after converting BT to radiance, according to the Stefan-Boltzmann Law, and converted it back to BT at the coarse resolution. Lastly, Sentinel-3 BT images were resampled to 960 m, to match the aggregated resolution of Landsat 8. The Sentinel-3 BT images were re-projected and geo-registered to the same reference system of the UTM Zone to match Landsat 8 thermal imagery. The Sentinel-2A dataset was atmospherically corrected with sen2cor software [39]. Band 8a (near-infrared, 20-m pixel size) and band 4 (Red, 10-m pixel size) were used to calculate NDVI.

2.6. TsHARP Performance Assessment

To evaluate the sharpening accuracy of Sentinel-3 BT images (BT_sentinel3) to a finer resolution based on VNIR data acquired by Sentinel-2, fine-resolution BT images acquired by Landsat 8 (BT_landsat) were used. To evaluate the uncertainty caused by comparing results from different sensors, a two-step analysis was performed. First, BT_landsat was aggregated to the coarse resolution of BT_sentinel3 on which the sharpening algorithms were applied. The sharpened images were then compared to the original BT_landsat. After that, BT_sentinel3 were sharpened and compared to BT_landsat. This allowed the isolation of errors that were solely due to differences between the two sensors, which are irrespective of the sharpening procedure.

The level of agreement between the reference and the sharpened temperatures was estimated by root-mean-square error (RMSE), mean absolute error (MAE), bias, the coefficient of determination (R²), and the normalized root-mean-square error (nRMSE); Equations (5)–(9), respectively [40].

$$\mathrm{RMSE}=\lfloor {\mathrm{n}}^{-1}{\displaystyle \sum }_{\mathrm{i}=1}^{\mathrm{n}}{\left({\mathrm{T}}_{\mathrm{sharp}}-{\mathrm{T}}_{\mathrm{ref}}\right)}^2{\rfloor }^{1/2}$$

(5)

$$\mathrm{MAE}=\lfloor {\mathrm{n}}^{-1}{\displaystyle \sum }_{\mathrm{i}=1}^{\mathrm{n}}\left({\mathrm{T}}_{\mathrm{sharp}}-{\mathrm{T}}_{\mathrm{ref}}\right)\rfloor$$

(6)

$${\mathrm{R}}^2=1-\frac{\sum {\left({\mathrm{T}}_{\mathrm{sharp}}-{\mathrm{T}}_{\mathrm{ref}}\right)}^2}{\sum {\left({\mathrm{T}}_{\mathrm{sharp}}-{\mathrm{T}}_{\mathrm{ref}\_\mathrm{mean}}\right)}^2}$$

(7)

$$\mathrm{bias}=\frac{{\sum }_{\mathrm{i}=1}^{\mathrm{n}}\left({\mathrm{T}}_{\mathrm{sharp}}-{\mathrm{T}}_{\mathrm{ref}}\right)}{\mathrm{n}}$$

(8)

$$\mathrm{nRMSE}=\frac{\mathrm{RMSE}}{{\mathrm{T}}_{\max }-{\mathrm{T}}_{\min }}$$

(9)

where T_sharp is the sharpened temperature, T_ref and T_{ref_mean} are the reference temperature and the average of the reference temperature in the entire image, respectively, and T_max and T_min are the maximum and minimum temperature values, respectively.

3. Results and Discussion

3.1. Sensor Intercalibration of Brightness Temperature

The surface temperature rapidly changes, especially during the mid-morning. The typical overpass time of many satellites including Landsat and Sentinel. A comparison between sensors is, thus, challenging, and, unless the acquisition time is exactly simultaneous, some of the differences should be attributed to the time gap. As mentioned above, time acquisition was the primary criterion for choosing the sites for this study, and, even with this careful screening, some gaps in time acquisition were inevitable (Table 1).

Figure 4 displays the results of the correlation between the Sentinel-3 BT and the reference aggregated Landsat-8 BT images, and

Table 2. Quantitative analysis of the differences between the simulated Landsat 8 thermal data and Sentinel-3 TM data at a coarse spatial resolution. The statistics include: the regression coefficients slope and intercept, coefficient of determination (R²), root mean square error (RMSE), mean absolute error (MAE), mean bias (Bias = Sentinel-3 (S3) minus Landsat 8 (L8)), and normalized RMSE (nRMSE).

Sites	Date of Image Acquisition	Acquisition Time Difference (L8–S3) (min)	Intercept (°C)	Slope	R2	RMSE	MAE	Bias	nRMSE
1	13-Jul-17	0	0.75	0.99	0.85	0.92	0.72	0.42	7.4%
2	20-Jun-17	0	1.73	0.96	0.85	1.11	0.83	0.55	4.5%
3	16-Jul-17	5	0.97	0.99	0.92	0.71	0.61	0.57	5.9%
4	16-Jul-17	0	1.60	0.98	0.96	1.28	1.0	0.71	4.2%
5	9-Jul-17	15	0.10	1.0	0.93	1.38	1.0	0.18	4.4%
6	17-Oct-17	0	1.41	0.98	0.95	1.08	0.90	0.80	5.8%

depicts the associated statistics. Strong positive correlations were found between the images with the intercepts ranging from 0.10–1.73 °C and the slopes from 0.96–1.0. The highest intercepts were detected for areas with a wide altitude range: Sites 2, 4, and 6.

RMSE and MAE ranged from 0.71–1.37 and 0.61–1.01 °C, respectively. The normalized RMSE showed that all sites had nearly the same error range with values from 4.2% to 7.4% (Table 2). The smallest errors were found for homogeneous and relatively flat regions (Sites 1 and 3), despite the time difference between image acquisitions.

The positive biases signify that the Sentinel-3 derived BTs were slightly higher than those derived by Landsat 8. The near-unity slopes and the relatively high R² suggest that, despite the uncertainties related to the calibration and different viewing geometries, the responses of both sensors are similar. The regression results also indicate that it is possible, to a reasonable degree, to calibrate the derived temperatures, i.e., “to correct” the Sentinel-3 data to better match the Landsat 8 data. The quotation marks are added to indicate that we do not intend to define which of the two sensors is more accurate. Rather, we inter-calibrate them for the purpose of testing the TsHARP algorithm.

To the best of our knowledge, the relationship between Sentinel-3 and Landsat 8 thermal bands has not yet been reported. Studies comparing land surface temperature from MODIS and Landsat platforms [41] revealed a significant difference between sensors induced by sensor measurement errors and systematic errors that have led to alterations in radiance at the sensors. Liu et al. [42] found a discrepancy between MODIS and aggregated ASTER surface temperature of about 3 °C due to the difference in the retrieval algorithm, atmospheric correction, and sensor performance. Merlin et al. [43] also reported errors between MODIS and ASTER data of about 2.4 °C. Weng et al. [44] pointed out that thermal data from different sensors at a close acquisition time can be comparable after applying pre-processing procedures, such as radiometric calibration, geometric detection, and atmospheric correction. Nevertheless, biases can be observed due to the differences in acquisition time, bandwidth, view geometry, swath width, geo-location errors, and a spectral response function [44,45]. Satellites’ altitude can also lead to temperature discrepancies [46].

Despite the relatively high R², which indicates that it is feasible to apply the inter-comparison regression coefficients and use the Landsat 8 high-resolution images to evaluate the accuracy of the sharpening approach, we conducted one additional intermediate step. To assess the accuracy of TsHARP for the study sites, the sharpening procedure was first applied to the aggregated Landsat scenes, using only one source of data. This allowed the assessment of the founding errors of the method itself, excluding additional errors due to the differences between the sensors.

3.2. TsHARP Validation

Figure 5 and Figure 6 present the results of sharpening the 960-m resolution BT images from both aggregated Landsat 8 (TsHARP_Landsat) and Sentinel-3 (TsHARP_Sentinel) to 60 m, alongside the ‘original’ coarse resolution image and the original Landsat 8 scene at 60 m, for Sites 1 and 2, respectively (see Figure 3 for the RGB and false color images of the sites). A visual qualitative assessment reveals that the spatial distribution of the reference image and disaggregated images showed a similar spatial distribution both when using a single (Landsat 8) platform and when using two platforms for the sharpening (Sentinel-2 and 3) and comparing the results to a third platform (Landsat 8). The maps were comparable in regions with high NDVI values, and the sharpened thermal maps showed more spatial detail than the coarse resolution BT imagery. However, the thermal maps obtained from a single platform demonstrated a much higher contrast between low and high temperatures compared to the reference image.

The TsHARP_Landsat and TsHARP_Sentinel images were compared with the reference BT images at a 60-m resolution by computing the error difference (reference BT minus downscaled BT). Maps of the differences between the three images (the two sharpened temperatures TsHARP_Landsat minus TsHARP_Sentinel and the Landsat reference image) for Sites 1 and 2, are presented in Figure 7 and Figure 8, respectively. In Site 1, both TsHARP_Landsat and TsHARP_Sentinel imagery underestimated the temperature over forests and dense vegetation and overestimated the temperature over sparse vegetation fields and bare soils compared with the reference temperature. Differences >3 °C were observed in 0.09% of the area (1,283 pixels out of a total of 1,357,185 pixels in the scene) for TsHARP_Landsat, and, in 7.2% of the area (98,216 pixels), the errors ranged from 1.5 to 3 °C, mostly along the borders of fields and riparian areas. This can be explained by the high correlation between BT and NDVI (Figure 7a). Agam et al. [21] reported that sharpening of homogeneously vegetated areas results in smaller errors due to the accurate fitting of the BT-NDVI regression model. For TsHARP_Sentinel errors > 3 °C were observed in 1.4% of the area (19,128 pixels) and errors ranging from 1.5-3 °C were observed in 20% of the area (270,511 pixels) (Figure 7b). This is likely due to loss of accuracy between different satellite platforms. Over this site, the Sentinel-based sharpened temperatures were mostly lower than those from TsHARP_Landsat (Figure 7c).

Sharpening over Site 2 yielded less accurate (±3 °C) temperatures for both TsHARP_Landsat and TsHARP_Sentinel. The lowest errors were found over agricultural landscapes for both platforms and sites. However, densely vegetated areas and bare soils resulted in larger errors due to the variation in NDVI values in different land covers. In this case as well, most errors were generally along the borders of fields and riparian areas.

The error statistics analysis indicates that the accuracy of the sharpening algorithm decreases as the target resolution becomes finer (

Table 3. Statistics of the disaggregated thermal maps in comparison with the referenced thermal imagery.

		Site 1			Site 2
		240 m	120 m	60 m	240 m	120 m	60 m
MAE	TsHARPLandsat	0.53	0.59	0.64	0.64	0.78	0.87
	TsHARPSentinel	0.88	0.92	0.95	0.92	1.02	1.09
R2	TsHARPLandsat	0.89	0.86	0.84	0.83	0.79	0.77
	TsHARPSentinel	0.80	0.78	0.76	0.69	0.67	0.65
RMSE	TsHARPLandsat	0.68	0.77	0.83	0.85	1.05	1.17
	TsHARPSentinel	1.12	1.17	1.20	1.20	1.34	1.45
BIAS	TsHARPLandsat	-0.003	-0.004	-0.004	0.007	0.009	0.013
	TsHARPSentinel	0.63	0.63	0.63	0.27	0.27	0.27

), i.e., that the greater the difference between the original resolution and the target resolution becomes, the greater the errors. Agam et al. [17] and Essa et al. [47] also observed a decrease in accuracy with a finer resolution due to the increasing degree of subpixel variability. RMSE for TsHARP_Landsat ranged from 0.68–0.83 °C and 0.85–1.17 °C, for Sites 1 and 2, respectively. The results of RMSE for TsHARP_Landsat are comparable with the results from Agam et al. [17] where RMSE ranged between 0.67 and 1.39 °C, depending on the date and sensor. At both sites, BT obtained from TsHARP_Sentinel was less accurate with RMSE ranging from 1.1–1.5 °C. The bias of TsHARP_Landsat was small and negative for Site 1, which signifies an overestimation of the sharpened temperature. The bias of TsHARP_Sentinel indicates that the sharpened temperature was considerably lower than the reference temperature for both sites. The MAEs for TsHARP_Landsat compared to the reference BTs were also lower than for TsHARP_Sentinel. Compared to TsHARP_Landsat, the MAE for TsHARP_Sentinel was higher by ~0.3 °C for Site 1 and by ~0.2 °C for Site 2 at 60-m spatial resolution. The R² values for TsHARP_Sentinel were always less than those estimated for Landsat-8 data.

An overall weaker performance was observed for Site 2 in comparison to Site 1 (Table 3, Figure 8). This is likely due to the weaker correlation between BT and NDVI for Site 2. The potential limitation of using NDVI-based sharpening approaches for agricultural areas was shown by Karnieli et al. [28]. The authors suggested that, at high latitude regions, where plant growth is limited by availability of energy rather than by availability of water, the NDVI-BT correlation is positive. Being located at 50 N latitude, this may feature an additional explanation to the larger errors observed for Site 2.

Most TsHARP research is based on disaggregating MODIS, ASTER, or Landsat images from TIR to VNIR resolutions. Most of these studies have tested the sharpening using the same satellite platform. Only a few of them attempted to sharpen thermal infrared images using multiple platforms [15,20,22,40,43,44]. Mukherjee et al. [25] pointed out that using TsHARP to downscale a MODIS image from 1000 m to 250 m resulted in an RMSE of about 1.59 °C and R² = 0.82, while downscaling aggregated Landsat 7 from 960 to 120 m yielded an error of 0.78 °C. Our results are consistent with studies by Bisquert et al. (2016a) for whom RMSE values ranged from ± 1.6-± 1.7 °C using one Landsat platform and ±1.8-±2.1 °C using MODIS versus Landsat satellites.

Lastly, to emphasize the advantage of TsHARP over a simple resampling, the Sentinel-3 coarse resolution images for both sites were resampled to 60 m without applying a sharpening algorithm and compared to the Landsat reference scenes (Figure 9).

The resulting lower R² (0.53 vs. 0.75 and 0.30 vs. 0.65, for Sites 1 and 2, respectively) indicates that TsHARP outperforms a simple sharpening. One should also bear in mind the possibility that the errors obtained are, in fact, smaller than the ones reported in this paper. The lack of ground measurements prevents us from determining whether the Sentinel-3 or the Landsat-8 thermal images better agree with the actual surface temperatures. Thus, it is possible that we compared the TsHARP results to a less accurate reference. This remains an open question.

4. Conclusions

The mean absolute differences between Sentinel-3 and Landsat 8 BT images were up to 1 °C in all six sites, with the Sentinel-3 temperatures being typically higher. The smallest errors were found for homogeneous and relatively flat regions despite the time difference between image acquisitions. In comparative studies, such as the one presented in this paper, it is possible to intercalibrate between these two sensors when needed.

Applying TsHARP on Sentinel-3 BT data using Sentinel-2 VNIR data resulted in mean absolute errors of ~1 °C, with errors increasing as the difference between the native and the target resolutions increases. Part of the error is attributed to the discrepancy between the BT images acquired by the two platforms. This is reflected by the smaller errors observed when sharpening was applied on synthetic-aggregated Landsat-8 BT images, with errors ranging between 0.53 and 0.87 °C. These results show the potential for sharpening Sentinel-3 BT images using Sentinel-2 VNIR data for a range of applications. Further research is needed to test additional sites and conditions, and potentially additional sharpening methods, which can be applied to the Sentinel platforms.