Harmonized Landsat and Sentinel-2 Data with Google Earth Engine

Abstract

Continuous and dense time series of satellite remote sensing data are needed for several land monitoring applications, including vegetation phenology, in-season crop assessments, and improving land use and land cover classification. Supporting such applications at medium to high spatial resolution may be challenging with a single optical satellite sensor, as the frequency of good-quality observations can be low. To optimize good-quality data availability, some studies propose harmonized databases. This work aims at developing an ‘all-in-one’ Google Earth Engine (GEE) web-based workflow to produce harmonized surface reflectance data from Landsat-7 (L7) ETM+, Landsat-8 (L8) OLI, and Sentinel-2 (S2) MSI top of atmosphere (TOA) reflectance data. Six major processing steps to generate a new source of near-daily Harmonized Landsat and Sentinel (HLS) reflectance observations at 30 m spatial resolution are proposed and described: band adjustment, atmospheric correction, cloud and cloud shadow masking, view and illumination angle adjustment, co-registration, and reprojection and resampling. The HLS is applied to six equivalent spectral bands, resulting in a surface nadir BRDF-adjusted reflectance (NBAR) time series gridded to a common pixel resolution, map projection, and spatial extent. The spectrally corresponding bands and derived Normalized Difference Vegetation Index (NDVI) were compared, and their sensor differences were quantified by regression analyses. Examples of HLS time series are presented for two potential applications: agricultural and forest phenology. The HLS product is also validated against ground measurements of NDVI, achieving very similar temporal trajectories and magnitude of values (R² = 0.98). The workflow and script presented in this work may be useful for the scientific community aiming at taking advantage of multi-sensor harmonized time series of optical data.

1. Introduction

Continuous and dense time series of satellite remote sensing data are needed for several land monitoring applications, including vegetation phenology [1], fast crop cycles [2], and improving land use and land cover classification [3]. Supporting such applications at medium to high spatial resolution may be challenging with a single optical satellite sensor, as the frequency of good-quality observations usually is low [4]. With the increasing number of Earth observation satellites providing free imagery archives at medium spatial resolution, combining multiple satellite observations into a single dataset can significantly enhance temporal resolutions, improving our ability to detect fine-grained, subtle landscape change patterns across large areas. This, in turn, has become feasible due to advances in affordable data storage and the increased availability of cloud computing structures, such as the NASA Earth Exchange computing facility [5], Google Earth Engine (e.g., [6]), and specialized data cubes software (e.g., [7]).

Landsat and Sentinel-2 data represent the most widely accessible 10–30 m multi-spectral satellite measurements [8]. In combination, data from the currently operational Landsat-7 ETM+, Landsat-8 OLI, and Sentinel-2 MSI can provide global observations with a better than 3-day revisit [9,10]. This potential for synergistic use is due to their similarities in terms of spectral, spatial, and angular characteristics [5,10,11,12,13]. In fact, the spectral and spatial configurations of the S2 MSI sensors were designed to correspond to analogous bands in the Landsat (and SPOT) series of sensors [14]. Nevertheless, there are also a number of differences between these sensor data that should be addressed before the data can be reliably used together [10,11]. Through a process called harmonization, these differences can be mitigated to generate time series data as though they came from a single sensor or from a virtual constellation [15]. Zhang et al. [10] describe harmonization as the process of transforming the data from one sensor so that they are more similar to the other sensor’s data.

Significant and continuous efforts have been made to develop methodologies for the harmonization of Landsat and Sentinel-2 data [5] as well as commercial sensors like the PlanetScope constellation [16]. The NASA Harmonized Landsat and Sentinel (HLS) product integrates L8 OLI and S2 MSI data to deliver a consistent dataset of surface reflectance at a 30 m scale, with a temporal frequency of 2–4 days [5], and PlanetScope tries to match Sentinel MSI data. The harmonization workflow includes atmospheric correction, cloud/shadow masking, geometric resampling, spatial co-registration, corrections for bandpass difference, and bidirectional reflectance distribution function (BRDF) effects [5]. Although the NASA HLS product can be used with confidence in some parts of the world (e.g., in North America [17]), “HLS is still a research product” [18], and the global coverage HLS product (version 2 at the time of writing) still needs better understanding as some technical challenges remain [19], including improving validation and sorting the ‘known issues’ (~20 reported at the time of writing [20,21]). Another challenge is the requirement to download the harmonized data, which can become increasingly difficult with larger volumes, potentially limiting its widespread use. There is an ongoing effort to alleviate this issue by integrating HLS into the GEE catalogue. However, currently, GEE offers only the Landsat component (HLSL30) (the sentinel component HLSS30 is produced separately), meaning that NASA HLS is not fully operational within GEE. Also, NASA HLS does not harmonize Landsat 7 data, which may be important to increase a series density of cloud-free satellite observations, especially prior to the launch of Sentinel 2 B in 2016.

Working on the same baseline principles as the NASA HLS initiative, a few other projects have been advancing the Landsat and Sentinel-2 data integration mainly through developing new tools. Frantz [7] developed the FORCE (Framework for Operational Radiometric Correction for Environmental Monitoring), a freely available software that can produce harmonized Level-2 (and beyond) analysis ready data (ARD) from Landsat and Sentinel-2 archives; a challenge to use this software may be handling the considerable data volume that can be generated (e.g., downloading and storing input images, saving outputs to disc). The “sen2like” tool, proposed by ESA, has been designed to produce L8 and S2 harmonized Level-2 surface reflectance data at 10 m of resolution, primarily aiming at improving agricultural monitoring capabilities [22]; Sen2like is currently in a pre-operational phase [23]. Nguyen et. al [24] proposed a stream processing for the generation of harmonized L7, L8, and S2 at 30 m spatial resolution but using mostly the Google Earth Engine (GEE) cloud platform and its built-in functions; however, unlike our study, the atmospheric correction is conducted outside the GEE web-based code editor, demanding the installation of software and storing of the corrected data, which can be prohibitive with large datasets.

Overall, these initiatives aim at turning Earth Observation (EO) data consistently and systematically into valuable information layers, which is an ongoing challenge for the EO community [25] with the ever-increasing number of available EO data.

Emphasizing web-based processing instead of download-driven approaches is a current trend and challenge in the field of big EO data [25]. Google Earth Engine (GEE) has become a popular cloud-based platform solution, offering a web-based integrated programming environment and an API for pixel-based image processing, enabling local-to-planetary scale studies without the need for extensive data management or in-house high-performance computing infrastructure [26]. In GEE, the most common archives of remote sensing data (e.g., Landsat and Sentinel) are available and can be processed without requiring download. GEE’s ability to handle multi-petabyte datasets and provide rapid data analysis and visualization has made it a valuable tool in EO research [27]. Also, the GEE code can be easily shared, allowing the reproducibility of methodologies and results.

We aimed to develop an ‘all-in-one’ GEE web-based workflow to produce harmonized surface reflectance data from the Landsat-7 Enhanced Thematic Mapper+ (L7 ETM+), Landsat-8 Operational Land Imager (L8 OLI), and Sentinel-2 MultiSpectral Instrument (S2 MSI) top of atmosphere (TOA) reflectance data. To the best of our knowledge, this is the first HLS product that is generated fully within the GEE. We propose and describe six major processing steps to generate a new source of near-daily HLS reflectance observations at 30 m resolution: (i) band adjustment; (ii) atmospheric correction—here lies a major advancement from similar products; (iii) cloud/cloud shadow masking; (iv) view and illumination angle adjustment; (v) co-registration; and (vi) reproject and resample. Our GEE HLS is validated against ground reference data and intercompared with existing NASA HLS. The proposed GEE workflow is applied at two different regions to test the transferability of the methodology and to show potential applications of HLS series of data.

2. Materials and Methods

Although L7, L8, and S2 data are generally similar, building harmonized surface reflectance data requires efforts to mitigate sensor differences. We chose to create HLS data for this study using GEE because it provides the ability to sort, filter, and mask clouds and cloud shadows, apply radiometric corrections, and analyze the entire data records of L7, L8, and S2 as well as ancillary MODIS data in an efficient manner without requiring the downloading and storing of large volume of image archives [11], all while using a user-friendly JavaScript coding editor.

2.1. Harmonizing Sentinel-2, Landsat-7, and Landsat-8 Imagery

2.1.1. Workflow Overview and Test Site

The workflow uses recent research on L7, L8, and S2 pre-processing to generate registered, surface nadir BRDF-adjusted reflectance (NBAR) sensor time series at a 30 m spatial scale. The processing flowchart (Figure 1) starts with orthorectified TOA reflectance data from L7 (L1T), L8 (L1T), and S2 (L1C) and generates three products:

• L7-30: ETM+ harmonized surface reflectance resampled to 30 m into the Sentinel-2 tiling system, and adjusted to Sentinel-2 spectral bands.
• L8-30: OLI harmonized surface reflectance resampled to 30 m into the Sentinel-2 tiling system, and adjusted to Sentinel-2 spectral bands.
• S2-30: MSI harmonized surface reflectance resampled to 30 m into the Sentinel-2 tiling system.

Because the L7-30, L8-30, and S2-30 products are gridded to the same resolution and projection system, they are “stackable” (data cube) for time series analysis.

The GEE’s JavaScript code is available as a GEE snapshot (https://code.earthengine.google.com/934b2252c5403a1b3af9a4a0ccb4c9e1, accessed on 22 July 2024) and also at Github (https://github.com/eliasberra/HLS_with_GEE.git, accessed on 22 July 2024).

Specifically, following [5], by “harmonized” it means that the products are as follows:

• Gridded to a common pixel resolution, map projection, and spatial extent.
• Atmospherically corrected to surface reflectance using a common radiative transfer algorithm.
• Normalized to a common nadir view geometry via bidirectional reflectance distribution function (BRDF) estimation.
• Adjusted to represent the response from common spectral bandpasses.

The proposed GEE workflow is applied in two different regions to test the transferability of the methodology and to show other potential applications of HLS series of data (Figure 2). The S2 tiles (S2 scene footprint) are defined as the region of interest to apply our HLS methodology. Firstly, an S2 footprint in southern Brazil was chosen because it intersects ground NDVI data over an agricultural field, data that are used for validation. The second area encompasses an S2 footprint in the northeast UK, from where a well-studied woodland site [4] is selected as an example application. Details of the two sites are given below. Our workflow is applicable worldwide, wherever L7, L8, and S2 data are available (and MODIS, as detailed below).

2.1.2. Satellite Data

The input data in the workflow are the top of atmosphere (TOA) reflectance satellite data, which are readily available in the Earth Engine Data Catalogue and are identified as follows:

• L7: “USGS Landsat 7 Collection 2 Tier 1 TOA Reflectance”.
• L8: “USGS Landsat 8 Collection 2 Tier 1 TOA Reflectance”.
• S2: “Harmonized Sentinel-2 MSI: MultiSpectral Instrument, Level-1C”.

Table 1. Input sensor specifications [28,29,30,31].

	L7 ETM+	L8 OLI	S2 MSI
Radiometric resolution	8-bit	12-bit	12-bit
Equatorial crossing time	10:00–10:15	10:00–10:15	10:30
Swath/field of view	183 km/15° (±7.5°)	183 km/15° (±7.5°)	290 km/20.6° (±10.3°)
Orbit altitude	705 km	705 km	786 km
Revisit frequency	16 days	16 days	4–5 days
Launch date	15 April 1999	11 February 2013	S2-A: 23 June 2015 S2-B: 7 March 2017

summarizes the overall characteristics of the three satellites and their sensors and

Table 2. Characteristics of the approximately equivalent Landsat-7 ETM+, Landsat-8 OLI, and Sentinel-2 MSI spectral bands from which the harmonization was applied in this study. The original spatial resolution and central wavelength are given in between brackets [28,29,30,31].

HLS Equivalent Name	Original Band Number [Spatial Resolution, Central Wavelength 1]
	L7 ETM+	L8 OLI	S2 MSI
Blue	B1 [30 m, 483 nm]	B2 [30 m, 482 nm]	B2 [10 m, 497 nm]
Green	B2 [30 m, 560 nm]	B3 [30 m, 561 nm]	B3 [10 m, 560 nm]
Red	B3 [30 m, 662 nm]	B4 [30 m, 655 nm]	B4 [10 m, 664 nm]
NIR	B4 [30 m, 835 nm]	B5 [30 m, 865 nm]	B8 [10 m, 843 nm]
SWIR1	B5 [30 m, 1641 nm]	B6 [30 m, 1609 nm]	B11 [20 m, 1613 nm]
SWIR2	B7 [30 m, 2206 nm]	B7 [30 m, 2201 nm]	B12 [20 m, 2200 nm]

¹ Nominal center wavelength at full width half maximum (FWHM).

summarizes the equivalent bands used in this study. The L7, L8, and S2 satellites are in circular sun-synchronous polar orbits. The L7 satellite is equipped with the ETM+ sensor, which is a fixed whisk-broom multispectral scanning radiometer with seven reflective wavebands (450 nm to 2350 nm) [28]. The L8 has the multispectral OLI push-broom sensor which has nine reflective wavebands (435 nm to 2200 nm) [29]. The same characteristics can be associated with Landsat 9 OLI-2, to be implemented in the future. The S2 mission is composed of two identical satellites, each carrying an MSI push-broom sensor capturing 13 reflective wavebands (443 nm to 2190 nm) [14] at different spatial resolutions; combined, the two out-of-phase S2 satellites provide a nominal revisit time of five days (at the equator) [30]. A more detailed description and intercomparison of these three sensors can be found in [11].

The six approximately equivalent spectral bands between ETM+, OLI, and MSI were used in the HLS workflow: blue, green, red, near infrared (NIR), shortwave-infrared 1 (SWIR1), and shortwave-infrared 2 (SWIR2); the key characteristics are summarized in Table 2. Despite the data gaps in the ETM+ imagery due to the failure of its scan-line corrector (SLC) in 2003 [32], keeping this series of satellite data may be important to increase the series density, especially across areas with frequent cloud cover and prior to launching Sentinel-2.

2.1.3. Inter-Sensor Band Adjustment

The small differences between the equivalent spectral bands can be adjusted if their relationships are known. In this study, setting the S2 MSI as a reference, we adjusted the L7 and L8 TOA reflectance values across the six equivalent bands (blue, green, red, NIR, SWIR1, and SWIR2) using the cross-sensor transformation coefficients derived from [11] (

Table 3. Cross-sensor transformation coefficients. Sentinel-2 MSI TOA reflectance as a function of Landsat-7 (L7) ETM+ TOA reflectance and Landsat-8 (L8) OLI TOA reflectance, according to [11].

HLS Equivalent Name	L8 Intercept	L8 Slope	L7 Intercept	L7 Slope
Blue	−0.0107	1.0946	−0.0139	1.1060
Green	0.0026	1.0043	0.0041	0.9909
Red	−0.0015	1.0524	−0.0024	1.0568
NIR	0.0033	0.8954	−0.0076	1.0045
SWIR1	0.0065	1.0049	0.0041	1.0361
SWIR2	0.0046	1.0002	0.0086	1.0401

). Briefly, using over 10,000 image pairs across the conterminous United States, [11] calculated the cross-sensor transformation equations for the six bands, as presented in Table 3.

2.1.4. Atmospheric Correction

The Landsat and Sentinel TOA reflectance data were corrected to surface reflectance using the same algorithm to reduce biases that may occur if different algorithms were used [10]. The Sensor Invariant Atmospheric Correction (SIAC) method [33], which has been adapted to the GEE JavaScript language, was used in this study. A detailed description of SIAC is given in [33], including the results of surface reflectance validation for Landsat-8 and Sentinel-2; SIAC has also been successfully tested in [34]. The basic idea of the SIAC method is to use the coarse-resolution spectral BRDF (500 m MODIS BRDF) to provide prior surface reflectance estimation and the Copernicus Atmospheric Monitoring Service (CAMS) 4.4 km data as a prior estimate of atmospheric composition [34]. The surface reflectance is then calculated from the TOA reflectance with the solved atmospheric parameters. Overall, SIAC is a sensor-agnostic approach to atmospheric correction but can only be applied from 2000 onwards, when MODIS data became available. Due to the lack of non-linear solver available in the existing GEE built-in functions, the Aerosol Optical Depth (AOD) estimation is derived from the MCD19A2.061 product, which has been validated to have an accuracy of R = 0.84 and RMSE of 0.12 [35]. Here lies a major difference from the work of [24], who also proposed a method to harmonize L7, L8, and S2 imagery. While we implement a single block of JavaScript code all within the GEE web-based code editor, Nguyen [24]’s workflow performs atmospheric correction (using a different algorithm) outside the GEE environment via Python API and Docker container, a step which may bring extra challenges for the user as it is necessary to install the software and download/upload imagery from/to GEE, therefore diminishing the effectiveness of GEE in terms of being a web-based approach. For this study, all image correction steps were performed using the GEE platform which does not require any external tools to create a streamlined processing pipeline.

Another difference regards the AOD. The algorithm used in [24] assumes that the AOD does not change over the entire region of interest, which may not be valid if a processing is over S2 100 km × 100 km image swath. It uses AOD from MOD08 AOD 550 (‘MODIS/006/MOD08_M3’), which is a monthly mean of the MODIS daily AOD product and does not represent the daily variation of AOD. To enhance these methods, we use the daily MCD19A2.061 product to retrieve AOD, which is a different source, but according to the validation, it produces a high level of accuracy when compared to ground measurements [35] at a higher spatial resolution, 1 km compared to the MOD08’s 1 × 1 degree grid.

2.1.5. Cloud and Cloud Shadow Masking

For L7 and L8, cloud and cloud shadow were identified and masked using the per-pixel bit-packed Quality Assessment Band (BQA) that accompanies the Landsat Collection 1 Level-1 product [36]. BQA is generated with the C Function of Mask (CFMask) algorithm, which is described in [37,38]. Pixels labeled as medium (34–66%) or high (67–100%) confidence cloud, cloud shadow, cirrus, or snow were masked.

For S2, clouds and cloud shadows are identified and masked using the recently available ‘Cloud Score+ S2_HARMONIZED’ product. This dataset includes two QA bands and is produced from the harmonized Sentinel-2 L1C Collection. Cloud Score+ outputs can be used to identify relatively clear pixels and remove clouds and cloud shadows from S2 L1C or S2 L2A imagery. In this study, we used the ‘cs’ QA band, values which are based on a spectral distance between the observed pixel and a (theoretical) clear reference observation [39]. The threshold for masking was set to 0.60, as values between 0.50 and 0.65 generally work well. An overview of the effect of applying the cloud and cloud shadow masking for S2 can be seen in the Appendix A (Figure A1).

2.1.6. View and Illumination Angles (BRDF) Adjustment

The Sentinel-2 and Landsat satellites acquire images at view angles ±10.3° (Sentinel-2) and ±7.5° (Landsat) from nadir that cause small directional effects over non-Lambertian surfaces [40]. Such directional reflectance effects, commonly described by the bidirectional reflectance distribution function (BRDF) (units of sr⁻¹), should be minimized to provide consistent data in applications such as the detection of surface change through space and/or time [40,41]. The BRDF correction of our HLS product was made using the MODIS-based c-factor approach [40,41], because it has been successfully evaluated with L5 TM and L7 ETM+ [41], S2 MSI [40], and S2 and L8 [10] data, and is recommended for view zenith angles near nadir [5]. The technique is deemed conservative, as it under-corrects BRDF effects [42]. The global BRDF coefficients for the isotropic, volumetric, and geometric kernels (definitions in [43]) are shown in

Table 4. Global 12-month fixed MODIS spectral BRDF model parameters f_iso(λ), f_geo(λ), and f_vol(λ) used for the c-factor approach [41]. Adapted from [40,41].

MODIS Band	fiso	fgeo	fvol	HLS Equivalent Name
3 (blue, 469 nm)	0.0774	0.0079	0.0372	Blue
4 (green, 555 nm)	0.1306	0.0178	0.058	Green
1 (red, 645 nm)	0.169	0.0227	0.0574	Red
2 (NIR, 858 nm)	0.3093	0.033	0.1535	NIR
6 (SWIR, 1641 nm)	0.343	0.0453	0.1154	SWIR1
7 (SWIR, 2130 nm)	0.2658	0.0387	0.0639	SWIR2

. The technique can only be applied on the L7, L8, and S2 bands equivalent to MODIS ones.

The bottom of the atmosphere (BOA) reflectance data are normalized for per-pixel view and per-tile illumination angles, which results in a nadir BRDF-adjusted reflectance (NBAR) product. The view angle (related to the sensor’s FOV) is set to nadir (0° view zenith) and the illumination geometry (zenith and azimuth) is defined based on the tile central latitude (see [5] for the equations). In summary, the NBAR products are surface reflectance normalized for the view and illumination angles. The BRDF correction GEE script was adapted from [24,44].

2.1.7. Spatial Co-Registration between Landsat and Sentinel-2

The objective of our HLS product, and in accordance with [5], is to maintain the geodetic accuracy requirement of the Sentinel-2 images (<20 m error, 2σ) and improve the multi-temporal co-registration between Sentinel-2 and Landsat images (<15 m, 2σ) for the 30 m products. The differences in the geometric reference datasets (among other factors, e.g., different viewing geometries [45]) used to create the S2 and Landsat products lead to residual misregistration between S2 and Landsat data products [46], which may demand a refining of the S2-Landsat registration for multi-sensor time series purposes [47].

The S2 dataset available in the GEE has different geolocation uncertainties as different geometric references are currently used in the series (at the time of writing). Since 30 March 2021 (processing baseline > 03.00), a geometric refinement is applied based on the S2 Global Reference Image (GRI), where preliminary results show an absolute geolocation better than 6 m and a multi-temporal co-registration better than 5 m at 95% confidence [48]. This is notably different from that of the so-called ‘unrefined products’ (no GRI; processing baseline < 03.00), where the absolute geolocation is close to 11 m (at 95%) and the multi-temporal registration is around 12 m (at 95%) for both S2 satellites, in accordance with previous studies [49]. Ongoing efforts are on the way to apply the geometric refinement with GRI to the entire S2 Collection [50].

Both Landsat Collection-1 and Collection-2 products are available in GEE. The Collection-2 data have significant geometric accuracy improvements due to the re-baselining of the Landsat-8 OLI GCPs to the European Space Agency’s Copernicus Sentinel-2 GRI [51]. The independent validation of the Landsat products processed using the GRI dataset indicated a global misregistration error of <8 m (CE90) (Collection-2), an improvement from the 25 m prior to the GRI (Collection-1) [47]. A more recent report calculated a global average registration accuracy of <26 m (CE90) for Landsat Collection-1 data [52]. The Landsat Level-1 (TOA reflectance) is available in both Landsat Collection-1 and Collection-2 within the GEE.

This work tests Landsat Collection-2 data and S2 data with processing baseline prior to 03.00, but the GEE code can be easily adapted to process Collection-1 and/or baseline > 03.00. This means that S2 and Landsat Level-1 products may not align to sub-pixel precision (depending on the Collection/baseline combinations), and may reach misregistration errors of 36–38 m (2σ) in some parts of the globe [5,46]. Therefore, a method to improve the registration between the S2 and Landsat data products is necessary.

In the GEE, the ‘displacement’ function was used to calculate the displacement between a pair of overlapped images (S2/L7 or S2/L8) which were captured at (approximately) the same time over our selected study areas (

Table 5. Pairs of overlapping Sentinel and Landsat imagery, which are cloud-free and acquired on the same date (±2 days) over the study area.

	Pair	Acquisition Time (GMT 1)	Image ID
Brazil	L8-S2	6 August 2019 13:24	‘LANDSAT/LC08/C01/T1_TOA/LC08_222080_20190806’
		6 August 2019 13:40	‘COPERNICUS/S2/20190806T133231_20190806T133228_T22JCP’
	L7-S2	18 November 2019 13:07	‘LANDSAT/LE07/C01/T1_TOA/LE07_222080_20191118’
		19 November 2019 13:40	‘COPERNICUS/S2/20191119T133219_20191119T133222_T22JCP’
UK	L8-S2	20 April 2020 11:09	‘LANDSAT/LC08/C01/T1_TOA/LC08_204021_20200420’
		19 April 2020 11:25	‘COPERNICUS/S2/20200419T112111_20200419T112443_T30UWG’
	L7-S2	21 April 2020 10:38	‘LANDSAT/LE07/C01/T1_TOA/LE07_203022_20200421’
		19 April 2020 11:25	‘COPERNICUS/S2/20200419T112111_20200419T112443_T30UWG’

¹ GMT = Greenwich Mean Time.

). More specifically, the displacement was calculated between pairs of equivalent NIR bands (Table 2). Then, the ‘displace’ function was used to warp and co-register the L7/L8 image to match an S2 (reference) image. Because the S2 absolute geodetic accuracy is better than L7/L8 [48], we aligned all Landsat images using a common base S2. The GEE documentation provides further explanation about the ‘displacement’ and ‘displace’ algorithms; complementarily, Nguyen [24] also provides an interpretation on this.

2.1.8. Reproject and Resample

The objective of this step is to put the L7, L8, and S2 data into a common coordinate grid. Our HLS has adopted the tiling system used by the Sentinel-2 Level-1C products, which are composed of granules (also called tiles) of 100 × 100 km ortho-images in the UTM/WGS84 map projection [30]. The S2 tiles (scene footprint; see Figure 2) are defined as the region of interest to apply our HLS methodology.

Because the L7/L8 and S2 data are defined in different tiling schemes with different map projections (UTM zones), the S2 images were reprojected and rescaled (via nearest neighbor) according to the L8/L7 30 m grid (UTM/WGS84), as the HLS result has a 30 m spatial resolution.

2.2. Descriptive and Statistical Analysis

In order to understand the effect of applying our methodology in the satellite data, an analysis was performed in two ways: (i) cross-comparing the TOA reflectance (input data) versus HLS (output) values for each of the equivalent spectral bands, and derived NDVI; (ii) focusing on the red, NIR and NDVI, a detailed cross-comparison of the effect of applying each processing step (Figure 1) in the reflectance and vegetation index values was made. This analysis focuses on comparing L8 × S2, as they are likely of the most relevance for the EO community, but examples are also produced for L7 × S2 (Appendix A). Comparing L8 × L7 was not reliable as their overlapping scenes (one day apart) fall into adjacent paths (border of the images); in addition, the gaps of ETM+ SLC-off images are noticeable along the edge. This resulted in a reduced number of L8 × L7 sampled pixels.

To sample the full range of potential spectral values across the study region (a S2 footprint), a simple random sample of point locations was used. We assembled the images into pairs of observations (S2 × L7, S2 × L8, L7 × L8) that were recorded within 1 day of a different platform-sensor image (e.g., for S2 × L8: an S2 image is paired with an L8 image, only if both images are acquired within ±1 day of each other) to minimize the impact of surface changes between the sensors’ observations. The pixel values that intersected the sample point location, for each of these pairs, were then extracted for the period of 2016–2020; this interval is to restrict the analysis to the S2 processing baseline prior to 03.00 (30 March 2021). These summed up five complete calendar years in which OLI, ETM+, and MSI data were compared. An example of a random sample of point locations can be seen in the Appendix A (Figure A2).

The extracted pixel values were shown on scatterplots. The degree of correspondence between the comparable Sentinel-2 and Landsat (7/8) sensor data was examined by ordinary least square (OLS) regression. The goodness of fit of the OLS regressions was defined by the coefficient of determination (R²). The root mean square difference (RMSD) between Sentinel-2 and Landsat (7 or 8) data was calculated as follows:

$$RMSD=\sqrt{{\displaystyle \frac{{\sum }_i^n{(v_i^{Sentinel}-v_i^{Landsat})}^2}{n}}}$$

(1)

where RMSD is the root mean square difference between corresponding Sentinel-2 MSI ($v_i^{Sentinel}$) and Landsat ($v_i^{Landsat}$) reflectance or NDVI values at every ith observation, out of n observations.

2.3. Ground Reference Data

The performance of our HLS surface reflectance product was evaluated by cross-comparing its HLS series values against independent in situ NDVI data at an agricultural field in southern Brazil (−28.228550°, −52.905090°). Ground NDVI measurements were retrieved from a couple of Spectral Reflectance Sensors (SRS—Decagon Devices, Inc., Pullman, WA, USA) (Figure 3), as described and validated in [53,54]. Briefly, the NDVI sensors are of two types. While the incident radiation was measured with an NDVI-hemispherical (up looking), the surface-reflected radiance was measured by an NDVI field stop (down looking). Both NDVI sensors measure the red and near-infrared spectra. The sensors provide continuous monitoring with recordings every 15 minutes, data which were stored with a CR1000 datalogger (Campbell Scientific, Inc., Logan, UT, USA). These NDVI sensors were installed in pairs at a height of 1 m above the soybean canopy.

2.4. Application Examples and Comparison with NASA HLS

The processing power of the developed framework is exemplified with four potential applications (four different land cover types) across the two test sites (Figure 2). Two examples are from southern Brazil: (i) an agricultural field to illustrate a crop monitoring application (same site where the ground NDVI data were available); and (ii) a wetland to illustrate a dynamic monitoring of wetlands and inundations (−28.49785°, −52.55475°). The other examples come from the northeast UK: (iii) a woodland site with deciduous-dominated tree species to illustrate a forest phenology monitoring (55.22055°, −1.69610°); and, (iv) a grassland to illustrate the temporal patterns of grassland dynamics (54.98799°, −2.60930°).

At these four sites, the HLS time series examples were extracted and graphs were produced. Our GEE-generated HLS was also intercompared with the existing NASA HLS data [20,21] to evaluate their degree of agreement in terms of magnitude of values and temporal trajectories. The NASA HLS product was downloaded using the Application for Extracting and Exploring Analysis Ready Samples (AppEEARS) [55], which is a web-based application that allows extracting the NASA HLS point samples of geographic coordinates. Only harmonized observations flagged without clouds (‘Fmask_Cloud_Description’ = No) were used in the comparisons with our HLS. It is important to remember that the NASA HLS does not harmonize Landsat 7 data.

3. Results

The cross-comparison of Landsat-8 vs. Sentinel-2 equivalent spectral bands and derived NDVI values, before (input) and after (output) applying the HLS method, is shown in Figure 4, while Figure 5 shows how the HLS workflow affects the red, NIR, and NDVI values for each processing step. A similar comparison for Landsat-7 vs. Sentinel-2 can be seen in the Appendix A (Figure A3 and Figure A4).

A shift towards smaller reflectance values and increasing in the data range occur particularly and most noticeably in the shortest wavelengths (blue, green, and red; Figure 4). This has been seen in similar studies [10,56] and is expected as the atmospherically corrected results compensate for the additive components and aerosol absorption of the atmosphere (see ‘Atm. cor.’ in Figure 5). The lowest correlations are also seen in the shortest wavelength band (blue in Figure 4; R² = 0.65 and R² = 0.52), which is also likely due to its higher sensitivity to such atmospheric influence.

Except for the blue band, the agreement metrics (R², RMSD, and regression slope) values are about the same when comparing ‘Original’ (TOA reflectance) vs. ‘HLS’ (harmonized NBAR at 30 m) for L8 × S2 (Figure 4). The best agreements are observed with NDVI with both ‘Original’ and ‘HLS’ products, with the NDVI HLS regression slope being closer to the 1:1 line. This is expected as the normalized difference between red and NIR-retrieved surface reflectance (NDVI) tends to be less affected by changes in illumination conditions [57], BRDF effects, and other geometric and inter-sensor specific differences, than the individual bands alone [10,58].

Except for the blue and green band, all of the OLS regressions had good fits for the original (0.81 ≤ R² ≥ 0.92) and HLS (0.79 ≤ R² ≤ 0.92) products (Figure 4). Similarly, all of the OLS regressions had good fits for red and NIR reflectance (R² ≥ 0.80) and NDVI (R² ≥ 0.92) comparisons, across the different processing steps of our workflow (Figure 5).

3.1. Comparing with Ground NDVI

The magnitudes of the ground (considered as reference) and our harmonized satellite NDVI values are all very close (R² = 0.98; N = 11; when considering only pairs of coinciding HLS and ground NDVI), following very similar temporal trajectories for the soybean plantation (not smoothed data; Figure 6). Some differences arise when comparing the two smoothed series as the satellite series contains relatively larger temporal gaps, when compared to the daily ground NDVI. Most of the data points come from Sentinel-2 (11/17), followed by Landsat-8 (4/17) and Landsat-7 (2/17).

Regarding the NASA HLS data, very similar values and temporal dynamics are observed when comparing against our HLS. Equally, NASA HLS values are in good agreement with the ground NDVI (R² = 0.99; N = 11).

3.2. Example of Applications

A longer time series example of HLS-derived NDVI time series is shown in Figure 7. The subtropical agricultural fields (Brazil) have shorter seasons, while the woodland with mixed deciduous trees (UK) and the grassland (UK) have longer seasons. The time series from the wetland (Brazil) has a pronounced drop in NDVI values due to episodic inundation events covering its vegetation. There is a noticeable increase in the number of good quality observations after Sentinel-2 became fully operational, particularly during the period of 2016–2017 (Figure A5). Figure 6 and Figure 7 show that the Landsat 7 data, despite their technical challenges, can be very useful to increase the temporal frequency of the series, particularly before the Sentinel-2 constellation (A and B) became fully operational in mid-2017 (Figure A5).

Overall, our GEE HLS and the NASA HLS time series follow a very similar temporal trajectory of NDVI data (Figure 8). The intercomparison showed a closer agreement in the sampled series from southern Brazil (crop: R² = 0.99; N = 102; wetland: R² = 0.85; N = 108) than the series from the UK (woodland: R² = 0.61; N = 80; grassland: R² = 0.75; N = 204). In terms of absolute NDVI values, the degree of correspondence varies depending on the site and time of year. A more pronounced difference is observed in the UK sites around January 2020 (wintertime). While our HLS retrieved a denser (yet noisier) time series, the NASA HLS product resulted in fewer data points. Additionally, the Landsat 7 data remain valuable during periods when high-quality L8 and S2 observations are unavailable (see Figure A6), thereby enhancing the density of the time series.

4. Discussion

In this study, we used GEE, a cloud computing platform, to develop an all-in-one web-based workflow to produce 30 m harmonized surface reflectance data from L7 ETM+, L8 OLI, and S2 MSI top of atmosphere reflectance data. The greatest benefit of our HLS product is a denser temporal resolution of optical data compared to a single satellite sensor alone. The additional temporal resolution provided by HLS products, like our workflow and other initiatives [5], can enhance observation-intensive methods in applications such as land surface phenology [1,58,59], agricultural phenology, crop monitoring, burned area mapping [42], land use and land cover mapping, and near real-time land cover change [60]. For vegetation phenology studies using medium spatial resolution satellites, this is particularly important as image availability can be significantly reduced due to cloud contamination, reducing the already lower dataset temporal resolution (compared to coarse spatial resolution), resulting in large temporal gaps and increasing prediction uncertainties of phenological metrics and trends [4,61].

The cross-comparison of HLS datasets from Landsat-8 and Sentinel-2 revealed overall good agreements between the equivalent spectral bands (except the blue band), but differences remained. In [10], the authors quantified uncertainties due only to spectral response function differences between Sentinel-2A MSI and Landsat-8 OLI, but only small disagreements were observed for reflectance across the equivalent spectral bands (RMSD < 0.015) and NDVI (RMSD = 0.0151). In [62], the authors found significant differences between MSI and ETM+ simulated images (from Airborne Prism Experiment (APEX)) across red, near infrared, and NDVI spectral values; the RMSE values ranged from greater than 8% in the red band to 2.3% in the NIR band, and an RMSE = 3.1% between NDVI values.

Our workflow would benefit from cross-sensor transformation coefficients developed specifically for our NBAR products (after BRDF correction) to diminish small non-random differences (e.g., residual calibration errors [63]) of coincident Landsat ETM+, Landsat OLI, and Sentinel MSI NBAR data. The development of such new correction factors’ coefficients for all comparable bands was beyond the scope of this study, as a systematic and geographically extensive investigation would be necessary to produce robust coefficients, as in [11] for TOA coefficients. Nevertheless, site-specific models could be developed for specific areas of study in a future version of our workflow, as in [12] and [10].

The spatial co-registration step in the GEE may be optional as both Landsat and Sentinel-2 datasets are expected to be processed using a common geometric reference in the near future: the GRI. Updated Collection 2 results (Phase 4) have yielded a Landsat/Sentinel-2 global average registration accuracy of <10 meters circular error at the 90th percentile (CE90) [52]. This substantial improvement in the Landsat/Sentinel-2 registration accuracy is expected to satisfy the geometric requirements of the analysis ready data [47].

The degree of correspondence between our HLS and NASA HLS datasets varied depending on the site and time of year. In the Brazilian subtropical land cover types (~28° S latitude), the agreements between the two HLS were good, whereas greater differences were observed in the UK land covers (~55° N latitude), mainly within the UK wintertime. The differences in atmospheric and illumination conditions are likely contributing to such differences, as the UK experiences frequent cloud cover (and its shadow) and haze [64] and a higher solar zenith angle (stronger directional and angular effects [65]). Also, the atmospheric threshold that we used in the NASA HLS seems more conservative, leading to a lower number of selected samples (Brazil: N = 191; UK: N = 202) when compared to our product (Brazil: N = 266; UK: N = 266). The combined effect of using different cloud/cloud shadows and atmospheric and BRDF correction methods between our HLS and NASA HLS workflows has also likely contributed to observed differences.

Currently, our workflow is designed to harmonize ETM+, OLI, and MSI image data only at local scales (S2 footprint) to overcome the processing limitations imposed by the GEE platform (for a comprehensive overview of GEE limitations, please see [66]). In this first version, our focus was to assemble an operational, yet scientifically sound, HLS workflow. Applying the workflow to larger extents (regional scales) is a known challenge that may be tackled in a subsequent version of the workflow. Planned improvements to the workflow are the following: (i) adding the Landsat 9 dataset in order to increase the HLS virtual constellation and keeping the HLS updated; (ii) adding a topographic correction algorithm, such as the [67] method or the c-factor method by [68]; (iii) a more comprehensive intercomparison of our NBAR product against the ground/reference reflectance from the RadCalNet sites [33] and with the existing NASA HLS data.

5. Conclusions

This paper presents a workflow for the harmonization of Landsat and Sentinel-2 (HLS) data into the GEE cloud platform. The GEE JavaScript code used to implement the HLS methodology is available and may allow the reproducibility of a harmonized product within a user-defined area of interest.

The HLS dataset results in a denser image time series at a 30-meter scale, imagery which may be useful for applications demanding the detection of rapid changes on the land surface.

With the increasing number of Earth observation satellites providing free imagery archives at medium spatial resolution, combining multiple satellite observations into a single dataset can significantly enhance temporal resolutions, improving our ability to detect fine-grained, subtle landscape change patterns.