Assessment of the Performance of the Atmospheric Correction Algorithm MAJA for Sentinel-2 Surface Reflectance Estimates

Abstract

The correction of atmospheric effects on optical remote sensing products is an essential component of Analysis Ready Data (ARD) production lines. The MAJA processor aims at providing accurate time series of surface reflectances over land for satellite missions, such as Sentinel-2, Venμs, and Landsat 8. The Centre d’Études Spatiales de la Biosphère (CESBIO) and the Centre National d’Études Spatiales (CNES) share a common effort to maintain, validate, and improve the MAJA processor, using state-of-the-art ground measurement sites, and participating in processor inter-comparisons, such as the Atmospheric Correction Intercomparison Exercise (ACIX). While contributing to the second ACIX-II Land validation exercise, it was found that the candidate MAJA dataset could not adequately be compared to the main reference dataset. MAJA reflectances were corrected for adjacency and topography effects while the reference dataset was not, excluding MAJA from a part of the performance metrics of the exercise. The first part of the following study aims at providing complementary performance assessment to ACIX-II by reprocessing MAJA surface reflectances without adjacency nor topographic correction, allowing for an un-biased full resolution comparison with the reference Sentinel-2 dataset. The second part of the study consists of validating MAJA against surface reflectance measurements time series of up to five years acquired at three automated stations. Both approaches provide extensive insights on the quality of MAJA Sentinel-2 Level 2 products.

1. Introduction

Space programs such as the Copernicus Sentinel missions are providing an ever-growing amount of Earth Observation (EO) data, allowing for the monitoring of the atmosphere, ocean, and land surfaces at unprecedented spatial and temporal resolutions. In the field of land surface observation, correcting optical remote sensing products for atmospheric effects is a prerequisite for quantitative uses of time series of satellite images, whose signal is affected by atmospheric quantities that change continuously along time and space. The purpose of the Atmospheric Correction (AC) is, therefore, to compute the radiative transfer terms, the absorption and scattering, mostly affected by water vapour and aerosols to retrieve the actual surface reflectance (SR) out of the Top-of-Atmosphere (TOA) reflectance acquired by in-orbit sensors ([1]).

This processing step is usually part of space agencies ground segment production lines, and relies on an AC processor, such as MAJA, an open-source software which performs both the cloud-screening and the atmospheric correction for high-resolution fixed-orbit satellite sensors ([2]). MAJA is operated by CNES to provide SR products for Sentinel-2, Ven$\mathsf{\mu }$s and Landsat-8 to the community through the THEIA data catalogue (https://www.theia-land.fr, accessed on 15 May 2023). Other AC processors include, e.g., ATCOR3 ([3]), FORCE ([4]), LaSRC ([5]), or sen2cor ([6]).

The surface reflectances made available by AC processor production lines, commonly identified as Level 2A Earth Observation products, address the needs of a broad community of users in demand for Analysis Ready Data (ARD), i.e., “the minimum level required to support time series analysis and data interoperability” (https://ceos.org/ard, accessed on 15 May 2023). Such data are a pre-requisite for downstream derived products describing, e.g., the land cover, the snow cover, or the state of the vegetation, which are essential inputs for thematic studies and land surface monitoring (e.g., [7,8]). The quality of these derived products is closely linked to the quality of the level 2A surface reflectances, and it is, therefore, essential to accurately quantify the uncertainties associated with the AC processor in use, raising the need for high-quality validation datasets ([9]).

Acquiring adequate high-quality in situ spectral reflectances for validation is all but straightforward. It requires both incoming and reflected radiances measurements that match the spectral sampling, spatial footprint, and the time of acquisition of the satellite product under consideration, either at the nadir if the surface Bi-Directional Reflectance Function (BRDF) is known, or at comparable angles of observation. For land surfaces, the French space agency CNES designed the Robotic Station for Atmosphere and Surface characterization protocol (ROSAS) ([10]), and maintains two observation stations that contribute to the Radiometric Calibration Network (RadCalNet, [11]). For water surfaces, ref. [12] designed the “pan-and-tilt hyperspectral radiometer system” (PANTHYR) that acquires hyper-spectral water reflectances, while [13] presented an on-going initiative to create the HYPERNETS worldwide network of reference measurements of hyperspectral surface reflectance. The main drawbacks of such protocols are their complexity and cost, which limit the number of stations deployed worldwide, and weather conditions, which limit the number of quality proofed observations. Such measurements are not without uncertainties, particularly related to the calibration of the photometer or to the homogeneity of the target, resulting in relative uncertainties of the order of less than 5% for the ROSAS protocol.

Another validation approach is to feed a state-of-the-art Radiative Transfer Model (RTM) with in situ measurements of atmospheric quantities to compute reference sensor-specific spectral surface reflectances used to validate atmospheric correction algorithms outputs ([14,15]). The significant advantage of this approach is that it can rely on a broad worldwide network measuring atmospheric vapour and aerosols content, such as the Aerosol Robotic Network (AERONET, [16]), covering a broader range of geographical and weather contexts than the aforementioned ROSAS-like protocols. However, such indirect estimates of surface reflectance are dependent on methodological assumptions, such as whether or not the surface is considered Lambertian, whether or not adjacency or topographic effects are taken into account, and of course the RTM used. The relative differences between RTMs tend to increase with a decrease in the wavelength, with differences in the visible spectrum of the order of, e.g., 3–4% between 6SV and libRadtran, and up to 4–11% between 6SV dans MODTRAN ([17]).

The later approach is the cornerstone of Atmospheric Correction Inter-comparison Exercise (ACIX). Following a first exercise ([17]), a second exercise (ACIX-II) started in 2019 to address the performance of 13 AC processors in their ability to accurately retrieve both the land surface reflectance, the total atmospheric water vapour content and the atmospheric optical depth (AOD) ([18]). While contributing to it, it was found that the candidate MAJA reflectances, corrected for adjacency and topography effects, could not adequately be compared to the ACIX reference, excluding MAJA from a part of the performance metrics of the exercise.

The motivation behind this study is to provide the readers with MAJA performances metrics that could not be published under [18], and to broaden the view of the quality of the MAJA level-2 using reference values from i. surface reflectances computed from Sentinel-2 Top-of-Atmosphere reflectance products spread over a large variety of areas worldwide, combining state-of-the-art algorithm and accurate measurements of the state of the atmosphere, which will hereafter refer to the AERONET-based approach; and ii. ground measurements of land surface reflectances acquired by ROSAS stations, with only three observation sites but with continuous time-series of up to five years.

2. Materials and Methods

2.1. The MAJA Atmospheric Correction Processor

The atmospheric correction of optical remote sensing images is based on a fine estimation of atmospheric properties, some of which are rather homogeneous in space and time when it comes to the gas mixture, while others are not, such as the water vapour and aerosol content. While the estimation of vapour content can benefit from dedicated bands centred on vapour absorption wavelengths, the estimation of atmospheric optical depth (AOD) is more difficult. Depending on the AC processors, the separation of the atmospheric and land spectra, needed to infer the composition of the atmosphere, is classically based on the “dark pixel” hypothesis ([19]) or on multi-spectral criteria, e.g., ([20]).

The MAJA processor is a cloud-screening and atmospheric correction algorithm developed by the CESBIO, CNES, and the Deutsches Zentrum für Luft- und Raumfahrt (DLR) since 2008 ([2]). This algorithm is distinctive in its ability to combine both multi-spectral and multi-temporal criteria. It is designed specifically for satellite sensors acquiring images at constant observation angle, such as Sentinel-2, Ven$\mathsf{\mu }$s, or Landsat. A coarse resolution detection of sudden changes in Top-of-Atmosphere reflectance (${\rho }_{\lambda }^{TOA}$) at wavelength $\lambda$ combined with time-dependant spectral thresholds provides reliable and highly conservative cloud masks ([21]), and benefits from a cirrus band whenever available. For Sentinel-2 MSI products, the coarse resolution is 240 m for MAJA versions 3.x, and can optionally be set to 120 m for the latest MAJA 4.6+ versions. This coarse resolution is kept for the output cloud mask, while surface reflectances are provided at band-specific full resolution.

The multi-temporal features of MAJA rely on the generation of two composite images which contain the most recent cloud free observation for each band, where one composite provides TOA reflectances, while the other provides surface reflectance. At the end of each L2A processing, the composite images are updated with the new cloud-free pixels. Variations in the reflectance of the composite are then used to derive the Atmospheric Optical Depth ($\tau$), following the assumption that a quick shift of reflectance is more probably linked to a change in the mix of atmospheric aerosols than to a change of surface properties. The amplitude of the thresholds for change detection criteria are time-dependent—the shorter the time-lag between two consecutive cloud-free observations, the narrower the expected reflectance change—but multi-spectral criteria are included to solve ambiguities. The absorption-corrected TOA reflectances are then converted to ground surface reflectance ${\rho }_{\lambda }^S$ using pre-processed Look-up Tables (LUT) computed using the Successive Order of Scattering (SOS) radiative transfer code ([22]) expressing ${\rho }_{\lambda }^S$ along sun and viewer angular configuration at the centre of the image, ground surface altitude, ${\rho }_{\lambda }^{TOA}$, and $\tau$.

Since MAJA version 3.3, a set of LUTs is provided to optionally benefit from the aerosol composition products from CAMS COPERNICUS ([23,24]) in place of the standard continental atmosphere. The CAMS global atmospheric composition forecasts are then used to account for the contribution of 7 hydrophobic and hydrophylic aerosols species (only 5 species before July 2019), given their respective proportion to the total atmospheric optical depth ([25]). Since the LUTs are computed per sensor band and per aerosol type for a set of relative humidities $Rh$ to account for the change of optical properties of sulfate, organic matter, sea salt, nitrate, and ammonium particles along water vapour content, their mixing ratio vertical profiles are used to derive a value of $Rh$ representative of the total atmospheric column, and to point to the adequate set of LUTs. The MAJA ground reflectances are corrected for topographic and adjacency effects by default, although these final steps can be deactivated. The resulting products will be referred to as L2A hereafter.

2.2. Aeronet Based Validation Methodology

The Atmospheric Correction Inter-Comparison Exercise (ACIX) is an open international initiative launched in 2016 aiming at evaluating existing AC algorithms in the frame of the Committee on Earth Observation Satellites (CEOS) Working Group on Calibration and Validation (WGCV) ([17]). A second exercise (ACIX-II) was initiated in 2019 to address the performance of 13 AC processors in their ability to accurately retrieve both the land surface reflectance, the total atmospheric water vapour content and the AOD ([18]). Both exercises focus on time series of Sentinel-2 (S2) and Landsat 8 (L8) products.

The protocol mostly relies of the Aerosol Robotic Network (AERONET), a global network of Cimel Electronique Sun–sky radiometers measuring the AOD with a high accuracy for nearly 30 years ([16,26]). Within ACIX-II, time series of 9 by 9 km subsets of S2 and L8 L1C centred on existing AERONET sites are processed using the 6SV radiative transfer code ([27]) and aerosol properties derived from on-site AERONET measurements to compute reference land surface reflectances (hereafter ${\rho }_{\lambda }^{S,Ref}$). This led to a collection of 3266 products over 127 sites worldwide, ranging from August 2017 to September 2018. When comparing surface reflectances obtained from a given AC processor to such a reference dataset, it is expected that most of the uncertainties will come from the discrepancies in the estimation of atmospheric quantities. However, as mentioned by [18], AC processors that do not rely on 6SV could be at a disadvantage with this validation criterion, as any difference with 6SV in the calculation of the radiative transfer is interpreted as an error.

CESBIO and CNES submitted the corresponding MAJA’s ground reflectance, but since they were corrected for adjacency and topographic effects while the ACIX-II ones were not, the AC performance metrics were not reflecting the actual quality of the atmospheric correction. Therefore, it was decided to remove them from the ACIX-II exercise. The first part of this study fills this gap with a recomputed MAJA dataset comparable to the ACIX-II one, and presents inter-comparison results following the ACIX metrics. It is worth mentioning that ACIX-II also included results from RadCalNet data, but for a significantly narrower time interval as compared to this study.

This first validation exposed in this paper relies on the one adopted in the frame of the ACIX-II Land exercise, and was required to recompute MAJA surface reflectances matching the entire collection of 3266 products over 127 sites worldwide provided within ACIX-II, without the default adjacency and topographic corrections. The obtained MAJA ground reflectances of each valid pixel i, hereafter ${\rho }_{i,\lambda }^{S_{MAJA}}$, rely on simulation conditions similar to the ACIX-II reference reflectances computed from 6SV, hereafter ${\rho }_{i,\lambda }^{S_{REF}}$, and are as such comparable.

It is worth mentioning that, by design, MAJA requires time series of ortho-rectified TOA reflectance (hereafter L1C) products as input to express its full potential, and performs in a degraded, multi-spectral mode only, for a single product. MAJA’s strategy to process a time series with an adequate cloud mask and surface reflectance composite from the first L1C is to start in reverse chronological order (backward mode), from the 8th to the 1st product, to then start in chronological order (forward mode) with proper initialisation values. Moreover, the code is intended to ingest an entire L1C product together with its metadata, and not a subset for a given region of interest. Therefore, the entire processing encompasses more L1C than initially provided by ACIX-II, to allow for a proper use of the backward mode.

In addition, some sites could not be processed when L1C with cloud coverage below 90% were too sparse along time (i.e., Ilorin, Minsk, Moscow, Yekaterinburg, Tomsk, Yakutsk). Moreover, MAJA’s approach that consists of a linear combination of the LUT computed for each CAMS aerosol model is not compatible with the occurrence of high absorbing aerosols due to pollution or biomass burning. The regions prone to the presence of such aerosols are processed with the default continental aerosol model. In the ACIX-II dataset, it corresponds to eleven sites (i.e., Alta Floresta, Beijing, Cairo, Chiang-Mai, Chiayi, Dalanzadgad, Dushanbe, Gandhi College, Luang Namtha, Mexico City, and Zinder Airport).

It is worth mentioning that four reference samples show a significant shift of the distribution of surface reflectances below zero in the blue band, while tending to overestimate the reflectances as compared to MAJA in the red and near infrared, as illustrated for Luang Namtha site for 6 March, 2018, in Figure 1, or, e.g., for the sample over Brussels of 5 February, 2018. The overcorrection of the reference surface reflectance is more likely to be related to the aerosol type than to the total AOD. Any error in the size distribution and refractive index of the aerosols will lead to significant biases. Although MAJA was not able to correctly retrieve the AOD for these specific cases, the use of CAMS auxiliary data can still help to use appropriate aerosol properties. Such marginal cases were excluded from the final ACIX II analysis.

Finally, it must be noted that some products were affected by co-registration issues for tiles located at the boundary of two UTM zones, as some images of the same tile could alternatively belong to one or the other UTM zone along the time series of products. This might have no consequence for other AC processors who can ingest L1C subsets provided by the ACIX-II organizers, but since MAJA requires entire L1C products, 116 final MAJA products affected by a registration mismatch with the reference were discarded from the analysis. As this represents only 116 images out of more than 3000, it is still considered acceptable to express the overall performance of MAJA reflectances estimates, since more than 1.27 billions cloud-free pixels remain available for the study for the 10 m resolution bands, and more than 318 millions for 20 m resolution bands. The 60 m resolution MSI bands 1, 10, and 11 are not part of the results since they are dedicated to coastal aerosols, water vapour, and cirrus detection, respectively.

The metrics used to assess MAJA’s performance against the reference are the accuracy, precision, and uncertainty (hereafter APU criteria) used within ACIX since the first experiment ([17]). For a given pixel i within the $n_{\lambda }$ cloud-free pixels available for a given band of wavelength $\lambda$, the residual $\Delta$ of surface reflectance ${\rho }^S$ is expressed as:

$$\Delta {\rho }_{i,\lambda }^S=\left({\rho }_{i,\lambda }^{S_{MAJA}}-{\rho }_{i,\lambda }^{S_{REF}}\right)$$

(1)

From which are computed the accuracy A, the precision P, and uncertainty U defined as:

$$A=\frac{1}{n_{\lambda }}\sum _{i=1}^{n_{\lambda }}\Delta {\rho }_{i,\lambda }^S$$

(2)

$$P=\sqrt{\frac{1}{\left(n_{\lambda }-1\right)}\sum _{i=1}^{n_{\lambda }}{\left(\Delta {\rho }_{i,\lambda }^S-A\right)}^2}$$

(3)

$$U=\sqrt{\frac{1}{n_{\lambda }}\sum _{i=1}^{n_{\lambda }}{\left(\Delta {\rho }_{i,\lambda }^S\right)}^2}$$

(4)

2.3. In Situ Measurement-Based Validation Methodology

The second validation approach proposed in this study relies on ground measurements of land surface reflectance following the RObotic Station for Atmosphere and Surface (ROSAS) protocol designed by CNES ([10]). It relies on the same CIMEL spectral photometers as the ones of AERONET, but installed on top of a 10 m high mast. Provided that the sky is cloud-free, the instrument performs a sequence of measurements of solar extinction, hemispherical downwelling (sky) and upwelling (ground) radiance, with a spectral sampling of 12 bands between 340 and 1640 nm.

Measurements are post-processed at CNES to provide estimates of atmosphere properties, bidirectional reflectance distribution function (BRDF) and to simulate the TOA reflectance for bands of a given imaging sensor. ROSAS was initially designed for satellite sensor calibration, with a first site deployed in La Crau, France, in 1997, over an homogeneous surface of gravel and low sparse grass. A second site of desert sand was installed in Gobabeb, Namibia in July, 2017 ([28]).

The land surface homogeneity, scarce vegetation, and low cloud coverage of both sites perfectly match with calibration purposes. In 2021, daytime cloud cover was 12% over Gobabeb and 47% over La Crau. Absolute clear days for the same year were 51% for Gobabeb and 25% for La Crau. However, such sites provide a rather small range of reflectance values compared to the wide variety of contexts that MAJA is supposed to deal with. To tackle this limitation, CESBIO and CNES deployed a third site over a cultivated field in Lamasquere, France, in 2021, referred hereafter as Fr-Lam station. The agricultural rotation of three crops every two years, maize, winter wheat, and intermediate cover, respectively, leads to a significant intra- and inter-annual variation of land surface reflectances, at the cost of a stronger cloud coverage as compared to the first two sites. The combination of these three datasets provide insightful results on the performance of MAJA.

The in situ measurements used in this second approach consist in time series of ROSAS acquisitions over La Crau and Gobabeb from July, 2017, to December, 2021, as well as the first set of acquisition of the Fr-Lam ROSAS station during a complete cycle of maize growth, senescence, and harvesting in the year 2021. Since the full daytime measurement sequence requires 140 min of cloud-free sky, any dataset affected by sparse cloud-cover in the data acquisition interval are removed at the post-processing stage. The ROSAS datasets over La Crau and Gobabeb have been reprocessed to account for an improved calibration of the photometer instrument water vapour band provided by CIMEL in 2022. This leads to 181 quality controlled measurements for Gobabeb and 166 for La Crau. As Fr-Lam is a recent site, and more prone to clouds than the other two sites, the number of samples available for validation is limited to 13 from June to November, 2021. To match Sentinel-2 MSI bands, the surface reflectances obtained from the photometer are resampled using a third order spline interpolation.

The ROSAS photometer performs measurements of upwelling radiance up to 60 degrees off nadir, which leads to a 34 m diameter circle footprint at ground level, and a 26 m diameter circle footprint at maximum maize canopy height for the Fr-Lam site. Within this area, the surface cover is assumed homogeneous by the processor although it might not be exact in reality, each measurement of angular configuration $(\theta ,\varphi )$ being assimilated to the observation of the exact same surface from a different point of view. A BRDF following Roujean’s model ([29]) is applied to derive the nadir surface reflectance comparable to MAJA’s output. The MAJA surface reflectance values of the 10 m resolution Sentinel-2 bands are averaged over 50 by 50 m square centred on the photometer mast.

The errors associated with ROSAS surface reflectances were estimated considering all the sources of uncertainties along the processing chain, photometer noise, calibration of the photometer collimators based on Rayleigh scattering and Bouguer–Langley methodology, BRDF fitting, and spectral interpolation. For clear sky conditions along a day, the surface reflectance estimation error varies from 3.5 to 5% for La Crau and 3 to 4.5% for Gobabeb depending on the spectral band. For a more turbid day, this budget is only slightly increased by less than 1% because of the data quality filtering before processing. The Region Of Interest (ROI) used to compare ROSAS with Sentinel-2 was a 50 m × 50 m square centred on the photometer. The error compared to the 30 m × 30 m ROI covered by the photometer was estimated using PLEIADE data and is on average <1% for La Crau and <0.5% for Gobabeb. The slightly higher error at La Crau compared to Gobabeb is explained by the higher heterogeneity within the instrument footprint at La Crau. For Fr-Lam, an early estimate using the first acquisitions suggests an error of up to 7%, but since the calibration improves along time, the error is expected to decrease as the data collection increases. The methodology for the ROSAS uncertainty estimation is presented in [30,31].

For the purpose of this comparison, we processed a time series of Sentinel-2 L1C products available from July, 2017, to December, 2021, for tile 31TFJ and 33KWP for La Crau and Gobabeb, respectively, and from June to November, 2021, for tile 31TCJ for Fr-Lam, applying the default adjacency and topographic correction. Therefore, the following MAJA L2A ground reflectances are nominal products (as opposed to the previously used ${\rho }_{i,\lambda }^{S_{MAJA}}$ uncorrected for adjacency and topographic effects). For the sake of comparison, two sets of L2A products are computed with and without using the CAMS aerosols auxiliary data.

3. Results

3.1. MAJA- and AERONET-Based Reference Inter-Comparison

The results of the inter-comparison between the reference and MAJA ground reflectances are summarized in Figure 2 for Sentinel-2 MSI band 2 (492 nm), band 3 (560 nm), and band 4 (665 nm) and 8 (842 nm). The Sentinel-2 60 metre resolution band 1 is not part of the MAJA Level 2A product, therefore no information is given below for this band. Any pixel from all the available products of all sites where both reference and MAJA Quality Analysis (QA) flags are valid and both ground reflectances values are positive is stacked in the computation of the APU criteria, leading to more than 1.27 billion samples for the 10 m resolution bands. Overall, uncertainties (U) of all bands are within the specifications defined by [15], such that $spec=0.05\rho +0.005$, and accuracies A showing slightly negative values mostly in the visible bands and for reflectances lower than 0.4, meaning that MAJA tends to underestimate these reflectances as compared to the reference. The uncertainties presented in the

Table 1. APU criteria obtained for all the ACIX-II sites together with total valid pixels counts according to quality filter criteria.

	Valid QA				Valid QA and ${\mathit{\rho }}_{\mathit{i},\mathit{\lambda }}^{\mathbf{REF}}>0$
	Samples	A	P	U	Samples	A	P	U
B2 492 nm	1.273 × ${10}^9$	−0.005	0.011	0.012	1.271 × ${10}^9$	−0.005	0.010	0.011
B3 560 nm	1.273 × ${10}^9$	−0.003	0.010	0.010	1.273 × ${10}^9$	−0.003	0.009	0.010
B4 665 nm	1.273 × ${10}^9$	−0.001	0.009	0.009	1.272 × ${10}^9$	−0.001	0.009	0.009
B5 705 nm	3.184 × ${10}^8$	−0.000	0.009	0.009	3.182 × ${10}^8$	−0.000	0.008	0.008
B6 740 nm	3.184 × ${10}^8$	−0.000	0.012	0.012	3.182 × ${10}^8$	0.000	0.010	0.010
B7 783 nm	3.184 × ${10}^8$	−0.003	0.012	0.012	3.182 × ${10}^8$	−0.003	0.010	0.010
B8 842 nm	1.273 × ${10}^9$	−0.000	0.011	0.011	1.273 × ${10}^9$	−0.000	0.009	0.009
B8A 865 nm	3.184 × ${10}^8$	−0.003	0.011	0.011	3.182 × ${10}^8$	−0.003	0.009	0.009
B11 1.6 µm	3.184 × ${10}^8$	−0.001	0.007	0.007	3.181 × ${10}^8$	−0.001	0.005	0.005
B12 2.2 µm	3.184 × ${10}^8$	0.001	0.006	0.006	3.179 × ${10}^8$	0.001	0.004	0.004

range from 0.011 for the blue band to 0.005 for medium infrared bands.

To evaluate the impact of negative values found in some reference ACIX-II samples, the statistics were computed with and without such negative values, leading in the later case to slightly better uncertainties results. Filtering such negative reflectances led to the removal of less than $0.2\%$ of the available pixels for the blue band B2. The Table 1 provides the samples count per band in both cases, together with statistics of APU criteria for two configurations, i. all valid pixels according to the QA flag, and ii. all valid pixels according to QA flag provided that ${\rho }_{i,\lambda }^{S_{REF}}>0$. The uncertainties obtained without filtering reflectances less than 0 still remain close from the specifications, with 0.012, 0.010, 0.009, and 0.011 for Sentinel-2 band 2, band 3, band 4, and band 8, respectively.

The worldwide distribution of the AERONET stations used to define the ACIX samples sites can depict very diverse contexts in terms of land cover, cloud occurrence, or aerosol mixture, and, therefore, very diverse MAJA processor performances too. Over arid sites with typical AOD of 0.2–0.5 corresponding to the AERONET stations of Banizoumbou (Nigeria), El Farafra (Egypt), Mezaira (UAS), Medenine (Tunisia), Mexico City (Mexico), Tabernas (Spain), Tamanrasset (Algeria), Carthage (Tunisia), and Sede Boker (Israel), the uncertainties U fall down to 0.006 for band 2, 0.005 for band 3, 0.006 for band 4, and 0.008 for band 8. Interestingly, MAJA’s estimations of the reflectances over arid sites are usually very accurate, despite poorer estimates of the $AOD$ as compared to vegetated areas. This is explained by the limited variation of the surface reflectance in these areas, which remain high over time, and, as such, are less sensitive to errors on $AOD$ ([25]). Agricultural areas surrounding stations, such as Alta Floresta (Brasil), Carpentras (France), Chilboton (UK), Evora (Portugal), Granada (Spain), Murcia (Spain), or Saada (Morocco), show comparable uncertainties to arid sites. Samples partly covered with forests, such as Noto (Japan), Seoul (South Korea), or Zvenigorod (Russia), show uncertainties lower than 0.011, 0.007, 0.005, and 0.005 for bands 2, 3, 4, and 8, respectively, with the notable exception of Luang Namtha (Laos), where it increases up to 0.021, 0.025, 0.035, and 0.031 for bands 2, 3, 4, and 8, respectively. The samples for Luang Namtha were processed without CAMS auxiliary data due to the occurrence of strong absorption, and MAJA over-corrected the reflectances as compared to the reference values.

Overall, the comparisons of MAJA’s reflectances against the ACIX reference show large uncertainties over urbanized areas subject to a strong contribution of absorbing aerosols to the $AOD$, with a nearly systematic over-correction of the MAJA processor. As an example, the uncertainties for the band 2 reach 0.018 for Pune (India), 0.019 for Xiang He (China), 0.021 for Cairo (Egypt), 0.027 for Kanpur (India), 0.029 for Chiang Mai (Thailand), 0.030 for Yellowknife (Canada), and even 0.042 for Dhaka (Bangladesh) samples. Since the samples of Cairo and Chiang-Mai were processed without CAMS auxiliary data, the resulting uncertainties can be explained by a mismatch between the default continental aerosol model MAJA is using in such cases and the aerosol content acquired by the AERONET sensor. The other samples correspond to locations with occurrence of high $AOD$ together with a mix of absorbing and non-absorbing aerosol species, where the assumption that the MAJA LUTs computed for each CAMS aerosol model can be linearly combined fails. Future research by our team will attempt to address the issue of combining aerosol types with a significant proportion of absorbing aerosols.

3.2. MAJA and ROSAS Inter-Comparison

In this case, the validation is site-specific, and relies on time series of in situ ROSAS acquisitions at La Crau, Gobabeb, and Fr-Lam stations. Both La Crau and Gobabeb datasets cover the period ranging from July, 2017, to December, 2021, while the recent Fr-Lam station dataset ranges from June to November, 2021. Only quality controlled ROSAS data are used for validation, leading to 166 samples on La Crau, 181 on Gobabeb and only 13 on Fr-Lam.

The results presented on Figure 3 compare MAJA L2A ground reflectances using auxiliary CAMS aerosols data, as well as topography and adjacency effects corrections (nominal settings of MAJA), with quality-controlled ROSAS ground reflectances. The overall RMSEs are well in line with the expected MAJA uncertainties for Sentinel-2 bands 2 to 4, but are not as good for higher wavelengths. The differences between MAJA and ROSAS surface reflectances account for both the calibration specification of Sentinel-2 (within 3% for TOA reflectances [32]), the error budget of the ROSAS protocol (within 5% [10]) and the errors from MAJA itself. MAJA’s multi-temporal approach assumes that the surface reflectance at the shortest wavelength does not vary rapidly with time, so MAJA tends to adjust the $AOD$ to match this assumption. It is therefore expected that the RMSE for the visible bands will be lower than for the shortwave infrared bands.

The slightly higher RMSE over La Crau as compared to Gobabeb can be explained by the stronger heterogeneity of the footprint of the ROSAS instrument. The higher RMSE for Sentinel-2 band 8 can also be linked to the spectral resampling needed to compare the MSI bands with the CIMEL sun-photometer of the ROSAS station, since Sentinel-2 band 8 is centred at 842 nm while the nearest CIMEL band is centred at 870 nm. Despite the use of hyperspectral data acquired in the footprint of the photometer to refine the interpolation process, the accuracy remains higher in the spectral regions close to the photometer bands. In addition, hyperspectral data are acquired very infrequently. Their validity may decrease over time for vegetated sites, such as FrLam, resulting in reduced interpolation accuracy.

Separating the results between Sentinel-2A and Sentinel-2B does not reveal any significant differences between both MSI sensors, as illustrated on Figure 4. The samples distribution is balanced, as Gobabeb counts 89 samples from Sentinel-2A and 92 samples from Sentinel-2B, while La Crau counts 94 samples from Sentinel-2A and 72 samples from Sentinel-2B. Larger differences in the RMSE over Gobabeb, from band 5 onwards, are mostly due to a single outlier. This is site specific and is explained by one significant underestimation of the surface reflectance by the ROSAS instrument, which is still consistent with the quality filter, as can also be seen from the point in September, 2021, on Figure 3. The differences in RMSE between Sentinel-2A and Sentinel-2B can be explained by, among other things, the $1\%$ calibration error between the two MSI sensors, differences in the actual aerosol load at the time of acquisition of each product, and the tendency of MAJA to compensate for variations in TOA reflectances by varying the $AOD$.

The $RMS{E}_{AOD}$ over Fr-Lam reached 0.095, which partly explains poorer RMSEs of spectral reflectances for this station, together with the local surface heterogeneity. During this period, the maize crop surrounding the Fr-LAm ROSAS mast grew from 10 cm early June to 2.6 m early August. With an inter-row spacing of 50 centimetres, the signature evolves from mostly clear clay bare soil to a dense green vegetation. Although the North–South pattern of the rows induces directional effects, these are smoothed by the BRDF model. Although not quantified in this study, the Rounjean BRDF might in such cases over-simplify the actual BRDF and, thus, contribute to the uncertainties of the derived surface reflectances. The harvesting occurred the 20th of September, before the maize reaches its senescence stage as it was intended for silage, and left the soil surface mostly covered by vegetation debris. According to the Figure 3, MAJA and ROSAS show better agreement when the vegetation is low over Fr-Lam.

It should be noted that the quality of the AOD estimates of MAJA can be affected by the presence of large gaps in the the time-series of Sentinel-2 images due to clouds. As the period from June to July was particularly cloudy over the Fr-Lam station, the hypothesis of a change in $TOA$ reflectance explained by a change in $AOD$, on which MAJA partly relies on, is weakened as the available observations become sparse. Although MAJA detected the sudden increase in $AOD$ on 7 September, it failed to catch its actual amplitude.

The use of the CAMS Global Atmospheric Composition Forecast with MAJA is considered the nominal configuration. However, users willing to process Level 1 products without it can still use the default continental aerosol definition, by deactivating the CAMS option. For the sake of comparison, the time series of products over La Crau, Gobabeb and Fr-Lam were processed with the default continental aerosol and compared to ROSAS.

Table 2. RMSEs obtained when comparing ROSAS stations reflectances and $AOD$ with both MAJA L2A products computed with the default continental aerosol setting and using CAMS auxiliary data.

	La Crau		Gobabeb		Fr-Lam
	Continental	CAMS	Continental	CAMS	Continental	CAMS
AOD	0.074	0.045	0.070	0.067	0.109	0.095
B2 492 nm	0.009	0.007	0.007	0.004	0.013	0.012
B3 560 nm	0.015	0.009	0.011	0.007	0.022	0.014
B4 665 nm	0.015	0.011	0.011	0.010	0.017	0.011
B5 705 nm	0.018	0.013	0.014	0.010	0.026	0.014
B6 740 nm	0.021	0.015	0.013	0.011	0.044	0.043
B7 783 nm	0.022	0.016	0.014	0.011	0.060	0.060
B8 842 nm	0.023	0.018	0.014	0.011	0.064	0.063
B11 1.6 µm	0.017	0.016	0.018	0.021	0.035	0.030

summarises the actual benefits of CAMS auxiliary data over the continental aerosol when compared to ROSAS stations data, with improvements in terms of RMSE of the order of $0.001$ to $0.008$ for reflectances, and up to $0.029$ for AOD over La Crau. The overall performance of MAJA without CAMS data remains acceptable for both Gobabeb and La Crau, which is to be expected as the default continental aerosol was chosen to match these validation sites.

4. Discussion

The performance metrics of this study do not cover the retrieval of the water vapour by MAJA, since it was already addressed in ACIX-II, together with detailed indicators on the uncertainties in the retrieval of the AOD. Ref. [18] reported an RMSE of MAJA’s AOD of 0.135 for all AERONET sites together. The RMSEs on AOD reported in Table 2 for the La Crau and Gobabeb time series of ROSAS acquisitions are overall lower, but do not cover the large variety of contexts addressed by the AERONET network. In addition, the Table 2 illustrates the benefit of the auxiliary CAMS aerosols data on the estimation of the AOD by MAJA, although more significant over La Crau than Gobabeb.

When compared to other AC processors within ACIX-II (Figure 5), the uncertainties obtained when comparing the MAJA products without adjacency and topographic correction to the reference dataset are overall slightly higher than the ones of FORCE, LaSRC, or Planet. Although the ACIX-II reference dataset should, by design, give a non negligible advantage to 6SV-based atmospheric correction processors, such as LaSRC or Planet, the performance of MAJA fits well within the expected specifications $spec=0.05\rho +0.005$ defined by [15]. The uncertainties of the SR computed by MAJA remain under 0.01 for the red, NIR and SWIR bands, and below 0.011 for the blue band. With the exception of a limited number of locations worldwide, the use of CAMS aerosols data provide SR in good agreements with the reference ones. Lower performances are expected in areas with strong aerosols, as observed over Asian cities sites. Moreover, rainforest areas are more prone to errors in the estimation of AOD and derived SR, mainly because of the strong and continuous cloud coverage which weakens the multi-temporal approach of MAJA.

The major interest of this first AERONET-based approach is to assess AC processors against a large variety of contexts in terms of both land cover and atmospheric properties, though it remains a comparison with indirect observations of ground reflectances. The occurrence of lower than zero reflectances in the reference dataset illustrates the presence of biases inherent to this approach, which remains sensitive to uncertainties in the estimation of atmospheric quantities by AERONET stations.

The second approach based on the ROSAS stations is complementary. The complexity and cost of the instrumental setup limit the number of observation sites to three stations. On the other hand, the continuous record of the surface reflectances for any angle of observation with a spectral sampling close to Sentinel-2 specifications provides unique time series of validation data. At the time of this study, no other in situ network provides equivalent datasets over land. The recent addition of the Fr-Lam station combined with stronger cloud occurrences over the Lamasquere region limited the number of quality-checked observations on this site, leading to poorer validations results. Further studies remain necessary to estimate the impact of heterogeneity within the instrument footprint, as the vegetation coverage of this station can range from bare soil to three meters high maize. The discrepancies observed between the wide Sentinel-2 842 nm band 8 and the corresponding narrower 870 nm CIMEL band suggest that the Fr-Lam instrument requires further calibration. This calibration is part of the ROSAS protocol, and is linked to the length of available time-series of quality-proofed observations, which was very limited for the period under consideration.

The nearly continuous five-years long records available for the La Crau and Gobabeb stations provide a significant amount of validation samples. Overall, the RMSEs obtained comparing ROSAS SR against MAJA SR are slightly lower than the uncertainties retrieved from the ACIX reference datasets, especially for the blue band 2. Moreover, the RMSEs tend to increase for NIR and SWIR bands, as opposed to the observed behaviour with the AERONET approach. Such higher RMSEs, as, e.g., for Sentinel-2 band 8, could be attributed to the spectral resampling of the CIMEL sensor used under the ROSAS protocol, whose bands central wavelength and width do not fit well with the Sentinel-2 one. La Crau and Gobabeb uncertainties however remain within reasonable bounds as compared to the results obtained for the Fr-Lam station, who will need further calibration.

The comparison of the surface reflectances between MAJA and ROSAS stations also demonstrates the benefits of the use of the CAMS aerosols auxiliary data for SR computation, mainly over La Crau. However, as suggested by the results in Table 2, the performances of MAJA using the single continental aerosol definition as a substitute to the use of CAMS remains within the aforementioned specifications. Although the use of the single continental aerosol is not considered as the nominal use of MAJA, it proved to remain an alternative for some very specific regions of the globe subject to strong absorbing and non-absorbing aerosols mixing. A new approach to better account for aerosols mixing is planned for an upcoming publication.

These validation results demonstrate the adequate performance of MAJA for the production of high-quality Sentinel-2 surface reflectance products, combined with high-quality cloud masks, as shown in [21]. Because of the combination of multi-spectral and multi-temporal criteria to estimate the Atmospheric Optical Depth, MAJA is designed to deal with time series of products rather than single acquisitions spread over time, making it particularly suitable for operational uses by institutional productions centres.