Next Article in Journal
An Explainable CatBoost Model for Crater Classification Based on Digital Elevation Model
Previous Article in Journal
Correction: Marchamalo-Sacristán et al. MT-InSAR and Dam Modeling for the Comprehensive Monitoring of an Earth-Fill Dam: The Case of the Benínar Dam (Almería, Spain). Remote Sens. 2023, 15, 2802
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Improving AOD Algorithm Evaluation: A Spatial Matching Method for Minimizing Quality Control Bias

1
Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100094, China
2
College of Resources and Environment, University of Chinese Academy of Sciences, Beijing 100049, China
3
School of Computer Science and Engineering, Northeastern University, Shenyang 110167, China
4
Space Star Technology Co., Ltd., Beijing 101399, China
5
Key Laboratory of Radiometric Calibration and Validation for Environmental Satellites, National Satellite Meteorological Center (National Center for Space Weather), China Meteorological Administration, Beijing 100081, China
6
Innovation Center for FengYun Meteorological Satellite (FYSIC), Beijing 100081, China
*
Author to whom correspondence should be addressed.
Remote Sens. 2025, 17(7), 1235; https://doi.org/10.3390/rs17071235
Submission received: 20 February 2025 / Revised: 26 March 2025 / Accepted: 28 March 2025 / Published: 31 March 2025

Abstract

:
Satellite-derived aerosol optical depth (AOD) products from MODIS and VIIRS sensors are vital for monitoring global aerosol distributions. However, inconsistencies in quality control algorithms and spatial resolution introduce errors that complicate validation processes and reduce the accuracy of satellite-to-ground comparisons. This study proposes the “optimal” spatial matching method to minimize these errors and enable a more accurate evaluation of retrieval algorithm performance. Using AERONET ground observations from 2012 to 2021, MODIS and VIIRS AOD products were systematically validated with three spatial matching methods—“direct”, “average”, and “optimal”. Results demonstrate that the “optimal” method consistently outperformed the other methods by selecting pixel values. The study highlights significant quality control disparities across AOD products and demonstrates that high-resolution products, with purer pixels, achieve superior accuracy under the “optimal” method. These insights provide valuable guidance for optimizing dataset applications and refining aerosol retrieval algorithms.

1. Introduction

Atmospheric aerosols, composed of a mixture of solid or liquid particles suspended in the atmosphere, have both direct and indirect effects on climate [1]. These aerosols significantly influence the Earth’s energy balance [2], contribute to climate change [3,4], and impact human health [5,6]. Aerosols are widely distributed, exhibiting notable spatial and temporal variability [7], which requires quantitative measurements to thoroughly understand their effects on climate and related fields. Aerosol optical depth (AOD) quantifies the extent of solar radiation attenuation as it passes through an aerosol-containing atmospheric column and serves as a critical parameter in the radiative transfer equation, essential for simulating aerosol impacts on atmospheric radiation [8,9]. Ground-based AOD measurements provide accurate point observations but suffer from limited spatial coverage. To overcome this limitation, satellite retrievals have become the primary method for obtaining AOD data, offering much broader spatial coverage than ground-based stations [10].
The MODerate Resolution Imaging Spectroradiometer (MODIS) sensors, aboard Terra (2000) and Aqua (2002), are widely recognized and extensively used in numerous applications due to their long-term, globally comprehensive data record [11]. They have exceeded their initial five-year design life, recording data for over two decades and now exhibiting signs of aging. To ensure the continuity of long-term observational data, Visible Infrared Imaging Radiometer Suite (VIIRS) sensors, which are similar to MODIS, were deployed. One VIIRS sensor has been operational on the Suomi National Polar-Orbiting Partnership (SNPP) spacecraft since 2011, with another operational on NOAA-20 since 2017. These sensors not only extend the MODIS observational record but also improve its quality and coverage [12,13,14].
To retrieve AOD, three primary algorithms have been developed over land for the MODIS and VIIRS sensors. The Dark Target (DT) algorithm, referring exclusively to the land algorithm in this paper, is the earliest developed approach. It relies on fixed statistical relationships of surface reflectance between visible and shortwave infrared channels over vegetated or dark-soiled land (dark in the visible spectrum) [15]. Identifying dark target pixels within designated “retrieval boxes” is essential for the DT algorithm. In contrast, the Deep Blue (DB) algorithm relies more heavily on historical data. On bright surfaces, the blue spectral region is sufficiently darker than the long-wavelength region, enabling differentiation between the surface and dust plumes. The DB algorithm uses this characteristic by applying the minimum reflectance method to establish a static, pre-calculated surface reflectance database [16]. Both the DT and DB algorithms retrieve AOD based on the Lambertian surface assumption, utilizing single-orbit data. In contrast, the Multi-Angle Implementation of Atmospheric Correction (MAIAC) algorithm retrieves AOD based on a non-Lambertian surface assumption, incorporating multi-orbit data over a span of 3 to 16 days [17]. Each algorithm generates one or more associated scientific datasets. To expand coverage for climate research, the MODIS aerosol product includes a merged 10 km dataset (DTB), which combines DT and DB retrievals [18].
Ensuring dataset accuracy is critical during algorithm updates and applications in research. Therefore, appropriate methods must be employed to compare site measurements with satellite products to verify data reliability. Due to differences in measurement methods between satellite and ground-based instruments, their products vary in wavelength, time, and spatial coverage. In the early stages of MODIS product development, a method based on temporal and spatial window averaging was proposed to compare satellite datasets with ground-based observations, and the findings indicated that spatial window selection had a relatively minor effect on validation outcomes [19]. This method has since been widely applied in the validation of AOD products retrieved using the DT and DB algorithms in MODIS and VIIRS. Some studies have refined the temporal and spatial window sizes, employing approaches such as computing the average AOD value within a specified spatial window (weighted by distance or pixel count) for comparison with ground-based measurements [17,20,21,22,23,24,25,26]. Additionally, to mitigate the potential influence of residual clouds on validation results, further data filtering has been performed within spatial windows. For instance, during the validation of the MAIAC algorithm, a 20 × 20 km2 window was used to filter 1 km resolution AOD retrievals, and the highest 50% of AOD values were removed before averaging [17].
However, averaging methods can inherently introduce biases, particularly when extreme values arise due to undetected clouds [27,28]. While quality control measures in DT and DB products flag high-quality retrievals [29], the detection and removal of subpixel clouds remain particularly challenging [30,31]. Ideally, AOD algorithm validation should be conducted under cloud-free conditions. However, due to the contamination of subpixel clouds, validation based on averaging methods primarily reflects the accuracy of the product rather than the accuracy of the retrieval algorithm. A more robust validation approach is needed to better isolate algorithmic performance from quality control effects.
To ensure that validation results reflect the maximum achievable accuracy of these algorithms, we applied a new processing method to address this issue, aiming to minimize the influence of quality control procedures by selecting optimal values within a specified window. This study evaluates the accuracy of MODIS and VIIRS AOD products using ground-based observations from the global Aerosol Robotic Network (AERONET) from 2012 to 2021, presenting validation results based on various matching methods. Since most AERONET sites are situated on land, the comparison focuses exclusively on AOD products over land. In this paper, Section 2 introduces the datasets, Section 3 outlines the matching method and verification indicators, Section 4 discusses and analyses the results, and Section 5 provides a summary of the conclusions.

2. Data Description

2.1. Satellite AOD Datasets

The datasets used in this study are listed in Table 1, comprising a total of 13 datasets. MODIS Collection 6, released in 2016 for all atmospheric products, was followed by a minor update to Collection 6.1 in 2017. VIIRS DB Version 1 and DT Version 1 were released in 2018 and 2020, respectively, with updates to Version 1.1 in 2021 and Version 2.0 in 2023. The “Alias” column provides convenient references for these datasets throughout the study. For the MAIAC, retrievals from the Terra and Aqua satellites are separated and labeled as MOD(MAIAC) and MYD(MAIAC), respectively.
The Quality Assurance (QA) flag indicates the reliability of the retrieved AOD. For the DT and DB datasets, higher retrieval quality is represented by a higher QA value, ranging from 0 to 3. The recommended QA flag range for DT datasets is restricted to a value of 3, while for DB and DTB datasets, values from 2 to 3 are recommended. In contrast, QA flags in the MAIAC datasets do not adhere to the 0 to 3 scale used in DB and DT datasets; thus, only AOD values with “Best quality” QA flags were included in this study. Each algorithm’s AOD product has its own independent quality assurance (QA) mechanism, tailored to its retrieval approach and assumptions. In this study, only data with the highest quality or recommended QA flags were used to ensure the reliability of the analysis.

2.2. AERONET

AERONET is a globally distributed ground-based sun photometer network that provides continuous, high-quality AOD observations for atmospheric research [32]. It has been extensively utilized for validating satellite-retrieved AOD products. AERONET offers point-based aerosol spectral data at multiple wavelengths, including 440, 500, 675, 870, and 1020 nm. AOD measurements are recorded at intervals of 5 to 15 minutes, with an uncertainty of ±0.02 in AOD [33]. These AOD data are categorized into three quality levels: L1.0 (unscreened), L1.5 (cloud-screened), and L2.0 (cloud-screened and quality-assured). The highest quality level, L2.0, has a publication latency of several months to a year due to the need for post-deployment calibration. In this study, L2.0 AOD data from AERONET version 3.0, spanning 2012 to 2021 and measured at 882 sites, are used for validation.

2.3. Others

The MCD12C1 product is an annual global land cover dataset derived from the MODIS sensors aboard the Terra and Aqua satellites. It provides land cover classification at a spatial resolution of 0.05 degrees (5.6 km at the equator). MCD12C1 includes multiple classification schemes; among them, the International Geosphere-Biosphere Programme (IGBP) classification is the most widely used and provides 17 land cover types representing major global vegetation and surface categories [34].

3. Methods

3.1. Overview of AOD Matching Method

A specific pairing between a satellite-retrieved AOD and a site-measured AOD is referred to as a matchup. Due to differences in measurement methods between satellite and ground-based instruments, their products vary in spectral wavelength, time, and spatial coverage. Direct matching between site-measured and satellite-retrieved AOD is not feasible; thus, some processing is required [19,22,25], as shown in Figure 1.
In terms of spectral wavelength, AERONET does not provide measurements at 550 nm. Consequently, multi-wavelength AOD values from a single measurement were interpolated to 550 nm using a quadratic polynomial [35].
Regarding temporal aspects, a time discrepancy exists between ground-based measurements and the satellite overpass. Thus, AOD values at 550 nm were averaged from measurements taken within a 30-minute window before and after the satellite overpass [19].
Spatially, satellites provide snapshot measurements, whereas ground stations offer point measurements. To reconcile these differences, this study employs three methods:
  • “Direct” Method: This method selects the satellite-derived AOD value from the pixel covering the ground station. Mathematically, the AOD value x direct is determined as
    x direct = x j , where j = arg min i W d ( ( l a t i , l o n i ) , ( l a t site , l o n site ) )
    where x i represents the AOD value retrieved from the satellite at pixel i, and ( l a t i , l o n i ) denotes the latitude and longitude coordinates of that pixel. The function d ( · ) computes the spatial distance between the satellite pixel and the ground station location ( l a t site , l o n site ) . The index j corresponds to the object satellite pixel within the predefined spatial window W.
  • “Average” Method: This common approach involves averaging AOD values within a spatial window centered on the ground station [19]. Mathematically, the AOD value x average is determined as
    x average = 1 | W | i W x i
    where x i represents the AOD value retrieved from the satellite at pixel i, and | W | denotes the total number of pixels within the spatial window W that contain valid retrievals.
  • “Optimal” Method: This method, proposed in this study, selects the retrieved AOD value that exhibits the smallest absolute error relative to the ground-based measurement. Mathematically, the AOD value x optimal is determined as
    x optimal = x j , where j = arg min i W | x i x site |
    where x i represents the AOD value retrieved from the satellite at pixel i, and x site denotes the AOD measured by the ground station. The index j corresponds to the object satellite pixel within the spatial window W.

3.2. Differences of Three Spatial Matching Methods

Discrepancies between ground-based observations and satellite overpass times can make it difficult to locate the same aerosol layer in satellite imagery. Aerosols are transported by wind. Variations in wind speed and topography at different altitudes cause changes in aerosol distribution between the times of ground-based measurement and satellite overpass. This effect is more noticeable during high-wind events, making direct spatial matching less precise.
Despite the general assumption that aerosol distributions are spatially homogeneous over limited regions, fundamental differences in measurement techniques introduce significant retrieval uncertainties. Ground-based instruments derive AOD by measuring direct solar radiation, where the radiation traverses the atmospheric column only once. In contrast, satellite retrievals estimate AOD based on reflected radiance, requiring radiation to pass through the atmosphere twice. This distinction makes satellite-based AOD more susceptible to radiative perturbations, including subpixel cloud contamination, which can introduce retrieval biases. Consequently, satellite-retrieved AOD often exhibits enhanced spatial variability relative to actual atmospheric conditions.
The “direct” method assigns the satellite-derived AOD value from the pixel containing the ground station. However, it is particularly susceptible to temporal mismatches, as the retrieved aerosol may not correspond to the same air mass observed by the ground station. Additionally, this method cannot mitigate the impact of subpixel cloud contamination.
The “average” method reduces the impact of retrieval errors and localized anomalies with the assumption that aerosol concentrations vary smoothly over small spatial scales. Unlike the “direct” method, which relies on a single pixel, this approach averages AOD values from a surrounding spatial window, thereby mitigating errors arising from aerosol transport over time. However, in cases where subpixel clouds introduce outliers within the spatial window, the averaging process can lead to AOD values that deviate significantly from ground-based observations, as Figure 2 demonstrates.
The “optimal” method proposed in this study selects the satellite retrieval that minimizes the absolute error relative to the ground-based measurement. By prioritizing the retrieval with the smallest discrepancy, this approach mitigates the impact of subpixel cloud contamination, as the selected retrieval is less likely to be influenced by local radiative disturbances. Figure 2 presents the application of this method using real data, demonstrating its effectiveness in reducing the influence of outliers compared to the “average” method. However, two limitations remain:
  • The selected pixel may not align with the true transport direction of aerosols, limiting the method’s physical interpretability and necessitating the assumption that aerosols are homogeneously distributed within the spatial window.
  • This method is susceptible to retrieval algorithm biases. When the atmospheric conditions observed by the satellite differ significantly from those at the ground station, larger algorithmic errors can sometimes make the retrieved AOD appear closer to the ground-based measurement. As a result, the “optimal” method may select a retrieval that matches the observation numerically but does not accurately reflect the true aerosol conditions. For example, in MOD(DT), certain QA = 0 AOD values have been observed to align more closely with ground-based measurements than QA = 3 values, despite QA = 0 AOD retrievals potentially violating key algorithmic assumptions.

3.3. Selection of Window Size and Matching Constraints

For low-resolution products, the number of pixels within the spatial matching window is typically set to an odd number to ensure that the ground-based site is located at the central pixel and that spatial coverage is balanced in all directions. In this study, given that the analyzed datasets include products with a spatial resolution of 10 km, two primary window sizes were considered: 30 × 30 km2 and 50 × 50 km2. Both window sizes were evaluated, and the resulting distributions are presented in Table 2 and Table A1. Although the 50 × 50 km2 window produced better statistical performance than 30 × 30 km2 window, the overall conclusions remained consistent regardless of window size. The 30 × 30 km2 window is evenly divisible by the spatial resolutions of all products, which helps minimize discrepancies in spatial coverage during validation. Therefore, it was adopted for spatial matching in all three approaches used in this study. The total number of AOD values within the same spatial window varies across products due to differences in spatial resolution, ranging from as few as 9 pixels for 10 km products to as many as 900 pixels for 1 km products.
To enhance the reliability of spatial collocation, we continue to assume that aerosol properties remain relatively uniform within a defined spatiotemporal window. To address temporal mismatches in both the “direct” and “optimal” methods, we average AOD values over a time window containing at least three valid observations. In the spatial domain, to prevent the “optimal” method from selecting retrievals that do not adhere to algorithmic assumptions, we still restrict the analysis to AOD values that meet the recommended quality assurance (QA) criteria or are flagged as “Best Quality” in the respective datasets. Although a higher number of high-quality pixels within the window expands the selection pool for the “optimal” method, imposing a threshold on the number of high-quality pixels may introduce artificial constraints, potentially leading to a favorable bias in the validation results rather than accurately reflecting real-world retrieval performance. AOD retrievals are independent of one another, and no such restriction is applied in the main text. Additionally, an alternative version, in which spatial matching is conducted only when the proportion of high-quality AOD retrievals within the spatial window exceeds 20%, is provided in Appendix A, Table A2. The application of this proportion-based quality threshold further enhances the performance of the “optimal” method.

3.4. Evaluation Metrics

Expected Level of Error (EE): EE, which combines absolute and relative error, is expressed as a confidence interval at approximately one standard deviation ( 67%). Greater consistency of data within this range when compared with ground measurements from AERONET indicates a more accurate AOD product [29]. For DT, the EE for AOD is defined as ±(0.05 + 20%), where QA = 3 [36], and for DB, it is ±(0.05 + 15%), where QA ≥ 2 [37,38]. This study employs the former, more stringent constraint for EE. The EE metric can be subdivided into the following three variables:
  • =EE: Percentage of total data within the expected error range (±(0.05 + 15%)).
  • >EE: Percentage of total data exceeding the expected value (+(0.05 + 15%)).
  • <EE: Percentage of total data below the expected value (−(0.05 + 15%)).
Correlation Coefficient (CC): Pearson’s correlation coefficient quantifies the linear relationship between satellite retrievals and AERONET measurements, providing an indication of retrieval consistency.
Root Mean Square Error (RMSE): RMSE measures the overall deviation between retrievals and measurements, emphasizing larger discrepancies.
Relative Mean Bias (RMB): RMB assesses systematic over/underestimation trends.
Fractional Bias (FB): FB provides normalized error assessment robust to extreme values. It represents a measure of the bias that allows symmetric analysis of overestimation or underestimation by the model relative to observations [39].
The RMB and FB metrics are computed as
RMB = M ¯ O ¯
FB = 2 N i = 1 N | M i O i | M i + O i
where N is the total number of pairs of modeled (M, i.e., MODIS) and observed (O, i.e., AERONET) values. The theoretical range of FB is from −200% and +200%, with 0 indicating the best value. RMB ranges from 0 to + , and 1 is the best value.
In summary, the EE metric serves as a comprehensive evaluation criterion by integrating both absolute and relative errors. The proportion of retrievals falling within the EE range (=EE) is considered the primary indicator of algorithm performance. Additionally, CC, RMSE, RMB, and FB complement EE by capturing different aspects of retrieval accuracy, including systematic bias, overall error magnitude, and relative performance.

4. Results and Discussion

4.1. Overall Results Under Three Matching Methods

As the site measurements, satellite products, and QA selections are identical across all methods, spatial matching serves as the sole variable. Accordingly, the observed differences in performance metrics can be attributed exclusively to the spatial matching strategy. Table 2 presents the validation results for different spatial matching methods grouped by datasets.
In Table 2, the =EE percentage for the “optimal” method is consistently higher than the “average” and “direct” methods across all datasets, with an improvement range of 10% to 30%. Substantial reductions are also observed in both RMB and FB. These results suggest that retrieval values near the validation sites are generally more accurate. Some high-QA points, however, are influenced by subpixel clouds or shadows, making them less optimal. In contrast, the surrounding points, which are unaffected by these factors, provide a more accurate AOD estimate.
It is noteworthy that even under the “optimal” spatial matching strategy, the >EE results consistently exceed the <EE results for each product. This indicates that, among the outliers, overestimations are more frequent than underestimations. Moreover, the RMB is consistently positive across all datasets, further confirming a systematic tendency toward AOD overestimation. This positive bias may result from several contributing factors:
  • First, although the “optimal” method aims to select retrievals spatially closest to the AERONET site, it cannot fully eliminate the influence of subpixel clouds or cloud shadows. Residual thin clouds or elevated humidity levels may be misclassified as aerosol signals, leading to inflated AOD estimates. This effect is particularly pronounced in regions with frequent convective activity or in humid seasons, where cloud contamination is more difficult to detect.
  • Second, most aerosol retrieval algorithms favor pixels with lower surface reflectance, as these conditions enhance the aerosol signal-to-noise ratio. However, this selection preference may introduce a bias: by preferentially using pixels with darker surfaces, the algorithm may adopt surface reflectance values that are lower than the actual conditions at the time of observation. This can lead to a systematic overestimation of aerosol loading. Additionally, in heterogeneous or dynamic surface environments, static surface reflectance databases may not accurately represent current surface conditions, further contributing to retrieval errors.
In both the “direct” and “average” method validations, the AOD products from the DT algorithm demonstrated the lowest accuracy among the three algorithms (DT, DB, MAIAC), with a potential accuracy difference of 10-20% compared to the others. However, in the “optimal” method validation, the accuracy of the MODIS DT algorithm improved, approaching that of the DB algorithm. This indicates that the accuracy difference between the DB and DT algorithms is minimal at the algorithm level. Nevertheless, the significant discrepancy in product accuracy highlights substantial room for improvement in the DT algorithm’s quality control. For both the NOAA and SNPP products, the DB algorithm outperformed the DT algorithm by 10-17% in accuracy when the optimal comparison method was applied. This suggests that, in terms of accuracy, the DB algorithm on VIIRS has undergone more significant improvements after transitioning to operational use compared to the DT algorithm.
Interestingly, the 3 km resolution AOD product generated by the DT algorithm exhibited the greatest accuracy variation among the three validation methods. The “optimal” method improved validation accuracy by 25% to 30%, exceeding the accuracy of both the 10 km DT and DB algorithms. This result highlights two key observations:
  • First, the DT algorithm is highly sensitive to dark target pixels, showing particularly high accuracy for pixels that meet the dark target criteria. From the perspective of optimal pixel selection, accuracy improves significantly when a sufficient number of pixels meet the dark target criteria.
  • Second, as the resolution increases to 3 km, the higher pixel resolution and purer dark target signals lead to improved retrieval accuracy. However, for pixels that do not fully meet the criteria, the accuracy is more heavily impacted, resulting in greater uncertainty.
The DTB product, combined from the DT and DB products, primarily expanded the coverage but did not lead to a significant improvement in accuracy. The results show that, in the “direct” method, the =EE percentage of the DTB product is only marginally higher than that of the DT product and 6% lower than the DB product from the same sensor. The quality control method applied in the DT product is more lenient when selecting high-quality points, which results in a larger average deviation between retrieval values and measurements. This process likely introduced additional biases during synthesis, which may have counteracted the strengths of the DB algorithm, leading to a reduction in overall accuracy. Therefore, in terms of accuracy, the DTB product did not meet expectations.
In the “optimal” results, the MAIAC product performs the best. MOD(MAIAC) achieves the highest percentage of =EE at 94%, with MYD(MAIAC) closely following at 93.5%. The high accuracy of the MAIAC product can be attributed to two main factors. First, it utilizes time-series data, assuming that surface reflectance changes are minimal over short periods. Under this assumption, the algorithm identifies the clearest day within a time series, using a series of operations to obtain accurate Bidirectional Reflectance Factor (BRF) values, which are then used to retrieve the AOD for the target time [17]. During this process, the design of the time-series filter and the minimization of the objective function effectively filter out many undetected cloud and cloud shadow pixels, thus reducing errors caused by these factors. Second, the MAIAC algorithm retrieves AOD at a 1 km resolution. The higher spatial resolution of the pixels allows for purer land surface characteristics, which are more consistent with the algorithm’s assumptions compared to lower-resolution pixels. As a result, some pixels achieve higher AOD accuracy in the “optimal” method.
In both the “average” and “optimal” methods, products with higher resolution typically yield more matching pairs, primarily due to the purity of their pixels. In low-resolution products, certain pixel areas may be assigned lower QA flags due to cloud cover. However, in higher-resolution products, the same area is subdivided into smaller pixels, where cloud-free pixels are assigned higher QA flags, thus increasing the number of matching pairs. Additionally, as the successor to MODIS, the VIIRS sensor has a similar revisit cycle but offers a wider swath and higher spatial resolution. These features enable the VIIRS sensor to provide more matching data. When prioritizing the number of matches, both the VIIRS DB product and the MODIS MAIAC product demonstrate distinct advantages in terms of matching pairs and retrieval accuracy, making them highly recommended for use.
Table 3 presents validation results across different EE ranges to further support the previous conclusions. Four EE ranges are provided in the table. Here, “max (0.03;10%)” refers to selecting the larger value between absolute error and relative error as the EE range, which is narrower than the “(0.03 + 10%)” range. The results show that, although the EE percentage decreases across all products as the envelope range narrows, the previous conclusions remain valid. Additionally, the DT products, particularly those from VIIRS, perform poorly across different EE ranges. In contrast, the MOD(MAIAC) dataset consistently achieves the highest percentage under the “optimal” method, while the SNPP(DB) dataset performs best under the “direct” and “average” methods.

4.2. Impact of Land Cover and Seasonal Variability on Retrieval Accuracy

4.2.1. Land Cover Variability

Figure 3 presents the validation results of various satellite AOD products, expressed in terms of the expected error (EE) metric, under different spatial collocation strategies and land cover types. Although the “optimal” method generally improves validation accuracy compared to the “direct” and “average” methods, retrievals still exhibit a systematic overestimation bias. This bias is particularly pronounced over bareland surfaces and for the DT product.
Among the three algorithms, the MAIAC product shows the best performance over bareland regions under the “optimal” method, followed by DB, and DT performs the worst. The DT algorithm’s poor performance mainly results from its invalid assumptions over bright surfaces. Compared to the “average” method, MAIAC shows an exaggerated improvement under the “optimal” method, suggesting that the MAIAC product exhibits large variance and uncertainty over bare surfaces. Moreover, the underestimation bias of MAIAC is more evident in bareland areas than in other products. In such regions, high surface reflectance reduces the aerosol signal strength. Moreover, significant temporal variations in reflectance—caused by factors such as precipitation, wind-blown dust, or soil moisture—may cause the temporal filtering mechanism of MAIAC to misinterpret low AOD values as valid retrievals, thereby introducing a systematic underestimation. Furthermore, the dominance of surface-reflected radiance in the satellite signal complicates the extraction of the aerosol scattering component. To minimize retrieval errors arising from surface reflectance uncertainty, the MAIAC algorithm may be biased toward lower AOD values. Improving the adaptability of AOD retrieval algorithms over bareland remains a key challenge for future product development.
Among all evaluated products, the SNPP (DT) product exhibits the poorest performance across land cover types. As shown in Figure 3, SNPP (DT) yields the lowest proportion of retrievals within the EE envelope and the highest proportion exceeding the EE threshold for all land types, indicating large retrieval errors. These findings further support the view that the migration of the DT algorithm to the SNPP platform has not been successful.

4.2.2. Seasonal Variability

Different aerosol optical depth (AOD) products exhibit notable seasonal variability; however, the seasonal patterns observed at individual sites vary by region. As shown in Figure 4, the results represent the overall retrieval bias across global AERONET sites. AOD biases tend to be smaller and more stable during autumn (SON) and winter (DJF), whereas larger biases and greater variability are observed in spring (MAM) and summer (JJA). This pattern can be attributed to stronger evaporation during summer, which induces rapid changes in atmospheric humidity and wind speed. These dynamic conditions, including intensified cloud formation, enhanced hygroscopic growth of aerosols, and rapid changes in surface reflectance, collectively increase the susceptibility of satellite retrieval algorithms to systematic biases.
Among the spatial matching approaches, the optimal method effectively reduces AOD bias and improves the fraction of retrievals within the expected error (EE). Nevertheless, it does not eliminate the underlying seasonal trend, which primarily reflects algorithmic limitations in adapting to rapidly evolving atmospheric and surface conditions.

4.3. Comparison of DB and DT Products at Site Level

To provide a more detailed and intuitive comparison of the differences between the three methods, a statistical analysis was conducted at the site level. For simplicity in presentation and description, only the DT and DB algorithm products from the same sensor were compared. These products share the same satellite overpass time, ensuring that the observed differences can primarily be attributed to the retrieval algorithms used, thereby strengthening the reliability of the comparison.
To assess the impact of the three matching methods on the results at different AERONET sites, a comparison of the accuracy between the DT and DB datasets of the same product was plotted (Figure 5). For the MODIS dataset, the higher-accuracy MYD dataset was selected, while the longer-duration SNPP dataset was used for the VIIRS data. Thus, the statistical analysis utilized the following datasets: MYD(DT), MYD(DB), SNPP(DT), and SNPP(DB). To improve statistical reliability, sites with fewer than 50 AERONET matchups were excluded, resulting in 335 sites used for analysis.
Figure 5 clearly demonstrates that the “optimal” method significantly enhances validation accuracy across most sites. In a 1:1 scatter plot, points positioned closer to the top indicate better performance in the vertical-axis metric, while those near the right end reflect better performance in the horizontal-axis metric. Under the optimal method, the AOD retrievals for MYD products exhibit a more pronounced concentration toward the upper-right region of the plot compared to other spatial matching methods. This suggests that both DT and DB products experience substantial improvements in the =EE metric, with the majority of sites showing enhanced accuracy.
As shown in Figure 5, the majority of scatter points lie above the 1:1 reference line, indicating that the DB algorithm generally provides more accurate AOD retrievals than the DT algorithm. The DB algorithm utilizes additional information from the deep blue spectral band, which is less sensitive to surface reflectance variations and, therefore, more effective over bright surfaces, such as arid or semi-arid regions. Its broader applicability to various surface types further contributes to its superior performance at a greater number of sites. For the SNPP product, the outperformance of DB over DT at nearly all sites suggests a potential systematic bias in the implementation of the DT algorithm. To further examine this issue, a detailed comparative analysis is carried out at two representative sites.
Figure 6 and Figure 7 present the results of three spatial matching methods at two representative sites: Mexico City and São Paulo. Both sites are located in regions where the MODIS DT algorithm performs optimally, generally yielding higher retrieval accuracy than the DB algorithm. In densely vegetated urban areas, the DT algorithm can identify a sufficient number of “dark” targets, enabling more accurate retrievals. In contrast, the DB algorithm is more susceptible to interference from “bright” pixels in these regions. Among the three evaluated methods, the “optimal” approach consistently improves accuracy metrics across different products. However, not all products achieve the expected accuracy after applying the “optimal” method, particularly when the retrieval algorithm itself is not well-suited to the region.
At the Mexico City site, the performance of the MODIS DB product is affected by persistent pollution [18], whereas the VIIRS product demonstrates significant improvement following the implementation of an updated smoke algorithm. As shown in the figures, inherent limitations in the MODIS DB algorithm persist even when the “optimal” method is applied, resulting in relatively low accuracy. Compared to the “average” method, the “optimal” approach effectively minimizes residual cloud contamination and reduces the influence of outliers, resulting in a tighter scatter distribution within the expected error envelope. The validation images generated by the “optimal” method are clearer, facilitating the identification of potential issues. Similarly, at the São Paulo site, the VIIRS DT algorithm does not achieve the desired accuracy even after applying the “optimal” method, suggesting that further refinement of the algorithm is necessary.
The site-level analysis highlights differences among the three spatial matching methods while also yielding additional insights. Although the DB product generally exhibits higher accuracy than the DT product in most regions, site-specific comparisons indicate that each algorithm has distinct advantages, and the DB product cannot entirely replace the DT product. As the first global aerosol retrieval algorithm implemented for MODIS, the DT algorithm inevitably involves a trade-off between retrieval quality and spatial coverage. To maximize its advantages, refining the quality control mechanism is recommended to enhance product reliability.
Combining the DT and DB products into a unified DTB product presents a promising strategy. Currently, the DTB product determines the applicability of DT or DB retrievals based on NDVI and utilizes QA values for synthesis in transition regions. However, this approach remains relatively simplistic and has not led to a significant improvement in overall accuracy [1]. Based on the above analysis, the following strategies could enhance the accuracy of the DTB product:
  • Optimization of surface feature selection: Existing site data can be leveraged to identify the most suitable surface characteristics for the DT algorithm. A global mask could then be developed to ensure that the DT algorithm is applied exclusively in these regions.
  • Improved resolution adaptation: Given the strict applicability assumptions of the DT algorithm, employing it at higher resolutions would allow for the selection of clearer pixels that meet retrieval criteria, thereby enhancing overall accuracy.

5. Conclusions

This study systematically validated MODIS and VIIRS AOD products against AERONET ground observations using three spatial matching methods: “direct”, “average”, and “optimal”. The results indicate that the “optimal” method significantly reduces errors associated with quality control algorithms but does not eliminate the inherent retrieval biases of the algorithms themselves. Consequently, the “optimal” method provides a more accurate assessment of retrieval algorithm performance.
It is important to note that the “optimal” method represents the upper bound of retrieval accuracy and, therefore, cannot entirely replace the “average” method. Instead, it should be considered a complementary approach. As a post-processing validation method, the “optimal” method is site-dependent and cannot be directly integrated into the retrieval process. However, it can be applied to filter existing datasets, thereby generating high-quality sample sets, which can be used as optimized inputs for deep learning models and other advanced analytical approaches.
In summary, this study highlights the unique advantages of the “optimal” spatial matching method in more accurately reflecting the intrinsic retrieval accuracy of AOD algorithms. These findings underscore the importance of enhancing quality control mechanisms to improve AOD product accuracy and reliability in satellite-based aerosol monitoring. For data users, these insights facilitate more optimized dataset applications, while for algorithm developers, they provide clearer pathways for algorithmic enhancement.

Author Contributions

Conceptualization, B.D.; methodology, B.D., B.Z., H.C. and S.W.; validation, B.D.; data curation, B.D., S.W. and H.C.; resources, B.Z., J.W., A.Y. and Q.L.; writing–original draft preparation, B.D.; writing–review and editing B.D., B.Z., H.C., Y.Q., X.W., Q.L., H.Z. and J.J.; funding acquisition, B.Z. and Q.L. All authors have read and agreed to the published version of the manuscript.

Funding

This research was supported by the National Key Research and Development Program of China (No. 2024YFF1306102).

Data Availability Statement

The datasets used in this study are publicly available. These data were derived from the following resources available in the public domain: MODIS and VIIRS data at https://search.earthdata.nasa.gov (accessed on 26 October 2024); AERONET data at https://aeronet.gsfc.nasa.gov/cgi-bin/webtool_aod_v3 (accessed on 13 May 2024).

Acknowledgments

We thank the reviewers for their insightful comments and suggestions, which helped improve our manuscript.

Conflicts of Interest

The Author Jingxiong Jiang was employed by the Space Star Technology Co., Ltd. The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest. The authors declare no conflicts of interest.

Appendix A. Validation Results

Table A1. Validation results of three methods with at least one high-quality pixel (50 × 50 km2 window).
Table A1. Validation results of three methods with at least one high-quality pixel (50 × 50 km2 window).
AliasMethod>EE(%)=EE(%)<EE(%)R2RMSECCRMBFB(%)PairsSites
MOD3K
(DT)
direct29.5362.098.380.5590.1560.831.2118.01111,304814
average37.159.113.790.6380.1290.8741.36536.78264,711854
optimal10.0889.450.480.90.0670.9551.11714.6270,330854
MOD
(DT)
direct25.664.739.670.5860.1470.8261.14212.68139,319821
average28.4466.345.220.7260.1150.8911.2324.11212,138848
optimal12.8385.41.770.8750.0770.9431.11614.75217,053848
MOD
(DB)
direct12.8973.5413.570.650.1320.8390.966−14.26182,715795
average14.0374.1311.840.7290.1160.8741.004−5.79226,706808
optimal5.589.395.110.8730.080.9380.988−3.29226,706808
MOD
(DTB)
direct22.5866.1811.240.5780.1450.821.1024.73179,540837
average23.6869.486.830.7230.1150.8851.17215.59244,392859
optimal9.5488.092.370.8810.0750.9441.0768.09249,095859
MOD
(MAIAC)
direct16.1974.819.00.7540.0950.8721.0286.82232,087829
average18.3474.327.340.750.10.8681.07216.05310,077760
optimal3.0295.921.060.9430.0470.9711.013.54309,173760
MYD3K
(DT)
direct24.8764.5510.580.4380.1750.7491.1016.0104,923824
average29.4766.044.490.6580.1250.8711.26822.33245,354850
optimal8.0991.320.590.8990.0670.9531.0899.87254,751850
MYD
(DT)
direct21.5366.2912.180.4350.1760.7281.030.55121,026824
average21.6372.415.950.7350.1120.8871.1510.77190,472839
optimal9.6988.162.140.870.0780.9381.0767.12197,096839
MYD
(DB)
direct12.8776.1910.940.6530.130.840.997−9.17155,875775
average13.9976.269.750.7440.1130.8811.026−1.98201,296797
optimal5.5890.264.150.8790.0780.941.002−1.24201,295797
MYD
(DTB)
direct19.9467.4912.570.4590.1670.7441.025−2.78157,158839
average19.1474.016.850.7340.1120.8861.126.34222,677850
optimal7.7489.892.370.8770.0750.9421.0543.43229,127850
MYD
(MAIAC)
direct21.0671.287.660.7580.0970.8751.08917.43251,899840
average22.271.156.650.7480.1020.871.1224.21320,728765
optimal3.5695.291.150.9390.050.9691.0175.02319,633765
NOAA
(DB)
direct13.178.138.770.660.1330.8371.02−2.41115,751561
average10.582.986.510.8220.0910.9131.0191.13151,796562
optimal3.095.681.320.9470.050.9731.012.04151,796562
NOAA
(DT)
direct24.3160.8714.810.2080.1990.671.09−2.37100,700581
average19.8770.589.540.6640.1240.8621.1111.12152,090584
optimal6.0491.922.040.8830.0710.9441.0411.7165,633587
SNPP
(DB)
direct12.179.328.580.6840.1260.8471.01−2.92239,122800
average9.6383.516.860.8320.0890.9171.0030.03313,791814
optimal2.6595.911.450.9470.050.9731.0041.8313,791814
SNPP
(DT)
direct37.255.37.51−0.0270.2260.71.39526.09203,907834
average38.4458.033.530.4550.1570.8611.43534.39325,935850
optimal13.0386.270.690.8050.0930.9261.16715.35336,799850
Table A2. Validation results of three methods with a 20% high-quality pixel constraint (30 × 30 km2 window).
Table A2. Validation results of three methods with a 20% high-quality pixel constraint (30 × 30 km2 window).
AliasMethod>EE(%)=EE(%)<EE(%)R2RMSECCRMBFB(%)PairsSites
MOD3K
(DT)
direct29.5362.098.380.5590.1560.831.2118.01111,304814
average27.5968.094.330.7560.1070.9131.23323.36157,466799
optimal2.6796.860.460.9720.0360.9871.0325.19162,100799
MOD
(DT)
direct25.664.739.670.5860.1470.8261.14212.68139,319821
average24.8669.435.710.7490.1110.9041.18817.28153,143810
optimal12.0485.292.670.8780.0770.9461.09510.36156,800811
MOD
(DB)
direct12.8973.5413.570.650.1320.8390.966−14.26182,715795
average12.975.6711.430.7560.110.8880.994−7.54187,093782
optimal5.7587.96.350.8680.0810.9360.973−6.5187,093782
MOD
(DTB)
direct22.5866.1811.240.5780.1450.821.1024.73179,540837
average21.3571.117.540.7350.1130.8931.1419.91190,897836
optimal9.8186.33.890.8670.0790.9391.063.89194,512837
MOD
(MAIAC)
direct16.1974.819.00.7540.0950.8721.0286.82232,087829
average12.4680.097.450.8140.0810.9021.0119.45270,861798
optimal0.6898.161.160.970.0320.9860.9851.08270,450798
MYD3K
(DT)
direct24.8764.5510.580.4380.1750.7491.1016.0104,923824
average20.0975.084.830.7720.10.9121.1518.69141,317805
optimal1.598.060.440.9750.0330.9881.0161.35148,716805
MYD
(DT)
direct21.5366.2912.180.4350.1760.7281.030.55121,026824
average19.3574.695.960.7680.1040.9061.1235.27132,668806
optimal8.6888.52.830.8890.0710.9491.0562.41137,172806
MYD
(DB)
direct12.8776.1910.940.6530.130.840.997−9.17155,875775
average12.6578.089.270.7730.1050.8961.018−3.47160,047759
optimal5.5889.54.920.880.0760.9420.991−3.74160,046759
MYD
(DTB)
direct19.9467.4912.570.4590.1670.7441.025−2.78157,158839
average17.8275.137.040.7530.1070.8981.1032.44167,487829
optimal7.7288.833.440.8790.0740.9441.04−0.5171,837829
MYD
(MAIAC)
direct21.0671.287.660.7580.0970.8751.08917.43251,899840
average17.5775.786.650.8080.0850.91.06919.44287,430805
optimal1.3197.41.290.9640.0370.9830.9923.33286,755805
NOAA
(DB)
direct13.178.138.770.660.1330.8371.02−2.41115,751561
average8.7785.186.050.8510.0850.9281.007−0.67115,163538
optimal1.5797.041.390.9570.0450.9790.991−0.64115,163538
NOAA
(DT)
direct24.3160.8714.810.2080.1990.671.09−2.37100,700581
average19.1871.699.130.6580.1220.8721.112−1.41107,791574
optimal3.7692.993.250.90.0640.9531.001−4.77116,924577
SNPP
(DB)
direct12.179.328.580.6840.1260.8471.01−2.92239,122800
average7.8385.926.250.8630.0810.9330.991−1.52239,053769
optimal1.397.231.470.9610.0430.980.986−0.77239,053769
SNPP
(DT)
direct37.255.37.51−0.0270.2260.71.39526.09203,907834
average36.4960.323.190.4390.1540.8741.42931.31233,619820
optimal11.2687.750.990.8420.0810.9441.14512.21241,513820

References

  1. Bilal, M.; Nichol, J.E.; Wang, L. New customized methods for improvement of the MODIS C6 Dark Target and Deep Blue merged aerosol product. Remote Sens. Environ. 2017, 197, 115–124. [Google Scholar]
  2. Haywood, J.; Boucher, O. Estimates of the direct and indirect radiative forcing due to tropospheric aerosols: A review. Rev. Geophys. 2000, 38, 513–543. [Google Scholar]
  3. Ramanathan, V.; Crutzen, P.J.; Kiehl, J.; Rosenfeld, D. Aerosols, climate, and the hydrological cycle. Science 2001, 294, 2119–2124. [Google Scholar] [CrossRef]
  4. Ramanathan, V.; Carmichael, G. Global and regional climate changes due to black carbon. Nat. Geosci. 2008, 1, 221–227. [Google Scholar]
  5. Brunekreef, B.; Holgate, S.T. Air pollution and health. Lancet 2002, 360, 1233–1242. [Google Scholar] [CrossRef]
  6. Pöschl, U. Atmospheric aerosols: Composition, transformation, climate and health effects. Angew. Chem. Int. Ed. 2005, 44, 7520–7540. [Google Scholar] [CrossRef]
  7. Dubovik, O.; Holben, B.; Eck, T.F.; Smirnov, A.; Kaufman, Y.J.; King, M.D.; Tanré, D.; Slutsker, I. Variability of absorption and optical properties of key aerosol types observed in worldwide locations. J. Atmos. Sci. 2002, 59, 590–608. [Google Scholar]
  8. Kaufman, Y.J.; Tanré, D.; Boucher, O. A satellite view of aerosols in the climate system. Nature 2002, 419, 215–223. [Google Scholar] [CrossRef]
  9. Kassianov, E.; Cromwell, E.; Monroe, J.; Riihimaki, L.D.; Flynn, C.; Barnard, J.; Michalsky, J.J.; Hodges, G.; Shi, Y.; Comstock, J.M. Harmonized and high-quality datasets of aerosol optical depth at a US continental site, 1997–2018. Sci. Data 2021, 8, 82. [Google Scholar]
  10. Papachristopoulou, K.; Raptis, I.P.; Gkikas, A.; Fountoulakis, I.; Masoom, A.; Kazadzis, S. Aerosol optical depth regime over megacities of the world. Atmos. Chem. Phys. 2022, 22, 15703–15727. [Google Scholar]
  11. Wang, Y.; Wang, J.; Levy, R.C.; Shi, Y.R.; Mattoo, S.; Reid, J.S. First retrieval of AOD at fine resolution over shallow and turbid coastal waters from MODIS. Geophys. Res. Lett. 2021, 48, e2021GL094344. [Google Scholar] [CrossRef]
  12. Murphy, R.E.; Barnes, W.L.; Lyapustin, A.I.; Privette, J.; Welsch, C.; DeLuccia, F.; Swenson, H.; Schueler, C.F.; Ardanuy, P.E.; Kealy, P.S. Using VIIRS to provide data continuity with MODIS. In Proceedings of the IGARSS 2001. Scanning the Present and Resolving the Future. Proceedings. IEEE 2001 International Geoscience and Remote Sensing Symposium (Cat. No. 01CH37217), Sydney, NSW, Australia, 9–13 July 2001; IEEE: New York, NY, USA, 2001; Volume 3, pp. 1212–1214. [Google Scholar]
  13. Levy, R.; Munchak, L.; Mattoo, S.; Patadia, F.; Remer, L.; Holz, R. Towards a long-term global aerosol optical depth record: Applying a consistent aerosol retrieval algorithm to MODIS and VIIRS-observed reflectance. Atmos. Meas. Tech. 2015, 8, 4083–4110. [Google Scholar]
  14. Sayer, A.M.; Hsu, N.C.; Lee, J.; Kim, W.V.; Dutcher, S.T. Validation, stability, and consistency of MODIS Collection 6.1 and VIIRS Version 1 Deep Blue aerosol data over land. J. Geophys. Res. Atmos. 2019, 124, 4658–4688. [Google Scholar]
  15. Kaufman, Y.J.; Wald, A.E.; Remer, L.A.; Gao, B.C.; Li, R.R.; Flynn, L. The MODIS 2.1-/spl mu/m channel-correlation with visible reflectance for use in remote sensing of aerosol. IEEE Trans. Geosci. Remote Sens. 1997, 35, 1286–1298. [Google Scholar]
  16. Hsu, N.C.; Tsay, S.C.; King, M.D.; Herman, J.R. Aerosol properties over bright-reflecting source regions. IEEE Trans. Geosci. Remote Sens. 2004, 42, 557–569. [Google Scholar]
  17. Lyapustin, A.; Wang, Y.; Laszlo, I.; Kahn, R.; Korkin, S.; Remer, L.; Levy, R.; Reid, J. Multiangle implementation of atmospheric correction (MAIAC): 2. Aerosol algorithm. J. Geophys. Res. Atmos. 2011, 116. [Google Scholar] [CrossRef]
  18. Sayer, A.; Munchak, L.; Hsu, N.; Levy, R.; Bettenhausen, C.; Jeong, M.J. MODIS Collection 6 aerosol products: Comparison between Aqua’s e-Deep Blue, Dark Target, and “merged” data sets, and usage recommendations. J. Geophys. Res. Atmos. 2014, 119, 13–965. [Google Scholar]
  19. Ichoku, C.; Chu, D.A.; Mattoo, S.; Kaufman, Y.J.; Remer, L.A.; Tanré, D.; Slutsker, I.; Holben, B.N. A spatio-temporal approach for global validation and analysis of MODIS aerosol products. Geophys. Res. Lett. 2002, 29, MOD1-1–MOD1-4. [Google Scholar]
  20. Mhawish, A.; Banerjee, T.; Sorek-Hamer, M.; Lyapustin, A.; Broday, D.M.; Chatfield, R. Comparison and evaluation of MODIS Multi-angle Implementation of Atmospheric Correction (MAIAC) aerosol product over South Asia. Remote Sens. Environ. 2019, 224, 12–28. [Google Scholar]
  21. De Leeuw, G.; Holzer-Popp, T.; Bevan, S.; Davies, W.H.; Descloitres, J.; Grainger, R.G.; Griesfeller, J.; Heckel, A.; Kinne, S.; Klüser, L.; et al. Evaluation of seven European aerosol optical depth retrieval algorithms for climate analysis. Remote Sens. Environ. 2015, 162, 295–315. [Google Scholar]
  22. Wei, J.; Li, Z.; Peng, Y.; Sun, L. MODIS Collection 6.1 aerosol optical depth products over land and ocean: Validation and comparison. Atmos. Environ. 2019, 201, 428–440. [Google Scholar] [CrossRef]
  23. Wei, J.; Li, Z.; Sun, L.; Peng, Y.; Liu, L.; He, L.; Qin, W.; Cribb, M. MODIS Collection 6.1 3 km resolution aerosol optical depth product: Global evaluation and uncertainty analysis. Atmos. Environ. 2020, 240, 117768. [Google Scholar] [CrossRef]
  24. Wang, Q.; Li, S.; Yang, J.; Zhou, D. Evaluation and comparison of VIIRS dark target and deep blue aerosol products over land. Sci. Total Environ. 2023, 869, 161667. [Google Scholar] [CrossRef] [PubMed]
  25. Qin, W.; Fang, H.; Wang, L.; Wei, J.; Zhang, M.; Su, X.; Bilal, M.; Liang, X. MODIS high-resolution MAIAC aerosol product: Global validation and analysis. Atmos. Environ. 2021, 264, 118684. [Google Scholar] [CrossRef]
  26. Su, Y.; Xie, Y.; Tao, Z.; Hu, Q.; Yu, T.; Gu, X. Validation and inter-comparison of MODIS and VIIRS aerosol optical depth products against data from multiple observation networks over East China. Atmos. Environ. 2021, 247, 118205. [Google Scholar] [CrossRef]
  27. He, Q.; Li, C.; Geng, F.; Zhou, G.; Gao, W.; Yu, W.; Li, Z.; Du, M. A parameterization scheme of aerosol vertical distribution for surface-level visibility retrieval from satellite remote sensing. Remote Sens. Environ. 2016, 181, 1–13. [Google Scholar] [CrossRef]
  28. Myhre, G.; Stordal, F.; Johnsrud, M.; Kaufman, Y.; Rosenfeld, D.; Storelvmo, T.; Kristjansson, J.E.; Berntsen, T.K.; Myhre, A.; Isaksen, I.S. Aerosol-cloud interaction inferred from MODIS satellite data and global aerosol models. Atmos. Chem. Phys. 2007, 7, 3081–3101. [Google Scholar] [CrossRef]
  29. Levy, R.; Mattoo, S.; Munchak, L.; Remer, L.; Sayer, A.; Patadia, F.; Hsu, N. The Collection 6 MODIS aerosol products over land and ocean. Atmos. Meas. Tech. 2013, 6, 2989–3034. [Google Scholar] [CrossRef]
  30. Henderson, B.G.; Chylek, P. The effect of spatial resolution on satellite aerosol optical depth retrieval. IEEE Trans. Geosci. Remote Sens. 2005, 43, 1984–1990. [Google Scholar] [CrossRef]
  31. Cai, H.; Zhong, B.; Liu, H.; Du, B.; Liu, Q.; Wu, S.; Li, L.; Yang, A.; Wu, J.; Gu, X.; et al. An improved deep learning network for AOD retrieving from remote sensing imagery focusing on sub-pixel cloud. GISci. Remote Sens. 2023, 60, 2262836. [Google Scholar] [CrossRef]
  32. Holben, B.N.; Eck, T.F.; Slutsker, I.; Tanré, D.; Buis, J.; Setzer, A.; Vermote, E.; Reagan, J.A.; Kaufman, Y.; Nakajima, T.; et al. AERONET—A federated instrument network and data archive for aerosol characterization. Remote Sens. Environ. 1998, 66, 1–16. [Google Scholar] [CrossRef]
  33. Dubovik, O.; Smirnov, A.; Holben, B.; King, M.; Kaufman, Y.; Eck, T.; Slutsker, I. Accuracy assessments of aerosol optical properties retrieved from Aerosol Robotic Network (AERONET) Sun and sky radiance measurements. J. Geophys. Res. Atmos. 2000, 105, 9791–9806. [Google Scholar] [CrossRef]
  34. Friedl, M.A.; McIver, D.K.; Hodges, J.C.; Zhang, X.Y.; Muchoney, D.; Strahler, A.H.; Woodcock, C.E.; Gopal, S.; Schneider, A.; Cooper, A.; et al. Global land cover mapping from MODIS: Algorithms and early results. Remote Sens. Environ. 2002, 83, 287–302. [Google Scholar] [CrossRef]
  35. O’Neill, N.; Eck, T.; Holben, B.; Smirnov, A.; Dubovik, O.; Royer, A. Bimodal size distribution influences on the variation of Angstrom derivatives in spectral and optical depth space. J. Geophys. Res. Atmos. 2001, 106, 9787–9806. [Google Scholar] [CrossRef]
  36. Levy, R.; Remer, L.; Kleidman, R.; Mattoo, S.; Ichoku, C.; Kahn, R.; Eck, T. Global evaluation of the Collection 5 MODIS dark-target aerosol products over land. Atmos. Chem. Phys. 2010, 10, 10399–10420. [Google Scholar] [CrossRef]
  37. Hsu, N.C.; Tsay, S.C.; King, M.D.; Herman, J.R. Deep blue retrievals of Asian aerosol properties during ACE-Asia. IEEE Trans. Geosci. Remote Sens. 2006, 44, 3180–3195. [Google Scholar] [CrossRef]
  38. Sayer, A.M.; Hsu, N.; Bettenhausen, C.; Jeong, M.J.; Holben, B.; Zhang, J. Global and regional evaluation of over-land spectral aerosol optical depth retrievals from SeaWiFS. Atmos. Meas. Tech. 2012, 5, 1761–1778. [Google Scholar] [CrossRef]
  39. Che, H.; Gui, K.; Xia, X.; Wang, Y.; Holben, B.N.; Goloub, P.; Cuevas-Agulló, E.; Wang, H.; Zheng, Y.; Zhao, H.; et al. Large contribution of meteorological factors to inter-decadal changes in regional aerosol optical depth. Atmos. Chem. Phys. 2019, 19, 10497–10523. [Google Scholar] [CrossRef]
Figure 1. AOD satellite-site matching framework.
Figure 1. AOD satellite-site matching framework.
Remotesensing 17 01235 g001
Figure 2. Schematic diagram of three spatial matching methods. Squares represent pixels, while circles indicate ground sites. The values within the squares correspond to the AOD (QA = 3) retrievals from MOD(DT) for the granule “MOD04_L2.A2015219.1120.061.2017322082600.hdf”. The site AOD represents measurements from the AERONET Granada site. In panel (a), different colors denote variations in AOD values between the site measurement and pixel retrievals. Panels (bd) illustrate the AOD values after applying different spatial matching methods. The closer color similarity between pixels and the site indicates a smaller retrieval–measurement difference.
Figure 2. Schematic diagram of three spatial matching methods. Squares represent pixels, while circles indicate ground sites. The values within the squares correspond to the AOD (QA = 3) retrievals from MOD(DT) for the granule “MOD04_L2.A2015219.1120.061.2017322082600.hdf”. The site AOD represents measurements from the AERONET Granada site. In panel (a), different colors denote variations in AOD values between the site measurement and pixel retrievals. Panels (bd) illustrate the AOD values after applying different spatial matching methods. The closer color similarity between pixels and the site indicates a smaller retrieval–measurement difference.
Remotesensing 17 01235 g002
Figure 3. Comparison of AOD retrieval accuracy across different land cover types using various satellite aerosol products. Each pie chart represents the proportion of retrievals falling within (=EE, gray), above (>EE, yellow), or below (<EE, red) the expected error range (EE). The x-axis categorizes IGBP land cover types from the MCD12C1 product, with three spatial matching methods applied: direct (d), average (a), and optimal (o).
Figure 3. Comparison of AOD retrieval accuracy across different land cover types using various satellite aerosol products. Each pie chart represents the proportion of retrievals falling within (=EE, gray), above (>EE, yellow), or below (<EE, red) the expected error range (EE). The x-axis categorizes IGBP land cover types from the MCD12C1 product, with three spatial matching methods applied: direct (d), average (a), and optimal (o).
Remotesensing 17 01235 g003
Figure 4. Seasonal variations in AOD bias for different satellite aerosol products and retrieval methods. Red, green, and blue represent the direct, average, and optimal spatial matching methods, respectively. The box plots illustrate the AOD bias distributions across four seasons: DJF (December–February), MAM (March–May), JJA (June–August), and SON (September–November). The right y-axis denotes the percentage of retrievals falling within the expected error (EE), with circular markers for each method.
Figure 4. Seasonal variations in AOD bias for different satellite aerosol products and retrieval methods. Red, green, and blue represent the direct, average, and optimal spatial matching methods, respectively. The box plots illustrate the AOD bias distributions across four seasons: DJF (December–February), MAM (March–May), JJA (June–August), and SON (September–November). The right y-axis denotes the percentage of retrievals falling within the expected error (EE), with circular markers for each method.
Remotesensing 17 01235 g004
Figure 5. Comparison of =EE between DT and DB AOD products for MODIS (Aqua) and VIIRS (SNPP) under different spatial matching methods. The color scale indicates the density of data points, with red representing higher densities. The diagonal black line represents the 1:1 reference line, where DT and DB products have equal EE percentages.
Figure 5. Comparison of =EE between DT and DB AOD products for MODIS (Aqua) and VIIRS (SNPP) under different spatial matching methods. The color scale indicates the density of data points, with red representing higher densities. The diagonal black line represents the 1:1 reference line, where DT and DB products have equal EE percentages.
Remotesensing 17 01235 g005
Figure 6. Validation charts of DT and DB products for Mexico City site.
Figure 6. Validation charts of DT and DB products for Mexico City site.
Remotesensing 17 01235 g006
Figure 7. Validation charts of DT and DB products for Sao Paulo site.
Figure 7. Validation charts of DT and DB products for Sao Paulo site.
Remotesensing 17 01235 g007
Table 1. AOD satellite datasets utilized in this study.
Table 1. AOD satellite datasets utilized in this study.
ProductVersionScientific DataSetAliasAlgorithmSpatial
Resolution
MOD04_3KC6.1Image_Optical_Depth_Land
_And_Ocean
MOD3K(DT)DT3 km
MOD04_L2C6.1Image_Optical_Depth_Land
_And_Ocean
MOD(DT)DT10 km
MOD04_L2C6.1Deep_Blue_Aerosol_Optical_Depth
_550_Land
MOD(DB)DB10 km
MOD04_L2C6.1AOD_550_Dark_Target_Deep_Blue
_Combined
MOD(DTB)DTB10 km
MYD04_3KC6.1Image_Optical_Depth_Land
_And_Ocean
MYD3K(DT)DT3 km
MYD04_L2C6.1Image_Optical_Depth_Land
_And_Ocean
MYD(DT)DT10 km
MYD04_L2C6.1Deep_Blue_Aerosol_Optical_Depth
_550_Land
MYD(DB)DB10 km
MYD04_L2C6.1AOD_550_Dark_Target_Deep_Blue
_Combined
MYD(DTB)DTB10 km
AERDB_L2_VIIRS
_NOAA20
V2.0Aerosol_Optical_Thickness
_550_Land
NOAA(DB)DB6 km
AERDT_L2_VIIRS_NOAA20V2.0Image_Optical_Depth_Land
_And_Ocean
NOAA(DT)DT6 km
AERDB_L2_VIIRS_SNPPV2.0Aerosol_Optical_Thickness
_550_Land
SNPP(DB)DB6 km
AERDT_L2_VIIRS_SNPPV2.0Image_Optical_Depth_Land
_And_Ocean
SNPP(DT)DT6 km
MCD19A2C6.1Optical_Depth_055MOD(MAIAC)
/MYD(MAIAC)
MAIAC1 km
Table 2. Validation results of matched pairs for the products using three matching methods.
Table 2. Validation results of matched pairs for the products using three matching methods.
AliasMethod>EE(%)=EE(%)<EE(%)R2RMSECCRMBFB(%)PairsSites
MOD3K
(DT)
direct29.5362.098.380.5590.1560.831.2118.01111,304814
average38.6457.93.450.60.1340.8751.39837.94260,031851
optimal13.785.690.610.8570.080.941.16118.4265,420852
MOD
(DT)
direct25.664.739.670.5860.1470.8261.14212.68139,319821
average27.766.95.40.7170.1160.8951.22621.36183,362830
optimal16.8980.252.860.8290.090.9281.14615.48187,019831
MOD
(DB)
direct12.8973.5413.570.650.1320.8390.966−14.26182,715795
average14.0774.4311.50.7320.1150.8781.01−6.17208,594797
optimal7.6585.46.950.8320.0910.920.991−5.24208,594797
MOD
(DTB)
direct22.5866.1811.240.5780.1450.821.1024.73179,540837
average23.5969.167.260.7070.1180.8851.17112.99219,959848
optimal13.5182.44.090.8230.0910.9221.0997.7223,574849
MOD
(MAIAC)
direct16.1974.819.00.7540.0950.8721.0286.82232,087829
average19.4173.557.040.7430.0980.8661.09216.72319,655804
optimal4.294.451.350.9270.0520.9631.025.06318,732804
MYD3K
(DT)
direct24.8764.5510.580.4380.1750.7491.1016.0104,923824
average30.9765.063.970.6320.1280.8741.323.92240,115843
optimal10.7888.470.750.8650.0770.9411.1211.99248,806843
MYD
(DT)
direct21.5366.2912.180.4350.1760.7281.030.55121,026824
average21.7272.555.730.730.1120.8951.1569.13160,902822
optimal12.884.063.140.8330.0870.9271.0996.65165,406822
MYD
(DB)
direct12.8776.1910.940.6530.130.840.997−9.17155,875775
average13.9676.739.310.7560.110.891.035−1.94181,428783
optimal7.7186.815.480.850.0860.9291.011−2.17181,427783
MYD
(DTB)
direct19.9467.4912.570.4590.1670.7441.025−2.78157,158839
average19.773.486.820.7210.1130.8891.135.29195,926837
optimal11.085.273.730.830.0880.9251.0752.7200,276837
MYD
(MAIAC)
direct21.0671.287.660.7580.0970.8751.08917.43251,899840
average23.570.166.340.7360.1020.8651.14225.11331,285811
optimal5.0493.531.430.9190.0560.9591.037.24330,109811
NOAA
(DB)
direct13.178.138.770.660.1330.8371.02−2.41115,751561
average10.8882.946.190.8260.090.9171.0311.57140,319557
optimal4.0393.912.060.9270.0590.9631.0111.09140,319557
NOAA
(DT)
direct24.3160.8714.810.2080.1990.671.09−2.37100,700581
average23.9367.778.30.5680.1370.8481.1866.7141,623579
optimal9.886.63.610.7940.0930.9111.0752.0151,289582
SNPP
(DB)
direct12.179.328.580.6840.1260.8471.01−2.92239,122800
average9.8483.746.430.8380.0870.9221.0140.73291,049805
optimal3.5694.282.160.9310.0570.9661.0040.88291,049805
SNPP
(DT)
direct37.255.37.51−0.0270.2260.71.39526.09203,907834
average42.0454.982.980.2580.1770.8431.52838.52302,995841
optimal20.0978.721.190.6360.1230.8881.26821.44311,172841
Table 3. Validation results across different EE ranges for AOD products.
Table 3. Validation results across different EE ranges for AOD products.
AliasMethod=EE
(0.05 + 20%)(0.05 + 15%)(0.03 + 10%)max
(0.03;10%)
MOD3K
(DT)
direct65.8162.0943.333.41
average61.3957.939.4730.32
optimal86.7985.6978.3274.09
MOD
(DT)
direct68.5664.7345.3735.13
average70.6466.946.9836.15
optimal82.4680.2566.9458.42
MOD
(DB)
direct76.8873.5454.9345.25
average77.8274.4355.7645.71
optimal87.585.473.8966.2
MOD
(DTB)
direct69.8366.1847.4537.75
average72.869.1649.7239.22
optimal84.5382.470.061.72
MOD
(MAIAC)
direct77.9174.8155.8945.41
average76.5573.5555.3344.9
optimal95.0794.4591.0588.74
MYD3K
(DT)
direct68.0464.5546.8236.83
average68.2965.0647.5437.49
optimal89.4688.4782.5578.64
MYD
(DT)
direct69.8166.2947.9937.72
average76.0172.5553.5542.26
optimal86.0984.0671.7762.94
MYD
(DB)
direct79.276.1958.748.96
average79.9376.7358.8448.54
optimal88.7886.8176.0468.61
MYD
(DTB)
direct70.967.4949.6539.92
average76.8773.4854.944.0
optimal87.2285.2773.765.26
MYD
(MAIAC)
direct74.3471.2852.3741.95
average73.1870.1651.5741.03
optimal94.2493.5389.4686.94
NOAA
(DB)
direct81.1378.1360.6850.23
average85.8382.9465.7554.33
optimal94.9193.9187.8982.77
NOAA
(DT)
direct64.3860.8742.5533.25
average71.2367.7748.8538.19
optimal88.1886.676.9970.34
SNPP
(DB)
direct82.2579.3262.4251.87
average86.6183.7467.0855.74
optimal95.2594.2888.483.37
SNPP
(DT)
direct58.3655.339.6131.56
average58.1254.9838.3330.24
optimal80.5678.7268.3161.97
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Du, B.; Zhong, B.; Cai, H.; Wu, S.; Qiao, Y.; Wang, X.; Yang, A.; Wu, J.; Liu, Q.; Jiang, J.; et al. Improving AOD Algorithm Evaluation: A Spatial Matching Method for Minimizing Quality Control Bias. Remote Sens. 2025, 17, 1235. https://doi.org/10.3390/rs17071235

AMA Style

Du B, Zhong B, Cai H, Wu S, Qiao Y, Wang X, Yang A, Wu J, Liu Q, Jiang J, et al. Improving AOD Algorithm Evaluation: A Spatial Matching Method for Minimizing Quality Control Bias. Remote Sensing. 2025; 17(7):1235. https://doi.org/10.3390/rs17071235

Chicago/Turabian Style

Du, Bailin, Bo Zhong, He Cai, Shanlong Wu, Yang Qiao, Xiaoya Wang, Aixia Yang, Junjun Wu, Qinhuo Liu, Jinxiong Jiang, and et al. 2025. "Improving AOD Algorithm Evaluation: A Spatial Matching Method for Minimizing Quality Control Bias" Remote Sensing 17, no. 7: 1235. https://doi.org/10.3390/rs17071235

APA Style

Du, B., Zhong, B., Cai, H., Wu, S., Qiao, Y., Wang, X., Yang, A., Wu, J., Liu, Q., Jiang, J., & Zhang, H. (2025). Improving AOD Algorithm Evaluation: A Spatial Matching Method for Minimizing Quality Control Bias. Remote Sensing, 17(7), 1235. https://doi.org/10.3390/rs17071235

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop