*Article* **Validation and Accuracy Assessment of MODIS C6.1 Aerosol Products over the Heavy Aerosol Loading Area**

### **Xinpeng Tian 1,2,\* and Zhiqiang Gao 1,2**


Received: 1 August 2019; Accepted: 12 September 2019; Published: 14 September 2019

**Abstract:** The aim of this study is to evaluate the accuracy of MODerate resolution Imaging Spectroradiometer (MODIS) aerosol optical depth (AOD) products over heavy aerosol loading areas. For this analysis, the Terra-MODIS Collection 6.1 (C6.1) Dark Target (DT), Deep Blue (DB) and the combined DT/DB AOD products for the years 2000–2016 are used. These products are validated using AErosol RObotic NETwork (AERONET) data from twenty-three ground sites situated in high aerosol loading areas and with available measurements at least 500 days. The results show that the numbers of collections (N) of DB and DT/DB retrievals were much higher than that of DT, which was mainly caused by unavailable retrieval of DT in bright reflecting surface and heavy pollution conditions. The percentage falling within the expected error (PWE) of the DT retrievals (45.6%) is lower than that for the DB (53.4%) and DT/DB (53.1%) retrievals. The DB retrievals have 5.3% less average overestimation, and 25.7% higher match ratio than DT/DB retrievals. It is found that the current merged aerosol algorithm will miss some cases if it is determined only on the basis of normalized difference vegetation index. As the AOD increases, the value of PWE of the three products decreases significantly; the undervaluation is suppressed, and the overestimation is aggravated. The retrieval accuracy shows distinct seasonality: the PWE is largest in autumn or winter, and smallest in summer. The most severe overestimation and underestimation occurred in the summer. Moreover, the DT, DB and DT/DB products over different land cover types still exhibit obvious deviations. In urban areas, the PWE of DB product (52.6%) is higher than for the DT/DB (46.3%) and DT (25.2%) products. The DT retrievals perform poorly over the barren or sparsely vegetated area (N = 52). However, the performance of three products is similar over vegetated area. On the whole, the DB product performs better than the DT product over the heavy aerosol loading area.

**Keywords:** MODIS aerosol product; Collection 6.1; heavy aerosol loading; AERONET

### **1. Introduction**

Atmospheric aerosols are small particles (0.001–100 μm) from both natural and human sources suspended in the atmosphere that can significantly influence the ecosystem [1], climate, and hydrological cycle [2] due to their effect on radiative forcing [3,4], precipitation and clouds [5]. Due to the high temporal and spatial inhomogeneity in aerosol concentrations and the complex relationship between the aerosol chemical and physical properties and cloud microphysics, the uncertainty in the estimation of the indirect aerosol forcing remains one of the highest in the climate studies today [6–8]. Satellite remote sensing is the most effective way of measuring characteristics of aerosol on the global scale. However, currently, remote sensing retrievals of aerosol properties generally achieve a low accuracy,

because the radiance measured by the satellite sensor at the top of path radiance is a mixed signal [9], including not only information about the composition of the atmosphere, but also information about the Earth's surface reflectance [10].

Aerosol effects are significantly latitude dependent, i.e., at mid and lower latitudes, the aerosol loading is very strong, with consequent air pollution [11,12]. In Asia, the most substantial aerosol sources are coal, biomass burning, and dust [13,14]. The Middle East and Northern Africa are located in the region most affected by the presence of Arabian and Saharan desert dust, respectively [15]. The mass of African dust transported in the atmosphere is large [16], and it has been suggested that the transported dusts have a substantial influence on the regional radiative budget [17]. In recent years, the concentration of atmospheric particulates has increased unprecedentedly over these regions. Among them, China, India, and Pakistan's mean aerosol optical depth measured at AErosol RObotic NETwork (AERONET) sites [18] exceeds global background levels by 4–5 times [19].

Aerosol optical properties such as aerosol optical depth (AOD) were obtained from satellite sensors including the Sea-viewing Wide Field of view Sensor (SeaWiFS) [20], the Multiangle Imaging Spectroradiometer (MISR) [21], the MEdium Resolution Imaging Spectroradiometer (MERIS) [22], the Visible Infrared Imaging Radiometer Suite (VIIRS) [23], and the MODerate resolution Imaging Spectroradiometer (MODIS) [24]. MODIS sensors observe the Earth system from on board two satellites: on Terra since 1999 and on Aqua since 2002. MODIS acquires top-of-the-atmosphere (TOA) data in 36 spectral bands ranging in wavelength from 0.4 μm to 14.4 μm over both land and ocean with near-daily global coverage. The operational MODIS AOD product over land is based on two algorithms, namely, the Dark Target (DT) and Enhanced Deep Blue (DB) algorithms. Over the past decades, MODIS aerosol algorithms have experienced modifications for many times. The Collection 5 (C5) DT, C6/C6.1 DT and the DB algorithms have been extensively evaluated on both global and regional scales [25–28]. In a previous study, we provided a useful assessment of both the DT and DB aerosol products over Beijing with a specific focus on the potential of retrieval over inhomogeneous urban surfaces [28]. To date, the MODIS C6.1 aerosol products have not yet been fully and effectively verified in the polluted background of medium- and low-latitude areas with heavy aerosol loading. Thus, this study focuses more on the comparative performance of MODIS C6.1 DT, DB and the merged DT/DB AOD products over the heavy aerosol loading area.

MODIS is the first satellite observation plan designed to provide aerosol optical characteristic globally of high spatial resolution. It can provide a long time series of AOD products and can be very useful for air quality studies, etc. The objective of this study is to provide a more detailed evaluation the Terra-MODIS AOD retrieval products using the ground observations from twenty-three AERONET sites in Asia (including China, India and Pakistan), the Middle East (including United Arab Emirates, Bahrain and Kuwait) and Northern Africa (including Egypt, Niger, Benin, Mali and Nigeria) regions. For this, the Terra-MODIS C6.1 DT, DB and DB/DT AOD products (MOD04) at 10 km resolution are collected over the period 2000–2016. Then, the performance of MODIS aerosol retrieval algorithms is validated and compared against AERONET AOD measurements at the site, local and continental scales, meanwhile, the sensitivity of land use types (i.e., urban, barren or sparsely vegetation and vegetation) to aerosol and seasonal variations on aerosol retrievals are also considered and discussed.

### **2. Datasets and Method**

### *2.1. AERONET Ground-Observed AOD*

The AERONET (AERosol RObotic NETwork) is a worldwide network of ground stations equipped with well-calibrated Sun photometers to assess aerosol optical properties, such as their optical depth [18]. It provides a dataset of spectral AOD in the range of 0.340–1.060 μm with low uncertainty (~0.01–0.02) and high temporal resolution (every 15 min) under cloud-free conditions [29]. AERONET observations have been widely adopted for the validation of satellite-retrieved AOD [28,30,31]. For the validation of MODIS aerosol products over the heavy aerosol loading area, this study used the

version 3 cloud-screened and quality-controlled level 2.0 AOD ground-based observations from 23 AERONET sites during 2000 to 2016, as shown in Figure 1, which are located in regions above the annual average WHO IT-1 value for PM2.5 of 35 μg/m3, and with at least 500 days of measurements. Figure 1 shows global decadal mean satellite-derived PM2.5; this data set was inspired by work by van Donkelaar et al. [32]. PM2.5 concentrations in large populated regions of northern India and eastern China, respectively, exceed 60 μg/m3 and 80 μg/m3. The summarized information of all collected sites is shown in Table 1.

**Figure 1.** The geographical location of the study area. (**a**) The locations of the 23 AERONET sites are used for the evaluation of the satellite-based AOD products by circulars, and (**b**) the spatial distribution of all AERONET sites in the world. Background map is global annual average surface-level PM2.5 concentration derived from MODIS and MISR AOD satellite data sets (2001–2010) [32].


**Table 1.** AErosol RObotic NETwork (AERONET) Level 2.0 data used in this study.


**Table 1.** *Cont.*

**Note**: The SR is the surface reflectance at blue channel, the data source is MODIS surface reflectance products during 2010-2015; and the land cover type is from MODIS Land Cover product in 2013.

### *2.2. Descriptions of MODIS Aerosol Algorithms and Products*

The MODIS sensor has 36 channels, ranging from 0.4 to 14.4 μm. Eight of these channels between 0.47 and 2.13 μm are used to retrieve aerosol properties over land and ocean areas by separate algorithms. The MODIS C6.1 aerosol products suite has recently been released to replace the C5 products. The Level 2 aerosol products (MOD04 for Terra and MYD04 for Aqua) are provided at a nominal spatial resolution of 10 km × 10 km at nadir. Spatiotemporally aggregated Level 3 products at 1◦ × 1◦ resolution and daily, 8-day, and monthly temporal resolution are also available. In addition to aerosol properties, the MODIS algorithm reports several diagnostic products, including a protocol for assessing the "Quality" of the retrieval known as the Quality Assurance (QA) plan [33].

### 2.2.1. The DT Aerosol Algorithm and Product

The C6.1 DT algorithm is based on the same algorithm principle as in C5, with subtle changes such as aerosol model, surface reflectance assumptions, cloud mask and pixel selection. First, 20 × 20 groups of pixels with a 500 m resolution at 0.47, 0.65, and 2.13 μm channels, are organized into a "retrieval box" of 10 × 10 km. All unsuitable pixels (e.g., cloud, desert, snow/ice, and inland water), and the darkest 20% and brightest 50% (at 0.65 μm surface reflectance) of pixels are discarded to reduce cloud and surface contamination. At this stage some of the brightest urban surfaces may be discarded but some pixels within the 20 × 20 kernel normally remain for computation of AOD for the 10 km pixel. Then, the surface reflectance at two visible channels can be parameterized, based on a dynamic relationship between visible channels of 0.47 and 0.65 μm and the infrared channel of 2.13 μm. The aerosol model and look-up table are employed in DT retrieval, which is conducted according to geolocation and season. Finally, the spectral AOD is obtained according to the matching values. The details of the 10 km DT algorithm were published in Levy et al. [34]. Recently, the C6.1 not only improved the process of the 10 km product, but also introduced a dataset with a 3 km spatial resolution [24]. It is also worth mentioning that a new surface reflectance relationship between shortwave infrared and visible wavelength bands were revised using a spectral surface reflectance product in the C6.1 DT algorithm [35]. The DT algorithm has been applied on the global scale, and the results show that more than 70.6% retrievals are within the estimated confidence envelope (expected error, EE) of 1 standard deviation which is ±(0.05 + 15% × AODAERONET) [36]. In this study, the DT product at 10 km was obtained from the MODIS Level-1 and Atmosphere Archive and Distribution System (http://ladsweb.nascom.nasa.gov) for the years 2000–2016, and the "Optical Depth Land and Ocean" scientific data set (SDS) was used, with QA = 3 indicating the retrievals of highest confidence.

### 2.2.2. The DB Aerosol Algorithm and Product

Unlike DT, the DB algorithm was developed to retrieve aerosol properties over bright desert surfaces [37]. It performs retrievals at 1 km resolution and then aggregates pixels into a 10 km retrieval box. These pixels are masked and screened to eliminate clouds and snow/ice surfaces. For the remaining pixels, the surface reflectance is prescribed by one of several methods, dependent on location, season, and land cover type, from a global surface reflectance database in visible bands (0.412, 0.47, and

0.65 μm) at 0.1◦ × 0.1◦ resolution, which was developed by the minimum reflectivity method. Unlike the DT uncertainty estimate, the EE of the DB AOD is approximately ±(0.05 + 20% × AODAERONET), and 79% proportion of retrievals agrees within EE of the AERONET observation [26]. In this paper, the MOD04 product with the SDS of "Deep\_Blue\_Aerosol\_Optical\_Depth\_550\_Land\_Best\_Estimate" is used.

### 2.2.3. The Merged DT/DB Algorithm and Product

In C6 and C6.1, the new DT and DB merged AOD products (DT/DB) are based on the DT and DB AOD retrievals. The inclusion of a merged SDS was motivated by a desire to provide a more gap-filled data set than is available from the individual algorithms alone. The logic behind the merge as implemented within the C6 reprocessing was that there is a longer heritage of user familiarity with DT data over densely vegetated regions, while DB is the only data set providing coverage over arid surfaces. Three classifications were determined by monthly normalized difference vegetation (NDVI) in the merge algorithm. Over land, where NDVI ≤ 0.2 in a given month, DB data are used to populate the merged SDS, and where NDVI ≥ 0.3, DT data are used. For intermediate NDVI areas which are usually transition zones between arid and vegetated land, the algorithm whose retrieval returns the higher QA flag is used, or if both return QA = 3, the mean value is used [38]. This result is stored in the SDS named "AOD\_550\_Dark\_Target\_Deep\_Blue\_Combined".

### *2.3. Spatio-Temporal Matching for Satellite-Retrieved AOD Products with Ground-Based Observations*

In this study, to match the instantaneous AOD value provided by satellites with the repeated measurements observed by AERONET, we followed the matchup methodology of Ichoku et al. [39]. The AERONET data averaged within 30 min of the MODIS overpass are extracted and compared with MODIS AOD data averaged within the 5 × 5 pixels surrounding the AERONET site. As satellite-retrieved AOD retrievals are at 550 nm, AERONET data are interpolated to 550 nm using the Angstrom exponent α, defined as:

$$a = \frac{\ln(\pi\_1/\pi\_2)}{\ln(\lambda\_1/\lambda\_2)}\tag{1}$$

where τ1, τ<sup>2</sup> are the AOD at wavelengths λ1, λ2. The nearest available pair of wavelengths from AERONET (normally 675 nm and either 440 or 500 nm) are used.

The statistical metrics considered in the evaluation are N is the number of collocated AODs; R and MR are the Pearson's correlation coefficient and the matching radio of the MODIS AOD product and the AERONET ground-observed data, respectively; RMSE, the root mean square error; MAE, the mean absolute error; RMB, the relative mean bias (the RMB > 1.0 and RMB < 1.0 indicate the overestimation and underestimation of the retrievals, respectively). PWE, PAE and PBE as percentages of collocations falling within, above and below EE envelopes. PAE and PBE represent the overestimation and underestimation of the algorithm, respectively.

$$MR = N\_{(MODIS)} / N\_{(AERONET)} \times 100\tag{2}$$

$$RMSE = \sqrt{\frac{1}{n} \sum\_{i=1}^{n} \left( AOD\_{(MODIS)i} - AOD\_{(AERONET)i} \right)^2} \tag{3}$$

$$MAE = \frac{1}{n} \sum\_{i=1}^{n} \left| AOD\_{(MODIS)i} - AOD\_{(AERONET)i} \right| \tag{4}$$

$$RMB = \overline{AOD\_{(MODIS)}} / \overline{AOD\_{(AERONET)}} \tag{5}$$

$$EE = \pm (0.05 + 0.15 \times AOD\_{AERONET}) \tag{6}$$

### **3. Results**

### *3.1. Validation of C6.1 DT Retrievals*

Table 2 provides the statistical parameters for the ground-observed and DT AOD retrievals. As shown in Table 2, we evaluate the DT retrievals at each site by analyzing the N, matching ratio (MR), R, RMSE, MAE, RMB, PWE, PAE and PBE. The MODIS C6.1 DT AODs show a high correlation (R = 0.804) with AERONET ground-observed measurements and very high accuracy (PWE = 45.6%), but have a significant overestimation over this region with high RMB (1.158, greater than 1.0) and PAE (40.7%) values. The estimation of surface reflectance (SR) is an important factor in DT algorithm over land. However, high SR values make it difficult to accomplish this discrimination as aerosol path radiance is often lower than surface radiance. It was reported by a previous study [40] that the large intercept between DT retrievals and ground-observed AOD is due to a large uncertainty in the surface reflectance estimation [41]. In the Asia region, the greatest uncertainty is observed at urban sites dominated by dust aerosols, including BJS, BJC, and LHR sites, with only 18.3%, 13.8%, and 30.6% of observations falling within the EE, respectively. This may be because the high SR (> 0.079, as shown in Table 1) of the three sites imposes a great challenge and introduces large uncertainty for the DT algorithm. There are no AOD retrievals at the KRC site with the highest SR (0.114). It is notable that the DT algorithm is not applicable in the Middle East region, where most of the land cover type is barren or sparse vegetation. The bright reflectance of dust at blue channel (SR = 0.095-0.162) limits the DT algorithm from retrieving AODs in all sites. Similar to the Middle East, the DT algorithm is only applicable to CSE, DJG and ILR sites with low reflectivity in the North Africa region. However, the N and MR are significantly smaller than those for sites in the Asian region. In addition, the DT algorithm is underestimated at DJG and ILR sites, which was attributed to overestimation of the SR and the use of inappropriate aerosol schemes in the algorithm.


**Table 2.** Statistical summary for validation of MOD04 DT AOD product.

**Note**: N is the number of collections; MR is the matching ratio of the MODIS AOD product and the ground-observed data; R is the correlation coefficient between the MODIS AOD product and the ground-observed data; PWE is the percentage within the EE; PAE is the percentage above the EE; PBE is the percentage below the EE; RMSE is the root-mean-square error; MAE is mean absolute error, it is the ratio of the satellite mean to the AERONET mean; and RMB is relative mean bias.

### *3.2. Validation of C6.1 DB Retrievals*

Table 3 shows a comparison between the DB AOD retrievals and the AERONET ground-based observations at all sites. Compared with the DT algorithm, there are much larger numbers of matched retrievals by DB algorithm, i.e., 10258, 3755 and 4632 over sites in Asia, the Middle East, and Northern Africa, respectively. For the Asia region, the statistical data show that the MR is 73.6%, approximately 57.5% of the collections falling within the EE, with the correlation coefficient of 0.702. In this region, the overestimation of AOD is slightly greater than the underestimation (PAE = 22.6%, PBE = 19.9%). In particular, there is serious overestimation at the KRC site, for which PAE = 55.2%. The performance of the DB algorithm is significantly better than that of the DT algorithm in the Middle East region, where the MR is 70.6%, with 47.3% of the collections falling within the EE. In the Northern Africa region, the DB AOD retrievals show a high agreement with AERONET AOD measurements and approximately 49.2% of the retrievals falling within the EE, with low RMSE (0.248) and MAE (0.166) errors. On the whole, the DB algorithm is significantly improved compared to the DT algorithm in this study, i.e., the RMSE and MAE errors are decreased by 14.5% and 20.8%, respectively. Additionally, approximately 53% of the DB retrievals fall within the EE. Although DB was slightly underestimated in comparison to DT, the overestimation of DB retrievals was significantly improved, with the PAE decreased by 63%. This indicates that the DB product is recommended for aerosol applications in this study.


**Table 3.** Statistical summary for validation of MOD04 DB AOD product.

### *3.3. Validation of C6.1 DT*/*DB Retrievals*

To evaluate the performance, a total of collocated 11,537 MODIS DT/DB aerosol retrievals were compared with AERONET ground-based measurements. As shown in Table 4, it is easy to find that AOD retrievals exhibit overall high correlations with AERONET AOD measurements (R = 0.841), with 53.1% of them falling within the EE, indicating good performances over the heavy aerosol loading areas. However, the accuracies vary greatly at different sites, where only five sites had more than 60% of the collections falling within the EE. In the Asia region, the DT/DB AOD retrievals have an underestimation

of 9.8% and an overestimation of 36.5%; the PWE is 53.7%. Similar to the DB algorithm, the DT/DB algorithm also underestimates at the KRC site, where PBE = 48.1%. However, the opposite trend is observed at sites in the North Africa region, where there is an average underestimation of 35.2% and an overestimation of 17.8%, with 47.0% of the collections falling within the EE. Compared with the DT algorithm, the DT/DB algorithm has improved in this region, but this is still not applicable at the DHB, MSF and BRN sites. For all sites, the DT/DB AOD retrievals achieved high correlation (R = 0.841) with AERONET ground measurements, with low RMSE (0.238) and MAE (0.157) errors. However, the DT/DB algorithm has an obvious overestimation in the three regions (PAE = 30.3%, RMB = 1.082).


**Table 4.** Statistical summary for validation of MOD04 DT/DB AOD product.

### *3.4. Comparison of DT, DB and DT*/*DB Products with Ground-Observed Data in Each AOD Bin*

Figure 2 shows the scatter plots of MODIS DT, DB and DT/DB AOD retrievals against AERONET measurements for the years 2000–2016 in the 0 to 4.0 AOD bins. The number of collections for DT retrievals is lower than that for DB and DT/DB retrievals because the DT algorithm is unable to retrieve AOD over bright and complex surface types. In the correlation comparison between satellite retrieval and ground observation, the slope of the linear regression is linked to systematic uncertainties such as assumptions of aerosol models, while the constant offset tends to be associated with factors such as deviation in surface reflectance estimation [26,42]. For DT retrievals, the slope is lower than DB and DT/DB and the intercept is higher than both. As shown in Figure 3, the PWE values of the three products are decreased significantly with increasing AOD. It is worth mentioning that the PWE value of DT algorithm is 51.1% when AOD > 2.0, which is due to the number of collections being very small (N = 47). In addition, the PAE and PBE values decreased and increased, respectively, indicating that overestimation was significantly suppressed, and underestimation increased with the increase of AOD. The DT retrievals have lowest PWE value, highest overestimation, and lowest underestimation in each AOD bin, except for AOD > 2.0. The DB and DT/DB retrievals have lower uncertainty when

AOD < 0.25 with PAE = 59.3% and 61.1%, respectively. However, the DB and DT/DB algorithms exhibit underestimation during polluted days (AOD > 1.0), approximately 30% of the collections above EE.

On the whole, the accuracy of the DT, DB and DT/DB algorithms over the heavy aerosol loading and high surface reflectance areas is lower than on the global scale. The verification results show that none of the aerosol products are suitable for application in research on atmospheric aerosols under heavy aerosol loading; thus, there is important theoretical and practical significance for proposing a high-precision method of aerosol retrieval.

**Figure 2.** Validation of MOD04 C6.1 DT (**a**), DB (**b**), and combined DT/DB (**c**) AOD retrievals at 10 km resolution against AERONET measurements for the years 2000–2016. The dashed lines = EE lines, black solid line = 1:1 line, and red solid line = regression line.

**Figure 3.** The percentage within in (PWE), above (PAE) and below (PBE) the expected error for the DT (**a**), DB (**b**), and DT/DB (**c**) products in each AOD bin over heavy aerosol loading area for the years 2000–2016.

### *3.5. Seasonal Di*ff*erences of DT, DB and DT*/*DB Products*

Previously, strong seasonality has been noted in the performance of Aqua MODIS products (including DT, DB and merged DT/DB) [43], which instigated us to examine the seasonality of Terra MODIS AOD. Figures 4 and 5 summarize the seasonal variation of retrieval accuracy for Terra MODIS and AERONET comparison of AOD at 550 nm, indicating that the statistical parameters of DT, DB and DT/DB aerosol retrievals exhibit strong seasonality and similar variation trends. The number of collections for three products is the greatest in spring, moderate in autumn and winter, and the smallest in summer. The value of PWE is highest in autumn, moderate in spring and winter, and lowest in summer. The variation of overestimation is similar, the largest in summer, the smallest in winter for DT and DT/DB products, but the smallest overestimation of DB product is in autumn. The PBE values of the three products are lower than those of PAE in different seasons, indicating that the underestimation of the aerosol retrievals is weaker than overestimation. The underestimation also has strong seasonality, the PBE values of DT and DT/DB are highest in winter and lowest in summer, while the PBE values of DB product are the opposite. In the four seasons, the number of collections of DB product is more than DT and DT/DB products. The accuracy of DB retrievals is superior to that of DT retrievals, due to the more accurate estimation of the surface reflectance. DT has the lowest accuracy, the largest overestimation and the minimum collections, which further demonstrates that the DT algorithm is not suitable for areas with high surface reflectance and pollution. We analyzed the single scattering albedo (SSA) value at Beijing AERONET site from 2001 to 2016 and found that the average values are 0.90, 0.92 at 440, 676 nm, respectively. For the study area, the high setting of the SSA value in the MODIS lookup table may be the one of the reasons for the uncertainty of AOD retrieval.

Figure 5 summarizes the error statistics for DT, DB, and DT/DB products against AERONET AOD ground-observed measurements in the four seasons, which indicates that the performance of the DB and DT/DB algorithms is slightly better than the DT algorithm in each season. Figure 5 also illustrates that three products show a strong seasonality when retrieving AOD. Correlations of the DB, DT/DB retrievals and AERONET AOD ground-observed measurements are up to approximately 0.80 in any season, but the correlation coefficient between the MODIS DT retrievals and the AERONET ground-observed AODs in winter decreases to 0.703 as shown Figure 5. Most values of the RMSE are much less than 0.30 in the four seasons for the three products, except DT in winter. The MAE error is higher in summer than other seasons. The RMB errors are greater than 1.0 in the four seasons, which indicates overestimation of the retrievals, except DT in winter, as shown in Figure 3, where PBE = 32.3%. In addition, the values of RMSE and MAE errors for DT product are larger than DB and DT/DB products and the RMB errors for DB and DT/DB products are close to 1.0. In general, the MODIS DT product shows poor ability to retrieve AOD as it has a low PWE of 37.4–50.5% with high RMSE of 0.233–0.380 and MAE of 0.166–0.283. The reason for seasonal variation in accuracy at these sites may be due to monsoon and downwind biomass burning sources [44,45].

**Figure 4.** The percentage within in (PWE), above (PAE) and below (PBE) the expected error and the number of collocations for the DT, DB and DT/DB products in the four seasons over heavy aerosol loading area for the years 2000–2016.

**Figure 5.** Error statistics for DT, DB, and DT/DB products against AERONET AOD ground-observed measurements in the four seasons.

### *3.6. Adaptability of DT, DB and DT*/*DB Products over Di*ff*erent Land Cover Types*

Due to the diurnal variation of the aerosol sources, components and particle size distributions, assessing the availability of DT, DB and DT/DB products over various land cover types is becoming imperative. To explore the adaptability of three products over different land cover types, in this study, the 23 AERONET sites are divided into three land cover types: (1) urban (8 sites: BJS, BJC, KRC, LHR, DHB, BRN, KUS, and CES); (2) barren or sparsely vegetated (9 sites: THS, HMM, MIS, MZR, MSF, EFS, DMS, ZAS, and AGF); and (3) vegetated (6 sites: XHS, XLS, GCS, KPR, DJG, and ILR), according to the MCD12Q1 Land Cover product in 2013 year and the MOD09A1 8-day composited Surface Reflectance product from 2010 to 2015. Figure 6 illustrated the numbers of collections and the value of PWE, PAE and PBE for three products over urban, barren or sparsely vegetated and vegetated areas and Figure 7 shows the error statistics.

There were totals of 1616, 6435 and 3432 MODIS/AERONET collections over urban area for the DT, DB and DT/DB products, respectively. It is easy to find that the DB retrievals have the highest PWE, the lowest PAE, high correlations with AERONET AOD measurements, and small RMSE and MAE errors, with a value of RMB close to 1.0. For DT retrievals, only 25.2% of the retrievals falling within the EE had large RMSE and MAE errors. The DT product has a higher PAE than the DB product, indicating that DT retrievals seriously overestimated aerosol loading. The DT/DB retrievals have a good correlation, small RMSE and MAE errors, with 46.3% of the collections falling within the EE. However, the DT/DB results exhibited an overestimation, the PAE is slightly higher than that for the DB product. The accuracy of DT/DB retrievals over urban areas is better than the DT product and worse than the DB product. On the whole, the DB retrievals have better applicability in urban areas. Unlike DT and DT/DB products, the DB retrievals are somewhat underestimated over this area.

For barren or sparsely vegetated areas, the collections N for the DT algorithm are much less than other algorithms over this area, with only a small amount at the THS site (N = 52, as shown in Table 2). The results show that the DT product has poor ability to retrieve AOD over barren or sparsely vegetated area. The accuracies of the DB and DT/DB products are similar, with 49.7% and 56.9% of the collections falling within the EE, respectively. However, the collections for DB retrievals with AERONET are almost 1.7 times those for DT/DB product.

We have collected 3432, 3571 and 4534 collections for DT and DB and DT/DB products over the vegetated area. Figures 6 and 7 show that the vegetated sites have highest PWE value for each product, with less overestimation, high correlations, small RMSE and MAE errors, which is not surprising, as the DT algorithm is tuned to vegetated targets.

**Figure 6.** The percentage within in (PWE), above (PAE) and below (PBE) the expected error and the number of collocations for the DT, DB and DT/DB products over different land cover types for the years 2000–2016.

**Figure 7.** Error statistics for DT, DB, and DT/DB products against AERONET AOD ground-observed measurements over different land cover types.

### **4. Discussion and Conclusions**

The objective of this study was to evaluate the Terra-MODIS C6.1 AOD retrieval product including Dark Target, Deep Blue and the combined DT/DB product at 10 km spatial resolution using ground-observed measurements from AERONET 23 sites over heavy aerosol loading areas during the period 2000 to 2016. The results showed the poorest performance for the DT algorithm over the study area, with only 45.6% of the collections falling within the EE, and with larger RMSE and MAE errors than the DB and DT/DB retrievals. The DB AOD retrievals were in good agreement with AERONET AOD measurements (R = 0.847) at most of the sites, and 53.4% of retrievals fall within in the EE with low RMSE and MAE errors, but the AOD was significantly underestimated when AOD > 1.0. The combined DT/DB AOD appeared to be superior to DT AOD due to the greater contribution of DB retrievals over bright-reflecting source regions. On the whole, over medium- and low-latitude land areas, the value of PWE for all algorithms was lower than the value on a global scale.

In general, the accuracy of the MODIS aerosol product is related not only to the land cover type but also to seasonal variation. For the urban and barren or sparsely vegetation areas, the accuracy of DB and DT/DB algorithms is similar, and better than that of DT retrievals. The performance of three products is similar over vegetation area. The accuracy of DT product is not significantly better than that of DB and DT/DB products, which may be related to both underestimation of surface reflectance and inappropriate use of an absorbing aerosol model in the study area. For the all products, the value of PWE is the largest in autumn, and the smallest in summer, overall, the performance of DB and DT/DB are better than DT in most seasons.

This study provided an overview and initial analysis of the three typical products of Terra-MODIS C6.1 over medium- and low-latitude land areas, where there is heavy aerosol loading and increasing air pollution. The analysis has shown that the DB algorithm has the best performance in this area, and while not optimal, the current merged DT/DB algorithm nevertheless does provide a data set which will be suitable for quantitative scientific applications.

**Author Contributions:** Conceptualization, X.T. and Z.G.; methodology, X.T. and Z.G.; data curation, X.T.; formal analysis, X.T.; validation X.T.; funding acquisition, Z.G.; project administration, Z.G.; supervision, Z.G.; writing of the original draft, X.T.; writing of review and editing, Z.G.

**Funding:** This work was supported by the National Natural Science Foundation of China (NSFC) fund project (41876107), NSFC-Shandong joint fund project (U1706219), Basic Special Program of Ministry of Science and Technology (2014FY210600) and Aoshan Science and Technology Innovation Program of Qingdao National Laboratory for Marine Science and Technology (2016ASKJ02).

**Acknowledgments:** The data for this paper are available at Level-1 and Atmosphere Archive & Distribution System (LAADS) Distributed Active Archive Center (DAAC) (http://ladsweb.nascom.nasa.gov) and AERONET Web (http://aeronet.gsfc.nasa.gov). The authors would like to thank GSFC and AERONET for kindly providing the data. In addition, a very special acknowledgement is made to the editors and referees who provided important comments that improved this paper.

**Conflicts of Interest:** The authors declare no conflict of interest.

### **References**


© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

MDPI St. Alban-Anlage 66 4052 Basel Switzerland Tel. +41 61 683 77 34 Fax +41 61 302 89 18 www.mdpi.com

*Atmosphere* Editorial Office E-mail: atmosphere@mdpi.com www.mdpi.com/journal/atmosphere

MDPI St. Alban-Anlage 66 4052 Basel Switzerland

Tel: +41 61 683 77 34 Fax: +41 61 302 89 18