Validation of MODIS 3 km Resolution Aerosol Optical Depth Retrievals Over Asia

Abstract

This study evaluates the new Aqua MODIS Dark Target (DT) Collection 6 (C6) Aerosol Optical Depth (AOD) (MYD04_3K) retrieval algorithm at 3 km resolution over Asian countries that have recently experienced severe and increasing air pollution. Retrievals showed generally low accuracy compared with the AErosol RObotic NETwork (AERONET), with only 55% of retrievals within the expected error (EE). The uncertainty appears mainly due to systematic overestimation at both low and high AOD levels. This is attributed to under-prediction of surface reflectance, similar to, but more severe than, the C6 DT product at 10-km resolution. This is because MYD04_3K observes more noise in the surface reflectance computations, due to retention of some bright pixels in the retrieval window which would be discarded at 10 km. Greatest uncertainty was observed at urban sites, especially those dominated by coarse aerosols. Results suggest that the DT at 3 km is less reliable than MODIS C6 AOD products at 10 km.

1. Introduction

Atmospheric aerosols are small particles from both natural and human sources suspended in the atmosphere. Since they have wide ranging impacts on climate forcing [1], visibility [2], and human health [3], their unprecedented increase over Asian countries in recent decades is a cause for concern. Asia is the largest source of aerosols and their precursor gases, with China, India, and Pakistan’s mean AOD measured at AErosol RObotic NETwork (AERONET) sites exceeding global background levels [4] by 4–5 times. The pollutant gases are involved in smog formation and are known to penetrate the lungs and circulatory system and act as precursors to fine particulates. Satellite remote sensing has been used to quantify and monitor aerosol distributions over much larger areas than can be covered by ground stations, and their optical properties including aerosol optical depth (AOD) can be derived from image visible wavebands.

MODIS on board the Terra and Aqua satellites has observed the earth since 1999 and 2002, respectively, creating an extensive geophysical dataset with 36 spectral channels, good temporal resolution of 1 to 2 days and moderate spatial resolutions of 250 m, 500 m, and 1000 m. MODIS has provided regular observations of AOD which have aided the understanding of the effects of anthropogenic aerosols at both local and global scales [5,6]. The current operational MODIS AOD product over land is known as Collection 6 (C6), which replaces Collection 5 (C5) and is based on two algorithms, namely the Dark Target (DT) [7,8,9] and Deep Blue (DB) [10,11] algorithms. The DT algorithm over land utilizes the generally bright appearance of aerosols over dark surfaces such as vegetation in the visible wavebands. If the surface reflectance is known, the residual image signal at the Top of Atmosphere (TOA) is due to atmospheric reflectance, including aerosol. The DB algorithm was introduced with C5 to retrieve AOD over bright surfaces using blue spectral bands which are not bright in this region. In C6 from 2013 onwards, it was expanded to retrieve over vegetated sites as well.

In recent years, many Asian cities have suffered severe deterioration in air quality with significant contributions from both natural and anthropogenic particulate sources. Thus, Asian aerosols are complex mixtures of coarse and fine particulates, of both light-absorbing and scattering types. The three main light-absorbing species are iron oxides in mineral dust, black carbon or soot (BC), and brown carbon (BrC), which is spectrally dependent. Light-scattering aerosols are mostly from anthropogenic sources, including sulphate, nitrate, and ammonium from coal and biomass burning, industrial, and vehicular sources [12]. As MODIS DT and DB algorithms at 10-km resolution are unable to resolve local aerosol gradients and city level features, an effective and higher resolution satellite aerosol retrieval is required. Therefore, in 2013, a global DT AOD product at a nominal resolution of 3 km (MYD04_3K) [13] was introduced in the operational C6 AOD product. This is in addition to the DT [14] and DB [15] AOD products at the standard 10-km resolution. Initially, the C6 product was available only for the Aqua satellite; therefore, in the current study, we only consider the Aqua–MODIS DT (3 km) product over land. Additionally, in a previous study [16], Terra-MODIS yielded similar results to Aqua-MODIS, with similar uncertainty. The MODIS C5 DT, and C6 DT and DB algorithms have been extensively evaluated over land [9,16,17,18,19], but the new MODIS C6 DT 3-km algorithm has not.

Therefore, this study evaluates the Aqua–MODIS C6 DT AOD algorithm at 3-km resolution over a period of 13 years—2002–2014—including low to very high aerosol loadings over seven Asian countries (China, Hong Kong, India, Japan, Pakistan, Taiwan, and Thailand) and 16 AERONET sites (

Table 1. AErosol RObotic NETwork (AERONET) Level 2.0 data used in this study.

Country	AERONET	Lat/Long	Time Period	Average AOD (2007–2013)
China	Beijing	39.97/116.38	2002–2014	0.48
Hong Kong	Hong_Kong_PolyU	22.30/114.18	2006–2013	0.42
India	Kanpur	26.61/80.23	2002–2014	0.65
	Gandhi College	25.87/84.13		0.64
Japan	Osaka	34.65/135.59	2002–2013	0.23
	Shirahama	33.69/135.36		0.20
Pakistan	Lahore	31.54/74.33	2007–2013	0.67
	Karachi	24.87/67.03		0.42
Taiwan	Chen Kung Uni.	23.00/120.22	2006–2014	0.42
	NCU Taiwan	24.97/121.19		0.27
	Taipei CWB	25.03/121.50		0.30
Thailand	Silpakorn Uni.	13.82/100.04	2006–2014	0.45
	Ubon Ratchathani	15.21/104.87		0.36
	Mukdahan	16.61/104.68		0.36
	Chiang Mai Met Sta.	18.77/98.97		0.47
	Pimai	15.18/102.56		0.45
France	Paris	48.87/2.33	2005–2014	0.16

). All are situated in the Asian monsoon region experiencing summer rainfall and cool dry winters; thus, most MODIS and AERONET collocations are in winter due to lower cloud cover. All five sites in Thailand as well as Gandhi College in India could be considered rural, and the rest urban or suburban, and all sites have moderate to high aerosol loading. For a comparison of MYD04_3K with areas experiencing significantly lower aerosol loadings, the Paris AERONET site, which has average AOD at 0.55 µm below 0.2 (Table 1), is additionally included.

2. Datasets

2.1. AErosol RObotic NETwork (AERONET)

AERONET is a worldwide network of calibrated ground-based Sun photometers [4,20], which provide cloud-screened and quality assured spectral AOD [21] in the range 0.340–1.060 µm with low uncertainty (0.01–0.02) and high temporal resolution (every 15 min). For validation of MYD04_3K, this study used cloud-screened and quality controlled level 2.0 AOD measurements from 16 AERONET stations for blocks of years between 2002 and 2014 (Table 1). Since AERONET AOD data at 0.55 µm are not available, AOD is interpolated into 0.55 µm using Equation (1) based on the Angström exponent (440–675α):

AOD_0.55µm = AOD_0.50µm (0.55/0.50^{)-440–675α}

(1)

2.2. The MODIS DT 3 km Resolution Aerosol Retrieval Algorithm

For the development of the MODIS Dark Target (DT) aerosol retrieval algorithm over land [8,16,22,23], reflectance at the top of the atmosphere (TOA) in the 2.11 $\mathsf{\mu }\mathrm{m}$ channel is calculated and corrected for gas absorption. The 1.38 $\mathsf{\mu }\mathrm{m}$ channel at 1000 m resolution is used to mask clouds and snow/ice pixels, and other very bright surfaces are avoided using reflectance at 2.11 $\mathsf{\mu }\mathrm{m}>\mathrm{0.25.}$ The 0.66 $\mathsf{\mu }\mathrm{m}$ and 0.86 $\mathsf{\mu }\mathrm{m}$ channels at 250 m are used to mask inland water bodies. The remaining pixels are aggregated at 500 m for further processing, and the only difference between the 10-km and 3-km product is in the selection of pixels in the retrieval window [13]. For retrievals at 3-km resolution, pixels are organized into retrieval windows of 6 × 6 (36 pixels). The darkest 20% and brightest 50% of the remaining pixels for the 0.66 $\mathsf{\mu }\mathrm{m}$ channel are deselected with, at most, 11 pixels remaining to perform aerosol retrieval. If there are at least five pixels remaining (13%), the spectral reflectance at 0.66 $\mathsf{\mu }\mathrm{m}$ is calculated for that window. The 10-km DT algorithm, on the other hand, uses a retrieval box of 20 × 20 (400 pixels) at 500 m, of which at most, 120 and a minimum of 51 (14%) remain after deselection. The DT AOD product at 3 km is then developed from the spectral reflectance using a similar look-up-table (LUT) and inversion based on the ratio of visible and shortwave infrared (VISvs2.1), as the 10-km product [13,14]. The MYD04_3K is expected to resolve aerosol gradients and pollution sources such as smoke plumes, which are missed at 10 km. However, since there are more pixels to select from in the deselection process at 10 km, dark or bright pixels discarded at 10 km might be retained at 3 km, which makes the DT 3 km potentially noisier than the 10-km product. Accordingly, the expected error (EE) over land for the DT algorithm at 3 km is ± (0.05% + 20%), which is slightly less stringent than the ± (0.05% +15%) for the 10-km product [13,14,24,25]. In general, the quality of the MODIS aerosol retrievals depends on accuracy of the surface reflectance and of the aerosol model, and over- or underestimation during both clear and polluted conditions is normally caused by error in these two factors [26,27,28,29]. The MODIS C6 DT 10-km AOD product has been observed to significantly and systematically overestimate AOD over Asian cities [19,30,31,32,33].

3. Methods

In this study, the DT C6 AOD product at 3 km (MYD04_3K) was generated from the MODIS Level-1 and Atmosphere Archive and Distribution System (http://ladsweb.nascom.nasa.gov) for the years 2002–2014 (Table 1). Only the highest quality flag (QF = 3) AOD observations [14] were utilized. Retrievals were validated using collocated MYD04_3K and level 2.0 AOD from AERONET sites. To increase the number of statistical samples for validation, collocation was defined as the average of at least two AERONET AOD measurements between 12:00 and 14:00 local time and at least two pixels of MODIS AOD observations within a sampling window of 3 × 3 pixels (average of 9 pixels) centered on the AERONET site, i.e., the average of a 9 km × 9 km spatial region for the MYD04_3K. The total collocated observations over all AERONET sites amount to 5522 observations. Deming regression, an orthogonal regression technique was used to estimate the slope and intercept of the datasets, and the accuracy of retrievals was evaluated using root mean square error (RMSE), mean absolute error (MAE), expected error (EE), and relative mean bias (RMB). For comparison purposes, the MODIS DT C6 AOD product at 10-km resolution was also downloaded and evaluated for all stations, with a total of 6487 collocations over all AERONET sites.

4. Results and Discussion

4.1. Performance of the DT 3-km AOD Algorithm

AOD retrieved from MYD04_3K is compared with AERONET AOD measurements (Figure 1 and

Table 2. Statistical summary for validation of MYD04_3K AOD observations over Asia.

AERONET Sites	Average (M – A) a	Slope	N b	% Above/Within/Below EE c
Beijing	0.71 – 0.37 = +0.34	1.14	588	93.20/06.46/00.34
Hong Kong	0.51 – 0.50 = +0.01	0.87	70	18.57/77.14/04.29
Kanpur	0.70 – 0.58 = +0.12	1.35	1165	38.71/58.80/02.49
Gandhi College	0.60 – 0.59 = +0.01	1.41	372	12.10/76.34/11.56
Osaka	0.61 – 0.29 = +0.31	1.49	132	75.76/21.97/02.27
Shirahama	0.29 – 0.22 = +0.07	1.13	671	37.70/62.00/00.30
Lahore	0.78 – 0.57 = +0.21	1.26	528	69.32/29.17/01.52
Karachi	0.73 – 0.35 = +0.38	1.50	110	94.55/04.55/00.91
Chen Kung Uni.	0.86 – 0.55 = +0.31	1.17	110	80.91/18.18/00.91
NCU Taiwan	0.49 – 0.32 = +0.17	1.30	95	74.74/25.26/00.00
Taipei CWB	0.61 – 0.37 = +0.24	1.08	97	68.35/31.65/00.00
Silpakorn Uni.	0.58 – 0.44 = +0.14	1.24	595	58.15/40.84/01.01
Ubon Ratchathani	0.41 – 0.39 = +0.02	1.27	163	12.27/78.53/07.20
Mukdahan	0.33 – 0.37 = −0.04	1.20	120	13.33/55.00/31.67
Chiang Mai Met Sta.	0.48 – 0.48 = 0.00	1.08	668	11.53/78.29/10.18
Pimai	0.28 – 0.37 = −0.09	1.00	56	08.93/41.07/50.00
Asia	0.59 – 0.45 = +0.14	1.24	5522	41.35/55.61/03.06
Paris	0.38 – 0.17 = +0.21	2.38	114	77.20/17.54/05.26

^a M = MYD04_3K and A = AERONET site; ^b N = Total collocations of MYD04_3K and AERONET AOD; ^c EE = ± (0.05% + 20%).

). Of the total of 5522 collocations at Asian AERONET sites, only 55% fall within the EE, but there is wide variation between sites (Table 2). The greatest uncertainty is observed at urban sites dominated by coarse aerosols including Beijing, Karachi, and Osaka, with only 6.5%, 4.5%, and 22% of observations, respectively, falling within the EE. These sites show a large positive offset from AERONET observations during both low and high aerosol loadings, indicating underestimation of the surface reflectance by the VISvs2.1 relationship. This is similar to the C6 DT algorithm at 10 km, which was also found to overestimate over cities in China and Pakistan, with a constant offset against AERONET [19,32], which was attributed to a large negative bias in surface reflectance estimation. This overestimation of AOD by the DT 3-km product was observed at 14 out of the 16 sites examined, the exceptions being the rural sites, which showed either underestimates (Mukdahan and Pimai) or no significant low or high bias (Gandhi College, Chiang Mai, and Ubon Ratchathani). Combining all sites shows a large and significant overestimation, with 41.3% above the EE. For Beijing and Karachi in particular, with only 4% and 6%, respectively, within the EE, both have high aerosol loadings; thus, using a correct aerosol model is important [14]. For both, the slope between MOD04_3K and AERONET is significantly greater than one, suggesting that there is too much absorption in the model [34]. These are cities downwind of large deserts, for which a coarse mode aerosol model, which is considered weakly absorbing, is used [14]. Mixing with non-absorbing fine mode aerosols from urban pollution would result in over-prediction of absorption, causing overestimation of AOD. This may also be true for Osaka, an urban site where coarse aerosols from Chinese dust also dominate, and the large offset and steep slope (1.49) between MYD04_3K and AERONET result in only 22% of observations within the EE. Levy et al. [14] also noted this potential problem in such areas.

The more rural vegetated sites—Gandhi College, Ubon_Ratchathani, Chiang Mai, and Mukdahan—show better retrievals, with 76%, 78%, 78%, and 55% within the EE, which is not surprising, as the DT algorithm is tuned to vegetated targets. However, at these sites, in addition to overestimation at high AOD levels, there is significant underestimation at low AOD levels, suggesting that there is too little absorption in the model, possibly because these sites are situated in rural areas, where carbon particles are released year-round from domestic fires, whereas the DT algorithm uses a strongly absorbing aerosol model only in the December–February biomass burning season [14]. The rural site of Pimai in Thailand is an exception, being a rural site with poor quality of retrievals with only 41% within the EE. Most retrievals for this site were in the dry season when agricultural fields are bare, giving a relatively bright surface, which is not ideal for the DT AOD algorithm.

The urban coastal site of Hong Kong shows the best results, having 77% of observations within the EE. The very dense and high rise environment around the AERONET site creates much shadow on images especially in winter, with a low sun angle, when most clear sky images are obtained. The relatively few collocations available for this site (only 70 over a 7-year period) is due to cloud cover in summer when the sun is overhead. The long shadows, which may remain in the pixel averages, even after DT’s elimination of the 20% darkest pixels, may result in less underestimation of the surface reflectance than at other urban sites, as the error in the VISvs2.1 relationship is greater for brighter surfaces.

Surprisingly, the Paris site where AOD levels are 2 to 5 times lower than the Asian sites shows the lowest correlation (R = 0.44) as well as very low accuracy, having only 17% within the EE. Overestimation is especially apparent at higher AOD levels above 0.5, which exceed AERONET by a factor of 2 or more. This site also has the highest RMB and steepest slope (2.38) among all sites, suggesting that the weakly absorbing fine mode aerosol model used may be inappropriate.

4.2. Performance of the DT 10-km AOD Algorithm

Reports of the performance of the 10-km DT algorithm (MYD04) suggest approximately four times greater confidence in the retrievals over Beijing and Karachi (Figure 2 and

Table 3. Statistical summary for validation of MYD04_10K AOD observations over Asia.

AERONET Sites	Average (M – A) a	Slope	N b	% Above/Within/Below EE c
Beijing	0.72 – 0.47 = +0.25	0.97	661	76.10/20.42/03.18
Hong Kong	0.47 – 0.48 = −0.01	0.94	178	06.18/85.96/07.87
Kanpur	0.65 – 0.58 = +0.07	1.22	1067	28.31/66.35/05.34
Gandhi College	0.61 – 0.60 = +0.01	1.34	402	12.44/78.11/09.45
Osaka	0.49 – 0.30 = +0.19	1.31	336	74.70/24.70/00.60
Shirahama	0.25 – 0.22 = +0.03	1.01	753	23.51/74.90/01.59
Lahore	0.72 – 0.61 = +0.11	1.45	661	42.81/49.92/07.20
Karachi	0.42 – 0.41 = +0.01	1.15	319	21.63/63.64/14.73
Chen Kung Uni.	0.57 – 0.47 = +0.10	1.15	222	44.59/49.10/06.31
NCU Taiwan	0.39 – 0.30 = +0.09	1.31	90	45.56/53.33/01.11
Taipei CWB	0.40 – 0.32 = +0.08	1.15	174	34.48/58.05/07.47
Silpakorn Uni.	0.58 – 0.44 = +0.14	1.15	552	59.24/39.67/01.09
Ubon Ratchathani	0.38 – 0.40 = −0.02	1.20	184	07.61/69.02/23.37
Mukdahan	0.32 – 0.38 = −0.06	1.08	116	06.03/57.76/36.21
Chiang Mai Met Sta.	0.44 – 0.48 = −0.04	0.98	706	07.65/71.67/20.68
Pimai	0.32 – 0.37 = −0.05	1.00	66	13.64/50.00/36.36
Asia	0.53 – 0.46 = +0.07	1.12	6487	30.78/63.22/06.00
Paris	0.26 – 0.17 = +0.09	1.29	349	45.56/52.72/01.15

^a M = MYD04_10K and A = AERONET site; ^b N = Total collocations of MYD04_3K and AERONET AOD; ^c EE = ± (0.05% + 20%).

) than for the 3-km results presented here. Munchak et al. [24] also attributed an observed positive error for the DT product at 3 km, to incorrect estimation of the surface reflectance. This effect appears more at 3 km than at 10 km, because, in the latter, most of the bright urban surfaces at 0.66 $\mathsf{\mu }\mathrm{m}$ in the DT retrieval box are discarded during the pixel selection process [24]. The 10-km retrievals for the 16 Asian sites support these observations, as overall 63% of retrievals fall within the EE, compared with 55% for the AOD product at 3 km. Similar to the 3-km product, uncertainties for most sites were due to overestimation of AOD (31% of total retrievals were overestimates), whereas rural stations were either unbiased or biased towards underestimation.

5. Conclusions

The objective of this study was to evaluate the new MODIS C6 AOD at 3 km, over Asia, including both bright urban surfaces and dark vegetated sites. Results indicate that the new MODIS DT C6 product at 3 km is not promising at a regional scale and is less reliable over Asian sites than our observed results for DT C6 at 10-km resolution. Only 55% of retrievals fell within the EE, compared to 63% within the EE for our testing of C6 DT at 10 km, and 69% reported by Levy et al. (2013) for global testing of C6 DT at 10 km over land [14]. Similar to DT 10 km, the error is mainly due to an underestimation of surface reflectance, especially over urban sites, leading to high AOD estimates for both low and high aerosol loading situations. Furthermore, only 4 out of 16 sites have above 67% accuracy, or approximately 1 SD within the EE, the standard test for the DT algorithm. The poorest results at Beijing, Karachi, and Osaka are urban sites dominated by coarse aerosols where uncertainty arises from both underestimation of surface reflectance and use of an absorbing aerosol model, which may not adequately represent both coarse dust and urban aerosols. The 10-km algorithm also performed poorly over Beijing and Osaka, with less than 25% of retrievals within the EE. The observed poor performance of MYD04_3K over Paris, an urban site with relatively low aerosol loadings, confirms the deficiency of DT’s surface reflectance assumptions when applied at 3-km resolution. Therefore, the MYD04_3K, which was devised mainly for finer detail of AOD retrieval over urban regions, appears unsuited to this task, as it has very poor retrieval quality over bright surfaces. An alternative may be to develop the DB algorithm at the finer 3-km resolution, as it is specifically targeted for retrieval over bright surfaces.