Detection of Air Pollution in Urban Areas Using Monitoring Images

Abstract

Air quality monitoring in polluted environments is of great significance to human health. Traditional methods use various pieces of meteorological equipment, which have limited applications in complex terrains and high costs. In this paper, a novel idea is put forward to solve the problem of air pollution monitoring in urban areas. We investigate whether air quality can be assessed visually by examining the haziness of photos from a far distance. Specifically, the correlation between the air quality indexes, such as the AQI, PM2.5, and PM10, of real outdoor scenarios and the haziness level evaluation scores of the monitoring images is calculated. The results show that the objective indicators can indeed reflect the level of air pollution, and the degree of correlation is invariant to the image size. To apply this new observation to a practical system, a novel method called fastDBCP (fast dark and bright channel prior) is developed. Based on a down-sampling strategy, a ratio is calculated between the dark and bright channel prior information in scaled images and adopted as the visual index of air pollution. Our experimental results show that the proposed metric not only demonstrates advantages in terms of its correlation degree and computational speed, but also shows a high level of classification accuracy compared to that of competing metrics.

1. Introduction

Outdoor haze is a well-known weather phenomenon. Traditionally, meteorologists monitor air quality with a variety of instruments and equipment. Popular indicators such as the AQI, PM2.5, and PM10 have been widely used to determine the quality of air. The advantage of these means is their high accuracy, while the disadvantage is that the distribution of the instruments is limited by the terrain, and it is difficult and costly in some areas, such as in mountains and deserts. In addition, the calculation of indicators requires data from various pieces of monitoring equipment. Hence, it is difficult to make real-time judgments. Although it may not be easy to solve the above problems by traditional meteorologic means, we can look at the problem from another angle. Researchers have been exploring various solutions in terms of vision. For instance, Sallis et al. suggested using vehicular sensors to detect air pollution and fog [1], Wong et al. used surveillance cameras to determine the temporal patterns of the PM10 concentration in the air [2], and Mukundan et al. proposed an air pollution detection method using snapshot hyperspectral imaging [3].

In this paper, we present a novel method to detect air pollution from monitoring images, which was inspired by the above vision-oriented methods. The differences are that our method does not need extra signals, such as sensors or hyperspectral data, and it covers a wider range of air quality categories. The rationale of the idea is twofold. First, thanks to the widespread development of imaging technology, monitoring videos/photos can be easily obtained through lenses, especially in urban areas. Second, in the field of image quality assessment, there are many effective objective evaluation indicators [4,5,6,7,8,9,10,11]. Of course, image quality cannot be simply treated as air quality. However, previous research studies have shown that certain image quality evaluation indicators can indeed characterize the haze level effectively [12,13,14,15,16], which is relevant to the air quality in hazy weather. Some implicit relationships between the subjective scores in hazy images and the corresponding environmental indicators were revealed in our previous work [17].

Based on the existing research findings, it is natural to ask: for hazy weather, does the image quality of a photo have a high correlation with the air quality at that time? Can such an indicator be used to characterize the level of air pollution? Our answer is yes, because physically the light source irradiates the object that is photographed, and the light reflected by the object is refracted or scattered by air particles to make an integral projection on the image perceptron. Assuming that the object and the light source are unchanged, the painting is equivalent to adding a filter to the reflected light of the object, and various components in the air will affect the performance of the filter, resulting in different images. Meanwhile, the composition of the air is related to the quality of the air. Hence, the quality of the air can be inferred by analyzing the differences in the images.

To confirm the aforementioned analysis, we performed experiments to explore the relationship between objective evaluation indicators and air quality indexes on our newly upgraded database called RHID_AQI [16]. The dataset includes real outdoor pictures taken in eight city environments. Compared to the former version, the upgraded dataset supplemented the air pollution level for each picture on the day it was taken. The experimental results are encouraging. A strong correlation between image quality and air pollution level is discovered. Hence, a fast evaluation system that reflects air quality through monitoring photos in urban areas is proposed in this paper. The main measure is to introduce a down-sampling strategy and utilize the dark channel and bright channel information in a hazy image. The new system is simple but effective, and the speed of the air quality evaluation is satisfactory.

The rest of the paper is organized as follows. The proposed method is described in detail in Section 2. In Section 3, we compare and analyze the performance of our metric compared to state-of-the-art indicators. In Section 4, possible applications in the field of air pollution monitoring and other types of real-time monitoring systems are investigated, and the conclusions are drawn in Section 5.

2. Dataset and Method

2.1. RHID_AQI Dataset

The RHID_AQI dataset was built by our team to provide a public benchmark to blindly evaluate the image quality of hazy images [16,17]. The source image set comes from a photographic exhibition held in China in 2014 [18], and we used a single-stimulus method to develop subjective evaluation experiments on these images. Specifically, the database is constructed with 301 images taken from real hazy environments in the urban areas of eight modern cities in China. The representative environments of each city are shown in Figure 1.

Differently from other image quality datasets, the RHID_AQI database provides information about air pollution. The environmental indexes include the air quality index (AQI), fine particulate matter (PM2.5) index, and inhalable particles (PM10) index. The air pollution level and category of each image are supplemented in the latest version. Specifically, according to the technical regulations on the ambient air quality index [19], there are six levels of air pollution, which are shown in

Table 1. AQI and air pollution category in the RHID_AQI dataset.

AQI	Air Pollution Level	Air Pollution Category	Classification Label	Classification Category
0–50	1	good	0	unpolluted
51–100	2	moderate	0	unpolluted
101–150	3	lightly polluted	1	polluted
151–200	4	moderately polluted	1	polluted
201–300	5	heavily polluted	1	polluted
>300	6	severely polluted	1	polluted

. For the purpose of classification, we use 0 and 1 as classification labels to represent unpolluted and contaminated conditions, respectively.

2.2. Subjective Index Versus Air Quality

Let us first review the relationship between subjective image quality and air quality. Our previous work noted that the subjective mean opinion score (MOS) of a hazy image is in good accordance with its visual quality, and the MOS values roughly negatively correlate with the air quality indexes represented by the AQI, PM2.5, and PM10 [17]. To visualize this statement, we select six pictures distributed in different air pollution levels of the same city in the RHID_AQI database and show their subjective scores and air quality index values respectively in Figure 2.

It can be intuitively observed that when the values of the environmental indexes such as the AQI, PM2.5, and PM10 are low, the air quality in the picture is better, and no haze is observed. When the environmental index values go up, the air pollution in the pictures gradually increases to a higher level. Hazy and heavy air pollution even shows up in the last couple of images. We also see that the subjective quality score of each picture decreases as the air quality goes down. It seems that a negative correlation between the subjective scores and the environmental indexes does exist.

To illustrate this more quantitatively, we demonstrate the scatter plots of the subjective MOS and the environmental indexes in Figure 3 and list their Pearson correlation coefficients (PCC) for comparison. The PCC is usually used to measure the degree of linear correlation between two signals, which is defined as follows:

$$PCC=\frac{{{\displaystyle \sum }}_{i=1}^M\left(x_i-\overline{X}\right)\left(y_i-\overline{Y}\right)}{\sqrt{{{\displaystyle \sum }}_{i=1}^M{\left(x_i-\overline{X}\right)}^2}·\sqrt{{{\displaystyle \sum }}_{i=1}^M{\left(y_i-\overline{Y}\right)}^2}},$$

(1)

where $M$ is the total number of images in the dataset; $x_i$ and $y_i$ denote the $ith$ image quality and air quality scores, respectively; and $\overline{X}$ and $\overline{Y}$ represent the mean values of the image quality and air quality scores in each image set, respectively.

Furthermore, any possible nonlinearity between the predicted image quality scores and air quality values should be eliminated before the evaluation index is calculated. In this paper, a five-parameter nonlinear logistic regression, which is recommended by the video quality expert group [20], is adopted:

$$y={\beta }_1\left[\frac{1}{2}-\frac{1}{1+e^{{\beta }_2\left(s-{\beta }_3\right)}}\right]+{\beta }_{4}s+{\beta }_5,$$

(2)

where $y$ represents the mapped objective score, $s$ denotes the predicted quality score, and ${\beta }_1$ through ${\beta }_5$ are the model-fitting parameters.

Figure 3 not only shows that the air quality indexes have a negative linear correlation with the subjective quality scores, but also provides a high correlation of 89% for the PM2.5. This high correlation is beneficial for air pollution studies because the concentration of fine particulate matter, i.e., PM2.5, is a major pollution source of concern for meteorologists [21,22] and has more public awareness due to its high relevance to pulmonary diseases.

2.3. Method

2.3.1. Motivation

Till now, we have found that the environmental indexes are well correlated with the subjective scores of the haze pictures. This implies that to some extent we can sense the air quality by observing the image quality. However, subjective evaluations of outdoor environments is time consuming, laborious, and inconvenient. In this section, we suggest substituting the subjective scores with automatically calculated objective scores to observe the air quality.

The question is, what type of metric is suitable for detecting air pollution? SSIM [23] is the most famous and widely used IQA metric, which can be adopted for full-reference and general-purpose image quality evaluation. Unfortunately, when we detect air pollution in hazy pictures, we do not have the original hazy free images to refer to. Hence, SSIM cannot be used in this scenario. For air pollution level evaluation tasks, the candidates should be no-reference metrics. There are many excellent no-reference image quality evaluation algorithms, which are either suitable for the degradation process for natural images [24,25,26,27,28,29,30], or suitable for special applications, such as with user-generated content [11], consumer images [10], and so on. However, there are few algorithms that have been designed to evaluate air pollution levels in real outdoor environments.

In our previous work, we presented a haze level evaluation strategy for real-world outdoor environments, which inspired us to apply a similar strategy to the problem of air pollution level detection. The contrast information between dark and bright channels in a single hazy image is still the main measure. Another consideration is the computation speed. As reported in Ref. [16], for a computer with an Intel(R) Core (TM) i7-4790 CPU (Dell (China) Company Limited, Xiamen, China) at 3.60 GHz and with 12 GB RAM, it needs 2.8 s to process a one-million-pixel photo. This might not meet the requirements of real-time monitoring and processing for high-resolution or low-power distributed supervisor control systems.

2.3.2. Insensitive Property

One intuitive solution is to use reduced images; thus, the computation speed can be greatly improved. However, the evaluation accuracy might be affected. In our previous studies [16], we drew the conclusion that the factors affecting the accuracy of haze level assessments include variations in the bright channel, changes in the contrast of the dark channel prior and bright channel prior, and the accurate division of high-contrast and low-contrast areas. Among them, $C^{\mathrm{db}}$, i.e., the contrast between the dark channel and bright channel in a single image, is the most critical factor. Hence, we use $C^{\mathrm{db}}$ as an example and illustrate the novel idea in Figure 4.

From Figure 4, we can observe that although the sizes of the hazy images in the first column are different, the histogram shapes of the corresponding $C^{\mathrm{db}}$ in the third column look very similar, and the segmentation thresholds are very close to each other. In other words, the shape of the histogram in different scales has self-similarity, and the segmentation threshold of the histogram is insensitive to image size. Therefore, if we use this insensitive property as a feature to assess the degree of air pollution, we are not vulnerable to the impact of image resolution reductions. In fact, the human visual system has a similar characteristic. People can observe and determine the haziness in an image, whether it is its original size or a reduced size.

2.3.3. fastDBCP

To conclude, there are two characters that are not affected by the size of the input image. One is the ratio of the dark channel prior to bright channel prior, and the other is the segmentation threshold of the high-contrast region and low-contrast region in the input image. These properties are beneficial to air pollution detection tasks. If the image is down-sampled to an appropriate size, the original evaluation accuracy can be maintained, and the operation speed can be greatly improved. We summarize the process of the presented method below. The new method is called fastDBCP, because it originates from the dark and bright channel prior (DBCP) metric recently published by our team in Ref. [16], but with the ability to detect air pollution levels at a faster operation speed.

The definition of fastDBCP is as follows:

$$\mathrm{fastDBCP}\triangleq \frac{1}{w}·\frac{1}{h}·{\displaystyle \sum }_{i=1}^w{\displaystyle \sum }_{j=1}^{h}dbcp\left(i,j\right),$$

(3)

where $w$ and $h$ represent width and height of an image, and $dbcp$ is the obtained air pollution level evaluation index. The $dbcp$ for image $I_{\mathit{D}}$ can be estimated as follows:

$$dbcp\left(I_{\mathit{D}}\right)\triangleq \{\begin{array}{ll}\frac{ C^{\mathrm{db}}\left(I_{\mathit{D}}\right) }{t_{\mathrm{d}}\left(I_{\mathit{D}}\right)},& I_{\mathit{D}}\in {\mathsf{\Omega }}_h \\ \mathrm{T}\left[C^{\mathrm{db}}\left(I_{\mathit{D}}\right)\right],& I_{\mathit{D}}\in {\mathsf{\Omega }}_l\end{array},$$

(4)

where $C^{\mathrm{db}}$ is the contrast between the dark channel and bright channel of the reduced image $I_{\mathit{D}}$ [12,31,32], ${\mathsf{\Omega }}_h$ and ${\mathsf{\Omega }}_l$ represent the high-contrast area and low-contrast area of $C^{\mathrm{db}}$, respectively, $t_{\mathrm{d}}$ is the refined transmission map of $I_{\mathit{D}}$ [33], and $\mathrm{T}\left[·\right]$ is the global image threshold of $C^{\mathrm{db}}$ using Otsu’s method [34]. The down-sampled image $I_{\mathit{D}}$ is defined as follows:

$$I_{\mathit{D}}\triangleq I\left(\frac{i}{n},\frac{j}{n}\right),$$

(5)

where $I\left(i,j\right)$ represents the original input hazy image, $\left(i,j\right)$ is the 2D column and row index of each pixel, and $n$ is the down-sampling parameter.

3. Experimental Results

3.1. Parameter Selection

There are two parameters that need to be empirically determined in fastDBCP. The first is the down-sampling parameter $n$. The candidate should meet the requirements of delivering both a high correlation performance and real-time processing. To compare the influence of $n$, we sample the input hazy images to 1/$n$ of the original sizes to calculate their fastDBCP indexes. Then, the correlation coefficient between the objective index and the three air quality indicators, i.e., the AQI, PM2.5, and PM10, is calculated, respectively, and demonstrated in Figure 5.

The results in Figure 5 exceed our expectation. We find that the correlation between our visual indicator and the air quality indexes does not decrease with respect to the image size reduction. It is even better than that of the original size. This phenomenon applies for all the three air pollution indicators in the whole RHID_AQI dataset with cross-scene images. From Figure 5, we know that when an original 1177 × 786 image is scaled down 16 times, the correlation reaches its peak. Hence, the recommended image resolution for a reduced image $I_{\mathit{D}}$ would be around 48 × 48, in which case the output of the algorithm could reflect the level of air pollution best.

The second parameter needs to be determined is the size of the template window for dark channel prior and bright channel prior calculation in a single image. The effect of window size on haze level evaluations has been analyzed in [16], and the conclusion was that the performance is not sensitive to the variation in window size. However, for fastDBCP, the conclusion might be different. After the original image is scaled down, the window size should also be appropriately reduced to match it; additionally, this can save operation time. Here, we select different window sizes from 3 × 3 to 9 × 9 to observe the influence of this parameter in Figure 6, and suggest a window size of 5 × 5 due to its optimal performance.

3.2. Performance Comparison

We compute the correlation between visual indexes and air quality indicators, respectively, and list the results in

Table 2. Comparison of PCC between visual indexes and air pollution indicators on the RHID_AQI dataset.

Method	AQI	PM2.5	PM10
MOS	0.71	0.70	0.70
$R$ [35]	0.68	0.66	0.71
$\overline{t}$ [36]	0.32	0.36	0.30
$C$ [12]	0.68	0.66	0.74
$\mu$ [37]	0.34	0.36	0.30
$Average$ [38]	0.19	0.09	0.31
$Variance$ [38]	0.53	0.48	0.60
$Entropy$ [38]	0.64	0.62	0.67
$D$ [14]	0.65	0.64	0.71
DBCP-I [16]	0.68	0.66	0.73
DBCP-II [16]	0.69	0.68	0.72
DBCP-III [16]	0.71	0.69	0.75
CNN [13]	0.68	0.66	0.72
DIQaM-NR [15]	0.75	0.72	0.75
fastDBCP	0.75	0.73	0.74

. The visual indicators used for the comparison include subjective assessment scores provided by the RHID_AQI database, and a series of image quality evaluation indicators based on haze environment characteristics or deep learning. The results are satisfying. Some of the objective indicators even outperform the subjective MOS value. Among them, the deep learning method DIQaM-NR and our metric have an equally leading position. However, the proposed fastDBCP algorithm is better overall because it is much quicker, which will be demonstrated in the following section.

3.3. Computation Cost

To further compare the computational cost, in Figure 7, we examine the average calculation time per image. As can be seen from the figure, the operation time decreases in an exponential trend. Taking $n=1$ and $n=16$ as an example, the processing time for each image is reduced from 2.58 s to 0.08 s, which means that the computational efficiency is 32 times better. This acceleration of speed is beneficial for an air quality real-time monitoring system. We also notice that the calculation speed in Figure 7 is not much different around $n=16$. Therefore, the size of the down-sampled image $I_{\mathit{D}}$ is suggested to be no smaller than 48 × 48 for both the performance and speed consideration.

Based on the performance of the objective metrics in Table 2, we choose relatively outstanding candidates to compare their computational speed. The configuration of our CPU computer is an Intel(R) Core (TM) i7-4790 CPU at 3.60 GHz with 12 GB RAM. The comparison results of the average computing time per image are shown in

Table 3. Comparison of computation cost among different image quality evaluation metrics on the RHID_AQI dataset.

Method	$\mathit{R}$	$\mathit{C}$	$\mathit{E}\mathit{n}\mathit{t}\mathit{r}\mathit{o}\mathit{p}\mathit{y}$	$\mathit{D}$	DBCP-I	DBCP-II	DBCP-III	CNN	DIQaM-NR	fastDBCP
Computation time (s)	1.34	0.07	0.02	1.86	2.79	1.54	2.80	0.33	0.81	0.08

. It can be seen clearly that with the same performance, fastDBCP is ten times faster than the neighboring deep learning method.

4. Discussion

4.1. Application in Air Pollution Classfication

To further explore the potential of the proposed fastDBCP, we apply it to remote air pollution monitoring. So far, we have verified that the output of the metric can reflect the air pollution level visually in real-time. Furthermore, can we set a monitoring threshold so that when the real-time monitoring value falls below the threshold, the air pollution monitoring system will issue an alarm to alert the staff? The answer is yes. We take Beijing in the RHID_AQI database as an example and plot the calculated values of fastDBCP for each outdoor picture in Figure 8. The predicted air quality value for each image is displayed as a different shape and color, depending on its actual air pollution classification label. In addition, we calculate the optimal classification threshold and represent it by the horizontal demarcation line.

As can be seen from Figure 8, the distribution of the fastDBCP values is quite consistent with that of air pollution label. Except for one case, the pictures with unpolluted air quality are basically above the 0.59 demarcation line, while the pictures with polluted air quality are all below the threshold. The calculated threshold is helpful to group most of the data points into their own categories. For a quantitative illustration of this, the confusion matrixes are plotted in Figure 9, which shows that the proposed fastDBCP has good class spacing in its sample space and can segment the two types of the sample data in most cases.

To further compare the classification performance of different algorithms, we calculate the classification accuracy according to the following definition:

$$Acc=\frac{TP+TN}{P+N},$$

(6)

where $Acc$ represents the accuracy of classification, $TP$ denotes the true positives, i.e., the number of unpolluted pictures that are correctly classified, $TN$ denotes the true negatives, i.e., the number of polluted images that are correctly classified, and $P$ and $N$ are the number of pictures for the positive and negative samples, respectively.

So far, we know that the data calculated using fastDBCP have a clear class spacing and are easy to classify, but we do not know whether other indicators can obtain similarly good results. Therefore, we calculate the optimal threshold and corresponding classification accuracy of nine competing algorithms and list the results in

Table 4. Comparison of classification accuracy among different image quality evaluation metrics on the RHID_AQI dataset.

Method	$\mathit{R}$	$\mathit{C}$	$\mathit{E}\mathit{n}\mathit{t}\mathit{r}\mathit{o}\mathit{p}\mathit{y}$	$\mathit{D}$	DBCP-I	DBCP-II	DBCP-III	CNN	DIQaM-NR	fastDBCP
$Acc$ (%)	78.74	75.42	77.74	54.49	77.74	79.40	81.06	54.49	54.49	82.39

. It can be seen that the fastDBCP method has a high classification accuracy and outperforms the other algorithms.

4.2. Application in Real-Time Monitoring System

The previous section mainly introduces the application of the proposed method in the field of air pollution monitoring. In fact, the fastDBCP approach has the potential to be applied to many more types of real-time monitoring systems. For example, in an autonomous driving scenario [39], it can effectively monitor the air quality of the current road surface. When hazy weather is a serious threat to driving safety, rapid warnings can be provided.

In addition, the fastDBCP index has a strong correlation with image visual quality reflecting the haziness degree in real environments, as shown in

Table 5. Comparison of PCC between different image quality indexes and subjective scores on the RHID_AQI dataset.

Method	$\mathit{R}$	$\mathit{C}$	$\mathit{E}\mathit{n}\mathit{t}\mathit{r}\mathit{o}\mathit{p}\mathit{y}$	$\mathit{D}$	DBCP-I	DBCP-II	DBCP-III	CNN	DIQaM-NR	fastDBCP
PCC	0.79	0.81	0.70	0.82	0.88	0.93	0.92	0.85	0.93	0.91

. According to the data in Table 3 and Table 5, the performance of the proposed fastDBCP is comparable to DBCP-II, DBCP-III, and the deep learning method DIQaM-NR, but has a ten-fold or greater advantage in terms of computing speed. This fast-processing capacity is a significant feature that enables it to be used in real-time monitoring systems. As shown in Figure 10, it can serve as a supplementary visual measure to detect other types of catastrophic weather having similar visual characteristics as hazy weather, such as forest fires [40], tornadoes [41], and so on. In these application scenarios, the speed of fastDBCP will be its advantage.

5. Conclusions

In this paper, we investigate an open question of whether we can sense air quality, especially the degree of air pollution, through visual information. After an analysis and verification, the following conclusions are obtained. First, the traditional image quality evaluation methods, especially the objective measurement index of image haze characteristics and deep learning methods, can reflect the air quality to a certain extent, with the same effect as the subjective index or even better. Second, the proposed air pollution evaluation method, i.e., fastDBCP, has a faster speed than the previous methods, and has a higher degree of correlation with the air quality indexes, so it can be applied to customized air pollution surveillance systems. Third, the presented novel metric could be integrated into an actual air pollution warning scheme to warn the aggravation of air pollution and could be used in hazy weather monitoring for automatic driving, tornado monitoring for weather forecasting, or forest fire predictions for disaster warnings.

Of course, our current research still has much room to improve. For example, fastDBCP cannot be compared and validated against other databases because we cannot find other publicly available datasets with both subjective measurements of images and air quality indicators. In addition, we hope to apply our indicators in a video surveillance system, but at present, there are few outdoor image and video quality evaluation databases for hazy weather [45], and no air quality indicator data are provided. Future studies should discover and build outdoor image or video databases that provide more accurate air quality indicators so that we can validate the performance of fastDBCP on these datasets and prove or verify the internal relationship between and mechanism of air quality and vision.