Multi-Site and Multi-Pollutant Air Quality Data Modeling

Abstract

This study proposes a new method for predicting air quality in major industrialized cities around the world. In some big cities, multiple air quality measurement stations are deployed at different locations to monitor air pollutants, such as NO₂, CO, PM _2.5, and PM ₁₀, over time. At every monitoring timestamp t, we observe one station × feature matrix ${\mathbf{x}}_t$ of the pollutant data, which represents a spatio-temporal process. Traditional methods of prediction of air quality typically use data from one station or can only predict a single pollutant (such as PM _2.5) at a time, which ignores the spatial correlation among different stations. Moreover, the air pollution data are typically highly non-stationary. This study has explicitly overcome the limitations of these two aspects, forming its unique contributions. Specifically, we propose a de-trending graph convolutional LSTM (long short-term memory) to continuously predict the whole station × feature matrix in the next 1 to 48 h, which not only captures the spatial dependency among multiple stations by replacing an inner product with convolution, but also incorporates the de-trending signals (transforms a non-stationary process to a stationary one by differencing the data) into our model. Experiments on the air quality data of the city of Chengdu and multiple major cities in China demonstrate the feasibility of our method and show promising results.

1. Introduction

The air quality problem is not only a health issue but also an urgent issue for sustainable development. With the rapid development of the global economy, the challenge of air pollution has become increasingly apparent. Air quality issues are complex and receive increasing attention. In most cities, multiple monitoring stations are located at different locations to report air quality indicators in real-time. Typically, levels of air pollutants are recorded hourly by multiple stations (such as the data used in this article). This means that at every timestamp, one air quality matrix with the shape of station × feature can be collected by all the stations, where features include NO₂, CO, PM_2.5, PM₁₀, etc. Based on this air quality data matrix, an air quality index (AQI) can be calculated to inform the public of the air quality at present [1]. However, the general public is more interested in predicting future air quality rather than real-time reporting. This prediction not only benefits people’s daily activities (such as developing travel plans or avoiding routes with poor air quality), but also improves their health by wearing masks to reduce exposure to air pollution. It also provides policy implications for the government.

There are a large number of works for air quality prediction in the literature [2,3,4,5,6]. Most of them are based on the temporal dependency between future states and historical data, such as time series models [4,7,8] and deep neural networks [3,9,10,11]. However, there are several limitations to the existing methods. First, many models [4,9,10] take the air quality prediction as a single-pollutant regression problem; for example, focusing on only the particulate matter PM_2.5. To predict the level of another pollutant, e.g., carbon monoxide CO, a different model needs to be trained. Second, to improve the performance for prediction, some methods [8,12] choose to incorporate extra knowledge, such as the weather forecasting results [12] or the traffic data [8]. In practical applications, it is not convenient to collect additional information and synchronize it with air quality data. When data from multiple stations are available, the geographic correlation between these stations is expected to contribute to air quality prediction [8,12]. However, most existing methods can only process data from one station at a time, leaving spatial correlations between multiple stations ignored [3,4,10] or partially considered [8,12]. Last but not least, air quality data usually represent a high degree of non-stationarity, as shown in Figure 1, where the mean value of the data varies over time, making the modeling problem even more difficult. Ignoring the non-stationarity in the data can lead to unacceptable prediction errors and severely weaken the predictive power of the model. Therefore, learning potential spatio-temporal features from non-smooth processes is particularly important for prediction.

We propose to solve the aforementioned problems using a non-stationary diffusion graph convolutional LSTM (long short-term memory). In detail, the spatio–temporal characteristics of air quality data from multiple sites motivate us to use the diffusion convolutional LSTM network [13]. The diffusion convolution [14] captures the spatial dependency using bidirectional random walks on the meteorological monitoring sites graph $\mathcal{G}=(V,E,A)$ as shown in Figure 2b. In addition, we add a de-trending step to the diffusion convolutional LSTM to accommodate the non-stationarity of the data. This de-trending operation can better capture the true characteristics of the data, thus improving the accuracy of the prediction. In time series analysis, the de-trending step is usually implemented via a differencing procedure ${\mathbf{x}}_t-{\mathbf{x}}_{t-1}$ [15,16]. As a result, the input of the proposed model involves both ${\mathbf{x}}_t$ and ${\mathbf{x}}_t-{\mathbf{x}}_{t-1}$. We name the proposed model as the long–short de-trending graph convolutional network (LS-deGCN).

Motivated by the vast applications of LSTM in the area of natural language processing (NLP), we propose two variants of the LS-deGCN, as shown in Figure 3d,e. At every timestamp, there is one station × feature matrix of data $\mathbf{x}\in {\mathbb{R}}^{M\times N}$ observed, where M is the number of stations and N is the number of features. Therefore, the final dataset is a three-dimensional tensor $\mathcal{X}\in {\mathbb{R}}^{M\times N\times T}$ by stacking all $\mathbf{x}$’s along the time, where T is the number of timestamps. Given a window length, for example, 3, which is a tuning parameter in our model, we can slice the samples along the third dimension T. For ease of exposition, we omit the first two dimensions and denote $\mathcal{X}[0:3]$$=\mathcal{X}[:,:,0:3]$. The sliced samples are in the form of ${\mathbf{x}}_1=\mathcal{X}[0:3],{\mathbf{x}}_2=\mathcal{X}[1:4]$, etc. We propose two different ways of defining the target $\mathbf{y}$, which correspond to the two variants of our models. One is a sequence-to-frame model, that is, ${\mathbf{y}}_1=\mathcal{X}\left[4\right],{\mathbf{y}}_2=\mathcal{X}\left[5\right]$, etc., and the other is a sequence-to-sequence model, that is, ${\mathbf{y}}_1=\mathcal{X}[1:4],{\mathbf{y}}_1=\mathcal{X}[2:5]$, etc.

The contributions of this research are threefold:

1.

Firstly, we introduce a de-trending operation into the traditional LSTM model to effectively eliminate the long-term trend in non-stationary data. This improvement enables our model to more accurately capture the changing patterns in non-stationary data.

2.

Secondly, we utilize a diffusion graph convolution to extract the spatial correlations present in the air quality data across multiple stations. This innovative method not only improves prediction accuracy, but also has important implications for understanding and predicting air pollutant spread and impacts.

3.

Lastly, we propose two distinct models based on LS-deGCN for multi-site air quality prediction and evaluate them on air quality data from Chengdu and seven other major cities. The experimental results demonstrate that the proposed models significantly outperform other existing methods in terms of prediction accuracy and stability.

The rest of the paper is structured as follows. Section 2 presents a brief review of related works and research gaps. In Section 3, we introduce the proposed LS-deGCN and its two variants: the sequence-to-frame model and the sequence-to-sequence model. The experimental results are shown in Section 4. Section 5 concludes this paper with some remarks.

2. Literature Review and Research Gaps

Traditionally, meteorologists often make air quality predictions based on their empirical knowledge of meteorology. With the development of statistics and machine learning, data-driven methods for air quality prediction are becoming increasingly popular nowadays, which can be typically divided into statistical approaches [4,17,18,19,20,21] and deep learning approaches [9,10,12,13,22,23].

Some existing deep learning models treat air quality forecasting as a single pollutant regression problem; thus, they only predict one pollutant at a time [4,9,24]. As a result, separate models need to be trained for different pollutants if all of the pollutants are of interest, where each model only focuses on one pollutant. Zhang et al. [4] conducted a statistical analysis of the PM_2.5 data in the years 2013–2016 from the city of Beijing based on a flexible non-stationary hierarchical Bayesian model. Mukhopadhyay and Sahu [20] proposed a Bayesian spatio-temporal model to estimate the long-term exposure to air pollution levels in England. Since the air quality records are typically monitored over time, Ghaemi et al. [18] designed a LaSVM-based online algorithm to deal with the streaming of the air quality data. Along another direction, the Granger causality has been proposed to analyze the correlations among the air pollution sequences from different monitoring stations [7,8,12]. Suppose that the sequence of air pollutant records from one station is denoted by ${\mathbf{x}}_t$, and the sequence of a factor (such as the geographical correlation) from another station by ${\mathbf{y}}_t$, then the mathematical representation of the Granger causality is given by

$${\mathbf{x}}_t=\sum _{k=1}^K{a}^k{\mathbf{x}}_{t-k}+\sum _{k=1}^K{b}^k{\mathbf{y}}_{t-k}+{ϵ}_t,$$

(1)

where $a^k$ is a weight indicating how the width of time window k affects the future evolution, $b^k$ represents the correspondent weight for ${\mathbf{x}}_t$ and ${\mathbf{y}}_t$, and ${ϵ}_t$ is the residual for time series ${\mathbf{x}}_t$. If $a^k\ne 0$ and $b^k=0$, it means that the sequence ${\mathbf{x}}_t$ is caused by its own history. Wang and Song [12] combined the Granger causality with deep learning models and there are also some connections between the Granger causality and LSTM [25].

Deep learning, more specifically, the recurrent neural network (RNN) and LSTM [25] have achieved vast success in the area of NLP [26] and video analysis [27]. The LSTM can capture both the long and short contextual dependency of the data via different types of gates. In the classical LSTM, the well-designed gates, for example, the input gate and the forget gate, make the network very powerful to model the temporal correlations of the sequential data. Thus, much literature has applied LSTM and RNN to the field of air quality prediction.

Guo et al. [28] proposed a multi-variable LSTM based on both temporal- and variable-level attention mechanisms, which was used to predict the PM_2.5 level in Beijing. Fan et al. [11] also used the LSTM as a framework to predict air quality in Beijing based on air pollution and meteorological information. Compared with [28], the data used in [11] are collected from multiple stations, while the data from different stations are analyzed separately by ignoring the spatial correlation. In big cities, there is usually more than one station deployed to monitor air pollutants and meteorological information. The correlations among readings from different stations are highly informative in forecasting future air quality; thus, the spatial information should be incorporated. Xu et al. [29] proposed a multi-scale three-dimensional tensor decomposition algorithm to deal with the spatio-temporal correlation in climate modeling. In deep learning, a convolutional neural network (CNN) has advantages in extracting spatial features, while the RNN has superior performance in processing sequence data. Therefore, it is expected to achieve more accurate prediction by combining convolution and RNN in the analysis of spatial and temporal data. Huang and Kuo [9] proposed to stack a CNN over LSTM to predict the level of PM_2.5. However, this modification results in a non-time series model, which may result in a loss of power when quantifying sequential air quality data.

Although the classical RNN and LSTM possess powerful capabilities in modeling time series data, they may not be suitable to capture the spatial correlation in the air quality data [11]. Let us look at the specifics. The air quality data are hourly recorded by multiple stations. At each timestamp, the observed data can be represented by a station × feature matrix $\mathbf{x}\in {\mathbb{R}}^{M\times N}$, where M is the number of stations and N is the number of features. The features refer to air pollutants (e.g., CO₂, PM_2.5) and meteorological parameters (e.g., air pressure, air temperature, and air humidity). As shown in Figure 2, the air quality data of the city of Chengdu involves nine monitoring stations and nine features. The entire data, thus, can be treated as a three-dimensional tensor $\mathcal{X}\in {\mathbb{R}}^{M\times N\times T}$, with respect to the three axes station × feature × time. The third dimension T corresponds to the number of timestamps, which indicates the sequential nature of the data $\mathcal{X}$.

The air quality data in Chengdu has a typical spatio-temporal patternn, and the data from different meteorological stations have a strong spatial correlation. Figure 2a shows the locations of different monitoring stations. Figure 2c shows the weekly readings of two types of pollutants, namely CO and PM_2.5, from Station 4 and Station 5, respectively. In the records of CO and PM_2.5, there is a significant change from four stations to five stations, and their patterns are similar. The area where Station 5 is located seems to be more polluted than the area where Station 4 is located, and this information is useful for government policy making in different regions. The similar oscillating patterns in the records of the two stations imply that the rows and columns in the matrix station × feature are correlated and further series correlation over time can also be observed. However, there seems no obvious daily periodicity or “weekends/holidays” effect from the data.

In this paper, we propose to model the air quality data from multiple stations with an LS-deGCN, which replaces the fully connected layer in the classical LSTM with graph diffusion convolution. According to the pairwise geographic distances between different stations, we compute the undirected graph $\mathcal{G}=(V,E,A)$ with a thresholded Gaussian kernel [30], where V, E, and A denote the vertex (station) set, edge set, and adjacent matrix of the graph, respectively. $a_{ij}=exp(-\frac{\parallel l_i-l_j\parallel }{{\sigma }^2})$, as shown in Figure 2b, where $a_{ij}$ is an element of the adjacent matrix A, $l_i$ and $l_j$ are the geographical locations of the $i,j$-th nodes, and $\parallel l_i-l_j\parallel$ calculates the distance between them. The LS-deGCN architecture can be well adapted to the spatio-temporal data. As far as we know, LS-deGCN was first proposed to analyze the two-dimensional radar echo map [13]. In addition, our model can accommodate the non-stationarity of the air quality data. Compared with the work of Wilson et al. [31], our method is easier to be implemented and trained. A similar diffusion convolutional recursive neural network was proposed in [14] to model the traffic flow.

3. Proposed Models

Figure 3 illustrates the overall framework of the proposed method.

3.1. Non-Stationary Diffusion Convolutional LSTM

We propose to model the non-stationary station × feature × time data with the de-trending diffusion convolutional LSTM (LS-deGCN), which aims to incorporate both the spatio-temporal correlations [13] and non-stationarity into our model. In contrast to the classical LSTM where all the gates are implemented via fully connected neural networks, the proposed model replaces them with diffusion operations [14]. In addition, the differences between the input signals ${\mathbf{x}}_t-{\mathbf{x}}_{t-1}$ and corresponding hidden signals ${\mathbf{h}}_{t-1}-{\mathbf{h}}_{t-2}$ are input for de-trending as follows,

$$\begin{array}{cc}\hfill f_t& =\sigma \left(W_f\ast [{\mathbf{x}}_t,{\mathbf{h}}_{t-1}]+W_{df}\ast [{\mathbf{x}}_t-{\mathbf{x}}_{t-1},{\mathbf{h}}_{t-1}-{\mathbf{h}}_{t-2}]+W_{cf}\circ {\mathbf{c}}_{t-1}+b_f\right)\hfill \end{array}$$

(2)

$$\begin{array}{cc}\hfill i_t& =\sigma \left(W_i\ast [{\mathbf{x}}_t,{\mathbf{h}}_{t-1}]+W_{di}\ast [{\mathbf{x}}_t-{\mathbf{x}}_{t-1},{\mathbf{h}}_{t-1}-{\mathbf{h}}_{t-2}]+W_{ci}\circ {\mathbf{c}}_{t-1}+b_i\right)\hfill \end{array}$$

(3)

$$\begin{array}{cc}\hfill {\mathbf{c}}_t& =f_t\circ {\mathbf{c}}_{t-1}+i_t\circ tanh\left(W_c\ast [{\mathbf{x}}_t,{\mathbf{h}}_{t-1}]+W_{dc}\ast [{\mathbf{x}}_t-{\mathbf{x}}_{t-1},{\mathbf{h}}_{t-1}-{\mathbf{h}}_{t-2}]+b_c\right)\hfill \end{array}$$

(4)

$$\begin{array}{cc}\hfill {\mathbf{o}}_t& =\sigma \left(W_o\ast [{\mathbf{x}}_t,{\mathbf{h}}_{t-1}]+W_{do}\ast [{\mathbf{x}}_t-{\mathbf{x}}_{t-1},{\mathbf{h}}_{t-1}-{\mathbf{h}}_{t-2}]+W_{co}\circ {\mathbf{c}}_t+b_o\right)\hfill \end{array}$$

(5)

$$\begin{array}{cc}\hfill {\mathbf{h}}_t& ={\mathbf{o}}_t\circ tanh\left({\mathbf{c}}_t\right),\hfill \end{array}$$

(6)

where * and ∘ represent the diffusion convolution and Hadamard product, respectively; $[\mathbf{a},\mathbf{b}]$ is the concatenation of vectors $\mathbf{a}$ and $\mathbf{b}$; $\sigma (·)$ is the $sigmoid$ function; and $tanh(·)$ is the hyperbolic tangent function. Equations (2), (3), and (5) correspond to the implementations of the forget gate, input gate, and output gate. Equation (4) is the updating mechanism of the cell state ${\mathbf{c}}_t$, and the hidden state ${\mathbf{h}}_t$ is updated by Equation (6). Note that $\{W_f,W_{df},W_{cf},b_f\}$ are the weights (corresponding to the feature, de-trending term, and hidden state) and bias of the forget gate, and $\{W_i,W_{di},W_{ci},b_i\}$ and $\{W_o,W_{do},W_{co},b_o\}$ are the counterparts of the input gate and the output gate, respectively. Furthermore, $\{W_c,W_{dc},b_c\}$ are the weights (corresponding to the feature and de-trending term) and bias of the hidden state.

The diffusion graph convolution, which is shown in Figure 3b, is specifically designed to capture the diffusion process of information via the graph structure. This assumes that information is transferred from one node (monitoring station) to one of its neighbors (other monitoring stations) with a specific transfer probability, resulting in the information distribution reaching equilibrium after several iterations. This process facilitates information exchange and updates between nodes and their adjacent ones; moreover, it could incorporate information spread over multiple hops or steps in the graph, thereby enabling the capture of spatial correlations among different stations. The specific definition is as follows [14],

$$\widehat{\mathbf{x}}=\sum _{m=1}^M\sum _{k=0}^{K-1}(w_{k1}{\left(D_O^{-1}A\right)}^k+w_{k2}{\left(D_I^{-1}A\right)}^k)\mathbf{x}$$

(7)

where $\mathbf{x}$ is an input matrix and $\widehat{\mathbf{x}}$ is the corresponding output. A is an adjacent matrix of the M stations as shown in Figure 2b, and $D_O$ and $D_I$ are the out-degree and in-degree diagonal matrix of A. $D_O^{-1}A$ and $D_I^{-1}A$ represent the transition matrices of the diffusion process and the reverse one, respectively.

The frame differencing term ${\mathbf{x}}_t-{\mathbf{x}}_{t-1}$ is governed by the corresponding coefficient matrices. In Equation (5), we observe that not only does the current output ${\mathbf{o}}_t$ depend on the current input ${\mathbf{x}}_t$, the current cell state ${\mathbf{c}}_t$, and the previous hidden state ${\mathbf{h}}_{t-1}$, but it also requires the input of the differencing signal ${\mathbf{x}}_t-{\mathbf{x}}_{t-1}$. We also concatenate the hidden differencing signal ${\mathbf{h}}_{t-1}-{\mathbf{h}}_{t-2}$ with the input differencing ${\mathbf{x}}_t-{\mathbf{x}}_{t-1}$.

The main differences between the classical LSTM and our LS-deGCN lie in the diffusion graph convolutional operations and the extra de-trending items ${\mathbf{x}}_t-{\mathbf{x}}_{t-1}$ and ${\mathbf{h}}_{t-1}-{\mathbf{h}}_{t-2}$. If the graph convolution in Equations (2)–(6) are substituted with inner products and set ${\mathbf{x}}_t-{\mathbf{x}}_{t-1}=0,{\mathbf{h}}_{t-1}-{\mathbf{h}}_{t-2}=0$, then the LS-deGCN reduces to the classical LSTM. The great success achieved by the graph convolutional neural network is largely owed to the graph convolution, which can effectively capture the spatial information of data. This is also the rationale underlying our LS-deGCN to characterize the spatial correlations.

As discussed earlier, the air quality data are typically collected from multiple stations over time, which represent both the spatial and temporal characteristics. The spatial correlations among different stations depend on many complicated factors such as geographical distances, directions, and meteorological features, which are very important for air quality prediction [7,8,12]. In the existing works, the spatial correlations are typically inferred manually by resorting to some complicated statistical processes. In this paper, we propose to model the spatial correlations with diffusion graph convolution. In other words, the spatial correlations among different stations can be extracted via the LS-deGCN, which is the most notable difference between our method and the existing models for air quality prediction. This distinctive feature makes our model coherent in the deep learning framework and more powerful for air quality forecasting.

3.2. Two New Models

According to the scheme of generating samples and targets as shown in Figure 3d,e, we develop two variants of our modeling framework. To help further elaboration, we first introduce some necessary notation. We first select ${\Delta }_t$, the width of a slicing window or the length of the samples, which is an analogy to the sentence length in NLP. The value of ${\Delta }_t$ represents how many historical sample observations are used for prediction. Let l denote the time lag from a sample to the target; that is, our goal is to make a prediction of air quality for l hours later. We then propose the following two modeling frameworks.

In the sequence-to-frame (seq2frame) model, we take samples to be slices of the three-dimensional tensor $\mathcal{X}\in {\mathbb{R}}^{M\times N\times T}$ along the dimension T. The sample length is ${\Delta }_t$, the width of the slicing window. The target is taken to be the single frame after the corresponding samples. Since the time lag from a sample to its target is l, the training and testing samples can be generated as follows,

$$\begin{array}{cc}\hfill {\mathbf{x}}_i& =\mathcal{X}[i-1:{\Delta }_t+i-1],\hfill \\ \hfill {\mathbf{y}}_i& =\mathcal{X}[{\Delta }_t+i+l-1].\hfill \end{array}$$

Figure 3d illustrates the way to generate ${\mathbf{x}}_i$ and ${\mathbf{y}}_i$ in the seq2frame model, with ${\Delta }_t=5$ and time lag $l=1$. As a result, we can obtain the first sample and its target, i.e., ${\mathbf{x}}_1=\mathcal{X}[0:5]$, and ${\mathbf{y}}_1=\mathcal{X}\left[6\right]$.

The sequence-to-sequence (seq2seq) model is motivated by the seq2seq model in NLP [26], under which the samples and targets are generated as in Figure 3e with both the sample and target of length ${\Delta }_t$ slices. We use the same way of slicing samples $\mathbf{x}$ as that for the seq2frame scheme and define a time l-shifted sequence after $\mathbf{x}$ to be the target $\mathbf{y}$. This data-generating scheme leads to

$$\begin{array}{cc}\hfill {\mathbf{x}}_i& =\mathcal{X}[i-1:{\Delta }_t+i-1],\hfill \\ \hfill {\mathbf{y}}_i& =\mathcal{X}[i+l-1:{\Delta }_t+i+l-1].\hfill \end{array}$$

As illustrated in Figure 3e, with ${\Delta }_t=5$ and time lag $l=1$, the first sample is ${\mathbf{x}}_1=\mathcal{X}[0:5]$ and its target is ${\mathbf{y}}_1=\mathcal{X}[1:6]$.

3.3. Selection of Tuning Parameters l and ${\Delta }_t$

The time lag l and window width ${\Delta }_t$ are critical tuning parameters for the proposed models. If the value of l is too small, the overlapping of the information would be too much. In contrast, a larger value of time lag l would make the prediction more difficult. Theoretically, the proposed model can predict the future air quality for any length of the window by setting l to be large enough. In practice, we explore the value of l from 1 h to 48 h, which means that we aim to predict the air quality till the day after tomorrow. Intuitively, the more past data are used, the better performance would be expected. In other words, the performance is usually monotonically increasing with the value of ${\Delta }_t$.

4. Experiments and Results

4.1. Baselines

We choose the following three models as the baselines for comparisons.

1.

Linear regression: This is one of the most commonly used approaches to modeling the relationship between a dependent variable y and covariates $\mathbf{x}$.

2.

Support vector regression: Equipped with a radial basis kernel, it extends linear regression by controlling how much error in regression is acceptable.

3.

LSTM sequence-to-scalar (seq2scalar): Samples under this model are constructed in the same way as those under the seq2seq model. The difference is that we take the target y as one of the nine pollutants one by one; that is, we need to train nine separate models for the nine pollutants.

It is worth noting that neither linear regression nor support vector regression can capture spatial correlations with other stations. This means that each station needs to be predicted separately with this method. In addition, all three baseline methods are single-pollutant regression procedures, that is, we need to train separate models on each pollutant at a time. Taking the linear regression as an example, to compare with the proposed model that predicts the whole station × feature map, we need to separately train nine linear regression models corresponding to the six air pollutants (NO₂, CO, SO₂, O₃, PM_2.5, and PM₁₀) and the three meteorological measurements.

We use the root mean squared error (RMSE), accuracy, and mean absolute error (MAE) as assessment criteria for prediction [9],

$$\begin{array}{ccc}\hfill \mathrm{RMSE}& =& \sqrt{\frac{{\sum }_{i=1}^n{({\widehat{\mathbf{y}}}_i-{\mathbf{y}}_i)}^2}{n}},\hfill \\ \hfill \mathrm{Accuracy}& =& 1-\frac{{\sum }_{i=1}^n|{\widehat{\mathbf{y}}}_i-{\mathbf{y}}_i|}{{\sum }_{i=1}^n{\mathbf{y}}_i},\hfill \\ \hfill \mathrm{MAE}& =& \frac{{\sum }_{i=1}^n|{\widehat{\mathbf{y}}}_i-{\mathbf{y}}_i|}{n},\hfill \end{array}$$

where ${\mathbf{y}}_i$ is the ground truth, ${\widehat{\mathbf{y}}}_i$ is the predicted value, and n is the size of the testing dataset.

4.2. Data Description and Preprocessing

The Chengdu dataset is composed of the air quality records from 1 January 2013 to 31 December 2016, from nine monitoring stations in the city of Chengdu. Since stations 3 and 9 were very close to each other and their records were the same, we removed the redundancy by dropping the data from station 9. Moreover, we deleted the data from station 8 because over 40% of readings for CO, SO₂, and O₃ were missing, which mainly occurred between June 2014 to January 2016 due to the sensor dysfunction. As a result, we only used data from seven stations and there are 35,064 instances for each station. Each air quality instance consists of the concentration of six air pollutants: NO₂, CO, SO₂, O₃, PM_2.5, and PM₁₀, and three meteorological measurements including air pressure, air temperature, and air humidity. Therefore, the observed data are in the form of $\mathcal{X}\in {\mathbb{R}}^{M\times N\times T}$, where $M=7$, $N=9$, and $T=$ 35,064.

Since the missing values in the data are mostly concentrated between two points rather than large areas, we use linear interpolation to fill in the missing values of $\mathcal{X}$ on the time domain for each column. For example, we can interpolate the missing ${\mathbf{x}}_i$ with the values of ${\mathbf{x}}_{i-1}$ and ${\mathbf{x}}_{i+1}$. The specific steps are as follows [32]: (1) Calculate the slope $m=({\mathbf{x}}_{i+1}-{\mathbf{x}}_{i-1})/(t_{i+1}-t_{i-1})$ between these two points. (2) Use the formula $\mathbf{x}=mt+b$, where m is the slope, and the $\mathbf{x}$-intercept b is obtained. (3) Substitute $t_i$ into the formula and we can obtain an estimated value for the missing value ${\mathbf{x}}_i$. After interpolation, we normalize all the data into the range of $[0,1]$ using the min–max normalization. The specific formula is ${\mathbf{x}}^{\ast }=(\mathbf{x}-{\mathbf{x}}_{min})/({\mathbf{x}}_{max}-{\mathbf{x}}_{min})$.

The training and testing samples for baselines are generated in the same way as those for the seq2frame model. The difference is how to extract the target. In the seq2frame architecture, the network receives one frame (matrix) as a target, while the targets of the baseline models are all scalars. Instead of using the whole frame as the target, we take a statistic (e.g., the mean) of the measurement of one pollutant within that frame as the target. Taking the linear regression as an example, we model the air quality prediction as a one-pollutant linear regression problem. Suppose that we fit a linear regression model to predict the PM_2.5 emission, then we can use the mean or the median of the PM_2.5 level within that frame as the target.

4.3. Training

After data normalization, we extracted the training samples and testing samples from dataset $\mathcal{X}$. We set the window width to be ${\Delta }_t=\{24,48\}$ h to strike a balance between the length of contextual information and the LSTM model complexity. As a result, every sample is a consecutive record of air quality measurements for one day or two days. We partitioned the data into training, testing, and validation sets with a similar ratio according to [33] as shown in

Table 1. Partition for training, validation, and testing datasets.

Datasets	Description
Training	Data from 1 January 2013 to 31 December 2015
Validation	Data from 1 January 2016 to 1 June 2016
Testing	The remaining data

. We constructed a two-layer LS-deGCN by stacking two one-layer LS-deGCNs.

Figure 4a,b reports the training and validation errors under the seq2frame and seq2seq models, respectively. Both models converged in 10 epochs, while the patterns of their convergence are different. Under the seq2frame model, both the training error and validation error drop sharply within the first three epochs, and then the gap between them gradually decreases. By contrast, the gap between the training error and validation error under the seq2seq model remains at a stable level even after 15 epochs. The different error patterns may be due to the different ways of extracting the target variable $\mathbf{y}$ under the two models, as shown in Figure 3d,e.

4.4. Visualization of Predictions

To visualize part of the predicted results, Figure 5 shows four $stations\times feature$ air quality maps randomly selected from the testing dataset, as well as the corresponding predicted station × feature frames by the seq2seq model with $l=24$ and ${\Delta }_t=48$. For each square of the station × feature matrix, the dark blue color represents a larger value and the light blue color indicates a smaller value.

The far right-end three columns (corresponding to the three meteorological measurements: air pressure, air temperature, and air humidity) of the ground-truth map are almost identical across seven stations. In contrast, the levels of air pollutants recorded by different stations are quite different. For example, the levels of O₃ (the third column) reported at different locations show very different patterns, and in the third frame, stations 2 and 5 reported a much higher value than other stations. The proposed model can make an accurate prediction of O₃ for each station. We also observe that the values of air pressure in the third frame are much lower than the other three frames, and the level of suspended particulate matter, PM₁₀ and PM_2.5, in the last frame is serious, with the level of PM₁₀ from stations 1 and 6 being the top two highest. These trends are all correctly predicted by our proposed seq2seq model. From the visualization results, we conclude that (1) the patterns of the nine air quality measurements are different from one another and (2) the proposed model can make an accurate prediction based on all of the nine measurements from seven stations.

To examine the proposed air quality model across different cities, we conducted another experiment with seven stations from seven major cities in China, including Beijing, Shanghai, Chengdu, Wuhan, Guangzhou, Xi’an, and Nanjing. The data were collected daily (instead of hourly) on seven pollutants (AQI, PM_2.5, PM₁₀, SO₂, CO, NO₂, and O₃) from 2 December 2013 to 29 February 2020 for each city. Figure 6 shows that the performance of air quality prediction for seven cities is not as good as that for the seven stations from the same city of Chengdu. One possible reason is that the spatial correlation among seven stations in Chengdu is much stronger than the seven cities that are far away from each other. This is consistent with our general understanding, thus proving the effectiveness of the proposed LS-deGCN in capturing spatial associations.

4.5. Evaluation with Three Metrics

RMSE. As discussed in Section 3.2, the proposed model can predict the future level of air pollution from 1 h to 48 h by varying l from 1 to 48. Moreover, the data used for prediction can be a collection of samples on the last day or the last several days, for which ${\Delta }_t$ is used to control the length of samples. In other words, we can vary the parameter ${\Delta }_t$ to obtain different training and testing datasets for different experiments. We explore six experiments by combinations of $l=\{1,24,48\}$ and ${\Delta }_t=\{24,48\}$. Therefore, we utilize the data collected from yesterday and the last two days to predict the air quality for one hour later, one day later, and two days later, respectively.

Table 2. Comparison of the root mean squared error (RMSE) among different methods based on the Chengdu testing dataset, where a smaller RMSE indicates a better result.

	Models	$\mathit{l}=1$	$\mathit{l}=24$	$\mathit{l}=48$
	Linear regression	1.78	1.82	2.23
	Support vector regression	1.61	1.93	1..98
${\Delta }_t$ = 24	LSTM seq2scalar	1.04	1.12	1.25
	Non-stationary LS-deGCN seq2frame	0.58	0.78	1.1
	Non-stationary LS-deGCN seq2seq	0.56	0.77	0.87
	Linear regression	1.67	1.79	2.03
	Support vector regression	1.52	1.76	1.95
${\Delta }_t$ = 48	LSTM seq2scalar	0.87	0.92	0.95
	Non-stationary LS-deGCN seq2frame	0.45	0.62	0.67
	Non-stationary LS-deGCN seq2seq	0.39	0.56	0.57

displays the RMSEs from the testing set for $l=\{1,24,48\}$ concerning ${\Delta }_t=\{24,48\}$. The RMSEs of the non-stationary LS-deGCN seq2frame and seq2seq models can be calculated directly based on the station × feature matrix. However, for the other three models, we need to conduct training and testing for each pollutant separately, and then calculate the average RMSE over all the pollutants. Our proposed models demonstrate significant improvements compared with the baseline methods and, in particular, the non-stationary LS-deGCN seq2seq model achieves the best performance under all six scenarios. The LSTM-based methods outperform the traditional linear regression and support vector regression. Moreover, the RMSEs of our models increase as the time lag l increases and decreases as ${\Delta }_t$ takes a larger value. Intuitively, predicting air quality two days in the future is more challenging than predicting it one hour in the future. However, as the sample size increases, the accuracy of prediction improves, as more historical information can help us capture long-term trends in the data.

Accuracy.

Table 3. Comparison of accuracy among different methods based on the Chengdu testing dataset, where a higher value of accuracy indicates a better result.

	Models	$\mathit{l}=1$	$\mathit{l}=24$	$\mathit{l}=48$
	Linear regression	0.5634	0.5367	0.5278
	Support vector regression	0.5763	0.5598	0.5557
${\Delta }_t$ = 24	LSTM seq2scalar	0.7021	0.7167	0.7198
	Non-stationary LS-deGCN seq2frame	0.7234	0.7545	0.7517
	Non-stationary LS-deGCN seq2seq	0.7365	0.7652	0.7482
	Linear regression	0.5612	0.5423	0.5186
	Support vector regression	0.5834	0.5654	0.5521
${\Delta }_t$ = 48	LSTM seq2scalar	0.6825	0.7212	0.7237
	Non-stationary LS-deGCN seq2frame	0.7235	0.7866	0.7655
	Non-stationary LS-deGCN seq2seq	0.7655	0.8123	0.7785

compares the accuracy of air quality prediction among the five methods for combinations of $l=\{1,24,48\}$ and ${\Delta }_t=\{24,48\}$. The proposed non-stationary LS-deGCN seq2frame and seq2seq models yield much higher accuracy than the others, and the non-stationary LS-deGCN seq2seq model achieves the highest accuracy among all.

MAE.

Table 4. Comparison of the mean absolute error (MAE) among different methods based on the Chengdu testing dataset, where a smaller MAE indicates a better result.

	Models	$\mathit{l}=1$	$\mathit{l}=24$	$\mathit{l}=48$
	Linear regression	0.0175	0.0211	0.0234
	Support vector regression	0.0158	0.0147	0.0186
${\Delta }_t$ = 24	LSTM seq2scalar	0.0148	0.0167	0.0166
	Non-stationary LS-deGCN seq2frame	0.0092	0.0091	0.0101
	Non-stationary LS-deGCN seq2seq	0.0077	0.0089	0.0093
	Linear regression	0.0178	0.0198	0.0211
	Support vector regression	0.0153	0.0132	0.0201
${\Delta }_t$ = 48	LSTM seq2scalar	0.0136	0.0142	0.0154
	Non-stationary LS-deGCN seq2frame	0.0080	0.0079	0.0091
	Non-stationary LS-deGCN seq2seq	0.0071	0.0078	0.0083

shows the MAE of all the methods for the six scenarios with $l=\{1,24,48\}$ and ${\Delta }_t=\{24,48\}$. In those experiments, the two proposed models, the non-stationary LS-deGCN seq2frame and seq2seq models, report much lower MAE than the others, and the non-stationary LS-deGCN seq2seq model outperforms all the rest of the competitors.

For the three metrics, we also observe that the scenarios with ${\Delta }_t=48$ outperforming those with ${\Delta }_t=24$, especially for the LSTM-based methods (i.e., LSTM seq2scalar, non-stationary LS-deGCN seq2frame and seq2seq). This is expected as the prediction of air quality with the historical data of the last two days is a better strategy than that of only utilizing the data of yesterday. Apart from that, models that take into account spatial dependence (Non-stationary LS-deGCN seq2frame and Non-stationary LS-deGCN seq2seq) perform better than models that do not consider spatial dependence (Linear regression, Support vector regression, and LSTM seq2scalar), which demonstrates the necessity and effectiveness of considering spatial correlation.

For the results of RMSE and MAE in Table 2 and Table 4, we observe that the performances continue to decrease when the value of l increases. However, for the accuracy results in Table 3, we observe that the performance does not monotonically decrease with an increasing value of l for our models. In contrast, the accuracy achieves a peak at $l=24$ and the performance deteriorates for both $l=1$ and $l=48$. This indicates that the three evaluation metrics do not exactly match with each other. As discussed earlier, the larger the time lag l, the more difficult it is to make a prediction. However, if l is too small, the overlapped information under the non-stationary LS-deGCN seq2seq model would be large. The model may not fully capture the dynamic changes in the data, resulting in overly simple weight updates. At the same time, the error information may be concentrated on recent data, making the gradient update too drastic, which affects the model’s prediction performance.

5. Conclusions

In this paper, we train a non-stationary LS-deGCN to fit the spatial–temporal data, a multi-station air pollution dataset of Chengdu from 1 January 2013 to 31 December 2016. On one hand, the de-trending long short-term structure of LS-deGCN can capture the non-stationary temporal dependency between the predicted values and the historical records. On the other hand, the diffusion graph convolution on the station × feature grid gains more power to extract the spatial dependency among stations than a fully connected network. According to the scheme of extracting samples and targets, we propose two sub-models, the non-stationary LS-deGCN seq2frame and non-stationary LS-deGCN seq2seq. The target of the former model is a frame while the latter is a l-shifted sequence. We evaluate both models based on the RMSE, accuracy, and MAE and demonstrate their outstanding performances in comparison with the existing works. However, air quality prediction involves many other factors such as geographical information and traffic information. In future work, we will incorporate other data and integrate them into the framework of the non-stationary LS-deGCN.