The Functional Spatio-Temporal Statistical Model with Application to O₃ Pollution in Beijing, China

Abstract

In recent years, with rapid industrialization and massive energy consumption, ground-level ozone (${\mathrm{O}}_3$) has become one of the most severe air pollutants. In this paper, we propose a functional spatio-temporal statistical model to analyze air quality data. Firstly, since the pollutant data from the monitoring network usually have a strong spatial and temporal correlation, the spatio-temporal statistical model is a reasonable method to reveal spatial correlation structure and temporal dynamic mechanism in data. Secondly, effects from the covariates are introduced to explore the formation mechanism of ozone pollution. Thirdly, considering the obvious diurnal pattern of ozone data, we explore the diurnal cycle of ${\mathrm{O}}_3$ pollution using the functional data analysis approach. The spatio-temporal model shows great applicational potential by comparison with other models. With application to ${\mathrm{O}}_3$ pollution data of 36 stations in Beijing, China, we give explanations of the covariate effects on ozone pollution, such as other pollutants and meteorological variables, and meanwhile we discuss the diurnal cycle of ozone pollution.

1. Introduction

As one of the major pollutants, ground-level ozone (${\mathrm{O}}_3$) has received a lot of public attention. Lots of studies have shown that ${\mathrm{O}}_3$ could have detrimental effects on human health, including exacerbation of cardiovascular and respiratory dysfunction, and even premature mortality [1,2]. Additionally, tropospheric ozone, as a greenhouse gas, plays an important role in climate change, and further affects, for example, agricultural crop production [3,4]. In recent years, as the consequence of rapid industrialization and alarmingly increasing energy consumption, China has encountered severe air pollution [5,6,7,8]. Particularly, ozone becomes one of the serious and worsening pollutants in major areas of China, such as Beijing–Tianjin–Hebei urban agglomeration, and the Pearl River delta [9,10]. With a population of over 20 million, Beijing is one of the world’s largest mega cities. Due to coal burning, fugitive dust, and more recently a rapid increase in vehicular emissions, Beijing faces serious air pollution problems, and especially, studies regarding photochemical ozone pollution are attracting more and more attention [11,12].

The Chinese government identifies the urgency for air quality assessment and emission control, and has built a large monitoring network since 2013. Now, there are over 1500 national pollution monitoring stations in over 300 cities. Hourly readings of air pollutants are regularly recorded and directly transferred to China National Environmental Monitoring Center (CNEMC). The real-time observation and recording of the air pollution data provide a solid basis for studying the dynamic changes of pollutants and the underlying causes. Air quality data are collected over space and time; thus, the amount of data are large, and the analysis is complex. One important and common statistical characteristic of such data worthy of our notice is that the nearby (both in space and time) observations tend to be more alike than those far apart. Consequently, an assumption that spatio-temporal data follows the “independent and identically distributed” (iid) statistical paradigm should typically be avoided. Based on the underlying spatio-temporal structure of the pollution data, spatio-temporal statistical model, which simultaneously considers both the spatial covariance and temporal dependence, is thus a sensible and reasonable choice [13]. Moreover, ${\mathrm{O}}_3$ data show a clear diurnal cycle. It peaks during the day and reaches a minimum at night. Since ozone data are sampled at a high frequency in time, it provides an overview of the daily cycle of pollutant concentrations.

A spatio-temporal statistical model is powerful to reveal spatial correlation structure and temporal dynamic mechanism in data. Huang and Cressie (1996) [14] introduced a dynamic random field with a separable spatio-temporal covariance structure, which is widely used in the environmental field. When the spatio-temporal dependencies become complicated, the power of the hierarchical statistical modeling (HM), which is capable of decomposing an uncertainty source of data, becomes apparent. The HM’s strength is well discussed in Cressie et al. [15]. Moreover, the daily pattern of ozone pollution needs more exploration. To do this, we divided the collection time into two parts, one related to intra-day fluctuations and the other related to intra-day changes. Geographic space is defined by latitude and longitude, with the date being the third dimension, and the intra-day hour is regarded as the fourth dimension, which gives a four-dimensional representation of the data. In this way, the functional data analysis (FDA) approach [16] is used to model the intra-day variation of the measurement data, and the remaining dimensions are processed according to the classic spatio-temporal data modeling. To summarize, in addition to the dynamic random field and the hierarchical modeling, the third building block is based on the functional representation of daily profiles of atmospheric pollution through a functional data analysis approach, which is the main innovation of the method.

In the present study, we propose a functional spatio-temporal statistical model, which is also a two-level hierarchical spatio-temporal model. A fruitful approach is based on the representation of random functional objects as linear combinations of the basis functions with Gaussian random coefficients. This allows for representing a functional model as a random components model and inheriting the related inferential machinery, e.g., Wood [17]. Based on the Kalman filter and expectation–maximization (EM) algorithm, a model inference for parameter estimates is implemented [18,19]. In addition, from the marginal likelihood function, an information matrix is obtained to measure the uncertainty of the model parameters [20]. The proposed model has the following advantages: (i) the dynamic random field is used to describe the spatio-temporal characterization of emissions of air pollution; (ii) and covariate effects are incorporated to analyze the underlying formation mechanism of atmospheric pollutants; (iii) in addition, the main innovation is the introduction of the functional data analysis approach, which is performed to explore the daily pattern of pollutants. In the paper, we show the capability of the model by using ${\mathrm{O}}_3$ pollution data from 36 pollution monitoring stations in Beijing, China.

The paper is organized as follows. In Section 2, we describe the data in the study region, and introduce the Fourier basis functions, and the functional spatio-temporal statistical model, including the implementation of model estimation and cross-validation. In Section 3, we first show the selection of covariates and basis numbers. After comparing our model with others, we show the outstanding model capability, and finally give a comprehensive interpretation of the results. Conclusions are in Section 4.

2. Material and Methods

In this section, we first describe the data in the study region. Then, we introduce the Fourier basis function, and describe the functional spatio-temporal statistical model. In particular, model equations, model estimation, and cross-validation are discussed.

2.1. Data Description

The World Health Organization (WHO) set a guideline of 100 $\mathsf{\mu }$g/m${}^3$ for a maximum daily 8-h average exposure to ground-level ${\mathrm{O}}_3$; otherwise, adverse impacts on human health may occur [21]. Considering the increasing public concern on ozone, we attempt to analyze the effects from other pollutants and meteorological variables on ozone pollution, and provide some insight into the diurnal cycle of ${\mathrm{O}}_3$, which peaks in the mid-day and reaches minimum at night-time.

In this study, we collect hourly concentration of the ground-level ozone in spring, summer, and autumn of year 2017, from thirty-six pollution monitoring stations in Beijing, China, which are directly managed by the Ministry of Environment and Protection (MEP). We also collect four other pollutant gases—particulate matter (${\mathrm{PM}}_{10}$), sulfur dioxide (${\mathrm{SO}}_2$), nitrogen dioxide (${\mathrm{NO}}_2$), and carbon monoxide ($\mathrm{CO}$). All of the pollutant gases are measured in $\mathsf{\mu }$g/m${}^3$. The oxides of nitrogen (${\mathrm{NO}}_{\mathrm{x}}$) and the volatile organic components (VOC) constitute are known to be the important precursors of the ground ozone generation [22]. However, components of VOC are not measured by the air quality monitoring network.

We also collect meteorological data: barometric pressure ($PRES$, in hectopascal), air temperature ($TEMP$, in degree celsius), dew point temperature ($DEWP$, in degree celsius), integrated rainfall ($IRAIN$, in millimeter), and integrated wind speed ($Iws$, in meter per second) from nine weather stations of China Meteorological Administration (CMA). All the measurements are recorded hourly. We match between air quality stations and meteorological stations by the geodesic distance. Figure 1 displays the spatial locations of the air quality stations with red dots as well as the meteorological stations with blue triangles [23]. In addition to these meteorological variables, ultraviolet radiation is also a significant meteorological factor that influences ${\mathrm{O}}_3$ generation. Therefore, we download the data of UVB (in J/m${}^2$) with wavelengths between 200 and 440 nanometers from the European Centre for Medium-Range Weather Forecasts (ECMWF, https://cds.climate.copernicus.eu). The UVB data are provided at a grid size of $0.25°\times 0.25°$ at hourly frequency available over the study region. Since the UVB data vary greatly during the day and night, we take their log-transform before adding to the model. Note that the integrated rainfall and integrated wind speed are respectively calculated by:

$$\begin{array}{c}\hfill \begin{array}{c}\hfill IW{S}_t=\left\{\begin{array}{cc}W{S}_t,\hfill & W{D}_t!=W{D}_{t-1},\hfill \\ IW{S}_{t-1}+W{S}_t,\hfill & W{D}_t==W{D}_{t-1}.\hfill \end{array}\right.\end{array}\end{array}$$

(1)

$$\begin{array}{c}\hfill \begin{array}{c}\hfill IRAI{N}_t=\left\{\begin{array}{cc}RAI{N}_t,\hfill & RAI{N}_t=0,\hfill \\ IRAI{N}_{t-1}+RAI{N}_t,\hfill & RAI{N}_t!=0.\hfill \end{array}\right.\end{array}\end{array}$$

(2)

2.2. Fourier Basis

The basic philosophy of functional data analysis is to think of observed data functions as single entities, rather than merely as a sequence of individual observations. In practice, functional data are usually observed and recorded discretely as n pairs $(t_j,y_j)$, and $y_j$ is a snapshot of the function at time $t_j$, possibly blurred by measurement error. Time is so often the continuum over which functional data are recorded that we may slip into the habit of referring to $t_j$ as such, but certainly other continua may be involved, such as spatial position, frequency, weight, and so forth:

$$\begin{array}{c}\hfill y_j=x\left(t_j\right)+{ϵ}_j\end{array}$$

(3)

In functional data analysis, we need a strategy for constructing functions, which balances the model fitting and complexity. We built a set of functions where ${\varphi }_k,k=1,\dots ,K$ are called basis functions, and their linear combination is defined as a function:

$$\begin{array}{c}\hfill x\left(t\right)=\sum _{k=1}^K{c}_k{\varphi }_k\left(t\right)={\mathit{c}}^{\prime }\mathit{\varphi }\left(t\right),\end{array}$$

(4)

the expansion of the basis function, where the parameters $c_k,k=1,\dots ,K$ are the expansion coefficients to be estimated. In effect, basis expansion methods represent the potentially infinite dimensional world of functions within the finite-dimensional framework of vectors like $\mathit{c}$. The functional data analysis is simplified to multivariate data analysis.

The basis functions used for data modeling mostly belong to two categories: periodic and non-periodic. Most functional data analyses involve either a Fourier basis for periodic data, or a B-spline basis for non-periodic data. Since we are interested in the diurnal variations of ozone, we introduce the Fourier basis functions in detail. In order to express the repeated pattern in long-term sequences, basis functions need to be repeated within a certain time period T. The famous basis function extension for periodic data provided by the Fourier series is:

$$\begin{array}{c}\hfill \widehat{x}\left(t\right)=c_0+c_{1}sin\left(\omega t\right)+c_{2}cos\left(\omega t\right)+c_{3}sin\left(2\omega t\right)+c_{4}cos\left(2\omega t\right)+\dots \end{array}$$

(5)

where $\omega =2\pi /T$. Defining a Fourier basis system requires two pieces of information: the number of basis functions K and the period T. Figure 2 shows the Fourier basis system with $K=5$ and $T=1$. Followed by the constant, the Fourier basis functions are arranged in consecutive sine/cosine pairs:

We select the ozone data from one of the pollution stations—Wanliu Monitoring Station, which is located at Haidian District, Beijing, for preliminary analysis. The time span is one week from 21 May 2017 to 27 May 2017. We capture the daily variation of ozone data by using five Fourier basis functions. The mean square error (MSE) of fitted residuals is 14.79 $\mathsf{\mu }$g/m${}^3$. As shown in Figure 3, the predicted value at hour 24 matches the predicted value at hour 0 in the next day, guaranteeing the periodic nature of the daily cycle.

2.3. Model Equation

Let $\mathit{s}=(s_{lat},s_{lon})$ be the generic spatial location on the Earth’s sphere with sample size n, and $t=1,\dots ,T$ the day index, and domain $\mathcal{H}=\left[h_1,h_2\right]\subset \mathbb{R}$ the time within the day expressed in hours. The model for ozone observations $O_3(\mathit{s},t,h)$ is:

$$\begin{array}{cc}\hfill O_3(\mathit{s},t,h)& =\mathit{x}{(\mathit{s},t,h)}^{{}^{\prime }}\mathit{\beta }\left(h\right)+\mathit{\varphi }{\left(h\right)}^{\prime }\mathit{z}(\mathit{s},t)+\epsilon (\mathit{s},t,h),\end{array}$$

(6)

$$\begin{array}{cc}\hfill \mathit{z}(\mathit{s},t)& =\mathit{G}\mathit{z}(\mathit{s},t-1)+\mathit{\eta }(\mathit{s},t).\end{array}$$

(7)

This model is referred to as the functional dynamic spatio-temporal model. In Equation (6), $\epsilon$ is a zero-mean Gaussian measurement error independent in space and time with functional variance ${\sigma }_{\epsilon }^2\left(h\right)$, which implies that $\epsilon$ is heteroskedastic across the domain $\mathcal{H}$. The variance is modeled as

$$\begin{array}{c}\hfill log\left({\sigma }_{\epsilon }^2\left(h\right)\right)=\mathit{\varphi }{\left(h\right)}^{\prime }{\mathit{c}}_{\epsilon },\end{array}$$

(8)

where $\mathit{\varphi }\left(h\right)$ is a $p\times 1$ vector of basis functions evaluated at h while ${\mathit{c}}_{\epsilon }$ is a vector of coefficients to be estimated. In Equation (6), $\mathit{x}(\mathit{s},h,t)$ is a $b\times 1$ vector of covariates while $\mathit{\beta }\left(h\right)={({\beta }_1\left(h\right),\dots ,{\beta }_b\left(h\right))}^{\prime }$ is the vector of functional parameters modeled as

$$\begin{array}{c}\hfill {\beta }_j\left(h\right)=\mathit{\varphi }{\left(h\right)}^{\prime }{\mathit{c}}_{\beta ,j},\end{array}$$

(9)

and ${\mathit{c}}_{\beta }={\left({\mathit{c}}_{\beta ,1}^{\prime },\dots ,{\mathit{c}}_{\beta ,b}^{\prime }\right)}^{\prime }$ is the $pb\times 1$ vector of coefficients to be estimated. Additionally, $\mathit{z}(\mathit{s},t)$ is a $p\times 1$ latent space-time variable with Markovian dynamics given in Equation (7). Matrix $\mathit{G}$ is a diagonal transition matrix with diagonal elements in the $p\times 1$ vector $\mathit{g}$. The vector $\mathit{\eta }$ is described by a multivariate Gaussian process independent in time but correlated across space with matrix spatial covariance function given by

$$\begin{array}{c}\hfill \mathsf{\Gamma }(\mathit{s},{\mathit{s}}^{\prime };\mathit{\theta })=diag\left(v_1\rho (\mathit{s},{\mathit{s}}^{\prime };{\mathit{\theta }}_1),\dots ,v_p\rho (\mathit{s},{\mathit{s}}^{\prime };{\mathit{\theta }}_p)\right),\end{array}$$

(10)

and $\mathit{v}={\left(v_1,\dots ,v_p\right)}^{\prime }$ is the vector of scale coefficients while $\rho (\mathit{s},{\mathit{s}}^{\prime };{\mathit{\theta }}_j)$ is a valid spatial correlation function for locations $\mathit{s},{\mathit{s}}^{\prime }\in {\mathbb{S}}^2$ parametrized by ${\mathit{\theta }}_j$, and $\mathit{\theta }={({\mathit{\theta }}_1,\dots ,{\mathit{\theta }}_p)}^{\prime }$. The unknown model parameter vector is $\mathit{\psi }=\left({\mathit{c}}_{\epsilon }^{\prime },{\mathit{c}}_{\beta }^{\prime },{\mathit{g}}^{\prime },{\mathit{v}}^{\prime },{\mathit{\theta }}^{\prime }\right)$.

In Figure 4, we summarize the methodology. The main innovation is to incorporate the function data analysis approach to the classic spatio-temporal statistical model, which facilitates exploring the intra-day fluctuations of ozone pollution as well as the functional effects of covariates. Note that, in order to ease the notation, the same p-dimensional basis functions $\mathit{\varphi }\left(h\right)$ are used to model ${\sigma }_{\epsilon }^2$, ${\beta }_j$ and $\mathit{\varphi }{\left(h\right)}^{\prime }\mathit{z}(\mathit{s},t)$ in Equations (6) and (7). In the empirical analysis, we choose different numbers of basis functions for modeling according to the model criteria, such as the mean square error (MSE), and $R^2$ (see Section 2.5 for details).

2.4. Model Estimation

The estimation of $\mathit{\psi }$ and the latent space-time variable $\mathit{z}(\mathit{s},t)$ is based on the maximum likelihood approach and Kalman filter. At a specific location ${\mathit{s}}_i$ and time t, q measurements are taken at hour points $\mathit{h}={\left(1,2,\dots ,q\right)}^{\prime }$ and collected in the vector

$$\begin{array}{c}\hfill {\mathit{y}}_{{\mathit{s}}_i,t}={(O_3({\mathit{s}}_i,t,1),\dots ,O_3({\mathit{s}}_i,t,q))}^{\prime },\end{array}$$

(11)

where $q=24$ as pollutants are hourly recorded. Daily profiles of ozone data observed at time t across spatial locations $\mathcal{S}$ are then stored in the vector ${\mathit{y}}_t={({\mathit{y}}_{s_1,t}^{\prime },\dots ,{\mathit{y}}_{s_n,t}^{\prime })}^{\prime }$. Accordingly, Equations (6) and (7) are rewritten as

$$\begin{array}{cc}\hfill {\mathit{y}}_t& ={\tilde{\mathit{X}}}_t{\mathit{c}}_{\mathit{\beta }}+{\mathsf{\Phi }}_{\mathit{z},t}{\mathit{z}}_t+{\mathit{\epsilon }}_t,\end{array}$$

(12)

$$\begin{array}{cc}\hfill {\mathit{z}}_t& =\tilde{\mathit{G}}{\mathit{z}}_{t-1}+{\mathit{\eta }}_t,\end{array}$$

(13)

where ${\tilde{\mathit{X}}}_t={\mathit{X}}_t{\mathsf{\Phi }}_{\mathit{\beta },t}$ is a $nq\times bp$ matrix, with ${\mathit{X}}_t$ the matrix of covariates and ${\mathsf{\Phi }}_{\mathit{\beta },t}$ the basis matrix for $\mathit{\beta }$. ${\mathsf{\Phi }}_{\mathit{z},t}$ is the $nq\times np$ basis matrix for the latent $np\times 1$ vector ${\mathit{z}}_t={(\mathit{z}{({\mathit{s}}_1,t)}^{\prime },\dots ,\mathit{z}{({\mathit{s}}_n,t)}^{\prime })}^{\prime }$. ${\mathit{\eta }}_t={(\mathit{\eta }{({\mathit{s}}_1,t)}^{\prime },\dots ,\mathit{\eta }{({\mathit{s}}_n,t)}^{\prime })}^{\prime }$ is the $np\times 1$ innovation vector, while ${\mathit{\epsilon }}_t$ is the $nq\times 1$ vector of measurement errors. Additionally, $\tilde{\mathit{G}}={\mathit{G}\otimes \mathit{I}}_n$ is the $np\times np$ diagonal transition matrix.

The complete-data likelihood function $L(\mathit{\psi };\mathit{Y},\mathit{Z})$ can be written as

$$\begin{array}{c}\hfill L(\mathit{\psi };\mathit{Y},\mathit{Z})=L({\mathit{\psi }}_{{\mathit{z}}_0};{\mathit{z}}_0)\prod _{t=1}^{T}L({\mathit{\psi }}_{\mathit{y}};{\mathit{y}}_t|{\mathit{z}}_t)L({\mathit{\psi }}_{\mathit{z}};{\mathit{z}}_t|{\mathit{z}}_{t-1}),\end{array}$$

(14)

where $\mathit{Y}=\left({\mathit{y}}_1,\dots ,{\mathit{y}}_T\right)$, $\mathit{Z}=\left({\mathit{z}}_0,{\mathit{z}}_1,\dots ,{\mathit{z}}_T\right)$, ${\mathit{\psi }}_{\mathit{z}}=\left({\mathit{g}}^{\prime },{\mathit{v}}^{\prime },{\mathit{\theta }}^{\prime }\right)$, ${\mathit{\psi }}_{\mathit{y}}=\left({\mathit{c}}_{\epsilon }^{\prime },{\mathit{c}}_{\beta }^{\prime }\right)$, and ${\mathit{z}}_0$ is the Gaussian initial vector with parameter ${\mathit{\psi }}_{{\mathit{z}}_0}$. The model parameter set $\mathit{\psi }$ is initialized with starting values ${\mathit{\psi }}^{〈0〉}$ and then updated at each iteration $\iota$ of the EM algorithm. The algorithm terminates if any of the conditions is satisfied

$$\begin{array}{c}\hfill ∥{\mathit{\psi }}^{〈\iota 〉}-{\mathit{\psi }}^{〈\iota -1〉}∥/∥{\mathit{\psi }}^{〈\iota 〉}∥<ϵ,\end{array}$$

(15)

$$\begin{array}{c}\hfill \left|L({\mathit{\psi }}^{〈\iota 〉};\mathit{Y})-L({\mathit{\psi }}^{〈\iota -1〉};\mathit{Y})\right|/\left|L({\mathit{\psi }}^{〈\iota 〉};\mathit{Y})\right|<ϵ,\end{array}$$

(16)

where $∥∥$ is the $l_2$ norm, ${\mathit{\psi }}^{〈\iota 〉}$ is the parameter set at the $\iota -th$ iteration, $L({\mathit{\psi }}^{〈\iota 〉};\mathit{Y})$ is the observed-data likelihood function evaluated at ${\mathit{\psi }}^{〈\iota 〉}$, and $ϵ$ is a small positive number (e.g., ${10}^{-3}$).

The EM algorithm provides a point estimate of the parameter vector $\mathit{\psi }$ but without uncertainty information. Note that $\mathit{Y}$ is a vector with dimension $N=nqT$. Generally speaking, inverting the full variance–covariance matrix of the N-dimensional data vector $\mathit{Y}$ has a computational complexity in the order of $O\left(N^3\right)$, which is clearly unfeasible. Thanks to the state space representation of model, we estimate the variance–covariance matrix ${\widehat{\mathsf{\Sigma }}}_{\mathit{\psi }}=\mathbb{V}\left(\mathit{\psi }\mid \mathit{Y}\right)$ from the marginal likelihood, which may be used for model selection and inference.

2.5. Cross-Validation

We implement a 2-fold cross-validation by partitioning the original spatial locations $\mathcal{S}$ into subsets ${\mathcal{S}}_{est}$ and ${\mathcal{S}}_{xval}$. Data related to ${\mathcal{S}}_{est}$ are used for model estimation while data related to ${\mathcal{S}}_{xval}$ are used for cross-validation. The cross-validation mean squared errors are then computed by

$$\begin{array}{c}\hfill MS{E}_{\mathit{s}}=\frac{1}{B}\sum _{t=1}^T\sum _{h\in {\mathit{h}}_{\mathit{s},t}}{\left(O_3\left(\mathit{s},h,t\right)-\widehat{O_3}\left(\mathit{s},h,t\right)\right)}^2,\end{array}$$

(17)

where $\widehat{O_3}\left(\mathit{s},h,t\right)=E_{\widehat{\varphi }}\left(O_3\left(\mathit{s},h,t\right)\right|\mathit{Y})$ is the prediction of ozone data at the cross-validation stations, and B is the number of terms in each sum. We also obtain the cross-validation $R^2$ with respect to station $\mathit{s}$:

$$\begin{array}{cc}\hfill R_s^2& =1-\frac{MS{E}_s}{VAR\left(\{O_3\left(\mathit{s},h,t\right),t,h\}\right)}\end{array}$$

(18)

The choice of the numbers of basis functions is very essential for model estimation. Here, based on the cross-validated mean square error and other model criteria, we choose the reasonable numbers of basis functions to estimate ${\sigma }_{\epsilon }^2$, ${\beta }_j$ and $\mathit{\varphi }{\left(h\right)}^{\prime }\mathit{z}(\mathit{s},t)$ respectively. After implementing leave-one-station-out cross-validation, we take the average $\overline{MSE}$ and $\overline{R^2}$ as our criteria:

$$\begin{array}{c}\hfill \overline{MSE}=\frac{1}{n}\sum _{i=1}^{n}MS{E}_{{\mathit{s}}_i},\end{array}$$

(19)

$$\begin{array}{c}\hfill \overline{R^2}=\frac{1}{n}\sum _{i=1}^n{R}_{{\mathit{s}}_i}^2.\end{array}$$

(20)

3. Analysis of ${\mathrm{O}}_3$ Pollution in Beijing

In the paragraph, we first show the selection of covariates and basis numbers with application to ozone data in Beijing, and then focus on the summertime modeling. By comparing our proposed model with other models, we show the outstanding advantage of the functional spatio-temporal statistical model. Finally, we show the model results and interpret the parameter estimates, especially the functional effects of covariates.

3.1. Selection of Covariates and Basis Numbers

In the following text, we select the covariates in $\mathit{x}(\mathit{s},t,h)$ by using the Akaike information criterion (AIC).

Table 1. The selection of model covariates according to AIC.

	Iteration
Season	Iter 0	Iter 1	Iter 2	Iter 3	Iter 4	Iter 5	Iter 6	Iter 7	Iter 8	Iter 9
Spring	NO2	SO2	PM10	CO	PRES	UVB	Iws	IRAIN	TEMP	DEWP
Summer	NO2	PM10	SO2	TEMP	IRAIN	UVB	DEWP	PRES	CO	Iws
Autumn	NO2	SO2	TEMP	DEWP	PM10	UVB	CO	Iws	PRES	IRAIN

displays the results of forward selection based on AIC, which means starting with no covariates, and iteratively adding the most contributive covariates. For instance, in the summertime modeling, at the beginning (Iter 0), we select the variable ${\mathrm{NO}}_2$, which results in the best model performance with maximum AIC. Then, at the next iteration (Iter 1), the variable particulate matter (${\mathrm{PM}}_{10}$) is further selected. Table 1 shows that the importance of covariates varies among seasons, but the most important variables are ${\mathrm{SO}}_2$, ${\mathrm{NO}}_2$, and ${\mathrm{PM}}_{10}$.

The ozone concentrations display a significant seasonal pattern, being pretty high in summer, while meanwhile being moderate in winter [24,25]. Therefore, we focus on the analysis of ${\mathrm{O}}_3$ pollution in summer. Figure 5 shows the maximum AIC at each iteration for summertime modeling.

The improvement of model AIC is no longer significant after five iterations; therefore, we find the optimal subset of covariates—$N{O}_2$, $P{M}_{10}$, $S{O}_2$, $TEMP$, $IRAIN$, and $UVB$. Hence, the measurement equation for ozone data is

$$\begin{array}{ccc}\hfill O_3(\mathit{s},h,t)& =& {\beta }_0\left(h\right)+x_{NO2}(\mathit{s},h,t){\beta }_{NO2}\left(h\right)+x_{P{M}_{10}}(\mathit{s},h,t){\beta }_{P{M}_{10}}\left(h\right)\hfill \\ \hfill & & +x_{S{O}_2}(\mathit{s},h,t){\beta }_{S{O}_2}\left(h\right)+x_{TEMP}(\mathit{s},h,t){\beta }_{TEMP}\left(h\right)\hfill \\ \hfill & & +x_{IRAIN}(\mathit{s},h,t){\beta }_{IRAIN}\left(h\right)+x_{UVB}(\mathit{s},h,t){\beta }_{UVB}\left(h\right)\hfill \\ \hfill & & +\mathit{\varphi }{\left(h\right)}^{\prime }\mathit{z}(\mathit{s},t)+\epsilon (\mathit{s},h,t),\hfill \end{array}$$

(21)

where data are available at $h=1,\dots ,24$, $s\in \{s_1,\dots ,s_{36}\}$, and $t=1,\dots ,92$. Moreover, due to the circularity of time, Fourier basis functions are adopted. This implies that ${\beta }_j\left(h\right)$, ${\sigma }_{\epsilon }^2\left(h\right)$ are periodic functions. Under the different combinations of the numbers of basis functions, the model criteria $\overline{MSE}$ and $\overline{R^2}$ in Section 2.5 are obtained and shown in

Table 2. Criteria $\overline{MSE}$, $\overline{R^2}$, and $AIC$ under different numbers of Fourier basis.

$\mathit{\varphi }{\left(\mathit{h}\right)}^{\prime }\mathit{z}(\mathit{s},\mathit{t})$	$\mathit{\beta }\left(\mathit{h}\right)$	${\mathit{\sigma }}_{\mathit{\epsilon }}^2$	$\overline{\mathit{MSE}}$	$\overline{{\mathit{R}}^2}$	$\mathit{AIC}$
5	3	3	357.58	0.9206	−255,385
5	3	5	356.32	0.9209	−254,607
5	3	7	356.47	0.9208	−254,599
5	5	3	352.61	0.9215	−254,235
5	5	5	352.25	0.9216	−253,459
5	5	7	352.39	0.9215	−253,454
5	7	3	352.14	0.9215	−254,116
5	7	5	351.62	0.9217	−253,309
5	7	7	351.76	0.9217	−253,306
7	3	3	332.95	0.9259	−249,514
7	3	5	331.88	0.9261	−248,723
7	3	7	331.99	0.9261	−248,713
7	5	3	330.06	0.9264	−248,848
7	5	5	329.80	0.9264	−248,066
7	5	7	329.90	0.9264	−248,062
7	7	3	329.09	0.9266	−248,656
7	7	5	328.97	0.9266	−247,879
7	7	7	329.05	0.9266	−247,876
9	3	3	324.08	0.9278	−246,673
9	3	5	323.13	0.9280	−245,937
9	3	7	323.19	0.9280	−245,928
9	5	3	322.07	0.9281	−246,056
9	5	5	321.80	0.9282	−245,327
9	5	7	321.86	0.9282	−245,322
9	7	3	321.28	0.9283	−245,879
9	7	5	321.12	0.9283	−245,152
9	7	7	321.13	0.9283	−245,150

.

From the table, when the number of basis functions for estimating $\mathit{\varphi }{\left(h\right)}^{\prime }\mathit{z}(\mathit{s},t)$ increases, it significantly reduces the $\overline{MSE}$. When the number of basis functions increases from 5 to 7, the $\overline{MSE}$ is reduced more than that from 7 to 9. Considering such enormous calculation stress, we choose seven basis functions to estimate the latent component $\mathit{\varphi }{\left(h\right)}^{\prime }\mathit{z}(\mathit{s},t)$. However, increasing the number of basis functions for the variance ${\sigma }_{\epsilon }^2\left(h\right)$ of the residual $\epsilon (\mathit{s},h,t)$ does not significantly reduce $\overline{MSE}$ but is helpful to improve the AIC. We find that an increase from 3 to 5 has an improvement in AIC, but the improvement becomes very minor from 5 to 7. Thus, we choose five basis functions to estimate the variance ${\sigma }_{\epsilon }^2\left(h\right)$. Finally, we choose five basis functions to estimate the effects from covariates ${\beta }_j\left(h\right)$, considering the trade-off between the model interpretation and over-fitting problem. Based on the analysis above, the number of basis functions for ${\beta }_j\left(h\right)$, ${\sigma }_{\epsilon }^2\left(h\right)$ and $\mathit{\varphi }{\left(h\right)}^{\prime }\mathit{z}(\mathit{s},t)$ is chosen to be 5, 5, and 7, respectively.

3.2. Model Comparison

In the paragraph, we compare the five models, namely Equations (22), (23), (24), (25), and (26). Equation (22) is an ordinary regression model; Equation (23) is a regression model with functional $\beta \left(h\right)$ estimates; Equation (24) introduces the latent spatio-temporal variable $\mathit{z}(\mathit{s},t)$ to characterize the spatio-temporal correlation; Equation (25) is a simplified version of the proposed functional spatio-temporal statistical model that is $\beta \left(h\right)\equiv \beta$, ${\sigma }_{\epsilon }^2\left(h\right)\equiv {\sigma }_{\epsilon }^2$; Equation (26) is the functional spatio-temporal statistical model:

$$\begin{array}{cc}\hfill O_3& =X\beta +ϵ\end{array}$$

(22)

$$\begin{array}{cc}\hfill O_3\left(h\right)& =X\left(h\right)\beta \left(h\right)+ϵ\left(h\right)\end{array}$$

(23)

$$\begin{array}{cc}\hfill O_3(s,t)& =X(s,t)\beta +z(s,t)+ϵ(s,t)\hfill \\ \hfill z(s,t)& =Gz(s,t-1)+\eta (s,t)\hfill \end{array}$$

(24)

$$\begin{array}{cc}\hfill O_3(s,h,t)& =X(s,h,t)\beta +\mathit{\varphi }{\left(h\right)}^{\prime }\mathit{z}(\mathit{s},t)+ϵ(s,h,t)\hfill \\ \hfill z(s,t)& =Gz(s,t-1)+\eta (s,t)\hfill \end{array}$$

(25)

$$\begin{array}{cc}\hfill O_3(s,h,t)& =X(s,h,t)\beta \left(h\right)+\mathit{\varphi }{\left(h\right)}^{\prime }\mathit{z}(\mathit{s},t)+ϵ(s,h,t)\hfill \\ \hfill z(s,t)& =Gz(s,t-1)+\eta (s,t)\hfill \end{array}$$

(26)

Similar to the selection of the numbers of basis functions, the average $\overline{MSE}$ and $\overline{R^2}$, and AIC are used to assess the model performance. As shown in

Table 3. $\overline{MSE}$, $\overline{R^2}$, and AIC for the five models.

	Number of Basis			Model Criteria
	$\mathit{\beta }$	$\mathit{\varphi }{\left(\mathit{h}\right)}^{\prime }\mathit{z}(\mathit{s},\mathit{t})$	${\mathit{\sigma }}_{\mathit{ϵ}}$	$\overline{\mathit{MSE}}$	$\overline{{\mathit{R}}^{\mathit{2}}}$	AIC	logL 1	Npar 2
Equation (22)	0	0	0	1880.54	0.5863	−414,714	−414,700	7
Equation (23)	5	0	0	1171.55	0.7423	−395,874	−395,812	31
Equation (24)	0	0	0	552.7	0.879	−256,960	−256,940	10
Equation (25)	0	7	0	336.88	0.925	−252,426	−252,370	28
Equation (26)	5	7	0	329.8	0.9264	−248,066	−247,954	56

¹ log likelihood, ² number of parameters.

, our model Equation (26) is the optimal among the five models in view of the three model criteria. Equation (23) is much improved from Equation (22) in terms of $\overline{MSE}$ and $\overline{R^2}$, which means a better model forecast in general. Benefiting from the latent spatio-temporal variable $\mathit{z}(\mathit{s},t)$, Equation (24) has an unbeatable advantage over the ordinary regression models, accessing much smaller $\overline{MSE}$ and much larger $\overline{R^2}$ and AIC. Equation (25) introduces the functional data analysis approach, and characterizes the latent component as a linear combination of the basis functions and the latent random spatio-temporal variable $\mathit{z}(\mathit{s},t)$. Although the AIC is only a little increased, a smaller $\overline{MSE}$ and larger $\overline{R^2}$ are achieved. Eventually, when Equation (26) adds the functional covariate effects $\beta \left(h\right)$ and the functional variance of the residuals ${\sigma }_{ϵ}\left(h\right)$, $\overline{MSE}$, and $\overline{R^2}$ is not improved much. However, AIC is further improved, which benefits from the more capable interpretation of covariates and the flexibility of the residual variance.

In Equation (26), firstly, the latent hidden variable $\mathit{z}(\mathit{s},t)$ captures the spatial correlation by range parameter $\theta$, and variance parameter v, which shows that an average standard deviation of 48 $\mathsf{\mu }$g/m${}^3$ of ozone data are explained by $\mathit{z}(\mathit{s},t)$ (refer to Table 5). Secondly, the functional $\beta \left(h\right)$ shows that the covariate effects are both significant and nonlinear, indicating the complicated formation of ozone pollution by using the functional representation (refer to Figure 6). In summary, the hierarchical spatio-temporal statistical model, combined with functional data analysis approach, contributes to the high amount of $\overline{R^2}$.

3.3. Model Result

Figure 6 shows the estimated $\mathit{\beta }\left(h\right)$ and ${\sigma }_{ϵ}^2\left(h\right)$ for model Equation (21). Thanks to Fourier basis functions, the estimation result at the end of the day matches the beginning of the next day.Since, in general, the confidence bands of estimated $\mathit{\beta }\left(h\right)$ may contain zero, it may be useful to test the significance of covariates. The ${\chi }^2$ tests are introduced as follows:

$$\begin{array}{c}\hfill {\beta }_j\left(h\right)=\mathit{\varphi }{\left(h\right)}^{\prime }{\mathit{c}}_{\beta ,j},{\mathit{c}}_{\beta ,j}\sim N(0,{\mathsf{\Sigma }}_{\widehat{{\mathit{c}}_{\beta ,j}}}).\end{array}$$

(27)

Thus, ${\mathit{c}}_{\beta ,j}^{\prime }{\mathsf{\Sigma }}_{\widehat{{\mathit{c}}_{\beta ,j}}}^{-1}{\mathit{c}}_{\beta ,j}\sim {\chi }^2\left(rank\left({\mathsf{\Sigma }}_{\widehat{{\mathit{c}}_{\beta ,j}}}\right)\right)$. In Figure 6, $IRAIN$ fluctuates around zero. The results of ${\chi }^2$ tests for the significance of covariates are reported in

Table 4. ${\chi }^2$ tests for significance of fixed effects.

Covariate	${\mathit{\chi }}^2$ Statistic	p-Value
Cons	282.77	0
$P{M}_{10}$	2114.06	0
$S{O}_2$	1048.50	0
$N{O}_2$	29,032.23	0
$TEMP$	5554.16	0
$IRAIN$	0.91	0.96
$UVB$	30,934	0

, and indicate that the effect of variable $IRAIN$ is not jointly significant.

Therefore, it comes to the final model equation by excluding the $IRAIN$ variable:

$$\begin{array}{ccc}\hfill O_3(\mathit{s},h,t)& =& {\beta }_0\left(h\right)+x_{NO2}(\mathit{s},h,t){\beta }_{NO2}\left(h\right)+x_{P{M}_{10}}(\mathit{s},h,t){\beta }_{P{M}_{10}}\left(h\right)\hfill \\ \hfill & & +x_{S{O}_2}(\mathit{s},h,t){\beta }_{S{O}_2}\left(h\right)+x_{TEMP}(\mathit{s},h,t){\beta }_{TEMP}\left(h\right)\hfill \\ \hfill & & +x_{UVB}(\mathit{s},h,t){\beta }_{UVB}\left(h\right)+\mathit{\varphi }{\left(h\right)}^{\prime }\mathit{z}(\mathit{s},t)+\epsilon (\mathit{s},h,t).\hfill \end{array}$$

(28)

In

Table 5. Estimates and standard error of parameter $\mathit{g},\mathit{\theta }$, and $\mathit{v}$.

	Transition g		$\mathit{\theta }$ [km]		Variance v
	Est	Std.err	Est	Std.err	Est	Std.err
Basis 1	0.739	0.018	63.12	4.57	8422.14	549.12
Basis 2	0.229	0.026	50.94	0.96	3799.47	176.33
Basis 3	0.179	0.03	36.98	1.02	2027.63	106.59
Basis 4	0.034	0.032	36.34	0.54	896.86	50.61
Basis 5	0.106	0.034	39.75	0.84	702.64	41.65
Basis 6	0.043	0.043	31.92	0.87	191.09	13.53
Basis 7	−0.210	0.042	37.10	0.35	151.80	10.78

, we show the estimates and standard deviation of parameters relevant to the latent spatio-temporal variable $\mathbf{z}(s,t)$, which are the transition coefficient $\mathbf{g}$, range parameter $\mathit{\theta }$, and variance vector $\mathbf{v}$. Most estimates of $\mathbf{g}$ parameter are positive, and the absolute values are all within one, which guarantees the stability of the 7-variate spatio-temporal process $\mathbf{z}(s,t)$. Compared with the geodesic distance of Beijing (around 50 km), the values of $\mathit{\theta }$ parameter, ranging from $31.92$ km to $63.12$ km, indicate a strong spatial correlation within the city. The average $\mathbf{v}$ estimate is around 2313 (with standard deviation of 48 $\mathsf{\mu }$g/m${}^3$), and shows that the latent variable $\mathbf{z}(s,t)$ accounts for much more proportion of original $O_3$ variance than the unexplained term ${\sigma }_{ϵ}^2\left(h\right)$. Hence, introducing the latent spatio-temporal variable $\mathbf{z}(s,t)$ guarantees the advantage of the proposed model.

Finally, in Figure 7, we show the estimated $\mathit{\beta }\left(h\right)$ and ${\sigma }_{ϵ}^2\left(h\right)$. The last figure is the plot of functional variance ${\sigma }_{ϵ}^2\left(h\right)$, which represents the unexplained portion of $O_3$ variance. The plot shows that the model is more capable when explaining the situation during the daytime [26].

As shown in Figure 7, the coefficient curves of $TEMP$, $uvb$ and $P{M}_{10}$ are similar, increasing from early morning and attending the peak at 12:00 p.m.–2:00 p.m., then falling down. Focusing on daytime, we see that the three curves are consistent with the trend of temperature (or uvb), which implicates that the relationship between ozone and temperature (or uvb) might be quadratic [27], or there were interactions between temperature and uvb, that is, the ozone concentrations were dependent on $TEM{P}^2$, $uv{b}^2$, or $TEMP\times uvb$. The coefficients of $TEMP$ and $uvb$ in daytime are positive, which is consistent with the present research [28]. While the coefficient of $P{M}_{10}$ is negative at 5:00 a.m.–10:00 a.m. and positive during other time periods. The positive correlation between $P{M}_{10}$ and ozone may be caused by their common sources, secondary nature, and interactions of their precursors [29], and the negative correlation could be explained by PM’s consumption of hydroperoxy (HO2) radicals, which would otherwise react with $NO$ for ozone generation [30]. Furthermore, the positive correlation becomes the strongest at 3:00 p.m., at which time the ozone concentration attains the largest.

In addition, the coefficient curves of $N{O}_2$ and $S{O}_2$ both have two spikes, while the coefficient of $N{O}_2$ is negative and the other is positive. The negative relationship between $N{O}_2$ and ozone is consistent with results in many studies [31,32], and the positive correlation between $S{O}_2$ and ozone could be explained by their common dependences on meteorology [33]. The strongest correlation between $N{O}_2$ and ozone in daytime appears at about 11:00 a.m., and the weakest correlation appears at 5:00 a.m. and 6:00 p.m. In contrast, the correlation between $S{O}_2$ and ozone is the strongest at 9:00 a.m. and 8:00 p.m., in other words, approximately the end of morning/evening rush hours in Beijing, respectively, and such correlation is the weakest at 3:00 p.m.

4. Conclusions

In this paper, we propose a functional spatio-temporal statistical method to analyze air quality data, and explore the mechanism of pollution formation.

• The method has several advantages. First, as a hierarchical spatio-temporal statistical model, it is flexible enough to handle latent variable while capturing spatio-temporal dynamics. Second, the proposed model also takes covariates into consideration, thereby being efficient in discovering relational patterns from chemical reaction, and meteorological factors on the formation of ${\mathrm{O}}_3$ pollution. Third, in the framework of the spatio-temporal models, we are the first to explore the intra-day variation of ozone through the functional data analysis approach, which is the most innovative part of the model.
• The model has made the following progresses. First of all, our model outperforms other models in many ways, as shown in Section 3.2. Second, the latent spatio-temporal variable $\mathbf{z}(s,t)$ well captures the temporal dynamic and spatial structure of ozone data. Third, from the functional effects of the covariates, we explore the possible effects of air pollutants and meteorological variables on ozone data.
• Our model is flexible enough to model any kind of data with spatio-temporal structure; therefore, it can be applied in many fields, such as economy and agriculture, apart from the environment. The introduction of the functional data analysis approach in the functional spatio-temporal model is not restricted to model the daily pattern of the data, and provides us more capability to explore the nature of the data of our interest.