The Development and Application of Machine Learning in Atmospheric Environment Studies

Abstract

Machine learning (ML) plays an important role in atmospheric environment prediction, having been widely applied in atmospheric science with significant progress in algorithms and hardware. In this paper, we present a brief overview of the development of ML models as well as their application to atmospheric environment studies. ML model performance is then compared based on the main air pollutants (i.e., PM_2.5, O₃, and NO₂) and model type. Moreover, we identify the key driving variables for ML models in predicting particulate matter (PM) pollutants by quantitative statistics. Additionally, a case study for wet nitrogen deposition estimation is carried out based on ML models. Finally, the prospects of ML for atmospheric prediction are discussed.

1. Introduction

The atmospheric environment is closely related to human health, as a high level of air pollutants can cause various diseases. For example, excessive inhalation of PM increases the risk of respiratory and heart disease [1], and lengthy exposure to O₃ has a detrimental effect on human lung function, leading to asthma as well as other serious cardiopulmonary diseases [2]. Therefore, the prediction of the atmospheric environment is essential for guiding both policy-making and personal daily outings. Atmospheric prediction methods can be classified into two main types: statistical models (including machine learning (ML) models and typical statistical models such as Land-Use Regression [3] and Geographically Weighted Regression (GWR) [4,5]), and numerical models (e.g., chemical transport models [6], box models, Lagrangian/Eulerian Models, Computational Fluid Dynamics (CFD) models and Gaussian models [7]). As an important part of statistical models, typical statistical models are designed for specific regression tasks related to geographic space using geo-statistical modeling, such as local geographic weighted calculation in GWR [4], and land-use features derived from the Geographic Information System (GIS) [3,5,8]. This kind of model is cost-effective and useful, but the major disadvantage is the limited nonlinear-fitting capability [9]. Another part with great application potential is machine learning (ML) models, which includes tree models, artificial neural networks, etc. Numerical models were popular and convincing in the past according to scientific or empirical deterministic equations based on atmospheric physical and chemical mechanisms. However, due to the limited understanding of complex physical and chemical mechanisms, the development of numerical models has been slow. In addition, the computational costs of numerical models are high, and pollution prediction results are often not available in a timely fashion. In recent years, with the rapid development of computational hardware and algorithms, machine learning (ML) has aroused widespread interest and started to be applied in academia and industry due to its powerful model-fitting capability, universality, denoising capability, and portability [10,11,12]. ML models combine the advantages of high computational efficiency and better nonlinear-fitting capability, making it a suitable complementary tool when the performance of numerical models is not satisfactory. In view of the current limited understanding of atmospheric physical and chemical mechanisms, ML models provide an effective alternatively way to simulate the atmospheric environment, especially for time-limited applications. Owing to the increasingly important prospect of ML applications in the atmospheric environment, we conducted this review.

As a branch of ML, deep learning has received special research attention. Before the 2010s, the main form of deep learning was artificial neural networks (ANNs) with shallow layers [13]. After AlexNet [14] won the ImageNet competition in 2012, researchers started to realize the importance of “deeper” neural networks, opening a new era of deep learning innovation. The detailed development of this area is introduced in Section 3.

Many aspects are involved in studies of the atmospheric environment: the sources and sinks of atmospheric pollutants [15,16], meteorological influences [17,18], physical transport [19,20], chemical formation and transformation [21,22], and so on. In the above research fields, numerical models are generally a suitable study approach, while statistical and ML models are applied mostly to air-pollutant prediction (e.g., PM_2.5, O₃, and NO₂). Specifically, ML models are widely applied in remote sensing studies, which can be summarized into three main types:

• Remote sensing data processes. The processes include data fusion and downscaling [23,24,25,26], missing-information filling and reconstruction [27,28], image dehazing [29] and despeckling [30], and data registration [23,24,31,32];
• Classical application using remote sensing data. The application includes image classification and segmentation (e.g., land-use and land-cover classification [33,34]), object detection (clouds [35], buildings [36], vehicles [37], landslides [38], trees [39], and so on), and change detection [40];
• Further application in the earth system. As a kind of universal approximation estimation algorithm, ML models have gained wide application in earth-system studies by using remote sensing data, such as atmospheric-pollutant prediction (including gas [41,42,43] and particulate matter pollutants [44,45,46,47]) or atmospheric-parameter retrieval and correction (e.g., Aerosol Optical Depth (AOD) retrieval and error correction [48], planetary boundary layer height estimation [49,50], aerosol chemical composition classification [51,52]), agricultural and forest prediction (e.g., yield prediction for different crops [53,54], forest habitats [55]), other parameter estimation or prediction in the earth system (e.g., land surface temperature (LST) [56,57], precipitation [58], soil moisture [59], evapotranspiration [60], biomass [61,62]), and so on.

In this review, we focus on ML model applications to air-pollution prediction. Therefore, we selected the prediction of atmospheric pollutants, especially studies using remote sensing data and atmospheric parameters directly related to atmospheric pollution, such as aerosol chemical composition classification. In addition, as an important sink of air pollutants, deposition is closely associated with air pollutants and meteorological conditions, as in the process of washout of particulate chemicals [63] and the dry deposition of aerosols by turbulent diffusion [64]. Considering that few studies have applied ML models to deposition, and many studies have been conducted on applications to atmospheric pollutants, a case study applying ML models to simulate nitrate wet deposition was carried out as an innovative point in this review.

The main objective of this paper is to:

• Introduce the development of ML models, especially for prediction;
• Review the application of ML models to atmospheric pollutants, including model classification, ML model performance, and identification of key variables;
• Conduct a case study that applies ML to deposition, in the hope of gaining further insight into the suitability of ML models for deposition estimation;
• Discuss the prospects of ML models for the study of atmospheric pollution.

2. Literature Search

We used Web of Science and Google Scholar for a literature search and set 2000–2020 as the search period. There were three steps in the collection of literature. The first search keywords included three parts: machine learning (deep learning, artificial intelligence), atmospheric pollution (air quality, air pollutant, air pollution), and prediction (estimation, forecast). Furthermore, a supplementary search was conducted using new keywords based on the previous search results. The new keywords included two parts: models (e.g., tree model, neural network) and pollutants (e.g., PM_2.5, O₃). Particularly, since the aerosol feature (detection, classification) is an important research field directly related to atmospheric pollution state, we used keywords including two parts: machine learning (deep learning, artificial intelligence) and aerosol classification (identification). Finally, 276 publications were collected after the three-step search process for the following statistics and analysis.

3. Overview of Machine Learning Development

ML models can be classified into several types depending on the task objectives, such as regression, classification, reinforcement learning [65], generative models [66], and so on. Since this review gave priority to atmospheric pollution prediction, we introduced the general development timeline of ML models mainly for models that can be used in regression prediction, particularly current popular models.

Regarding ML models available for regression prediction, all ML models in the collected research were classified into 4 categories: traditional convex optimization-based models (TCOB models), tree models, linear regression (LR), and modern deep-learning structure models (modern DL structure). The development timeline with selected milestones according to our classification is shown in Figure 1.

• Traditional convex optimization-based model

Two main model types are included in the TCOB model group: Support Vector Machine (SVM) and artificial neural networks (ANNs). The optimization algorithms of SVM and ANNs are based mostly on convex optimization (e.g., a stochastic gradient descent algorithm). Essentially, these two models add nonlinear data transformation based on a linear model. In addition, the methods of data transformation are different in SVM and ANNs: SVM transforms the data by means of kernel functions, while ANNs use activation functions.

The development of SVM can be divided into two stages, non-kernel SVM and kernel SVM [67,68], the latter of which is commonly applied today. The kernel function transforms input features from a low dimension to a higher dimension, simplifying the mathematical calculations in the higher-dimensional space. In practice, linear, polynomial, and Radial Basis Function (RBF) kernels are three commonly used model kernels. Kernel selection depends on the specific tasks and model performance.

Multiple Layer Perceptron (MLP), also called Back Propagation Neural Network (BPNN) [69], is the simplest neural network in this model group. MLP contains three types of layers inside: the input layer, the hidden layer, and the output layer. The input layer is a one-dimensional layer that passes independent variables organized into the network. The hidden layer receives data from the input layer and processes by a feedforward algorithm. All parameters (including the weight and bias between two adjacent layers) in the network are optimized by a backpropagation algorithm. In the training stage, the prediction result is passed to the output layer after each epoch, and network parameters are updated to better fit the predictions. In the validation or testing stage, the network parameters are frozen and make predictions directly.

After MLP was proposed, a lot of artificial neural networks (ANNs) were developed from the 1970s to the 2010s, such as the Radial Basis Function Network (RBFN) [70], ELMAN network [71], General Regression Neural Network (GRNN) [72], Nonlinear Autoregressive with Exogenous Inputs Model (NARX) [73], Extreme Learning Machine (ELM) [74], and Deep Belief Networks (DBN) [75]. One distinctive characteristic of these models is that they are relatively shallow due to the limited computing power when the models were proposed and their artificial design. For example, RBFN contains a Gaussian activation function inside the network, which is not a suitable design for a “deep” network. Furthermore, among ANNs, more layers in the network do not always mean improved prediction performance; sometimes, performance even deteriorates. Even so, ANNs are currently still effective tools for atmospheric pollution prediction due to the simplicity of model application and powerful model performance.

2.

Tree models

The development of tree models went through two stages: basic models and ensemble models. Basic models include ID3 [76], C4.5 [77], and CART [78]. The differences between them lie in the method of selecting features and the number of branches in the tree. We will not introduce the algorithms mathematically here, as they can readily be found. As a further development of basic tree models, ensemble tree models are key to the maturity of this group of ML models. There were two ensemble ideas in the history of development: bagging and boosting. The representative bagging model is the random forest (RF) [79], which develops n sub-models from the original input data and makes a prediction by voting. The two main ideas in boosting are changing the sample weight, and fitting the residual error according to the loss function during the training stage. AdaBoost [80] uses the former idea, whereas the Gradient Boosting Decision Tree (GBDT) [81], also called the Gradient Boosting Model (GBM), uses the other idea. For now, GBDT has been improved and developed into different models, such as XGBoost [82], LightGBM [83], and CatBoost [84], which have been widely used for classification as well as regression tasks.

3.

Linear regression

This group includes multiple regression (MLR), the Autoregressive Integrated Moving Average model (ARIMA), ridge regression [85], Least Absolute Shrinkage and Selection Operator (LASSO) [86], Elastic Net [87], and Generalized Additive Model (GAM) [88]. These models were originally designed to solve regression tasks. From the perspective of ML, ridge regression, LASSO, and Elastic Net are for the regularization of linear regression. ARIMA is a time-series function transforming unstable time series into stable series for model fitting; GAM as described here refers specifically to GAM for regression, where the target variable is the sum of a series of subfunctions. The function can be expressed as follows:

$$y={\displaystyle \sum }_{i=1}^n{f}_i$$

(1)

$f_i$ can be any function here.

As can be seen in Figure 1, LR has a long history of development. However, the innovation of model algorithms has stagnated since Elastic Net was proposed. One important reason for this is the limited nonlinear-fitting ability of this group.

4.

Modern deep-learning structure models

Modern DL structure models are another important part of deep learning that evolved from the development of ANNs, which are redesigned based on MLP considering the characteristics of the prediction tasks and input data. Modern DL structure models include mainly a convolutional neural network (CNN) [89] and a recurrent neural network (RNN) [90]. CNN contains a feature-capturing filter module called a “kernel” to catch local spatial features, thus making substantial connections between neighboring layers that are sparser compared to the dense connections inside MLP. This design makes optimization and convergence of the network easier. CNN has developed many network structures with innovative model design concepts, such as AlexNet (network goes “deeper”) [14], VGG (doubles the number of layers, half the height and width) [91], ResNet (skip connection) [92], and GoogLeNet (inception block) [93]. These networks can not only be applied directly to prediction tasks, but also provide modern ideas for future network design.

Compared to CNN, RNN is better for capturing temporal relationships in a time series. This group of models retains historical data in the “memory” unit and passes them into the network in the following training. The classical RNN simply passes history information from the last time step into the network along with input data in the current time step. However, this original “memory” unit design leads to a terrible problem: a vanishing gradient, which hinders the successful training of the model. Advanced RNN-based structures such as the long short-term memory network (LSTM) [90] and gated recurrent units (GRU) [94] significantly alleviate this problem with structure modification. These advanced RNNs are now more widely applied compared to the original RNN.

During the development of modern DL structure models, several improved model components were proposed, which efficiently improved the performance of both ANNs and modern DL structure models. For instance, a sigmoid activation function was replaced by the Rectified Linear Unit (ReLU) [95] or LeakyReLU [96] in most regression tasks; the dropout method [97] was usually applied in the model training stage to alleviate overfitting; Adam [98] and weight decay regularization [99] are commonly used in network optimization.

4. Machine Learning Application to Atmospheric Pollution

The analysis of the application of ML models to atmospheric pollution includes three parts:

• Analysis of the ML application trend by the annual number of publications, and the pollutants of concern;
• Comparison of ML model prediction performance;
• Design of a scoring system to explore key variables in ML models.

4.1. ML Application Trend

The annual trend in the number of publications applying ML models to atmospheric pollution from 2000 to 2020 was analyzed according to the literature collection rules. Due to the stable trend during 2000–2015, the number of studies every five years is presented for this period. After 2015, since the proportion of model applications and total number of publications changed significantly from year to year, we depict the total number and model contributions for every year. In addition, an analysis of the proportion of air-pollutant species based on the research collection has also been conducted, and these are shown together in Figure 2.

As presented in Figure 2, the number of papers on ML application to atmospheric pollutants stabilized at around 10 or fewer until 2016, with TCOB models as the main ML model type in this period. After 2017, the research count started to increase steeply, while the share of different ML models changed significantly at the same time. The proportion of tree models increased rapidly in 2017–2020, from 15.8% to 23.4%. Compared with tree models, the growth of modern DL structure models appeared later after 2019, contributing 17.2% in 2020. In addition, the proportion of TCOB models decreased to less than 50% (26.6–44.4% in 2018–2020) after 2017, implying that the development of ML application to air pollution began to be more diverse. Another obvious increasing model type was ensemble models, from 5.3% to 28.1% during 2017–2020. It is worth noting that the ensemble models mentioned here do not include bagging or boosting tree models, but rather refer to the aggregation of multiple ML model types by voting, stacking, or bagging. As for LR, this model group accounted for a small proportion during the whole study period.

For atmospheric species, the three most studied species were PM_2.5, PM₁₀, and O₃, contributing 34.0%, 19.0%, and 17.8%, respectively. Other popular predicted pollutants included NO₂, AQI, SO₂, and CO. It is evident that the common predicted species in this review are important indicators for air quality monitoring networks regardless of country. On the one hand, these indicators represent the general pollution level in the atmospheric environment. On the other hand, indicators in the monitoring network suggest that data availability and quality control are guaranteed compared to other data, which is important for ML modeling. Detailed annual species proportion is depicted in Figure S1. The proportion of PM_2.5 increased after 2015, then stabilized during 2016–2020 (33.3–50.0%). The general proportion of PM₁₀ declined, especially in recent years (from 20.0% to 6.9% in 2018–2020). O₃ had a decreasing trend during 2010–2019 (from 50.0% to 7.8%), but the contribution was elevated in 2020 (18.4%), indicating rising concerns about ozone. Moreover, NO₂ and AQI have increased slightly since 2017. In general, due to the increased amount of research, the diversity of air-pollutant studies has increased compared to five or ten years ago.

4.2. Model Performance

From Section 3, it is found that different kinds of ML models, such as TCOB models, tree models, and modern DL structure models, are widely applied at present. For atmospheric pollution modeling, model performance with different pollutants needs to be explored, so as to provide a reference and guidance for future air-pollution prediction research. For this purpose, we completed a statistical analysis of model-evaluation metrics from the copious publications collected in this review.

Various metrics were used in different studies, such as root mean square error (RMSE), correlation coefficient (CORR), mean square error (MSE), mean absolute percentage error (MAPE), index of agreement (IOA), normalized root mean square error (NRMSE), and so on. According to the metric availability calculated, two indicators were available for model performance analysis: CORR (63.9%) and RMSE (73.8%). CORR was selected as a statistical indicator for the following reason. In our study, evaluation indicators were collected from different research based on different datasets from different regions. Indeed, absolute metrics are not comparable between unfixed datasets. For example, RMSE of 10 μg/m³ is probably not a significant error in a dataset averaging 1000 μg/m³, while it would be significant in another dataset averaging 20 μg/m³. Therefore, CORR rather than RMSE was selected as the indicator of model performance in our study. Furthermore, since most studies used absolute error as the modeling loss function, there was no need to worry about the situation in which CORR is high and the ratio between prediction and observation deviates from 1. Most studies adopted 1 day or 1 h prediction horizon (50.7% and 37.9%, respectively). The prediction time step in all collected metrics was 1 step.

All collected studies were divided according to model type, and average CORR values were calculated for three main atmospheric pollutants: PM_2.5, PM₁₀, and O₃, as shown in Figure 3. Clearly, modern DL structure models had the highest CORR values for all main pollutants, with 0.94, 0.87, and 0.89 for PM_2.5, PM₁₀, and O₃, respectively. The performance of the TCOB models and tree models was similar, with slight advantages and disadvantages with different species. From a species perspective, PM_2.5 was the most successfully modeled species, and two other models provided good prediction performance in addition to modern DL structure models (tree models 0.91, and TCOB models 0.87). Furthermore, three model types showed good performance in modeling O₃ in addition to modern DL structure models (tree models 0.86, and TCOB models 0.82). For PM₁₀, modern DL structure models performed the best, followed by TCOB models and tree models with the same metrics (0.80). Overall, modern DL structure models showed strong modeling capability for atmospheric pollution prediction, while TCOB models and tree models performed at a similar relatively high prediction level. Moreover, LR failed to provide good performance, especially for PM₁₀ and O₃ (0.67 and 0.69, respectively).

4.3. Key Variable Identification

As with numerical models, various input variables related to prediction are required for ML modeling. In the atmospheric environment, various factors (e.g., meteorological conditions, pollution emissions) affecting pollutant generation, transport, chemical transformation, and deposition during the atmospheric lifetime are strongly associated with atmospheric pollution [6,100,101,102]. These factors are significantly effective in atmospheric pollution modeling. Essentially, ML models make a prediction by exploring the connection between input variables and target pollutants. In numerical models, this process is accomplished by deterministic equations. Unlike the artificial equation design in numerical models, the goal of ML models is to simulate the interrelationship between factors in the atmospheric environment by adjusting model internal parameters based on the provided datasets. This process is called “learning”. Several kinds of factors are used as input variables for air-pollutant modeling:

• Meteorological variables, e.g., temperature, relative humidity, pressure, wind speed, precipitation, and so on.
• Pollutant variables. The most common variables are pollutant data from observation sites. Observation data are usually set as prediction targets. Due to the relationship between different pollutants, observations can also be used as input data for predictive models. Another kind of pollutant variable is satellite data, such as Aerosol Optical Depth (AOD), Top of Atmosphere (TOA) reflectance, and so on.
• Auxiliary variables, including temporal variables (e.g., month of the year, day of the month, and mathematical transformation), spatial variables (e.g., longitude, latitude, and mathematical transformation), elevation, land cover, and social and economic data (e.g., GDP, nightlight brightness, road density).
• Historical data, specifically referring to time-series data before the time point to be predicted, or spatial data near the location to be predicted. In this case, the observation values become both input variables and output targets. Whether they are used as input variables or output targets depends on the predicted time point and the station location. The number of previous time steps depends on your datasets, model types and the characteristics of your tasks. For example, several studies indicated that time series at shorter lags (e.g., one or two lags) are better for ML modeling [103,104,105,106]. However, for some ML structures with powerful capability of temporal information extraction (e.g., LSTM, GRU), suitable longer lags were better for the model performance [107,108].

Due to the “learning” nature of ML models, variables described above are not always necessary for pollution modeling. In addition, the “learning” of ML is not intuitive for humans, which makes it less convincing [109]. Therefore, it is important to identify key variables for model prediction, whether for better understanding of the model or for gaining better model performance.

In our study, the key variables for ML models identified in previous research were collected. However, the driving variables for ML models varied in different studies. Accordingly, a scoring system was designed for variable importance to quantitively present the importance of input variables. The detailed scoring function is shown below:

$$I{S}^i={\displaystyle \sum }_{j=1}^N{a}_j^i\times r_j^i$$

(2)

$I{S}^i$ presents the importance score of variable $i$; $a_j^i$ means the number of papers that rank variable $i$ as the $j$th important factor; $r_j^i$ presents the scoring point of variable $i$. In this study, the top-three most important variables were considered and assigned different scoring points: $r_1^i=3$, $r_2^i=2$, $r_3^i=1$, respectively. Finally, all scoring points were summed together for each collected variable.

Researchers tend to select variables during their study regardless of whether they are shown in their studies. Therefore, a new indicator was counted, $V{C}^i$, the number of times that factors were used in all research, to denote the popularity of a variable in pollutant prediction.

Considering the limited number of studies, PM_2.5 and PM₁₀ were combined as PM pollutants and statistical analysis was then conducted on the variable importance between different ML models based on two indicators: $I{S}^i$ and $V{C}^i$. According to the ML model classification, all research results were divided into four model groups: TCOB models, tree models, LR, and modern DL structure models. As presented in Figure 4, variable importance varied from model to model. Since PM_2.5 and PM₁₀ were combined, both the PM component (PM_2.5 or PM₁₀) and “historical data” existed simultaneously in the results of the same model. PM_2.5 meant that the prediction target was PM₁₀, and vice versa. For tree models, AOD from satellite data was the most important variable, followed by history data, day of the year (DOY), and temperature (T). TCOB models were slightly different from tree models, with history data, PM₁₀ (for PM_2.5 target), and wind speed (WS) as the top-three variables. For LR, the significant variables included WS, PM₁₀ (for PM_2.5), and history data. For modern DL structure models, the most significant variable was history data. Overall, AOD data, the PM component (including history data and another component inside), and WS were the most important variables. In addition, we need to pay attention to some variables with low $V{C}^i$ but relatively high $I{S}^i$, such as DOY, NO₂, and NO in tree models, and traffic data in TCOB models. These variables are probably important to PM-pollutant prediction but have received little attention in previous studies. A full list of variable names is included in Table S1.

In our study, we noticed that remote sensing data played an important role in pollutant modelling. Since many satellite products published in recent years (e.g., Himawari 8/9 [110], Sentinel 5p [111], HY2B, and MetOp-C [112]), many studies did not utilize remote sensing data in their studies. In our collection, 75.0% of the studies applying satellite data for modeling were conducted since 2018. Besides, among the studies that have conducted the analysis of variable importance, 64.0% identified remote sensing data as the most important variables. As more satellite data are publicly released, these kind of data have great potential to improve the model performance.

5. Case Study: ML Application to Nitrate Wet Deposition Estimation

From the systematic review of atmospheric pollutants in Section 4, ML has been increasingly applied in the prediction or estimation of air pollutants, obtaining good performance, especially for PM and ozone. It is well known that pollution processes in the atmospheric environment are very complex, including air-pollutant generation, transport, chemical transformation, decomposition, and deposition. However, most studies focus on common atmospheric pollutants, such as PM, O₃, NO₂, SO₂, and CO. As an important sink of atmospheric pollutants, deposition pollution has seldom been predicted or estimated by applying ML models, and the common simulation method for deposition has been numerical models such as the global 3-D model from the Goddard Earth Observing System (GEOS), GEOS-Chem [113], and the chemical transport model developed at Meteorological Synthesizing Centre-West (MSC-W) from the European Monitoring and Evaluation Programme (EMEP), the EMEP MSC-W chemical transport model [114].

Therefore, in this section, several ML models were applied to estimate nitrate wet deposition in Guangdong province in China, aiming at seeking the applicability of ML models to deposition simulation. We selected one model in each model group classified in Section 3 as a representative model. Furthermore, we ran a numerical simulation case for comparison, which coupled the EMEP MSC-W chemical transport model with the Weather Research and Forecasting Model (WRF, v3.9.1) (WRF-EMEP) in the same period in Guangdong province. Additionally, due to the discontinuity of the time series in the deposition dataset, RNN was not considered in this case study. Finally, CNN, MLP, RF, MLR, and WRF-EMEP were selected for deposition modeling.

5.1. Study Area and Data

5.1.1. Study Area

Guangdong province lies in the south of China, with an area of 1.79 × 10⁵ km². There are 21 cities in this area, including Guangzhou, Shenzhen, Zhuhai, Shantou, and others. Annual precipitation varies from 1000 to 2000 mm under the influence of a subtropical monsoon climate. In our study, hourly ${\mathrm{NO}}_3^{-}$ wet precipitation measurements were collected from 25 sites from 2010 to 2017 in this region, with reliable quality control complying with the Collection and Preservation of the wet precipitation sample (GB/T 13580.2-1992) and the determination of fluoride, chloride, nitrite nitrate, sulphate in the wet precipitation—ion chromatography (GB/T 13580.5-1992). For modeling, monthly fluxes were calculated based on the following equations:

$$C_w={\sum }_{i=1}^n\left(C_i{M}_i\right)/{\sum }_{i=1}^n{M}_i,$$

(3)

$$D_w=M_t{C}_w/100,$$

(4)

where $C_w$ is the volume-weighted mean concentration of wet N concentration (mg N L⁻¹) in a customized study period (a month, a year, or other period); $M_i$ is the amount of precipitation, and; $C_i$ is the concentration. $D_w$ is wet N deposition flux (kg N ha⁻¹) calculated by the factors of $M_t$, the total amount of precipitation (mm) over a period.

Finally, eight years (2010–2017) of monthly fluxes from 25 sites were obtained in Guangdong province as prediction targets in the present work. The site location can be seen in Figure S2.

5.1.2. Data

Meteorological data were obtained from the China Meteorological Forcing Dataset (CMFD) [115] (http://data.tpdc.ac.cn/en/data/1980e33d-8615-448c-80e3-cfcb635fb110/, accessed on 27 July 2021). CMFD is a high spatial–temporal resolution gridded near-surface meteorological dataset developed in China (0.1° × 0.1°). We selected seven variables at a monthly temporal resolution from CMFD as the input feature, including 2 m air temperature (temp), surface pressure (sp), specific humidity (shum), 10 m wind speed (wind), downward shortwave radiation (srad), downward longwave radiation (lrad), and precipitation rate (prep).

For satellite data, tropospheric NO₂ column vertical density (VCD) from the Peking University Ozone Monitoring Instrument NO₂ product (POMINO) [116], a satellite product retrieved from OMI (http://www.pku-atmos-acm.org/~acm/acmProduct.php/#POMINO, accessed on 20 July 2021), was selected. Specifically, the latest POMINO version, POMINO v2, was used as the input dataset, with higher accuracy than the previous version. Since the dataset was retrieved with 0.25° × 0.25° longitude–latitude coordinates, we resampled VCD at 0.1° × 0.1° resolution as the input feature.

The NO_x emission inventory from the Multi-resolution Emission Inventory for China (MEIC) [117,118], with a spatial resolution of 0.25° × 0.25°, was downloaded from Tsinghua University (http://meicmodel.org/, accessed on 26 July 2021). Like VCD, the NO_x emission inventory was resampled at 0.1° × 0.1° resolution.

For auxiliary variables, the month of the year (MOY) was selected as a temporal variable, and longitude and latitude as spatial variables. Considering the temporal continuity of the temporal variable, the month variables were transformed into sine form by Equation (5). Specifically, for month j:

$$MO{Y}^j=\sin \left(\frac{2\pi j}{12}\right),$$

(5)

We also considered the influence of topography. Elevation data (elev) were obtained from the Shuttle Radar Topographic Mission (SRTM, version 4) produced by NASA (https://srtm.csi.cgiar.org/, accessed on 9 August 2021).

5.2. Model Design

Convolutional neural networks (CNN) are one of the most popular deep-learning structures in the deep-learning family and have been widely used in computer vision areas, including image classification, target detection, semantic segmentation, and so on [119]. The greatest advantage of CNN is the spatial feature extraction capability due to the constraint of weight sharing in the well-designed convolutional filter. In this study, a CNN was developed to estimate nitrate wet deposition. The structure of CNN is presented in Figure 5. The CNN model was developed on Pytorch 1.9.1, and the version of Python was 3.7.11. The data preprocess and analysis were mainly based on Numpy and Pandas libraries.

Meteorological variables (temp, shum, wind, sp, prep, srad, lrad), emission data (NO_x emission), auxiliary parameters (lon, lat, elev, MOY), satellite data (VCD), and zero padding were grouped and reshaped into a 7 × 2 size for each grid point in Guangdong province. Then, observation data at each site and the grouped grid data were paired according to the smallest Euclidean distance as samples to construct the whole dataset. The prediction target (label) was the observation data, the monthly nitrate wet flux (one dimension); 30% of the dataset was used for validation by random sampling, and the remaining 70% was used for training. For the hidden layer, we designed three convolutional layers with 1 × 1 convolutional kernels, with variables in each sample consisting of the different types described in Section 4.3. The convolutional filters were initialized by Kaiming Initialization [120]. The number of filters doubled with the deepening of the layers (8, 16, and 32). After three convolutional layers, there were two fully connected layers with 64 neurons for each layer, in the hope of better fitting the prediction. Since the prediction in our study was a regression task, mean square error loss was selected as the loss function. In addition, a batch-normalization layer was added before each convolutional layer to reduce Internal Covariate Shift [121]. The Rectified Linear Unit (ReLU) was selected as the activation function in our network. Additionally, the Adam algorithm [98] was selected as the optimization method during the training step.

5.3. Performance Comparison

According to model selection at the beginning of this section, four ML models and one numerical model (WRF-EMEP) were trained with the same dataset introduced in Section 5.1. The performance of all models is shown in Figure 6 and

Table 1. Quantitative metrics of predictive models.

Model	CORR	RMSE	MSE	MAE
CNN	0.68	0.61	0.38	0.37
RF	0.65	0.64	0.41	0.38
MLP	0.64	0.64	0.41	0.39
MLR	0.59	0.68	0.46	0.41
WRF-EMEP	0.20	0.93	0.87	0.55

. Generally, all ML models showed significant correlation between observations and estimation fluxes (p-value < 0.01). CNN performed best (CORR = 0.68, RMSE = 0.61) compared to the other ML models (CORR = 0.59–0.65, RMSE = 0.64–0.68). For the fitting degree of the validation dataset, RF tended to overestimate or underestimate to a relatively large extent at some points. For MLR, more points with high values tended to be underestimated, as can be seen in Figure 6d. The numerical model (WRF-EMEP) performed worst (CORR = 0.20, RMSE = 0.93) in this case, significantly underestimating the deposition flux in most samples of the validation dataset, especially for some samples with high observation deposition flux. Therefore, for the case in this study, ML models provided more reasonable simulation results than the selected numerical model.

From Figure 6 and Table 1, it seems that the advantage of CNN was not significant compared to the other ML models. However, the robustness of each model was totally different, which can be reflected in the spatial simulation. Figure S3 shows a spatial estimation by the 4 ML models (Figure S3b–e), observations (Figure S3a), and WRF-EMEP model (Figure S3f) in July 2014. The estimation made by tree models (RF) showed some anomalous patches with high values and failed to show reasonable spatial distribution. Obviously, the RF model overfitted the local site in the central Pearl River Delta (PRD) despite the validation during model training. In addition, MLP and MLR presented high values in the margins of Guangdong province, which were not reasonable compared to the spatial distribution of observations. As for the result of numerical model WRF-EMEP, the high values in the central PRD were significantly underestimated. The estimation by CNN well reconstructed the spatial distribution of nitrate wet flux in Guangdong province, with the deposition center in the western and northern PRD. From the above analysis, most ML models failed to estimate nitrate wet flux well when the site estimation was generalized to area mapping estimation, except CNN, which could capture the spatial pattern of simulated nitrate wet deposition.

5.4. Spatiotemporal Distribution

Based on the model performance results, we selected CNN as the final ML model for nitrate wet deposition estimation. The annual mean spatial distributions of observations and model estimation are presented in Figure 7a,b, respectively. The estimation mapping result is well consistent with the spatial distribution of observation values, with the deposition center in the western and northern PRD. This general spatial pattern is similar to that in previous studies using numerical models [122]. Furthermore, spatial error analysis (RMSE) is conducted in Figure 7c. The main RMSE was concentrated at several sites with high deposition values (located mainly in the PRD), whereas errors at other sites were small. Moreover, annual total wet flux between the model estimation and observations is compared in Figure 7d. Overall, the model-estimated flux was slightly higher than observations, with smaller differences when the annual observation flux was high (2010, 2015, and 2016).

6. Future Prospects

The idea in current research applying ML models to atmospheric prediction is to stay at the application level. Most research has simply used ML models as a “black box” predictor or added sophisticated designs as a data processor, e.g., variable selection or transformation [123,124,125,126,127], predicted target decomposition [128,129,130,131,132], and spatiotemporal information addition [107,133,134,135,136]. Another application method is the ensemble approach [137,138]. Few studies have improved the internal structure of predictive models according to specific atmospheric pollution problems. In the artificial intelligence field, many classical DL structure models have been proposed based on specific problems, such as target detection (faster R-CNN algorithm) [139] and semantic segmentation (FCN algorithm) [140]. When considering atmospheric pollution prediction, ML models also need “customization” for model structure design, rather than simply designing the pre-process for input data and predictive targets, or hyperparameter optimization modification inside ML models.

One “customization” idea is coupling with numerical models. Today, numerical models have been well developed [141] and have become mainstream in atmospheric pollutant prediction, especially for regional- or national-scale prediction. Physics and chemical constraints inside the numerical models reflect atmospheric laws. Coupling these constraints to ML models (e.g., in a regularization-like way) is an important idea for improvement in the future. In fact, similar efforts have begun recently, such as solving partial differential equations [142] and emulating pollutants [143,144].

The addition of physical and chemical characteristics of atmospheric pollution to constrain the model will also improve model interpretability (the extent to which a cause and effect can be observed within a model) or explainability (the extent to which results can be explained in human terms). Due to the characteristic of “learning” in ML models, their interpretability is far from that of numerical models. However, quite a few studies have ignored interpretability or explainability, or they have explained model results based simply on variable importance [145,146,147]. For now, the effort toward model interpretability or explainability is not enough. This will become a crucial issue in the future when ML models are more widely studied and applied. Model designers should consider model interpretability when designing future ML models.