Modelling the Daily Concentration of Airborne Particles Using 1D Convolutional Neural Networks ^†

Abstract

This paper focuses on improving the prediction of the daily concentration of the pollutants, PM₁₀ and nitrogen oxides (NO, NO₂) in the air at urban monitoring sites using 1D convolutional neural networks (CNN). The results show that the 1D CNN model outperforms the other machine learning models (LSTM and Random Forest) in terms of the coefficients of determination and absolute errors.

1. Introduction

Air pollution is a major environmental problem with significant impacts on human health and the planet’s ecosystem. Air pollutants can be of natural or anthropogenic origin. One of the most commonly assessed air pollutants is particulate matter (PM), which can vary in size and composition from coarse to fine and from organic to inorganic constituents. Of the commonly measured airborne particles that pollute the air, PM_2.5 (diameter below 2.5 µm fine airborne particles) and PM₁₀ (diameter below 10 µm, coarse airborne particles) are smaller and thus more worrisome for human health [1]. PM_2.5 particles are very small and can penetrate deep into the lungs and cause respiratory problems and cardiovascular disease, while PM₁₀ particles are slightly larger and cause irritation to the eyes, nose and throat. In addition to the suspended particles mentioned above, the other parameters of air pollution include nitrogen oxides (NO, NO₂), sulphur dioxide (SO₂), carbon monoxide (CO), and ozone (O₃). All of these particles are classified as Group 1 carcinogens by the World Health Organisation (WHO) and can have a significant impact on human health [2], the environment, and climate change.

To reduce the negative effects of air pollution and thus protect the health of affected people and the environment, effective measures must be taken to control air quality. Advances in machine learning (ML), particularly in convolutional neural networks (hereafter, CNNs), have made it possible to model and predict the concentration of particles in the air. Such tools can help in (a) understanding the contributors and (b) understanding trends and seasonalities. Both can help in their control and countermeasures. In this work, a 1D CNN model is developed to predict the concentration of PM₁₀, NO, and NO₂ in the air. The data come from public sources of the Austrian government [3] and include measurements of the above variables in the period from 1 January 2014 to 17 March 2022.

Three ML models were created and trained on the data: a 1D CNN model, a Long-Short Term Memory (LSTM) model, and a Random Forest (RF) model. The coefficient of determination (R²) and mean absolute error (MAE) were used to compare and evaluate the models. The preliminary results indicate that the 1D CNN model outperforms the other machine learning models (LSTM and RF) in terms of the R² and MAE.

2. Related Work

The use of ML methods in air quality prediction is an active research field. Examples of such methods include Least Absolute Shrinkage and Selection Operator (LASSO) Regression [4,5], Support Vector Machines (SVM) [6,7,8], and Random Forest (RF) [9].

Six different ML models were compared to predict the PM₁₀ particle concentration in Ankara, Turkey, namely: LASSO, SVM, RF, kNN (k-Nearest Neighbours), xGBoost (eXtreme Gradient Boosting), and ANN (Artificial Neural Network) [10]. The spatial distribution of PM₁₀ was analysed taking into account the land use characteristics of the region, namely the traffic, industrial density, population density, natural gas use, and changes in income. Data for the years 2009–2017 of six stations in Ankara, Turkey, were used as input, and the PM₁₀ concentrations of the seventh station for the year 2018 were predicted. This procedure was repeated for each of the stations: the data of each station were predicted for the year 2018, by using models trained using 2009–2017 data from the other six stations. The best results were obtained using ANN.

In the investigation of the change in air pollutant concentrations during the COVID-19 lockdown in Graz, Austria, machine learning, in particular RF, was used to analyse various predictions and actual pollutant levels [11]. The models showed good generalization, with the predicted PM₁₀ and NO₂ levels exceeding those measured during the closure, while the O₃ was underestimated, which was related to lower NOx emissions due to lower traffic volumes. Other examples of PM₁₀ and NO₂ modelling and prediction using machine learning models can be found in [12,13].

Deep learning (DL) (a subfield of machine learning) models are also being used in air quality modelling and prediction. Deep neural networks, i.e., neural networks (NN) with multiple hidden layers, are able extract intrinsic features and patterns in large datasets. An overview of such applications can be found in [14]. For example, the authors in [15] used a convolutional neural network to estimate PM_2.5 particle concentrations using 2011 data from the United States of America. Data on aerosol optical depth (AOD), meteorological fields, and land use were integrated into this model. The model was tested and evaluated using overall, spatially separated, and temporally separated cross-validation methods to ensure the reliability of the results. It was found that the proposed CNN-based model outperformed all the benchmark models in estimating the daily 24 h averaged ground-level PM_2.5 concentration. In addition, a novel metric for the importance of predictions was developed based on the Layerwise Relevance Propagation (LRP) method. The authors noted that the estimation accuracy of the model is boosted by exploiting the spatial correlation of nearby predictors. Other examples of DL models used in air pollution prediction include an LSTM [16,17], a combination of LSTM and convolutional neural networks (CNN) [18], and gated recurrent units (GRU) [19].

3. Materials and Methods

3.1. Study Area and Data

Long term environmental, pollution, and weather data from 5 measuring stations from the Austrian city of Graz namely Sud, Nord, West, Ost, and Don Bosco were analysed. The measurements, taken hourly, covered the time period from January 2014 to May 2020. Graz is a medium-sized European city, which has much in common with respect to the size and layout with many other European urban areas. The Ost and Don Bosco measurement sites are situated on arterial roads with high traffic volumes, especially during morning and evening rush hours. The most polluted measurement site of Graz is Don Bosco, which struggles every year to comply with the NO₂ and PM₁₀ regulatory limits of the EU-Council directive 96/62/EC. This is primarily because of traffic-related emissions but also because of the emissions from a nearby steel and iron mill. Although Graz East is located at a heavily frequented commuter-arterial road, the mean pollutant concentrations are lower than at Don Bosco. A more detailed description of the monitoring sites, pictures (Figure 1), and a historical overview, as well as an overview of the dataset can be found can be found in [11,20]. NO, NO₂, and PM₁₀ are measured at all stations, with O₃ being measured at stations Nord and Sud as well. The processed data used in this paper consisted of 71,377 measurements, with a total of 64 input time variables and 17 output variables of the particle concentrations in the air. However, the focus of this paper is on the aforementioned 3 output variables: NO, NO₂, and PM₁₀. For the training of the ML models, the first 80% of the data was used as a training set (measurements from 1 January 2014 to 7 July 2020), and the remaining 20% of the data (from 8 July 2020 to 17 March 2022) was used as the testing set.

3.2. Machine Learning Models

As mentioned earlier, three ML models were created and trained: a 1D CNN model, an LSTM model, and an RF model. One-dimensional CNNs are a type of NN inspired by the well-known 2D CNNs used in image recognition. However, unlike classical CNNs, which process 2D grids of data (like pixels in an image), they are suitable for processing one-dimensional sequences, making them ideal for tasks involving time-series data or audio signals [21]. Similar to the classic 2D CNNs, 1D CNNs consist of convolutional layers, pooling layers, and fully concatenated layers. The convolutional layers extract features from the data. These layers slide a filter (kernel) along the sequence and capture patterns and local dependencies. The pooling layer, such as the max-pooling layer, reduces the spatial dimensions by selecting the maximum value in each region, while the fully linked layers process the extracted features for classification or regression. Since they deal with simpler data, 1D CNNs are generally faster and require less computing power than 2D CNNs. However, similar to 2D CNNs, hyperparameter tuning is required to achieve optimal performance.

LSTMs are specialized recurrent neural networks (RNNs) designed to deal with long-term dependencies in sequential data and are widely used in various applications, specifically in tasks such as language modelling, machine translation, sentiment analysis, and time series prediction [16,17,18]. Conventional RNNs have difficulties with long-term dependencies. Information far back in a sequence can fade (vanishing gradient) or explode, making it difficult to learn long-term relationships. LSTMs introduce a clever gating mechanism that controls the flow of information within the network. This allows them to remember for longer; i.e., LSTMs have a cell state that can store relevant information for longer periods of time. LSTMs also have selective learning, i.e., gates regulate which information is added to the cell, which is retained, and which is forgotten. LSTMs consist of memory units (or LSTM cells) that enable the flow of information over multiple time steps. The most important components of an LSTM cell are

• Input Gate—controls the flow of new information into the cell;
• Forget Gate—determines which information from the previous state of the cell should be retained or forgotten;
• Output Gate—controls the output of the cell;
• Cell State—represents the memory content;
• Hidden state—represents the output of the cell.

LSTMs learn to update and maintain the cell state over time, allowing them to capture long-range dependencies.

RF is a versatile ML algorithm that uses the collective decision making of multiple decision trees to create a more accurate and stable prediction model. It generates an ensemble of decision trees, which are inherently simple models that, when used alone, are prone to overfitting and can be sensitive to noise in the data. For classification problems, the mode of the classes of the individual trees is chosen as the output, while for regression problems, the mean prediction of the individual trees is used as the output. The algorithm introduces randomness into the tree building process, which ensures that the high variance of the individual trees is balanced in the final model. By training each tree on a different subset of the data, the trees are less correlated and therefore, on average, more robust to overfitting. Due to this ensemble approach, RF models are also able to handle complex data sets with high dimensionality. Although an RF model is not as easy to interpret as a single decision tree, the use of feature importance scores can provide insight into which variables have the greatest influence on the model’s predictions. In this paper, the RF was chosen due to its good generalization shown in previous research [22,23,24].

The coefficient of determination (R²) and mean absolute error (MAE) were used to compare and evaluate the models in this work.

4. Results and Discussion

Based on the input data, all the models were trained to predict the NO, NO₂, and PM₁₀ values at all five measuring stations; hence, each model can be considered to have 64 input or predictive variables and 15 output or target variables. Hyperparameter tuning was performed for all models, and the summary of the results of the best models on the test dataset are shown in

Table 1. Results of the machine learning models on the test dataset.

Stations	Particles	R2			MAE
		1D CNN	LSTM	RF	1D CNN	LSTM	RF
Ost	NO	0.9008	0.8835	0.7664	6.2100	6.7127	10.2912
	NO2	0.9064	0.8803	0.7905	3.5989	4.1327	5.8821
	PM10	0.8182	0.8071	0.7366	5.5130	5.3905	6.5191
West	NO	0.9137	0.9044	0.7685	4.4323	4.6856	8.0933
	NO2	0.9144	0.8947	0.8171	3.1927	3.6173	5.2387
	PM10	0.8222	0.7940	0.6976	4.3716	4.8000	6.2122
Nord	NO	0.8913	0.8716	0.7234	3.3497	3.5162	5.7668
	NO2	0.9191	0.8932	0.8122	2.6985	3.3064	4.7665
	PM10	0.7094	0.6527	0.6182	5.1069	5.5235	6.2155
Sud	NO	0.9103	0.8972	0.7685	7.2757	7.7969	13.3983
	NO2	0.8737	0.8620	0.7741	4.5680	4.7530	6.6125
	PM10	0.8723	0.8527	0.7545	4.3542	4.8811	6.5107
Don Boscoe	NO	0.9159	0.9037	0.7832	10.5084	11.0673	17.2618
	NO2	0.9062	0.8727	0.7779	4.0131	4.8782	6.7749
	PM10	0.8158	0.8074	0.7261	5.1260	5.2686	6.4453

Maximum R² values and minimum MAE values of each target value across all model types are shown in bold.

below.

In Table 1, the maximum R² values and minimum MAE values of each target value across all model types are shown in bold. When analysing the results, it can be seen that the 1D-CNN model performed the best among the three models tested when considering both metrics, with the exception of Ost-PM₁₀, where the 1D-CNN model was the best model when considering the R², and the LSTM model was the best model when considering the MAE. The time series plot of the overall best model (Nord-NO₂) considering both metrics is shown in Figure 2 below.

5. Conclusions

Over a period from January 2014 to May 2020, hourly measurements of long-term environmental, pollution, and weather data from five measuring stations in the Austrian city of Graz, namely Sud, Nord, West, Ost, and Don Bosco, were analysed. Three machine learning models, namely 1D CNN, LSTM, and RF models, were used to predict NO, NO₂, and PM₁₀ levels. The models were compared using R² and MAE. The 1D CNN model showed the best overall results followed by the LSTM model. For future research, it would be very useful to further investigate the architecture as well as perform extra hyperparameter optimization of the 1D CNN and LSTM models to achieve higher accuracy and reliability.