An Ensemble Model for PM2.5 Concentration Prediction Based on Feature Selection and Two-Layer Clustering Algorithm

Abstract

Determining accurate PM2.5 pollution concentrations and understanding their dynamic patterns are crucial for scientifically informed air pollution control strategies. Traditional reliance on linear correlation coefficients for ascertaining PM2.5-related factors only uncovers superficial relationships. Moreover, the invariance of conventional prediction models restricts their accuracy. To enhance the precision of PM2.5 concentration prediction, this study introduces a novel integrated model that leverages feature selection and a clustering algorithm. Comprising three components—feature selection, clustering, and integrated prediction—the model first employs the non-dominated sorting genetic algorithm (NSGA-III) to identify the most impactful features affecting PM2.5 concentration within air pollutants and meteorological factors. This step offers more valuable feature data for subsequent modules. The model then adopts a two-layer clustering method (SOM+K-means) to analyze the multifaceted irregularity within the dataset. Finally, the model establishes the Extreme Learning Machine (ELM) weak learner for each classification, integrating multiple weak learners using the AdaBoost algorithm to obtain a comprehensive prediction model. Through feature correlation enhancement, data irregularity exploration, and model adaptability improvement, the proposed model significantly enhances the overall prediction performance. Data sourced from 12 Beijing-based monitoring sites in 2016 were utilized for an empirical study, and the model’s results were compared with five other predictive models. The outcomes demonstrate that the proposed model significantly heightens prediction accuracy, offering useful insights and potential for broadened application to multifactor correlation concentration prediction methodologies for other pollutants.

1. Introduction

As nations continue to industrialize and expand transportation networks to keep pace with rapid urban modernization, there is an attendant rise in living standards. But coupled with growth are escalating air quality indices, signifying increased quantities of harmful substances discharged into the atmosphere and exacerbating environmental issues. Polluted air comprises detrimental particles such as PM2.5, PM10, CO, SO₂, NO_x, and O₃, which have been implicated in the onset of respiratory and cardio-cerebrovascular illnesses [1]. Among these pollutants, PM2.5, particulates with diameters under 2.5 μm, are particularly concerning due to their high toxic substance content, lengthy atmospheric residence time, and extensive transport distance. This pollutant critically impacts both human health and atmospheric quality. According to the United Nations Environment Programme’s Global Environmental Outlook 5 launched in 2012, PM2.5-induced respiratory diseases cause nearly 700,000 deaths annually, with almost 2 million premature deaths linked to particulate pollution. Recent estimates from the Global Burden of Disease Project attribute approximately a million deaths in China yearly to PM2.5 pollution. Consequently, investigating air pollutants, particularly PM2.5, stands out as a prime research focus. Notably, numerous countries globally have installed air quality monitoring stations for real-time pollutant surveillance, enhancing the practical significance of forecasting pollutant concentrations. Accurate PM2.5 concentration prediction has important implications for shaping air pollution prevention and mitigation strategies, providing a useful navigate and reference point.

1.1. Related Works

Current research on PM2.5 concentration prediction models largely falls into two main categories: deterministic methods, exemplified by chemical transport models (CTMs), and statistical methods, which primarily encompass machine learning models, multiple linear regression (MLR), and auto-regressive comprehensive moving average models (ARIMA) [2]. Deterministic methods, which account for the chemical reaction and transport process of air pollutants, formulate models based on chemical and kinetic expressions, enabling simulations of pollutant emission, migration, and transformation, producing respective predictive results [3]. However, this method’s efficacy is compromised by the intricacy of the model and the extensive time required for model construction and solution, posing calculation-result realization challenges [2]. In contrast, statistical models, forgoing pollutant chemical evolution considerations, focus solely on data aspects, simplifying the model construction process, and consequently garnering increased interest. These statistical methods can be broken down into traditional statistical methodologies, machine learning approaches, and integrated learning practices. Traditional statistical methods, including linear statistical models like MLR and ARIMA, have notable limitations in predicting PM2.5 concentration, chiefly stemming from their dependency on linear mapping ability in non-linear processes. This leads to a significant inefficacy in exploring the laws governing non-linear models. In reality, most air pollutant sequences are non-linear and irregular. On comparing, machine learning models prove superior with an enhanced non-linear fitting ability. Techniques like artificial neural networks (ANN), support vector regression (SVR), and random forests (RF) find extensive applications in air pollution prediction. For instance, Ren et al. proposed a PM2.5 concentration level prediction model, leveraging a random forest and characterized by Taiyuan meteorological data from 2013 to 2016, and the site’s PM2.5 concentration change time sequence, coupled with its temporal and spatial correlation to surrounding sites [4]. Similarly, Hong et al. put forth a novel approach for estimating global PM2.5 concentration variations through the integration of satellite imagery, ground measurements, and deep convolutional neural networks [5]. Wu et al. proposed an adaptive genetic algorithm (AGA)-based long short-term memory (LSTM) network prediction model, employing a copula entropy (CE) framework, to analyze the correlation between multiple meteorological factors and different atmospheric pollutants and PM2.5 [6]. Meanwhile, Pruthi et al. offered a deep learning model, integrating neural networks, fuzzy inference systems, and wavelet transforms, to predict Delhi’s major air pollutant, PM2.5 [7]. Zaini et al. proposed a hybrid deep learning models to predict the hourly PM2.5 concentration for an urban area in Malaysia [8]. Li et al. utilized the LSTM and the gated recurrent unit network (GRU) as the baseline models to predict the concentration of air pollutants [9]. Zhou et al. aimed at the long-term prediction of PM2.5 concentration, considering PM2.5 spatio-temporal correlation between multivariate data, and a TSMN prediction model was proposed [10]. Hu et al. proposed a hybrid machine learning model (WD-SA-LSTM-BP) based on simulated annealing (SA) optimization and wavelet decomposition to predict the hourly PM2.5 concentration [11]. Huang et al. designed a prediction model called RNN-CNN for PM2.5 hourly concentration prediction based on ensemble deep learning, using combined recurrent neural networks (RNN) and CNN as individual learners and utilizing the integrated learning technology stacking for PM2.5 hourly concentration prediction [12].

Although machine learning models robustly exploit the nonlinear ability of air pollution prediction, they are subject to inherent limitations (such as underfitting or overfitting). However, integrated models can counter these limitations by training multiple “weak learners”, which are subsequently converged via a specific strategy to form a “strong learner”. This approach mitigates the risk of underfitting or overfitting, resulting in enhanced predictive performance. For instance, Liu et al. employed an amalgamation of the Bagging method and the Gradient Boosting Decision Tree (GBDT) to prognosticate PM2.5 levels in Beijing, China; comparative experiments substantiated that an ensemble model attains lesser predictive errors than a singular machine learning model [13]. Similarly, S. Yin et al. utilized two boosting algorithms, namely the Modified AdaBoost. RT and Gradient Boosting, for hourly PM2.5 concentration forecasting [3]. Further, Liu et al. advanced a multi-objective and multi-resolution ensemble model that assimilates a diversity of information expressions to elevate model accuracy [14]. Liu et al. utilized the Extreme Gradient Boosting (XGBoost) algorithm to give the final pollutant concentration prediction model [15]. Joharestani et al. used RF, XGBoost, and deep learning to predict PM2.5 concentration, and the results showed that the model performance obtained by using the XGBoost algorithm was the best [16].

Aside from model selection, the identification of influential factors related to PM2.5 concentration significantly impacts predictive results. The generation and flow of PM2.5 are significantly related to the local climate environment [17]. Many studies favor the Pearson Correlation Coefficient (PCC) for correlation analyses, owing to its straightforward processing method for generating a correlation matrix of PM2.5 concentration indices. Liu et al. used PCC for correlation analysis to obtain the correlation matrix of six major AQI indicators in a city regarding available monitoring data [18]. Zeng et al. used PCC to analyze the correlation between PM2.5 concentration in summer and autumn in Beijing and six meteorological factors, including air temperature, relative humidity, wind speed, water vapor pressure, atmospheric pressure, and wind direction [19]. However, PCC’s reliance on linear Gauss may undermine its reliability when dealing with non-linear air pollutant sequences. Thus, effective selection of multiple PM2.5-impacting factors and the elimination of irrelevant ones can save precious resources for prediction and enhance accuracy [20].

Feature selection methods are often categorized into filtering, packaging, and embedding techniques. The filtering method, which includes PCC, scores each feature according to divergence or correlation and sets a threshold value or a limit for feature selection. Conversely, the packaging method leverages machine learning algorithms for evaluating the impact of feature subsets, detecting interactions between two or more features, and selecting optimally performing feature subsets. However, this method demands significant computational resources due to the need to train a model for each subset. To enhance computational efficiency, multi-objective optimization algorithms are often applied to packaging methods to undertake feature selection. For instance, Redkar et al. utilized a multi-objective optimization-based packaging method for feature selection to handle drug-target interaction (DTI) data’s imbalance and high dimensionality [21]. Similarly, Wu et al. employed a multi-objective feasibility-enhanced particle swarm optimization (MOFEPSO) algorithm to optimize maximum relevancy, minimum redundancy, and maximum interaction of features while selecting the ideal ones [22]. Got et al. proposed a multi-objective algorithm in which a filter and wrapper fitness functions are optimized simultaneously [23]. Han et al. proposed a feature selection algorithm based on multi-objective particle swarm optimization with adaptive strategies (MOPSO-ASFS) to improve the selection pressures of the population [24].

1.2. Novelty of the Study

Derived from the aforementioned literature review, we propose an innovative mixed model for PM2.5 concentration prediction composed of three modules: feature selection, clustering, and integrated prediction. By enhancing feature correlation, refining data irregularity, and improving model prediction ability, this model seeks to boost overall prediction performance.

Our study’s contributions and innovations manifest in the following ways:

(a)

We employ a multi-objective optimization algorithm for selecting features from atmospheric pollutant and meteorological factor datasets that influence PM2.5 concentration, thereby supplying valuable feature data input for subsequent modules. Specifically, we use the non-dominated sorting genetic algorithm-III (NSGA-III) to compute the weight coefficient between the multi-factor feature variables and PM2.5 concentration prediction. By comparing this with a defined threshold value, we select Pareto-optimal input feature variables.

(b)

The features selected using the multi-objective optimization algorithm are subsequently clustered, further mining the irregularity of the multi-factor dataset and establishing a weak learner for each class. This enables data with high similarity to be predicted under the same model. In our study, we adopt a two-layer clustering method (initially using the SOM neural network, followed by K-means clustering). This method’s primary advantage lies in noise reduction, as the prototype of SOM constitutes average data, exhibiting lower sensitivity to random changes than the original data [25].

(c)

In our model, we harness the reinforcement learning method in ensemble learning to curtail the bias of preceding weak learners through iterative training, dynamically adjust the weight distribution of multiple weak learners, and ultimately transform these trained weak learners into a robust learner through linear combination. Specifically, we utilize the AdaBoost algorithm for integrating the weak learner composed of multiple extreme learning machine models. The resulting integrated prediction model seeks to enhance prediction accuracy.

The paper’s structure is as follows: Section 2 delineates our proposed PM2.5 concentration forecasting method and offers a detailed introduction to each ensemble model algorithm. Section 3 applies these proposed models to actual PM2.5 concentration data prediction, followed by an analysis of the experimental results. Section 4 concludes our research.

2. Materials and Methods

2.1. Description of Experimental Data

The experimental dataset utilized in this study was obtained from the environmental cloud of Nanjing Yunchuang Big Data Technology Co., LTD. (Nanjing, China) We accessed hourly meteorological records (comprising weather condition, air temperature, felt temperature, air pressure, humidity, rainfall, wind direction, and wind speed) and hourly air quality monitoring data (PM10, CO, SO₂, NO_x, O₃) for Beijing, spanning from 1 January 2016 to 31 December 2016. The air quality monitoring data refer to the hourly data from 12 monitoring locations, resulting in a total of 8784 records for each monitoring point, though some data are missing due to uncontrollable factors. Missing data are mainly missing meteorological data, including missing meteorological data on a certain day and hour or missing wind direction data on a certain day and hour. If the entire meteorological data of a certain hour on a certain day are missing, the data are deleted. If the wind direction data of a certain hour on a certain day are missing, the linear interpolation method is used to complete the data. Thus, the final number of data records used was 8120. The locations of the 12 monitoring points are shown in Figure 1. Text descriptions of weather conditions, wind direction, and wind speed were encoded, as outlined in

Table 1. Weather condition attribute coding rules in meteorological data.

Weather Conditions	Code	Weather Conditions	Code
Clear	1	Fog	10
Haze	2	Rain and snow	11
Cloudy	3	Snow	12
Yin	4	Moderate to heavy snow	13
Light rain	5	Heavy Snow	14
Moderate to heavy rain	6	Heavy to blizzard	15
Heavy rain	7	Floating dust	16
Showers	8	Medium Rain	17
Thundershowers	9	Rainstorm	18

,

Table 2. Wind attribute coding rules in meteorological data.

Wind Direction	Code
North Wind	1
Northeast Wind	2
East Wind	3
Southeast Wind	4
South Wind	5
Southwest Wind	6
West Wind	7
Northwest Wind	8

and

Table 3. Wind power attribute coding rules in meteorological data.

Wind Power	Code
Breeze	1
Level 1	2
Level 2	3
Level 3	4
Level 4	5
Level 5	6

.

In the endeavor to construct and assess the predictive model, the dataset for each of the 12 monitoring points was partitioned into three categories: a predictive training set, predictive validation set, and a test set. The training set, which is integral to the three modules, comprises records 1 to 7000 from the dataset. The validation set is composed of data records 7001 to 7200, and the test set consists of records 7201 to 8120. Equipped with an Intel(R) Core(TM) i7-8565U CPU (Intel, Santa Clara, CA, USA) at 1.80 GHz, 8 GB of memory, and Windows 10 operating system, we used Python 3.7.8 (Guido van Rossum, Holland) as our programming tool for this experimental setup.

2.2. Framing

The conceptualized prediction model for PM2.5 concentration, dubbed as NSGA-III-SOM-Kmeans-Elm-Adaboost (NSKEA), is an integrated apparatus that primarily consists of three modules. Its main components are as follows:

• The utilization of NSGA-III for feature selection;
• Implementation of a two-layer clustering method (SOM-Kmeans) to cluster data post feature selection;
• Use the ELM model for the clustered data resulting from each cluster;
• Lastly, integration with AdaBoost is facilitated to realize the prediction of the PM2.5 concentration.

The inner structure of the proposed NSKEA model is conveniently illustrated in Figure 2. Therefore, a detailed investigation aids in understanding the intricate operations of this model.

2.3. Feature Selection: Multi-Objective Optimization: NSGA-III Algorithm

This study employs meteorological factors (such as air temperature, apparent temperature, air pressure, humidity, rainfall, wind direction, and wind speed) along with other air pollutants (PM10, CO, SO₂, NO_x, O₃) as characteristic data in the feature selection process. It was found that these selected features had a strong correlation with PM2.5.

The aim of feature selection is identifying the optimal feature subset. It enables the elimination of irrelevant or redundant features, thus reducing the count of features, bettering the model’s precision, and decreasing the execution time.

Feature selection methodologies can be categorized into filtering method, wrapping method, and embedding method. Out of these, the wrapping method’s basic approach involves training the model for each feature subset to be selected on the training set. The feature subset is then chosen on the test set, based on the error magnitude.

In our feature selection module, we have taken the approach of the wrapping method, applying the NSGA-III algorithm for multi-objective optimization (MAE, MSE, and SD). It is used to compute the correlation between the predicted results of PM2.5 concentration for each feature and its actual value. Feature selection is performed by setting a specific threshold value.

The NSGA-III algorithm was first introduced by Kalyanmoy Deb and Himanshu Jain in 2014. This algorithm is an enhanced version of the multi-objective algorithm NSGA-II. It abandons the crowding-distance sorting mechanism often used in NSGA-II and introduces a new sorting mechanism based on reference points [26]. The NSGA-III is especially designed to deal with multi-objective optimization issues that have three or more objectives [27]. When compared with the NSGA-II algorithm, the NSGA-III not only significantly reduces the computational complexity but also excels in preserving diversity. This makes it an efficient tool for complex multi-objective optimization tasks.

The basic idea of using the NSGA-III algorithm for feature selection is as follows:

• The primary variables include PM10, CO, SO₂, NO_x, O₃, air temperature, sensible temperature, air pressure, humidity, rainfall, wind direction, wind power, and wind speed, totaling 13 factors $\left\{x_i\left(t\right),t=1,2,3,\cdots ,13\right\}$
• The focus for prediction is the PM2.5 concentration, identified as our target feature $x_{PM2.5}\left(t\right)$
• The Extreme Learning Machine (ELM) model is used individually with each feature $x_i\left(t\right)$ as an input for prediction. Each feature’s predictive capability $\left\{{\widehat{x}}_i^{PM2.5},i=1,2,3,\cdots 13\right\}$ is assessed and an aggregated prediction is achieved using weighted reconstruction (Equation (1)),

$${\widehat{x}}_{PM2.5}={\displaystyle \sum _{i=1}^{13}{\omega }_i{\widehat{x}}_i^{PM2.5}},$$

(1)

• The evaluation is objective using the NSGA-III algorithm, taking into account the mean square error (MSE), Mean Absolute Error (MAE), and standard deviation (SD). These metrics, outlined in Equation (2) through (4), are deployed to measure the divergence between the predicted results and the actual values accurately,

$$MSE=\frac{1}{N}{\displaystyle \sum _{j=1}^N{\left({\widehat{x}}_{PM2.5}\left(j\right)-x_{PM2.5}\left(j\right)\right)}^2},$$

(2)

$$MAE=\frac{1}{N}{\displaystyle \sum _{j=1}^N\left|{\widehat{x}}_{PM2.5}\left(j\right)-x_{PM2.5}\left(j\right)\right|},$$

(3)

$$SD=\sqrt{\frac{1}{N}{\displaystyle \sum _{j=1}^N{\left({\widehat{x}}_{PM2.5}\left(j\right)-x_{PM2.5}\left(j\right)-mean\left({\widehat{x}}_{PM2.5}\left(j\right)-x_{PM2.5}\left(j\right)\right)\right)}^2}},$$

(4)

where $N$ indicates the total number of the true value of PM2.5 concentration.

• To realize multi-objective optimization featuring Mean Absolute Error (MAE), Mean Square Error (MSE), and Standard Deviation (SD), we harness the capabilities of the NSGA-III algorithm. Our strategy embodies an iterative search for a weight-set $\left\{{\omega }_i,i=1,2,3,\cdots ,13\right\}$ that concurrently minimizes MAE, MSE, and SD. Given the defined threshold value—$T$, only those features $\left\{x^{\prime }\left|{\omega }_i\ge T,i=1,2,3,\cdots ,13\right.\right\}$ that comply with the condition ${\omega }_i\ge T$ are selected. This precision-guided approach enables us to optimize feature selection effectively.

2.4. Two-Layer Clustering

Emphasizing the identification of anomalies in multifaceted datasets, we employ clustering algorithms, each providing a unique set of characteristics for the resulting clusters. Herein, we report the use of a two-layer clustering method employing a combination of Self-Organizing Maps (SOM) and K-means algorithms.

Initially, the SOM algorithm is applied to learn the data from the input space, which comprises 12 monitoring points; this serves to excavate similarities among them. Following acquisition of the prototype vector via this initial stage, the K-means algorithm is employed to cluster this vector, further extracting features of the training dataset. Our two-layer clustering methodology not only brings high-dimensional data visualization to fruition but also upholds the topological structure of input space while reducing dataset noise.

SOM, also known as Kohonen maps, are effectively used to visualize and explore data properties, projecting the input space onto a low-dimensional, regular grid prototype. This form of unsupervised learning is utilized to cluster data. Rooted in an uncomplicated concept, this type of neural network, with solely the input and competition layer, uses a “competitive learning” method during training. Each input sample seeks the most compatible node within the competitive layer, referred to as its activation node or “winning neuron” [28]. The parameters of this active node are updated via random gradient descent, while those of the nodes located near to the active node are suitably updated based on their proximity to it.

2.5. Prediction Model

To predict PM2.5 concentration levels, we have established a third module: a predictive model enhanced through ensemble learning methods. Unpackaging the outcomes from the two initial modules, we generate multiple cluster-resultant datasets, each accompanied by its respective ELM predictive model (Equation (5)).

$$f_L\left(x\right)={\displaystyle \sum _{i=1}^L{\beta }_i{g}_i\left(x\right)={\displaystyle \sum _{i=1}^L{\beta }_{i}g\left({\omega }_i\times x_i+b_i\right),i=1,2,\cdots ,N}}.$$

(5)

In this context, “$L$” symbolizes the quantity of hidden units, while “$N$” represents the count of training samples. “${\beta }_i$” designates the $i$th weight vector interlinking the primary hidden layer and the output layer. The $i$th input vector is represented by “$x_i$”. “${\omega }_i$” stands for the weight vector connecting the $i$th input layer to the output layer. The activation function is represented by “$g_i$”. “$b_i$” signifies a bias vector.

An ELM, a single-hidden-layer feedforward neural network (SLFN), is employed to expedite the training process [29]. This training methodology is notably superior to traditional SLFN algorithms, with ELM selecting random weights for input layers, hidden-layer bias, and output-layer weight, determined through minimization of a loss function—a sum of the training error term and a regular term reflecting the output-layer weight norm [30].

Despite the randomized generation of hidden-layer nodes, ELM preserves the fundamental approximation capacity of the SLFN. This network structure is depicted in Figure 3. ELM provides a rapid learning speed, robust generalization ability, and reduced parameter training dependency. Nonetheless, ELM disadvantages prevail; its exclusive focus on empirical risk inspires an overfitting proclivity remedied by introducing ensemble learning in this study.

Introduced by Yoav Freund and Robert Schapire in 1997, the Adaptive Boosting (AdaBoost) algorithm emerged as an innovative variant of the ensemble learning method, Boosting [31]. The adaptive nature of this algorithm comes to the fore in its weighting strategy; training errors result in increased sample weights, whereas correctly classified samples see a decrease, readying them for training in succeeding basic classifiers. Moreover, with each iterative cycle, a new weak classifier is incorporated into the ensemble until either achieves a predefined minimum error rate or reaches the maximum number of iterations.

The procedure for an AdaBoost ensemble prediction model utilizing an Extreme Learning Machine (ELM) as a base classifier unfolds as follows:

Step 1: Initiate the weight distribution for each ELM base classifier such that each classifier assumes an equal weight of ${\omega }_i=\frac{1}{k}$. This process generates the initial weight distribution, denoted as $D_1\left(i\right)$, across the training sample set. The setup is defined by Equation (6).

$$D_1\left(i\right)=\left({\omega }_1,{\omega }_2,\cdots ,{\omega }_k\right)=\left(\frac{1}{k},\frac{1}{k},\cdots ,\frac{1}{k}\right),$$

(6)

Step 2: Iterative Process:

(a)

From the collection of weak classifiers, identify the classifier $h$ with the lowest current error rate. This is designated as the $t\left(t=1,2,\cdots ,T\right)$-base classifier, $H_t$, and calculate its value $h_t:X\to \left\{-1,1\right\}$. The corresponding error $e_t$ and distribution $D_t$ of the weak classifier are enumerated in Equation (7).

$$e_t=P\left(H_t\left(x_i\right)\ne y_i\right)={\displaystyle \sum _i^k{\omega }_{ti}I\left(H_t\left(x_i\right)\ne y_i\right)},$$

(7)

(b)

Ascertain the weight ${\alpha }_t$ of the weak classifier within the final classifier ensemble (Equation (8)).

$${\alpha }_t=\frac{1}{2}\ln \left(\frac{1-e_t}{e_t}\right),$$

(8)

(c)

Update the weight distribution $D_{t+1}$ (Equation (9)) for the training sample set.

$$D_{t+1}=\frac{D_t\left(i\right)\exp \left(-{\alpha }_t{y}_i{H}_t\left(x_i\right)\right)}{{{\rm Z}}_t},$$

(9)

Here, it is essential to note that ${{\rm Z}}_t$ represents the normalization constant ${{\rm Z}}_t=2\sqrt{e_t\left(1-e_t\right)}$.

Step 3: Compile each weak classifier ELM according to its weight. In the final stage, this integration generates the robust final classifier, $H_{final}$ (Equations (10) and (11)), which then delivers the predictive output.

$$f\left(x\right)={\displaystyle \sum _{t=1}^T{\alpha }_t{H}_t\left(x\right)},$$

(10)

$$H_{final}=sign\left(f\left(x\right)\right)=sign\left({\displaystyle \sum _{t=1}^T{\alpha }_t{H}_t\left(x\right)}\right).$$

(11)

3. Case Study

3.1. Evaluation Criteria

To assess the efficacy of the prediction model, we conduct an inclusive evaluation using a threefold metric of MAE, Root Mean Square Error (RMSE), and the index of agreement (IA, a nondimensional and bounded measure of the model prediction error degree with values closer to 1 indicating a better match). The calculated formulae for these parameters can be found in

Table 4. Evaluation criteria.

Criteria	Definition
RMSE	$RMSE=\sqrt{\frac{1}{n}{\displaystyle \sum _{i=1}^n{\left({\widehat{y}}_i-y_i\right)}^2}}$
IA	$IA=1-\frac{{\displaystyle {\sum }_{i=1}^n{\left(y_i-{\widehat{y}}_i\right)}^2}}{{\displaystyle {\sum }_{i=1}^n{\left(\left|{\widehat{y}}_i-{\overline{y}}_i\right|+\left|y_i-{\overline{y}}_i\right|\right)}^2}}$

. Within these formulae, the variables ${\widehat{y}}_i$ and $y_i$ signify the predicted and observed PM2.5 concentrations, respectively, the variable ${\overline{y}}_i$ is the average value of $n$ observed PM2.5 concentrations, whereas $n$ stands for the total number of predicted values.

3.2. Experimental Design

3.2.1. The NSGA-III Based Feature Selection Method Analysis

To validate the feasibility and precision of the feature selection method proposed herein, we designed three prediction models for comparative analysis—the NSGA-III-ELM-Adaboost (NEA) model; the single factor-ELM-AdaBoost (SFEA), a unifactorial ensemble model centered solely on PM2.5 concentration features; and the all features-ELM-Adaboost (AFEA) model, an ensemble model incorporating all features. For consistency and to ensure the legitimacy of our comparative results, all three models employ an ELM-Adaboost model and keep the integrated model parameters consistent.

The NEA model’s parameter settings can be found in

Table 5. Parameter setting of the NSGA-III-ELM-Adaboost model.

Models	Parameters	Values
NSGA-III	Maximum number of iterations	400
	Population size	100
	Mutation percentage	0.5
	Crossover percentage	0.5
	Mutation rate	0.03
	Cross parameter	20
	Mutation parameter	20
ELM	Number of hidden neurons	20
	Activation function	sigmoid
AdaBoost	Learning rate	1, there is a trade-off between learning rate and maximum integration number
	Maximum integration number	50

. Figure 4 depicts the results of the Pareto fronts and the selected point from the one-step predicted dataset. We chose the intermediate solution of Pareto fronts to harmonize the benefits from the three objective functions. Figure 5 represents the optimal weight results for each feature in a 12-point monitoring dataset.

Table 6. Comparison of evaluation criteria of six prediction models.

Monitoring Point Dataset	Model	Criteria			Monitoring Point Dataset	Model	Criteria
		MAE	RMSE	IA			MAE	RMSE	IA
1	NSKEA	13.418	21.401	0.95	7	NSKEA	13.418	21.401	0.95
	NEA	15.241	22.127	0.92		NEA	14.521	22.064	0.92
	SFEA	17.727	25.671	0.87		SFEA	16.702	25.717	0.88
	AFEA	16.502	24.518	0.90		AFEA	16.217	24.601	0.91
	RNN	17.031	25.126	0.89		RNN	17.258	25.102	0.89
	LSTM	16.557	24.673	0.82		LSTM	16.343	24.720	0.83
	MLR	20.826	28.855	0.71		MLR	20.520	28.728	0.73
	SVR	17.782	26.703	0.79		SVR	17.681	26.537	0.80
	RF	16.364	26.114	0.81		RF	16.305	26.246	0.81
2	NSKEA	13.418	21.401	0.95	8	NSKEA	13.418	21.401	0.95
	NEA	15.026	22.536	0.92		NEA	14.414	23.212	0.93
	SFEA	17.838	25.603	0.82		SFEA	17.515	25.372	0.82
	AFEA	16.882	24.126	0.87		AFEA	16.683	24.505	0.85
	RNN	17.524	25.331	0.83		RNN	17.518	25.002	0.83
	LSTM	16.266	24.791	0.85		LSTM	16.206	24.751	0.88
	MLR	20.244	28.855	0.71		MLR	21.206	28.506	0.70
	SVR	17.313	26.703	0.77		SVR	17.534	26.643	0.80
	RF	16.654	26.114	0.81		RF	17.028	25.811	0.84
3	NSKEA	13.418	21.401	0.95	9	NSKEA	13.418	21.401	0.95
	NEA	14.857	22.822	0.92		NEA	14.138	23.222	0.92
	SFEA	17.685	26.101	0.81		SFEA	17.371	25.618	0.80
	AFEA	15.863	25.237	0.88		AFEA	16.617	24.379	0.85
	RNN	17.106	25.639	0.84		RNN	17.113	25.338	0.82
	LSTM	16.313	24.604	0.86		LSTM	16.653	24.472	0.88
	MLR	20.933	28.360	0.73		MLR	20.637	28.537	0.75
	SVR	17.462	26.813	0.80		SVR	17.073	26.612	0.81
	RF	16.578	25.807	0.81		RF	16.511	25.377	0.84
4	NSKEA	13.418	21.401	0.95	10	NSKEA	13.418	21.401	0.95
	NEA	14.776	21.702	0.93		NEA	14.538	22.502	0.91
	SFEA	17.371	25.419	0.83		SFEA	17.219	25.421	0.82
	AFEA	15.013	24.665	0.87		AFEA	16.619	24.315	0.87
	RNN	17.077	25.028	0.85		RNN	17.787	24.801	0.83
	LSTM	16.326	24.552	0.89		LSTM	16.175	24.390	0.85
	MLR	21.212	28.527	0.75		MLR	20.667	28.677	0.77
	SVR	17.650	26.539	0.80		SVR	17.751	26.414	0.80
	RF	16.283	25.367	0.84		RF	16.566	25.361	0.84
5	NSKEA	13.418	21.401	0.95	11	NSKEA	13.418	21.401	0.95
	NEA	14.872	22.108	0.91		NEA	14.716	22.667	0.92
	SFEA	17.983	25.433	0.78		SFEA	17.131	25.366	0.80
	AFEA	16.382	24.712	0.81		AFEA	16.618	24.212	0.81
	RNN	17.321	25.114	0.80		RNN	17.812	25.771	0.80
	LSTM	16.505	24.536	0.83		LSTM	16.326	24.405	0.83
	MLR	20.831	28.312	0.74		MLR	20.335	28.378	0.73
	SVR	17.624	26.647	0.78		SVR	17.402	26.412	0.80
	RF	16.414	25.557	0.83		RF	16.613	25.534	0.82
6	NSKEA	13.418	21.401	0.95	12	NSKEA	13.418	21.401	0.95
	NEA	14.761	23.134	0.92		NEA	14.515	23.521	0.91
	SFEA	17.382	25.662	0.81		SFEA	17.617	25.780	0.83
	AFEA	16.287	24.404	0.86		AFEA	16.280	24.271	0.87
	RNN	17.680	25.311	0.78		RNN	17.428	25.263	0.79
	LSTM	16.627	24.573	0.87		LSTM	16.750	24.542	0.84
	MLR	20.622	28.602	0.75		MLR	21.137	28.732	0.73
	SVR	17.371	26.221	0.79		SVR	17.772	26.467	0.80
	RF	16.505	25.516	0.81		RF	16.379	25.205	0.81

presents a comparative analysis of the evaluation indicators from the three prediction models.

Employing the tenfold cross-validation method, the threshold for the NSGA-III feature selection stage was determined as $T=0.4$. Each feature’s weight was compared with this threshold, and those with a weight (${\omega }_i$) equal to or surpassing 0.4 were selected, as exemplified by the results of the 12-point monitoring study (Figure 4). Consequently, seven features—namely SO₂, PM10, NO₂, O₃, humidity, temperature, and barometric pressure—were chosen as the input data for the prediction model.

The experimental evaluations from three distinct models, presented in Table 6, underscore that the NEA predictions at 12 monitoring points are most favorable after feature selection via NSGA-III. This finding substantiates that a feature selection strategy premised on the NSGA-III algorithm can indeed enhance the accuracy of the prediction model.

3.2.2. Ensemble Model Analysis

In this section, we analyze the experimental outcomes of a two-layer clustering method (SOM+Kmeans). As delineated in Section 3.2, the SOM algorithm is initially employed for cluster learning on datasets subject to feature selection, contingent on NSGA-III. This is subsequently followed by using the K-means algorithm to cluster the prototype vector, thus further mining the features of the training dataset.

To critically assess the number of clusters, we utilize evaluation metrics such as F-measure, Accuracy, and Normalized Mutual Information. These indicators range between [0, 1], where a larger value signifies that the clustering outcome is commensurate with expectations.

Table 7. Evaluation index of clustering results.

Criteria	Definition
F-measure (FM)	$Precision=\frac{TP}{TP+FP}$ $\mathrm{Re}call=\frac{TP}{TP+FN}$ $F-measure=\frac{2\mathrm{Re}call\times Precision}{\mathrm{Re}call+Precision}$
Accuracy (ACC)	$ACC=\frac{TP+TN}{TP+TN+FP+FN}$
Normalized Mutual Information (NMI)	$NMI=\frac{I\left(U,V\right)}{\sqrt{H\left(U\right)H\left(V\right)}}$

encapsulates the calculation formulas of these three metrics. True Positive (TP) denotes the positive predicted sample count, while True Negative (TN) indicates the negative predicted count. False Positive (FP) represents instances where the predicted class number is wrongly marked as positive, whereas False Negative (FN) refers to samples falsely predicted as negative. Entropy of correct classification is represented as H(U), and H(V) stands for the entropy of results obtained via the algorithm.

Data collated from three evaluation indicators, for differing cumulative cluster numbers, are presented in

Table 8. Results of three evaluation indexes with different number of clusters.

Cluster Number	FM	ACC	NMI
2	0.00289	0.03401	0.07301
3	0.00443	0.33880	0.15358
4	0.01354	0.63023	0.16661
5	0.00576	0.42640	0.18444
6	0.00583	0.38350	0.19668

. Figure 6, meanwhile, portrays the variance in clustering efficacy in relation to fluctuating cluster numbers. An impartial analysis of both Table 8 and Figure 5 highlights that the number of clusters yielding the most optimal result across all three indices, and the most conducive clustering effect, when $k=4$.

3.3. Discussion

In this section, in order to prove the predictive superiority of our forecasting method, five existing air pollution forecasting models (RNN, LSTM, MLR, SVR, and RF) and three models given in 4.2.1 (NEA, SFEA, and AFEA) are compared with the proposed model. All models were tested using the data in Section 2.1, and the average error index data were obtained, as shown in Table 6.

In all comparative experiments, the evaluation index results, as measured by NSKEA at every monitoring point, are identical. Specifically, during the second phase, data from the twelve monitoring points are aggregated, amplifying similarities among these points, to yield a single, unique final prediction result. As can be seen from Table 6, in the experimental dataset, the error evaluation index data of NSKEA algorithm are the smallest, while IA index data are the largest, and the values of MAE, RMSE and IA are 13.418 µg/m³, 21.401 µg/m³ and 0.93, respectively. It shows that the prediction effect of NSKEA algorithm is the best.

By comparing the index data of the NEA, SFEA, and AFEA models in Table 6, it is found that the three indicator data of NEA are the best. This suggests that feature selection within the original dataset positively influences the model’s predictive accuracy, further hinting at a nonlinear relationship existing between the attributes. This highlights the need for machine learning methods to probe deeper into these mutual feature relationships.

When comparing NSKEA, RNN, and LSTM, NSKEA emerged supreme. This suggests that contrasted with standalone models, a comprehensive learning model that dynamically selects the dataset’s beneficial features and adjusts the weak prediction model’s weight ratios using the proposed prediction mechanism, is more efficient. By being adaptable, the predictive model can deliver more accurate PM2.5 concentration forecasts.

Because of the space limitation of the paper, we only give the prediction results of nine models on the first observation point dataset, as shown in Figure 7. By comparing the observed values and prediction results of each model on the dataset, it can be seen that the prediction results of the NSKEA model are the most consistent with the observed values, and the accuracy rate is 95.6%. Accordingly, the proposed model significantly enhances the overall prediction performance.

4. Conclusions

Rapid global climate change has led to escalating concerns surrounding air pollution, with adverse effects increasingly infringing upon daily life. As awareness of environmental conditions grows, so too does the demand for improved air quality. This increased societal pressure makes the urgent task of meticulous pollution prevention and control management necessary. Simultaneously, while sustaining rapid economic development, minimizing industrialization’s environmental and climatic impact has emerged as a shared objective sought by global academicians. Therefore, the scientific, accurate monitoring of air quality and pollutant concentrations, alongside understanding the pollution variation laws and environmental impacts of air pollution severity, offers strategic advantages. This knowledge promotes precisely guided pollution control measures and is critical for fostering healthy urban development.

This manuscript introduces a multifactorial model predicting PM2.5 concentration levels within the atmosphere. The described method integrates feature selection, clustering, and ensemble learning techniques to deep-mine original dataset in-house features, thereby augmenting the model’s predictive precision. Key findings from the experimental outcomes highlight:

(1)

The model outlined herein enhances PM2.5 concentration prediction accuracy. Demonstrating significant adaptability, the NSKEA model capably mines data, evident in its performance amidst PM2.5 seasonal fluctuations.

(2)

Implementing multi-objective optimization for multi-factor feature selection supports enhanced diversity preservation, consequential in advancing the predictive model’s precision.

(3)

The study employed the ELM as a weak learner without considering variations in the prediction model in light of diverse basic learners. Future research will explore this area further, focusing on identifying the optimal basic learner to enhance the robustness and accuracy of the integrated predictive model.