Transient Emissions Forecasting of Off-Road Construction Machinery Based on Long Short-Term Memory Network

Abstract

Off-road machinery is one of the significant contributors to air pollution due to its large quantity. In this study, a deep learning model was developed to predict the transient engine emissions of CO, NO, NO₂, and NO_x, which are the main pollutants emitted by off-road machinery. A portable emission measurement system (PEMS) was used to measure the exhaust emission features of four types of construction machinery. The raw PEMS data were preprocessed using data compensation, local linear regression, and normalization to ensure that the data could handle transient conditions. The proposed model utilizes the preprocessing PEMS data to estimate the CO, NO, NO₂, and NO_x emissions from off-road machinery using a recurrent neural network (RNN) based on a long short-term memory (LSTM) model. The experimental results show that the proposed method can effectively predict the emissions from off-road construction machinery under transient conditions and can be applied to controlling the emissions from off-road construction machinery.

1. Introduction

Off-road construction machinery plays an irreplaceable role in infrastructure construction, land development, and agricultural production [1]. However, the accompanying increase in emissions from off-road machinery has exacerbated the environmental impacts, particularly the impact of atmospheric pollution [2,3]. Vehicle exhaust emissions have long been recognized as a significant factor affecting air quality, with the increasing proportion of off-road machinery emissions in the global vehicle emission total drawing heightened attention [4]. Therefore, the accurate prediction and control of transient emissions from off-road construction machinery have become urgent requirements for current environmental protection and climate change mitigation efforts.

Off-road machinery encompasses construction machinery, agricultural machinery, etc., with their emission of hydrocarbons (HC), carbon monoxide (CO), nitrogen oxides (NOₓ), and particulate matter (PM) accounting for 18–29% of total global vehicle emissions [5]. Statistics indicate that the emissions from construction machinery constitute 28.2%, 31.3%, and 32.5% of the total emissions from off-road machinery [6]. Among these pollutants, nitrogen oxides are significant atmospheric pollutants with crucial implications for ozone layer depletion and acid rain formation [7,8]. Studies have shown that the emissions of off-road mobile machinery, such as excavators, loaders, and forklifts, increase with engine power, with diesel engine emissions accumulating the highest amount of exhaust pollutants [9,10]. Hence, establishing a comprehensive evaluation and control system for off-road mobile machinery emissions that can reduce the overall pollutant emissions and improve air quality has become a pressing environmental concern.

In general, the methods used to measure vehicle emissions include chassis and engine dynamometer testing [11], road tunnel measurements [12], optical remote sensing systems [13], plume chasing measurements [14], and portable emission measurement systems (PEMSs) [15]. In recent years, PEMSs have been utilized to measure the emission characteristics of off-road mobile machinery, proving to be an effective method for characterizing the actual emissions levels [16,17,18]. To accurately assess the pollution emissions from mobile machinery on actual roads, researchers have proposed various deep learning models for predicting mobile machinery emissions. Singh and Dubey [19] proposed an LSTM-based CO₂ emission prediction model that can utilize vehicle onboard diagnostic (OBD) port data for the remote emission monitoring of vehicles. Yu et al. [20] introduced a method for the accurate prediction of nitrogen oxide emissions from diesel vehicles using adaptive noise (CEEMDAN) and long short-term memory (LSTM) networks. Shin et al. [21] presented an LSTM-based emission prediction model for the accurate forecasting of transient NOx concentrations in diesel engines. Zhang et al. [22] developed a time series model based on wavelet transform and LSTM networks to predict engine emissions, including CO, HC, and NO emissions. Wang et al. [5] utilized a PEMS to measure and analyze the exhaust pollutant emission characteristics of engineering machinery under different operating conditions. Zhang et al. [23] utilized PEMS and global positioning system (GPS) data to investigate vehicle emission characteristics and developed an LSTM-based model for predicting carbon dioxide emissions. Xie et al. [24] proposed a parallel attention-based LSTM emission prediction model integrating PEMS and OBD systems.

The role of PEMS in assessing vehicle emissions is critical, but there are some challenges with their practical application. Firstly, the datasets collected by PEMS can be affected by outliers and zero-level offsets, especially over long sampling periods [24], which can significantly affect the accuracy and reliability of the emission data. Secondly, the high cost of PEMS has limited its accessibility to research institutions. To the best of our knowledge, research on the exhaust emissions of off-road machinery has not been fully carried out. Moreover, traditional models often lack the ability to learn from time series data, as they do not retain past information, thus limiting their predictive power for long-term time series data.

To effectively reduce emissions from off-road machinery, this study analyzes the relationship between variations in the external operating conditions of off-road construction machinery and changes in the emission characteristics. It also explores suitable emission prediction methods and effective emission reduction measures for different types of off-road mobile machinery under transient conditions. Based on LSTM network models, a predictive method tailored to the transient emission characteristics of off-road construction machinery has been developed. By integrating comprehensive data preprocessing, including data compensation and local linear regression, with LSTM predictive models, this study can enhance the accuracy of predicting vehicular emissions, particularly for off-road machinery. This research may provide new insights for controlling emissions from off-road construction machinery and support environmental monitoring and informed policy making to mitigate the environmental footprint of off-road machinery.

2. Materials and Methods

2.1. Recurrent Neural Network

Recurrent neural networks (RNNs) are a class of deep neural network designed for processing sequential data. They have been widely utilized in various fields, such as natural language processing, image recognition, and time series forecasting [25,26,27]. Unlike conventional feedforward neural networks, RNNs are capable of retaining and utilizing past information when processing sequential data, making them well suited to handling temporally dependent data. An RNN is a fully connected neural network with self-recurrent feedback, as illustrated in Figure 1.

2.2. Long Short-Term Memory Network

Conventional RNNs have limited ability to capture long-term dependencies due to issues with vanishing or exploding gradients during training lengthy sequences. To address this problem, the long short-term memory network (LSTM) was developed as a specialized type of RNN [28,29]. The LSTM has been widely used for processing sequential data and has found application in various domains of vehicle emission prediction. As shown in Figure 2, the LSTM is equipped with three unique “gate” structures within the neurons. These “gate” structures enable the addition and removal of information, facilitating the flow of states for long-term memory retention. Each LSTM cell contains three “gate” structures: input gate i_t, forget gate f_t, and output gate o_t to manage the flow of information.

The forget gate f_t determines the information to be discarded and the information to be retained using a sigmoid function that takes the input x_t, output from the previous unit h_t−1, and internal state outputs. It can be presented by

$$f_t=\sigma (W_f{x}_t+U_f{h}_{t-1}+V_f{c}_{t-1}+b_f)$$

(1)

The input gate i_t also uses a sigmoid layer to decide the new information to be added to the cell, and the input g_t creates a new state set. The process can be presented by

$$i_t=\sigma (W_i{x}_t+U_i{h}_{t-1}+V_i{c}_{t-1}+b_i),$$

(2)

$$g_t=\tanh (W_c{x}_t+U_c{h}_{t-1}+b_c).$$

(3)

The output gate o_t, determined by a sigmoid layer, produces the cell output h_t. The process can be presented by

$$o_t=\sigma (W_o{x}_t+U_o{h}_{t-1}+V_o{C}_t+b_o),$$

(4)

$$h_t=o_t\otimes \tanh (c_t),$$

(5)

where W, U, and V are the parameter matrices, b is the bias, $\otimes$ is the element-wise multiplication, σ(·) and tanh(·) are the activation functions, and the subscript t denotes the cell state at time t. Thus, the LSTM is designed to retain useful information for a long time while discarding irrelevant information quickly.

2.3. Dropout for Neural Networks

Overfitting is an important problem in multilayer LSTM models. Multilayer LSTM model training and learning is essentially optimization and generalization. Debugging the multilayer LSTM model parameters minimizes the loss function value in the training sample set. The performance of a fully trained predictive model in the test sample set reflects the model’s generalization capabilities. Overfitting can affect the training model’s performance and thus reduce generalization. To prevent models such as multilayer LSTM neural networks from overfitting the training sample set, regularization techniques are adopted to limit the complexity of the model. A regularization term for the model parameters is added to the objective function. The dropout method is one of the most effective regularization tools for deep learning [30], and the standard network and the dropout-based network are shown in Figure 3.

2.4. Loss Function and Optimization

The loss function is a mathematical tool employed to measure the difference between the output value of the neural network and the actual value, which is essential for evaluating the accuracy of the model’s predictions. The specific formulation of the loss function for LSTM is presented as follows:

$$F_{loss}=\sqrt{\frac{1}{m}{{\displaystyle \sum _{t=1}^m\left[\tilde{y}(t)-y(t)\right]}}^2},$$

(6)

where $\tilde{y}(t)$ is the predicted value, and y(t) is the measured value.

The optimization of the LSTM training process is to obtain the system parameters that can minimize the loss function by iteration. Neural network optimizers are tools used to minimize the loss function by adjusting parameters. Common optimizers include Stochastic Gradient Descent (SGD), momentum optimizer, the Adaptive Gradient Algorithm (AdaGrad), Root Mean Square prop (RMSprop), and Adaptive Moment Estimation (Adam). The Adam optimizer combines momentum and RMSprop, providing adaptive learning rates and momentum adjustments. It adjusts the learning rates for each parameter by computing first- and second-moment estimates of the gradient. Adam converges faster and demonstrates good generalization, making it widely used in practice.

3. Data Preprocessing

Emissions data preprocessing was used to eliminate the data anomalies caused by equipment malfunctions and sensor anomalies. In addition, it helps to make the data more standardized, which is essential for the study of emission prediction models for construction machinery.

3.1. Data Compensation

Due to sensor failures or partial data loss during monitoring, the recorded emission data in the monitoring system are incomplete. Some data are lost during the data transmission and updating process. Therefore, it is necessary to compensate for the missing data before analyzing the emission data. Compensating for incomplete data records could significantly enhance the prediction performance. The process of compensating for missing data mainly consists of three steps:

(1)

Perform data inspection on the original data sequence L to identify the positions of missing data points (x_i, y_i).

(2)

Select the key data points of (x₁, y₁) and (x₂, y₂) nearest to the missing value (x_i, y_i) from the data sequence L, and perform linear interpolation using these two points. Subsequently, add this interpolated data point (x’_i, y’_i) to the data sequence L to replace (x_i, y_i), resulting in an updated dataset L’. The coordinates of the updated data points are as follows:

$$x_i^{\prime }=x_i,$$

(7)

$$y_i^{\prime }=y_1+\frac{y_2-y_1}{x_2-x_1}\left(x_i-x_1\right),$$

(8)

(3)

Repeat steps (1) and (2) until there are no incomplete data points in data sequence L.

3.2. Outlier Detection

The emissions data of off-road construction machinery typically exhibit strong non-stationarity, which may stem from various factors such as varying operating conditions, mechanical wear and tear, differences in fuel quality, maintenance practices, and environmental influences. In addition, frequent acceleration, deceleration, and aerodynamics often put the engine in transient operating conditions. Due to the non-stationarity of the emissions data, selecting appropriate outlier detection methods is crucial. Techniques like local linear regression can better adapt to the non-stationarity of the data, thereby enhancing the accuracy and reliability of outlier detection.

Local linear regression is a non-parametric regression method that, in making predictions, takes into account the local data points surrounding each data point to fit a local linear model. This approach enables the more effective capture of the local characteristics and nonlinear relationships present in the data. Quadratic local linear regression is a variant of local linear regression that fits a quadratic polynomial model around local data points to capture the nonlinear relationships in the data and improve the data quality and analysis reliability. The model for quadratic local linear regression can be presented as follows:

$${\widehat{y}}_i ={\beta }_0(x_i)+{\beta }_1(x_i)(x-x_i)+{\beta }_2(x_i){(x-x_i)}^2+{ϵ}_i,$$

(9)

where ${\widehat{y}}_i$ is the response variable of the i-th data point, x_i is the predictor variable of the i-th data point, β₀(x_i), β₁(x_i), and β₂(x_i) are the local quadratic linear coefficients at x_i, and ϵ_i is the error term.

The objective of quadratic local linear regression is to minimize the following local weighted square error:

$$min{\displaystyle {\sum }_{i=1}^n{w}_i{(y_i-{\beta }_0(x_i)-{\beta }_1(x_i)(x-x_i)-{\beta }_2(x_i){(x-x_i)}^2)}^2},$$

(10)

where w_i is a weight function that adjusts the influence of each data point during the fitting process, which can be defined using Gaussian or triangular kernel functions. After the iterative parameter adjustments, the optimal fitting model is derived for prediction and analysis.

3.3. Normalization

The emission data were normalized to ensure that all features were on the same scale. Normalization ensures smooth training and not performing normalization might also cause a vanishing gradient problem while training out the LSTM Network. Min–max normalization has been used for normalizing the data. The formula for min–max normalization is as follows:

$$y_i^j=\frac{x_i^j-\min (x)}{\max (x)-\min (x)},$$

(11)

where $x_i^j$ is the j-th feature value of the i-th input sample and $y_i^j$ is the corresponding normalized value.

4. LSTM-Based Emissions Forecasting Model

The transient emission concentration from off-road construction machinery can vary significantly due to the dynamic and unpredictable nature of construction sites. Factors such as load variations, speed changes, and idling periods contribute to irregular patterns and sudden changes in emission concentrations, making accurate prediction challenging. The complexity of predicting emissions also lies in capturing the intricate nonlinear relationships among factors like the engine load, operating conditions, fuel quality, and environmental influences. The non-stationary nature of emission data further complicates prediction efforts, as traditional models may struggle to adapt to the rapid changes and irregular patterns inherent in off-road machinery emissions.

Deep learning techniques for predicting emission concentrations from off-road machinery offer notable advantages over traditional methods. Deep learning models excel in automatically learning complex features, capturing nonlinear relationships, handling temporal dynamics, scaling with data size, and adapting to changing patterns. These capabilities enable deep learning models to effectively capture the dynamic and non-stationary nature of emission concentration variations, facilitating the accurate predictions crucial for enhancing environmental monitoring and management practices in off-road settings.

The emission prediction model for off-road construction machinery is shown in Figure 4. The LSTM network used in this research includes one input layer, n hidden layers, and one fully connected output layer. To improve the prediction accuracy, the Adam optimization algorithm is implemented to optimize the LSTM model. Moreover, dropouts are set after each hidden layer to prevent over-fitting. The activation functions of the hidden layer are the sigmoid and tanh functions, and the activation function of the output layer is the linear function.

5. Experimental Results and Discussion

5.1. Data Collection and Setup

Portable emissions measurement systems (PEMSs) can evaluate engine emissions by conducting real-time tests and provide instantaneous emissions, fuel consumption, vehicle speed, temperature, and other parameters based on the equipment capabilities. The data obtained from the analysis of engine exhaust gases by various analyzers within PEMS, in conjunction with environmental and GPS parameters, are input into the data acquisition system and ultimately recorded and stored in a PC. The process of exhaust gas sampling and data processing by PEMS is shown in Figure 5.

An off-road construction machinery dataset was collected and compiled to train and evaluate the proposed model. The dataset was created using PEMS data collected from 19 pieces of off-road machinery in four different types of construction vehicles. The datasets were divided into training and test sets in a 6:2:2 ratio, as shown in

Table 1. Experimental datasets.

Datasets	Size
	Training	Validation	Test	Total
Forklift	5794	1931	1931	9656
Loader	5629	1877	1877	9383
Tractor	4594	1531	1531	7656
Excavator	4415	1472	1472	7359

. The ten input features, which are F₁, F₂, …, and F₁₀, and four output features, which are F₁₁, F₁₂, F₁₃, and F₁₄, are given in

Table 2. Input data and output data of the LSTM model.

	Symbol (Units)	Description
Input data	F1 (km/h)	Vehicle speed
	F2 (°C)	Ambient temperature
	F3 (%)	Ambient relative humidity
	F4 (kPa)	Atmospheric pressure
	F5 (m3/min)	Emission flow rate
	F6 (°C)	Emission temperature
	F7 (Pa)	Emission pressure differential
	F8 (°C)	Analyzer temperature
	F9 (°C)	Exhaust pipe ambient temperature
	F10 (%)	Exhaust pipe ambient absolute humidity
Output data	F11 (%)	CO
	F12 (ppm)	NO
	F13 (ppm)	NO2
	F14 (ppm)	NOx

. The LSTM-based emission prediction model was developed using MATLAB R2022b, and detailed information on the experimental environment is shown in

Table 3. Experimental environment.

Parameters	Value
CPU	Inter(R) Core (TM) i7-10875H CPU @ 2.30 GHz
GPU	NVIDIA GeForce RTX 2060
Number of GPU cards	1
Memory (RAM, VRAM)	16.0, 6.0 GB

.

5.2. Correlation Analysis

The input features of the time series are expected to be highly correlated with the output results, while the multiple input features are expected to have a low correlation with each other. The absolute value of the Pearson correlation coefficient (|r|) is used to show the correlation between the input and output features. These correlation results are shown in Figure 6. As depicted in Figure 6, the autocorrelation coefficient of each feature is 1.

5.3. Parameter Settings and Evaluation Metrics

The performance and complexity of an LSTM model are directly affected by its hyperparameters, as shown in

Table 4. Parameter setting of the LSTM model.

LSTM Parameters	Value
Initial learning rate	0.001
Learning rate drop period	125
Learning rate drop factor	0.2
Validation frequency	5
Minimum batch size	10
Maximum epochs	500
Gradient threshold	1
Number of hidden layers	2
Number of hidden units	256
Dropout rate	0.2

. These hyperparameters include the initial learning rate, minimum batch size, number of hidden layers, number of hidden units, and validation frequency. The batch size and the number of learning epochs influence the efficiency of model training. A thorough search using training and validation data is performed to ensure that the hidden layers can capture intricate details and align with the computational capabilities of the system. An LSTM model with three hidden layers and 256 neurons in each layer is found to be the optimal architecture. After evaluating various learning rates and monitoring changes in the iteration times and loss, a learning rate of 0.001 is selected. In the hyperparameter optimization process, the number of epochs is set to 500 to enhance accuracy. A dropout rate of 0.2 in each hidden layer is added to the LSTM model to ensure that the nodes are trained equally.

To assess the effect of the proposed method on predicting off-road machinery emissions, three distinct quantitative evaluation metrics were employed: the root mean squared error (RMSE), mean absolute error (MAE), and mean absolute percentage error (MAPE). These metrics were calculated as follows:

$$\mathrm{RMSE}=\sqrt{\frac{1}{N}{\displaystyle \sum _{t=1}^N{[y(t)- \hat{\mathrm{y}}(t)]}^2}},$$

(12)

$$\mathrm{MAE}=\frac{1}{N}{\displaystyle \sum _{t=1}^N\left|y(t)-\widehat{y}(t)\right|},$$

(13)

$$\mathrm{MAPE}=\frac{1}{N}{\displaystyle \sum _{t=1}^N\left|\frac{y(t)-\widehat{y}(t)}{y(t)}\right|\times 100\%}.$$

(14)

where y(t) is the true value, $\widehat{y}(t)$ is the predicted value, and N is the number of samples for the test set.

5.4. Emissions Prediction for Four Types of Construction Machinery

The PEMS test data from four different types of off-road construction vehicles were utilized to train and analyze using the method presented in this paper. Figure 7 shows the training and validation losses of the proposed model with a loss function. The LSTM models for forklifts, loaders, tractors, and forklifts reached their minimum training losses at the 50th, 40th, 60th, and 40th epoch, respectively, while achieving their minimum validation losses at the 40th, 40th, 60th, and 60th epoch, respectively. These results indicate that the final models were not over-fitted, and their accuracies were optimized.

The LSTM neural network model trained on the training set was evaluated for its predictive accuracy on the test set, as depicted in Figure 8, Figure 9, Figure 10 and Figure 11. These figures illustrate comparisons between the experimental and predicted results for the four types of off-road construction machinery in the testing set.

Figure 8 shows the analysis results of the LSTM model on the forklift test set, demonstrating the model’s ability to effectively forecast the emission concentrations of CO, NO, NO₂, and NO_X under transient conditions with high prediction accuracy. Figure 9 and Figure 10 present the LSTM model predictions of the CO, NO, NO₂, and NO_X emission concentrations on the loader and tractor test sets. It is noteworthy that the proposed model exhibits a certain level of predictive accuracy even for scenarios with excessively rapid transient changes, albeit with a slightly reduced accuracy compared to the other three types of off-road machinery, as shown in the analysis results for the excavator in Figure 11. This phenomenon is understandable as LSTM networks may struggle with capturing long-term dependencies when dealing with non-stationary data, given the sudden fluctuations in and volatility of the data. The results indicate the effectiveness of the proposed LSTM emission prediction model for off-road machinery under various operating conditions, and its ability to provide accurate predictions even in the presence of challenging transient conditions.

Table 5. Performance indicators of the LSMT emission prediction model.

Emissions		RMSE	MAE	MAPE
Forklift	CO	0.0017	0.0014	0.1541
	NO	143.6624	111.4938	0.6676
	NO2	35.8758	25.1891	3.4717
	NOx	164.1304	131.6780	0.5866
Loader	CO	376.9817	318.8271	54.8420
	NO	56.3650	41.8254	0.2550
	NO2	50.2226	37.9645	0.2319
	NOx	87.2692	65.7238	0.1958
Tractor	CO	0.0023	0.0018	0.3149
	NO	245.4075	199.6804	7.3622
	NO2	152.0607	124.3559	48.3758
	NOx	371.7461	291.2663	5.9550
Excavator	CO	0.0030	0.0020	0.07600
	NO	46.7109	36.2914	0.3589
	NO2	28.6745	21.1496	0.2393
	NOx	66.9997	50.6744	0.2456

shows the performance indicators of the proposed LSTM emission prediction model. It can be noticed that the loader has the highest RMSE, while the forklift has the lowest RMSE for CO, NO, NO₂, and NO_x emission forecasting. When forecasting the NO emissions of the four off-road construction machinery, the forklift has a high RMSE of 143.6624, much higher than the 56.3650 of the loader. Table 5 also confirms the prediction accuracy of the LSTM model in terms of MAE and MAPE. Considering the three indicators of RMSE, MAE, and MAPE comprehensively, it can be observed that the proposed model performs the best when predicting the emissions for loaders and the worst when predicting the emissions for tractors. From the trend in the emissions data for tractors, it can be inferred that the reason for this may be that the concentration of emissions fluctuates significantly. Overall, the proposed LSTM-based emission prediction model can reasonably forecast the exhaust emission concentrations of various types of off-road construction machinery.

5.5. Discussion

In

Table 6. Comparison of different emission forecasting methods.

References	Vehicle Type	Deep Learning Models	Data Acquisition	Forecasting Time (min)	Forecasted Emissions
Singh et al. [19]	On-road	LSTM	OBD	400	CO2
Yu et al. [20]	On-road	LSTM	OBD	33	NOX
Shin et al. [21]	N/A	DNN & LSTM	Fast analyzer	30	NOX
Zhang et al. [22]	On-road	Wavelet + LSTM	Monitoring System	180	CO, HC, NO
Zhang et al. [23]	On-road	LSTM	PEMS	20	CO2
Xie et al. [24]	N/A	LSTM	PEMS + OBD	175	CO, NO, NO2, THC
This study	Off-road	LSTM	PEMS	30	CO, NO, NO2, NOX

, the proposed method is compared with other approaches. It can be seen that by employing an LSTM model and leveraging PEMS data for off-road vehicles, the current method not only provides a long period of 30 min of prediction for a variety of pollutants such as CO, NO, NO₂, and NO_X, but also addresses the unique challenges associated with off-road environments. The findings underscore the potential of the proposed method to offer robust and actionable insights for emission control and policy making in sectors that extensively use off-road machinery.

It should also be noted that the vibration of construction machinery significantly contributes to dust dispersion. The proposed LSTM model can be further enhanced to explore the mechanisms of dust generation and improve the model’s predictive accuracy.

6. Conclusions

A long short-term memory network model is proposed to predict the emission of CO, NO, NO₂, and NO_x from four kinds of off-road construction machinery. The model considers nine input features, which are the vehicle speed, ambient temperature, ambient relative humidity, emission flow rate, emission temperature, and emission pressure differential. The predicted results of the model are CO, NO, NO₂, and NO_x. When some emission data samples are incomplete, a method based on linear interpolation is used to compensate for the missing data. Local linear regression is applied to smooth the raw emission time series, which exhibit strong non-stationary characteristics, and to detect outliers. For each outlier, a local linear fit is used to update the database. The preprocessed data are then input into the proposed LSTM model to obtain the emission concentrations of CO, NO, NO₂, and NO_x. The emission data measured using a portable emission measurement system are used to train the proposed LSTM-based emission prediction model, and the predicted results are compared with the experimental results of four types of off-road construction machinery, including a forklift, loader, tractor, and excavator. The experimental results show that the proposed LSTM-based model can predict the emission concentrations of different off-road construction machinery under transient operating conditions.