LSTM for Modeling of Cylinder Pressure in HCCI Engines at Different Intake Temperatures via Time-Series Prediction

Abstract

Modeling engines using physics-based approaches is a traditional and widely-accepted method for predicting in-cylinder pressure and the start of combustion (SOC). However, developing such intricate models typically demands significant effort, time, and knowledge about the underlying physical processes. In contrast, machine learning techniques have demonstrated their potential for building models that are not only rapidly developed but also efficient. In this study, we employ a machine learning approach to predict the cylinder pressure of a homogeneous charge compression ignition (HCCI) engine. We utilize a long short-term memory (LSTM) based machine learning model and compare its performance against a fully connected neural network model, which has been employed in previous research. The LSTM model’s results are evaluated against experimental data, yielding a mean absolute error of 0.37 and a mean squared error of 0.20. The cylinder pressure prediction is presented as a time series, expanding upon prior work that focused on predicting pressure at discrete points in time. Our findings indicate that the LSTM method can accurately predict the cylinder pressure of HCCI engines up to 256 time steps ahead.

1. Introduction

Model-based engine studies offer significant time and cost savings in enhancing engine performance. Traditionally, physics-based models have been employed to represent the physical processes of engines, enabling the prediction of cylinder characteristics [1,2,3,4]. These physics-based models are also utilized for control design purposes [5,6,7]. In addition, computational fluid dynamics based studies have been conducted to predict cylinder pressure [8,9].

Recently, artificial intelligence technologies, such as machine learning-based models, have emerged as a popular alternative for predicting cylinder pressure [10,11]. Unlike physics-based models, machine learning models do not require knowledge of the underlying physical processes due to their data-driven nature [12]. In general, deep learning is a subset of machine learning techniques based on neural networks. Deep learning involves training neural networks with more than three hidden layers [13]. Recently, deep learning methods have gained popularity as a research topic within the machine learning field [14]. These techniques are considered efficient and suitable tools for analyzing complex and non-linear systems such as engines [15].

Mohammad et al. [16] developed a machine learning-based emission model for compression ignition engines. Liu et al. [17] proposed four different machine learning techniques—artificial neural network (ANN), support vector regression (SVR), random forest (RF), and gradient boosted regression trees (GBRT)—to model an ammonia-fueled engine. The study found that the ANN and SVR algorithms predicted IMEP more accurately than RF and GBRT; however, ANN was the most appropriate algorithm when data noise was randomly distributed.

In a comparative study between a wavelet neural network (WNN) with stochastic gradient algorithm (SGA) and an ANN with non-linear autoregressive with exogenous input (NARX), both techniques were used to investigate engine performance and emissions [18]. The WNN with SGA demonstrated better results than the ANN-NARX combination. In the context of HCCI engines, a multi-objective optimization algorithm based on genetic algorithms was used to examine various performance parameters [19]. Another study proposed regression models based on machine learning techniques to predict the start of combustion in methane-fueled HCCI engines [20].

Shamsudheen et al. [21] investigated two machine learning techniques, K-nearest neighbors (KNN) and support vector machines (SVM), to classify combustion events in an HCCI engine. The study found that SVM achieved a 93.5% classification accuracy, compared to KNN’s 89.2% accuracy. Vijay et al. [22] used principal component analysis and neural networks with black box models to predict the non-linear combustion behavior during transient operation in an HCCI gasoline engine. The results indicated that neural networks are a powerful tool for predicting the non-linear behavior of HCCI engines. In another study, a combination of machine learning techniques, including random forest, ANNs, and physics-derived models, was employed to predict combustion performance parameters such as SOC, CA${}_{50}$, and indicated mean effective pressure (IMEP) for a reactivity controlled compression ignition engine [23]. Zhi-Hao et al. [24] conducted a study on the progress and development of machine learning-based analysis in combustion studies across various fields, industries, and engines. Luján et al. [25] predicted the volumetric efficiency of a turbocharged diesel engine using an adapted learning algorithm with increased learning speed and reduced computational effort.

In this paper, we develop a machine learning model based on the Long Short-Term Memory (LSTM) architecture to predict cylinder pressure at different intake temperatures for an HCCI engine. Unlike previous work, the present study conducts pressure prediction in HCCI engines as a time series forecast using an LSTM model. Our results demonstrate that the LSTM model can better handle time correlations for cylinder pressure predictions compared to the DNN-based prediction proposed by Yacsar et al. [10]. The mean average error (MAE) for the LSTM-based model is 0.37, lower than the DNN-based model for the data considering different intake temperatures. The mean squared error (MSE) for the LSTM-based model for the same data is below 0.20, while the DNN model’s MAE and MSE are 0.46 and 0.29, respectively.

The rest of the paper is organized as follows: Section 2 introduces the setup and methodology. Section 3 presents the results and discussion. Finally, Section 4 concludes the paper.

2. Setup & Methodology

2.1. Experimental Setup

Figure 1 presents the schematic diagram of the experimental setup. The setup includes a 500 cc single-cylinder, 4-stroke, port-injected gasoline engine manufactured by the Taiwan-based company CEC. The engine specifications are provided in

Table 1. Engine Specifications.

Parameters	Value	Unit
Bore	9	$\mathrm{cm}$
Stroke	9.8	$\mathrm{cm}$
Crankshaft length	3.9	$\mathrm{cm}$
Compression ratio	13	-
Connecting rod length	12.6	$\mathrm{cm}$
Intake valve open	5	$°$$\mathrm{aBDC}$
Intake valve close	58	$°$$\mathrm{aBDC}$
Exhaust valve open	47	$°$$\mathrm{bBDC}$
Exhaust valve close	11	$°$$\mathrm{aTDC}$

. An air heater (Leister LHS21L) is used to preheat the intake air entering the cylinder. K-type thermocouples are installed at the intake and exhaust manifolds to measure temperature. An intake manifold pressure sensor (SP–Ax102–B) is utilized to measure the pressure within the intake manifold.

To measure in-cylinder pressure, an AVL spark plug sensor, ZI21U3CPRT, is used in conjunction with a charge amplifier (KISTLER 5011B). The volumetric flow rate of the intake air is recorded using a Schmidt flow sensor SS 30.301, with a measuring range of up to 229 N · $\mathrm{m}$³/hour. The test engine is coupled to an FR 50 eddy current dynamometer with a torque rating of 150 N · $\mathrm{m}$ and a power rating of 40 kW. An angle encoder (HENGSTLER incremental encoder RI38) is installed to measure the crank angle, with a maximum speed limit of 10,000 rpm. The engine is fueled with 92 AKI unleaded gasoline. Figure 2 illustrates the pressure curve within one cycle, along with the start of combustion timing.

2.2. Long Short-Term Memory (LSTM)

First introduced by Hochreiter and Schmidhuber in 1997 [26], LSTMs have proven to be powerful tools for predicting certain outputs from given input data in various domains, such as natural language processing [27], event prediction [28], traffic forecasting [29], and stock market prediction [30]. Recently, LSTMs have also found applications in engine failure prediction. Aydin and Guldamlasioglu [31] collect sensor data to predict engine lifetimes and determine suitable times for engine maintenance. Yuan et al. [32] apply LSTMs to aero engines for performance prediction, testing different LSTM variations on a health monitoring dataset of aircraft turbofan engines provided by NASA. The standard LSTM outperforms other variations.

LSTMs set themselves apart from other machine learning methods due to their memory component, making them particularly useful for time series prediction [33]. LSTMs improve upon other structures, such as recurrent neural networks, by introducing gate functions into the cell [34]. This feature enables them to handle long-term relationships effectively. LSTMs can use inputs from different time steps and employ these inputs for their predictions. When applied to the domain of engine data, LSTMs have previously been used to predict engine failure.

LSTM architectures can vary greatly. A basic building unit of an LSTM is shown in Figure 3. $x_t$ represents the input vector, and $h_t$ denotes the output prediction. Yellow rectangles represent neural network layers, while red circles symbolize pointwise operations, with x standing for multiplication and + representing addition. Black arrows indicate information flow, which can be concatenated in the case of merging vectors or copied in the case of diverging vectors.

Figure 3 includes four interacting neural network layers, represented by yellow rectangles. They can be further classified into three sigmoid (abbreviated as $\sigma$ in the figure) activation layers and one $tanh$ activation layer. The sigmoid gates output a value between 0 and 1, while the $tanh$ activation function outputs a value between −1 and 1. The gates are also known as the forget, input, and output gates [35]. Each gate has a unique function. The leftmost sigmoid layer is the forget gate, which determines how much of the previous state should be retained. The gate can be described with Equation (1).

$$f_t=\sigma (W_f[h_{t-1},x_t]+b_f)$$

(1)

where $b_f$ describes the bias at the forget gate.

The next two layers, sigmoid and $tanh$, form the input gate, which decides about the new information that should be added to the current cell state. Equation (2) presents the sigmoid part of the input gate, while the $tanh$ layer is described in Equation (3). When multiplied together, they are added to the cell state created from the multiplication of the forget gate output $f_t$ and the cell state $C_{t-1}$ in Equation (4) to update the current cell state.

$$i_t=\sigma (W_i[h_{t-1},x_t]+b_i)$$

(2)

$$C_t=tan(W_c[h_{t-1},x_t]+b_c)$$

(3)

$$C_t=f_t\ast C_{t-1}+i_t\ast C_t$$

(4)

Finally, the fourth gate, the output gate, filters the cell state and generates the output. The filter value is created by Equation (7). The filter value is then multiplied with the output of the $tanh$ operation from the current cell state $C_t$, as seen in Equation (8). The horizontal line at the top represents the cell state.

$$o_t=\sigma (W_o[h_{t-1},x_t]+b_o)$$

(5)

$$h_t=o_t\ast tanh\left(C_t\right)$$

(6)

In summary, the LSTM cell uses forget, input, and output gates to determine how to update its internal cell state and generate output predictions. By handling long-term dependencies and time correlations effectively, LSTMs have been successfully applied to various domains, including engine performance prediction and engine failure prediction [31,32,36].

$$o_t=\sigma (W_o[h_{t-1},x_t]+b_o),$$

(7)

$$h_t=o_t\ast tanh\left(C_t\right)$$

(8)

Equations (7) and (8) describe the output gate of an LSTM cell. $\sigma$ represents the sigmoid function $\frac{1}{1+e^{-x}}$, $tanh$ is the tanh function $\frac{e^x-e^{-x}}{e^x+e^{-x}}$. $W_f$, $W_t$, $W_c$, $W_{\sigma }$ are the weights of the neural network layer and $b_f$, $b_i$, $b_c$, $b_o$ are the bias vectors. The output gate determines the information that should be output from the cell state. This is done by applying the sigmoid function to the input data and previous hidden state, followed by element-wise multiplication with the tanh of the current cell state.

The performance of the LSTM model is measured using mean average error (MAE) and mean squared error (MSE), as described in Equations (9) and (10). n describes the number of sample points in the test set, $Y_i$ is the groundtruth and $\widehat{Y_i}$ is the prediction of the machine learning model. These metrics provide a quantifiable way to assess the accuracy of the model’s predictions.

$$\begin{array}{cc}\hfill MAE& =\frac{1}{n}\sum _{i=1}^n\mathrm{abs}(Y_i-\widehat{Y_i})\hfill \end{array}$$

(9)

$$\begin{array}{cc}\hfill MSE& =\frac{1}{n}\sum _{i=1}^n{(Y_i-\widehat{Y_i})}^2\hfill \end{array}$$

(10)

The pressure curve in this study has the characteristics of a time series, where observations are used to predict future pressure states over a defined time interval. More specifically, the input size of the LSTM represents the number of observations that the LSTM receives with the input vector $x_t$. The output size defined by the output vector $h_t$ of the LSTM is the prediction horizon. The prediction horizon specifies, how many time steps ahead with respect to the latest observation in the input vector $x_t$ are predicted. This makes the LSTM model an appropriate choice for predicting cylinder pressure in HCCI engines. By evaluating the model’s performance using the MAE and MSE metrics, we can assess the effectiveness of the LSTM model and its potential for use in similar applications.

3. Results & Discussion

In this section, the results of the study are organized into three main parts. First, different LSTM configurations are explored for predicting time-series data, including various combinations of input and output sizes and hidden layer sizes. Next, the performance of the LSTM model is compared to that of a DNN. Finally, pressure data points are predicted for future time steps at different intake temperatures.

The experiments are conducted using an 80/20 train/test protocol on a dataset with 1,048,576 measurement points. The first 80% of the measured data is used for training; the last 20% is used for testing. This 2-stage process is a common machine learning practice in most machine learning applications. The training set consists of 838,861 samples, while the test set has 209,715 samples. The training data is only used to train the neural network during the training phase. The hold-out test set is then used for evaluation purposes on unseen data to prove the ability of the neural network to cope with unseen data. The training process spans 10 epochs with MAE and MSE loss and a learning rate of 2 × ${10}^{-3}$. The loss function calculates the error between prediction and groundtruth. According to the loss, the weights in the neural network are updated until the loss does not change any more and the network converges during training. Convergence of the models is verified by monitoring the loss curve during training, as illustrated in Figure 4. After the convergence of the loss, the trained neural network can then further be used for inference on test data.

Adam optimization [37] is used during the training process. All experiments were performed on a machine with an Intel^® Core™ i7-8700K CPU at 3.70 GHz, 32.00 GB of random access memory, an NVIDIA GeForce GTX™1080 Ti GPU, and a Windows 10 operating system. The PyTorch framework [38] is used for implementing the models.

The results of these experiments will provide insights into the optimal LSTM configuration for predicting cylinder pressure in HCCI engines, as well as a comparison of the LSTM model’s performance to that of a DNN.

3.1. LSTM for Pressure Time-Series Prediction

In this initial analysis, the performance of the LSTM architecture is evaluated at different parameters, such as input and output size, hidden layer size, and input features like pressure, temperature, and crank angle. Different configurations of the input and output size of the network produce varying MAE and MSE values, as shown in

Table 3. Input and output sizes of the LSTM.

Input Size	Output Size	LSTM
		MAE	MSE
2	2	0.00136	1 × ${10}^{-5}$
4	2	0.00268	1 × ${10}^{-5}$
8	2	6.7 × ${10}^{-4}$	1 × ${10}^{-5}$
2	4	0.00218	7 × ${10}^{-5}$
4	4	0.002	1 × ${10}^{-5}$
8	4	8.8 × ${10}^{-4}$	6 × ${10}^{-5}$
2	8	0.00403	0.0001
4	8	0.00351	0.00017
8	8	0.00172	0.00017
2	16	0.00691	0.00021
4	16	0.00525	0.00025
8	16	0.003	0.00024
2	32	0.01043	0.00035
4	32	0.00822	0.00035
8	32	0.00429	0.00032
2	64	0.01708	0.00042
4	64	0.01012	0.00044
8	64	0.006	0.00037

and Figure 5. Several trends can be identified:

(a)

As the input size increases, the MAE and MSE values generally decrease. This trend can be attributed to the fact that a larger input size provides more information to the LSTM, enabling it to make better forecasts.

(b)

Conversely, as the output size increases, the MAE and MSE values tend to increase. This outcome occurs because a larger output size corresponds to predictions that are further away from the current time step. For example, when the output size is 64, the LSTM predicts pressure states more than 64 time steps away from the input pressure states. Predicting these distant pressure states accurately becomes more challenging due to the cycle-to-cycle variation of cylinder pressure data.

(c)

A comparison of MAE and MSE loss in Figure 6 reveals that the MSE is much smaller than the MAE. This observation indicates that most of the predictions are very close to the actual results, and there are not many outliers. This is due to the fact, that the MSE loss carries a square term, as can be seen in Equation (10). Therefore the MSE loss punishes outliers [39].

These findings help to understand how the LSTM architecture’s performance is affected by changes in input and output sizes, and they provide valuable insights for optimizing the model’s configurations for predicting cylinder pressure in HCCI engines. For minimal errors, the LSTM should be fed with the largest input size available. The more data the LSTM receives, the better the predictions. The prediction horizon can be of reasonable size, as long as the MAE is still in the expected and acceptable regime. However, bigger output sizes might lead to higher errors and less precise modeling of the engine pressure.

Table 4. MAE & MSE results for different hidden layers.

Hidden Layer Size	Criteria
	MAE	MSE
1	0.00664	0.00044
2	0.00597	0.00041
4	0.00551	0.0004
8	0.00644	0.00041
16	0.00566	0.00039
32	0.00543	0.00039
64	0.00612	0.0042

summarizes the MAE and MSE losses at different hidden layer sizes. The hidden layer with 32 neurons shows the best results compared to other cases. Consequently, 32 neurons have been used in the hidden layer of the LSTM in this study.

The LSTM’s ability to predict a reasonable number of future pressure states given the current pressure state $x_k$ to $x_{k-1}$ is illustrated in Figure 6. k represents the current time step and defines the time step of the latest input. Figure 7 displays the output size of the LSTM within the range of 2 to 256 neurons. Each neuron predicts a distinct time step. As mentioned earlier, the MAE increases with an increase in the output size. The MAE for 256 time steps prediction is below 0.02, given the pressure states from x to $x_{k-7}$.

In the absence of any benchmark for predicting pressure in an engine as a time series, the performance of this study must be evaluated in terms of how well it contributes to research in cylinder pressure modeling. The error in this study is comparable to previous studies [2,10]. To provide a more comprehensive evaluation of the proposed method, the subsequent results will compare it with previously used methods, such as DNN [10].

3.2. Comparison between LSTM and DNN

To evaluate the performance of the LSTM in comparison to the previous study based on DNN [10], experiments are conducted with both DNN and LSTM architectures to gain insight into both methods for the given task of pressure prediction in an HCCI engine. Yacsar et al. [10] use a fully connected neural network with the crank angle and excess air coefficient as input to predict the cylinder pressure as output. Their DNN consists of 3 hidden layers, having 100, 150, and 120 neurons, and employs the ReLU activation function. In the present study, the input is the cylinder pressure of previous time steps to predict the cylinder pressure of current and future time steps for the experiments. The network architecture in this study is compared with the architecture presented by Yacsar et al. [10].

In the present study, the LSTM-based model consists of one hidden layer with 32 neurons. The results can be seen in

Table 5. Results of comparison between DNN and LSTM.

No.	Window Size	DNN		LSTM
		Train	Test	Train	Test
1	1	0.00339	0.00243	0.00242	0.00242
2	2	0.00085	0.00074	0.00096	0.00091
3	4	0.00082	0.00109	0.0006	0.00053
4	8	0.00086	0.00077	0.00053	0.00042
5	16	0.00103	0.00072	0.00043	0.00093
6	32	0.00119	0.00131	0.00038	0.0005

and Figure 8, which depict the comparison between LSTM and DNN. The LSTM yields better results overall with just one hidden layer against the 4 hidden layers in the case of DNN, which results in saving memory and reduced training time. In the previous study, training was found to be optimal for 500 epochs, but in the present study, training for 10 epochs shows comparable results. This might be due to the fact that the size of an epoch can vary. The experiments show that the LSTM can perform better in the scenario of varying input sizes. Each experiment has a different number of input neurons, which can be defined as input size. For the LSTM, the MAE decreases with increasing window size except for the input size of 16. This demonstrates the LSTM’s ability to learn from time series data effectively. It can also be seen that the LSTM outperforms the DNN in 4 out of 6 cases, whereas the DNN only performs better for input sizes of 2 and 16. The DNN is not sensitive to an increasing window size. The test accuracy remains more or less constant between window sizes of 2 and 16. Only for a window size of 1, a much worse performance can be observed. The above comparison is based on the MAE and a constant temperature, whereas in the following experiments, the DNN and LSTM test results are presented at different intake temperatures.

Table 6. Comparison between DNN and LSTM on predicting pressure at different Temperatures.

No.	Dataset	DNN		LSTM
		MAE	MSE	MAE	MSE
1	data 1	0.02604	0.00477	0.025	0.00583
2	data 2	0.00962	0.00065	0.00914	0.00078
3	data 3	0.0096	0.00091	0.00779	0.00074
4	data 4	0.02707	0.01103	0.04769	0.02539

provides MAE and MSE values for different datasets as defined in

Table 2. Dataset at different temperatures.

Dataset	Temperature $\mathbf{(}\mathbf{°}\mathbf{C}\mathbf{)}$
data 1	235
data 2	245
data 3	255
data 4	265
data 5	235, 245, 255, 265

for both DNN and LSTM architectures. Additionally, the experiments investigate how the two architectures, i.e., DNN and LSTM, cope with different temperature and crank angle as input features. The resulting MAE and MSE are presented in

Table 7. Prediction characteristics for Pressure at different Temperature states.

No.	Input	Dataset	DNN		LSTM
			MAE	MSE	MAE	MSE
1	P, CA	data 5	0.46	0.29	0.37	0.20
2	P, CA, T	data 5	0.46	0.29	0.45	0.26

and Figure 9. Regarding the computational time given the hardware introduced earlier in Section 3, the LSTM requires 2.73 s of computation during inference on the test set; the DNN requires 3.05 s. The training time of the LSTM is 7 min and 33 s; the training time of the DNN is 6 min and 16 s. Comparing the allocated memory for both architectures, the LSTM occupies 81 Kilobytes of memory space; the DNN occupies 181 Kilobytes. Therefore, it can be said, that the proposed LSTM architecture outperforms the DNN architecture both in inference time and in memory efficiency, even though the training time for the LSTM takes longer.

3.3. Intake Temperature

In this section, the LSTM-based prediction of cylinder pressure at different intake temperatures has been presented. Experimental pressure data at different intake temperatures has been recorded and organized as shown in Table 2. The intake temperature is maintained at the desired value using feedback control command to the electrical air heating device introduced in Section 2. The data is then fed into the LSTM-based machine learning model. Pressure data at four different intake temperatures are presented from Figure 10, Figure 11, Figure 12 and Figure 13, which shows that the SOC timing advances with the increase in intake temperature. The results are corroborated by the findings in the study by Chiang et al. [7]. Hence, the suggested LSTM model is capable enough to consider the effect of intake temperatures on one of the combustion characteristics, i.e., SOC. In the case of 235 $°$C intake temperature, the SOC is captured at around 28 ms, as shown in Figure 10. At an intake temperature of 245 $°$C, the SOC timing occurs at around 26 ms in Figure 11, 2 ms earlier than the previous case. The SOC timing advances further to 24.8 ms and 23.9 ms for the remaining two intake temperatures of 255 $°$C and 265 $°$C, respectively, as presented in Figure 12 and Figure 13.

4. Conclusions

In this study, we investigated the application of long short-term memory (LSTM) networks for predicting cylinder pressure in a homogeneous charge compression ignition (HCCI) engine under different operating conditions, specifically intake temperatures. The LSTM model’s performance was analyzed under various configurations, such as input and output sizes and hidden layer sizes. The results demonstrated that the LSTM model could effectively predict cylinder pressure, exhibiting lower mean absolute error (MAE) and mean squared error (MSE) in comparison to a deep neural network (DNN) model with three hidden layers. The accuracy of the LSTM model has been measured on the basis of MAE and MSE. Results shows that for output sizes up to 256, the MAE is below 0.02, which means that the present model can predict 256 time-steps ahead with an acceptable error. Increase in the input size from 2 to 8 results in reduction of the error by up to 50%. The hidden size of 32 gives the best performance in terms of lowest MAE of 0.22 and MSE of 0.049.

Additionally, the LSTM model proved capable of adapting to different window sizes between 4 and 32 and showed better overall performance than the DNN model. For input of different temperatures, the LSTM outperforms the DNN in terms of MAE and MSE. MAE for LSTM is reduced by 20%, on the other hand the MSE for LSTM is reduced by 32% in comparison to DNN for predicting cylinder pressure at different temperatures. The improvement on accuracy is also supported by an improvement on efficiency, since the LSTM requires less than half of the memory space and is faster during inference. Furthermore, the LSTM model successfully captured the impact of intake temperature on the start of combustion (SOC) timing, showcasing its ability to model complex non-linear relationships within the combustion process.

The results presented in this study highlight the potential of LSTM networks in predicting cylinder pressure in HCCI engines, enabling real-time monitoring and control of engine performance. The LSTM model’s ability to handle time-series data and consider the effects of intake temperature on combustion characteristics demonstrates its promise for various applications within the automotive industry. Future work could extend the LSTM model to other engine parameters and investigate its applicability in other combustion engine types. Moreover, exploring the integration of LSTM models with physics-based approaches could provide a more comprehensive understanding of engine performance and enhance the predictive capabilities of these models.