Research on an Ultra-Short-Term Working Condition Prediction Method Based on a CNN-LSTM Network

Abstract

Affected by factors such as complex production operation data, high dimensions, and weak regularity, the existing ultra-short-term working condition prediction method struggles to guarantee the prediction accuracy and operation speed. Therefore, we propose an ultra-short-term working condition prediction method based on a convolutional neural network (CNN) and a long short-term memory network (LSTM). Firstly, we use sliding window and normalized processing methods to carry out data processing, and use CNN to extract the characteristics of processed production operation data. Secondly, we then improve the LSTM gated structure and introduce L2 norm, learning the change law of the production operation data by means of the LSTM prediction layer, and then obtain the predicted value of the working condition. We use the Bayesian method to select the parameters of the CNN-LSTM model to improve the prediction accuracy. Finally, we apply our method to a real-world application to demonstrate that our ultra-short-term working condition prediction method achieves superior results for prediction accuracy and running speed when compared with other methods.

1. Introduction

Ultra-short-time working condition prediction (CL-UWCP) is a short-term trend analysis method based on production operation data [1], and it is also one of the auxiliary decision-making methods to ensure production safety and stable operation [2,3], thereby providing a reliable objective basis for the production decision-making department to adjust the production operation plan [4,5]. Due to the complex structure [6] of various production operation data models and the strong long-range correlation of data, the existing models have low accuracy, so the ultra-short time working condition prediction method [7,8,9,10] is regarded as one of the key problems in the field of industrial production.

With the rapid development of artificial intelligence [11,12], deep learning technology [13,14] has emerged in the time series data mining field [15,16,17] due to its advantages of robustness, scalability and versatility, and it has been proven to be able to solve the ultra-short-term working condition prediction problem in industrial production. Mainstream technologies include CNN and LSTM. CNN [18,19,20] adopts local connectivity and weight sharing, meaning that it can effectively mine the relationship between continuous and discrete data, so as to obtain effective representation. At present, it has been used to effectively extract sensitive features in complex structure data models, and implement data dimensionality reduction. LSTM [21,22,23] is equipped with a gated structure and a memory unit, so it has memory and nonlinear data processing capabilities to solve the problem of low wind power prediction [24]. The memory unit structure of the model can effectively deal with multi-time order characteristics and predict ultra-short period wind transmission [25,26,27]. Nonlinear data processing capability can solve the problem of predicting ultra-short term operating conditions of future gas load demand [28].

Therefore, drawing on the “divide and conquer, complementary advantages” hybrid intelligent algorithm design idea [26,27,28], and taking advantage of the excellent feature extraction ability of CNN and the advantages of LSTM in analyzing and processing long-range data dependencies, we propose an ultra-short-term condition prediction method combining CNN and LSTM. Firstly, we use CNN to perform feature extraction and dimension reduction of the input production operation data, thus forming valid eigenvectors. Secondly, we input the feature vector into the LSTM network for training to learn the change rule between production operation data, and then realize the ultra-short-term working condition prediction. Finally, we demonstrate the effectiveness of the method through experiments and analyze the applicability and accuracy of the ultra-short-time working condition prediction method.

The method proposed in this paper has been applied to real scenes and has good application effects. For example, when a gas group uses a gas pipeline real-time monitoring system, this method is chosen as the basis for early warning. In the process of practical application, the accuracy rate is significantly higher than that of the original method, which ensures the safe and stable operation of the pipeline and contributes to the research and development of the oil field.

We have structured our paper as follows. Section 2 provides the workflow and data processing method of the CL-UWCP method, and summarizes the key scientific problems we need to solve. Section 3 details the network model structure of the CL-UWCP method, and discusses its working mechanism and evaluation method. Section 4 describes the parameter selection method of CL-UWCP. Section 5 presents our experiments to demonstrate the effectiveness of our approach. Section 6 summarizes our results and presents directions for future work.

2. Preliminaries

The CL-UWCP method is based on a number of historical production operation data and working condition values to train the model, and the model ${\widehat{p}}_c$ is used to predict the working condition values of a specific period in the future. Specifically, at a given $t$, the production operation data $X_{t1}={\left[x_{t-t1-1},x_{t-t1},\cdots x_t\right]}^T$ of the previous $t_1$ moment is used to predict the working condition value of the next continuous period.

2.1. CL-UWCP Process

The CL-UWCP model includes four parts: data processing, a CNN feature extraction layer, a LSTM prediction layer and model evaluation, as shown in Figure 1.

Step 1. Data processing: mobile smoothing and normalization techniques are used to process the original data to eliminate the influence of different dimensions.

Step 2. CNN feature extraction layer: after data processing, the production operation characteristics are extracted, and the data dimensions are reduced.

Step 3. LSTM network prediction layer: the data processed by CNN are input into the LSTM network for training, so as to learn the fitting relationship between the data and obtain the predicted value.

According to the CL-UWCP process, this paper needs to solve the following three problems.

I. How to design data models and data processing methods.

II. How to build an ultra-short-term working condition prediction model based on a CNN-LSTM network.

III. How to automatically select CL-UWCP model parameters.

2.2. Data Model and Data Processing Method

Production operation data are multidimensional time series data, and we take a group of production running data and express them as $X_t$—that is, $X_t={\left[x_1,x_2,\cdots x_{m-1},x_m\right]}^T$, wherein $X\in R^{m\times n}$ and $t\in \left[1,m\right]$. $x_t$ is the set of all attribute values at time $t$, wherein $x_t\in R^n$, and $x_{ti}$ is the ith attribute value at time $t$; that is, $x_t=\left[x_{t1},x_{t2},\cdots x_{t(n-1)},x_{tn}\right]$. $X_t$ is an M-order N-dimensional vector, and the expansion is shown in Formula (1).

$$X_t=\left(\begin{array}{cccc}{x}_{11}& x_{12}& \cdots & x_{1n}\\ x_{21}& x_{22}& \cdots & x_{2n}\\ ⋮& \cdots & \ddots & ⋮\\ x_{m1}& x_{m2}& \cdots & x_{mn}\end{array}\right)$$

(1)

Since the CL-UWCP model needs to have high prediction accuracy, the time series data need to meet the following three conditions.

Precondition 1. 

$X$ can have missing values, but the missing rate of continuous data should be less than 5%.

Precondition 2. 

$X$ has some characteristics, such as high dimensionality, complexity, a weak correlation between variables, and weak laws.

Precondition 3. 

The intermediate station outages and pump stops are not considered.

The data processing steps are as follows.

Step 1. We define the production running data M as $M={\left[m_1,m_2,\cdots m_{n-1},m_n\right]}^T$. $M^w$ is obtained by performing N-order window moving smoothing processing according to $m_t^w=\frac{1}{N}{\displaystyle \sum _{i=t}^{t+N-1}{m}_i}$—that is, $M^w={\left[m_1^w,m_2^w,\cdots m_{n-N+1}^w\right]}^T$.

Step 2. We take a first-order difference on $M^w$, and the calculation rule is shown in Formula (2). The processed data are represented by $M^b$—that is, $M^b={\left[m_1^b,m_2^b,\cdots m_{n-N}^b\right]}^T$.

$$m_t^b=m_{t+1}^w-m_t^w$$

(2)

Step 3. We normalize the data according to $X_i^j=\frac{x_i^j-x_{\min }^j}{x_{\max }^j-x_{\min }^j}$, $M^b$ is normalized so that it adhere to the interval [−1, 1], and the transformed dataset is $X$—that is, $X={\left[x_1,x_2,\cdots ,x_{n-N}\right]}^T$.

Step 4. We use sliding windows to construct separated datasets. The CL-UWCP model input step size is $T_1$, the predicted step size is $T_2$, the length of the sliding window is $T_1+T_2$, and each slide gives $n-N-T_1-T_2+1$ sequences of length $T_1+T_2$.

Step 5. The processed data are divided into the training set and test set at a ratio of 7:3, as expressed in Formulas (3) and (4).

$$X_{train}^{\prime }=\left(x_1^{\prime },x_2^{\prime },\cdots ,x_n^{\prime }\right)$$

(3)

$$X_{test}^{\prime }=\left(x_{n+1}^{\prime },x_{n+2}^{\prime },\cdots ,x_m^{\prime }\right)$$

(4)

3. CL-UWCP Model Based on an Improved CNN-LSTM Network

CNN and LSTM networks show the following advantages when solving the ultra-short-term working condition prediction. A CNN can efficiently extract the production operation data characteristics and reduce interference information; an LSTM network can deeply analyze the correlation between production operation data. Therefore, the CL-UWCP model combined the advantages of CNNs and improved LSTM networks, thus improving the CL-UWCP model’s prediction accuracy.

3.1. Improved CNN-LSTM Network Model Design

A CNN model involves multi-layer stacking, and it can efficiently extract local features. By means of convolution and pooling layers, respectively, the data feature extraction and data dimension reduction can be implemented. The convolution operations and pooling operations are represented by Formulas (5) and (6), respectively.

$$P_j^i=f({\displaystyle \sum _{i=1}^N{P}_i^{l-1}\ast w_{ij}^l}+b_j^i)$$

(5)

$$P_j^l=f({\alpha }_j^l{F}_d(P_j^l-1)+b_j^l)$$

(6)

where $P_j^i$ represents the jth convolution mapping in the convolution layer, namely the production operation data features extracted by the convolution layer; $P_j^{l-1}$ represents the ith upper convolution map; $w_{ij}^l$ represents the weight obtained by the jth convolution kernel for the ith operation; and $b_j^i$ represents the bias of the jth convolution kernel at the convolution layer. $P_j^l$ represents the jth feature map in the lth pooling layer; ${\alpha }_j^l$ represents the multiplicative bias of the feature map; and $F_d(x)$ represents the downsampling function.

We select and improve the LSTM network, and the improvements are described as follows.

I. Based on the traditional LSTM network, we improve the gate control structure of the LSTM network, add a forget gate and an input gate, and make the newly added gate interact with the original gate to form a multi-level gated LSTM network.

II. We introduce the L2 norm and use the L2 norm to regularize the LSTM model to better solve the overfitting problem. The loss function is $L_s$—that is, $L_s=L(W)+\lambda {\displaystyle \sum _{i=1}^n{w}_i^2}$.

$f_{t1}$ represents the newly added forget gate, $i_{t1}$ represents the newly added input gate, $f_{t2}$ represents the original forgetting gate, $i_{t2}$ represents the original input gate, $f_t$ represents the total forgetting gate, and $i_t$ represents the total input gate. The improved LSTM network is shown in Figure 2, and the working process is shown in Formulas (7)–(10).

$$f_{t1}=\sigma ({\Vert W_{f1}\Vert }_2^2{x}_t+{‖W_{f1}‖}_2^2{h}_{t-1}+b_{f1})$$

(7)

$$f_{t2}=\sigma ({‖W_{f2}‖}_2^2{x}_t+{‖W_{f2}‖}_2^2{h}_{t-1}+b_{f2})$$

(8)

$$i_{t1}=\sigma ({‖W_{i1}‖}_2^2{x}_t+{‖W_{i1}‖}_2^2{h}_{t-1}+b_i)$$

(9)

$$i_{t2}=\sigma ({‖W_{i2}‖}_2^2{x}_t+{‖W_{i2}‖}_2^2{h}_{t-1}+b_i)$$

(10)

We set $u_t=f_{t1}\cdot i_{t1}$, and the total forgetting gate, the total input gate and the total output gate are shown in Formulas (11)–(13).

$$f_t=(f_{t1}-u_t)+f_{t2}\cdot u_t$$

(11)

$$i_t=(i_{t1}-u_t)+i_{t2}\cdot u_t$$

(12)

$$o_t=\sigma ({‖W_o‖}_2^2{x}_t+{‖W_o‖}_2^2{h}_{t-1}+b_o)$$

(13)

We obtain the total forget gate $f_t$ according to Formula (11), that is, $f_t=f_{t1}(f_{t2}\cdot i_{t1}+1-i_{t1})$, and the analysis results are as follows.

I. We set $f_{t1}=0$, and we obtain $f_t$—that is, $f_t=0$. When $f_{t1}$ chooses to forget the previous layer cell state $C_{t-1}$, $f_t$ must also forget.

II. We set $f_{t1}=1$, and we obtain $f_t$—that is, $f_t=f_{t2}\cdot i_{t1}+1-i_{t1}$. We set $i_{t1}=0$, and we obtain $f_t$—that is, $f_t=1$. We set $i_{t1}=1$, and we obtain $f_t$—that is, $f_t=f_{t2}$. If $f_{t1}$ retains the previous layer cell state $C_{t-1}$, and if $i_{t1}$ discards state ${\tilde{C}}_t$, then $f_t$ chooses to keep the previous layer cell state $C_{t-1}$; if $i_{t1}$ chooses to keep state ${\tilde{C}}_t$, whether $f_t$ discards state $C_{t-1}$ is determined by $f_{t2}$.

Therefore, compared with the LSTM network, the improved LSTM has higher information screening ability, can accurately extract useful information and can improve the model prediction accuracy. The structure of the improved CNN-LSTM ultra-short-term condition prediction model is shown in Figure 3.

3.2. CL-UWCP Model Training

The CNN network layer mainly extracts data features, and the LSTM network layer predicts ultra-short-term working conditions. According to the LSTM model structure [29,30], the CL-UWCP model’s training steps are as follows.

Step 1. We define production operation data as $X_t$, process data and divide them into a training set $X_{train}^{\prime }$ and test set $X_{test}^{\prime }$.

Step 2. We input $X_{train}^{\prime }$ into CNN, extract data features and perform data reduction through the convolution layer and pooling layer, so as to obtain an effective feature vector, and the operation process is shown in Formulas (5) and (6).

In the LSTM network, the parameters participating in the iteration are the input weight matrix $V\in R^{M_h\times m}$ of the gate controller and a temporary memory cell, the hidden layer weight matrix $W\in R^{M_h\times M_h}$ and the bias matrix $b\in R^{M_h\times 1}$, wherein h represents the input dimension, $M_h$ represents hidden layer neuron dimension, the objective function $L$ adopts root mean square error, and the calculation is shown in Formula (20).

Take the total forgetting gate $f_t$ weight matrix as an example:

$$W_f\cdot \left[\begin{array}{c}{h}_{t-1}\\ x_t\end{array}\right]+b_f=U\cdot \left[\begin{array}{c}{h}_{t-1}\\ x_t\end{array}\right]+b_f$$

(14)

$U=[V^{(f)}{W}^{(f)}{b}_f]$ is obtained from the input weight matrix and the hidden layer weight matrix, and the updated version of Formula (14) is as shown in Formula (15).

$$W_f\cdot \left[\begin{array}{c}{h}_{t-1}\\ x_t\end{array}\right]+b_f=\left[V^{(f)}{W}^{(f)}{b}_f\right]\cdot \left[\begin{array}{c}{h}_{t-1}\\ x_t\\ 1\end{array}\right]=W_f\cdot D_t$$

(15)

The chain rule is used to derive the gradient calculation of the objective function $L_t$ to the weight matrix $W_f$ of the total forgetting gate. The specific process is as follows.

$$\frac{\partial L_t}{\partial W_f}=\frac{\partial L_t}{\partial h_t}\cdot \frac{\partial h_t}{\partial C_t}\cdot \frac{\partial C_t}{\partial f_t}\cdot \frac{\partial f_t}{\partial W_f}$$

(16)

$\frac{\partial h_t}{\partial C_t}$, $\frac{\partial C_t}{\partial f_t}$ and $\frac{\partial f_t}{\partial W_f}$ are expressed as follows.

$$\frac{\partial h_t}{\partial C_t}=o_t\cdot \left[1-{\tanh }^2(C_t)\right]$$

(17)

$$\frac{\partial C_t}{\partial f_t}=C_{t-1}$$

(18)

$$\frac{\partial f_t}{\partial W_f}=f_t(1-f_t)D_t$$

(19)

We use Formulas (17)–(19) to calculate the gradient of the objective function $L_t$ with respect to $W_f$, as shown in Formula (20).

$$\frac{\partial L_t}{\partial W_f}=\frac{\partial L_t}{\partial h_t}\cdot o_t\cdot \left[1-{\tanh }^2(C_t)\right]\cdot C_{t-1}\cdot f_t(1-f_t)D_t$$

(20)

Step 3. We perform the working condition prediction, that is, ${\widehat{p}}_{c,t}=({\widehat{p}}_{c,1},{\widehat{p}}_{c,2},\cdots ,{\widehat{p}}_{c,n})$, and the objective function is the root mean square error, as shown in Formula (21). When the MSE is minimum, the training stops and we obtain a final CL-UWCP model.

$$MSE={\displaystyle \sum _{t=1}^n\frac{{({\widehat{p}}_{c,t}-p_{c,t})}^2}{n}}$$

(21)

where $p_{c,t}$ represents the true value and ${\widehat{p}}_{c,t}$ represents the predictive value.

3.3. CL-UWCP Model Evaluation Function

We used root mean square error (RMSE), mean absolute error (MAE) and mean absolute percentage error (MAPE) to evaluate the model prediction effect. The smaller the evaluation indexes values, the higher the model’s prediction accuracy. The specific formulas are as follows.

$$RMSE=\sqrt{\frac{1}{n}{\displaystyle \sum _{t=1}^n{({\widehat{p}}_{c,t}-p_{c,t})}^2}}$$

(22)

$$MAE=\frac{1}{n}{\displaystyle \sum _{i=1}^n\left|{\widehat{p}}_{c,t}-p_{c,t}\right|}$$

(23)

$$MAPE=\frac{1}{n}{\displaystyle \sum _{i=1}^n\frac{|{\widehat{p}}_{c,t}-p_{c,t}|}{p_{c,t}}}$$

(24)

4. Select CL-UWCP Model Parameters Based on Bayesian Method

If we manually adjust the parameters, not only is the process complicated, but also the selected parameter combination is often not optimal. Therefore, the automatic parameter selection method is another key problem that needs to be solved.

4.1. Parameters Selection Design Ideas

The parameter selection problem can be expressed as finding the optimal parameter in a set of parameter combinations. We set $S=\{s_1,s_2,\cdots ,s_n\}$ as the parameter combinations, and $\widehat{s}=\arg \min f(s_n)$ as the optimal parameter. $s_n$ represents the nth parameter value, and $f(s_n)$ represents the parameter evaluation result. The Bayesian algorithm is used to estimate the posterior distribution of the objective function, as shown in Formula (25). According to the objective function’s past evaluation results, we build a replacement function, so as to obtain the parameter combination that minimizes the objective function value [31].

$$p(f|S)=\frac{p(S|f)p(f)}{p(S)}$$

(25)

wherein $f$ represents the unknown objective function, $S$ represents the collection of parameters and observations, $p(f|S)$ represents the posterior probability of $f$, $p(f)$ represents the prior probability of $f$, $p(S|f)$ represents the likelihood distribution of $y$, and $p(S)$ represents the marginal likelihood distribution of $f$.

The Bayesian [32] method is used to solve the CL-UWCP parameter problem. Since the CL-UWCP model based on a deep neural network has many parameters, random forests have poor generalization ability. Therefore, we choose Gaussian process as the probabilistic surrogate model. This chapter proposes the parameter selection method based on Bayesian theory [33].

The pseudo-code form of the algorithm [34,35] is shown in Algorithm 1, wherein “/**/” indicates the annotation.

Algorithm 1: CL-UWCP model parameter selection algorithm based on Bayesian theory

Input: S: Parameter set to be selected, f: Bayesian objective function, l: Acquisition function, M: Gaussian model.

Output: D: Parameter combination. Begin 01  $D\leftarrow InitSamples(f,S)$; /* Initialize parameters. */ 02  For t $\in$ 1,…T do 03  $p(y|s,D)\leftarrow FitModel(M,D)$; /* Fit the model and evaluate the parameters. */ 04  $s_i\leftarrow \arg \max s_{s\in \chi }l(s,p(y|s,D))$; /* Get the parameters. */ 05  $y_i\leftarrow f(s_i)$; 06  $D\leftarrow D\cup (s_i,y_i)$; /* Update the parameter combination. */ 07  end for End

4.2. Parameters Selection Step

The main parameters involved in the CL-UWCP model are shown in

Table 1. CL-UWCP model parameters.

Parameter	Parameter Range	Explain
$\alpha$	0.01	Learning rate
Hidden layers	(1, 10)	Hidden layer
Hidden layer neurons	(2, 64)	Hidden layer neurons
Batch size	(1, 16)	Batch size
Epoch	(100, 800)	Iterations
Optimizer	Adam	Optimizer
Activate function	Relu	Activation function
Dropout rate	(0.1, 0.8)	Dropout rate

.

The steps to select CL-UWCP model parameters are as follows. The specific process is shown in Figure 4.

Step 1. Within the CL-UWCP model parameter range, sample points are randomly generated, the initialized sample points area input into the Gaussian process, and the CL-UWCP model is trained. According to the loss value output by the model objective function, the Gaussian model is adjusted to ensure that the Gaussian model is close to the true distribution of the function.

Step 2. After adjusting the Gaussian model, the sampling function is used to select the next group of evaluated sample points $x_i$, $x_i$ is input into the CL-UWCP model for training, and the new output value $y_i$ of the objective function is obtained, thereby updating the sample set $S=\{(x_1,y_1),(x_2,y_2),\cdots ,(x_n,y_n)\}$ and the Gaussian model.

Step 3. If the newly selected sample points $x_i$ corresponding to the objective function loss value meet the requirements, then the algorithm is terminated and the process is exited, and the best parameter combination and the model objective function loss value are obtained—that is, $(x_i,y_i)$.

Step 4. If the newly selected sample points $x_i$ corresponding to the objective function loss value do not meet the requirements, $(x_i,y_i)$ are updated to the set, and the Gaussian model continues to be adjusted until the requirements are met.

5. Example Verification

In the experimental part, by comparing similar algorithms’ performance indicators, we verify the effectiveness of the method. The experimental design is as follows.

I. The experimental preparations, experimental setup, pipeline operation data, and comparison algorithms are described.

II. The prediction accuracy and performance indicators of different forecasting models are compared, and the effectiveness of the CL-UWCP model is verified.

We analyze the effect of parameters on the CL-UWCP model’s prediction accuracy, including the hidden layer, hidden layer neurons and input step size.

5.1. Experiment Preparation

I. Experimental environment. We simulate a gas group′s gas pipeline digital master control platform and pipeline real-time monitoring system, and the simulation environment structure diagram is shown in Figure 5. The master control platform is responsible for intelligent control, the real-time monitoring system is responsible for monitoring the pipeline running real-time status, and the data center is responsible for providing data support.

II. Data Preparation. The experimental data come from a gas group′s pipeline operation data, and the pipeline operating parameters are shown in

Table 2. Pipeline operation parameters.

Notation	Explain	Notation
$p$	Pipeline pressure	$q$
$T$	Pipeline temperature	$T_e$
$p_d$	Pipeline inner diameter	$p_l$
$p_c$	Pipeline working condition	$p_r$
$p_w$	Pipeline wall thickness	$p_m$

. The time resolution of the dataset is 5 min, the sample data size is shown in

Table 3. Data volume statistics of each pipeline operation parameters.

Date	Pipeline A	Pipeline B	Pipeline C	Pipeline D	Date
April 2021–June 2021	171,800	156,800	168,900	158,900	April 2021–June 2021
October 2021–December 2021	125,600	145,600	156,200	1,465,100	October 2021–December 2021

.

III. The experiments include performance evaluation experiments of different models, establishing the effect of model parameters on prediction accuracy. We select CNN, LSTM, and CNN-LSTM network models for comparison, and the principle is similar to the CL-UWCP model.

5.2. CL-UWCP Model Performance Experiment

To verify the CL-UWCP model’s performance in handling the ultra-short-time working condition prediction problem, it is compared with CNN, a LSTM single prediction model and a CNN-LSTM network model.

5.2.1. Different Input Variable Performance Experiment

The first group of experiments adopted univariate input. We use a single-working-condition history sequence as the model input to predict the future working condition. Comparison curves between the real values of the different prediction models and the predicted value of the test samples are shown in Figure 6a,b, and the prediction performance index pairs are shown in

Table 4. Predictive performance metrics’ comparison for univariate input of different models.

Pipeline Name	Evaluation Indicators	Prediction Model
		CNN	LSTM	CNN-LSTM	CL-UWCP
Pipeline A	MAE	0.0854	0.0803	0.0637	0.0535
	MAPE	0.1771	0.1723	0.1323	0.1112
	RMSE	0.3625	0.3523	0.2704	0.2273
Pipeline B	MAE	0.0848	0.0824	0.0624	0.0533
	MAPE	0.1782	0.1734	0.1312	0.1125
	RMSE	0.3597	0.3498	0.2649	0.2265

.

According to Figure 6a,b and Table 4, in the case of univariate input, the CL-UWCP model has certain advantages in predicting the overall performance of ultra-short-term working conditions compared with the single CNN, LSTM model and CNN-LSTM model, and the trend of the predicted value and real value is closer. The prediction errors of RMSE, MAE and MAPE are the smallest, but the prediction accuracy still needs to be improved. The prediction performance of the hybrid CL-UWCP model and the CNN-LSTM model is significantly higher than that of the single CNN model and LSTM model, mainly because the hybrid model can combine the advantages of the two single models and fully explore the law between variables, so as to improve the accuracy of the prediction model.

The second group of experiments adopted multivariate inputs. We analyze the effect of multivariate inputs on CL-UWCP model performance. The model inputs are 10 pipeline running parameters. Figure 7a,b show the comparison curves between the real values of the different prediction models and the predicted values of the test samples. According to the evaluation indicators selected in Section 3.3, we quantify the four methods’ prediction of the error, and the prediction performance index pairs are shown in Table 4.

Analyzing the experimental results, the following conclusions are given.

(1) By comparing the univariate and multivariate prediction models and analyzing Table 4 and

Table 5. Predictive performance metrics comparison for the univariate input of different models.

Pipeline Name	Evaluation Indicators	Prediction Model
		CNN	LSTM	CNN-LSTM	CL-UWCP
Pipeline A	MAE	0.0628	0.0579	0.0483	0.0289
	MAPE	0.1321	0.1212	0.1023	0.0618
	RMSE	0.2663	0.2457	0.2048	0.1228
Pipeline B	MAE	0.0619	0.0572	0.0429	0.0334
	MAPE	0.1312	0.1235	0.1023	0.0625
	RMSE	0.2629	0.2428	0.1820	0.1415

and Figure 6a,b and Figure 7a,b, we can see that increasing the pipeline operating parameters greatly can improve the prediction accuracy. This shows that increasing the input variables can improve the model’s prediction performance.

(2) According to Table 5, all predictive performance indexes of the CNN-LSTM model are higher than those of the single LSTM model and the CNN model. Taking pipeline A as an example, its RMSE value is 0.0409 and 0.0615 lower than that of the LSTM model and CNN model, respectively. Compared with the LSTM model and the CNN model, the MAPE value decreased by 0.0189 and 0.0298, and the MAE value decreased by 0.0096 and 0.0145, respectively. The main reason is that the convolution layer in the CNN-LSTM model can extract the feature information in the working condition data, reduce the influence of redundant information on the prediction results, and form an effective feature vector. By using the advantages of the LSTM model in processing the sequential sequence, the law between the data can be fully mined, thus improving the prediction performance of the CNN-LSTM model.

(3) According to Table 5 and Figure 7a,b, we input multiple pipeline operation data, and the MAE, MAPE, and RMSE values of the CL-UWCP model proposed in this paper are the smallest. The prediction performance of CL-UWCP is the best, being higher than that of CNN-LSTM model. Compared with the CNN-LSTM model, it is reduced by 0.0194 and 0.0095, 0.0405 and 0.0398, and 0.082 and 0.0405, respectively. This shows that the optimal prediction performance of CL-UWCP model is mainly due to the following two aspects: first, an improved LSTM network is introduced to form a multilevel gated structure. A two-stage forgetting gate and input gate are used. The input information is screened and analyzed with the newly added gate at first, and then the retained result information is transmitted to the original gate for secondary screening and analysis, so as to extract the high-frequency and useful information more accurately and improve the information screening ability of the model. Secondly, an L2 norm regularized LSTM model is used to solve the overfitting problem better. Thirdly, partial hyperparameters in CL-UWCP model can be selected by means of the Bayesian method to make full use of historical information, and the optimal hyperparameter combination of the model can be selected quickly to further improve the prediction accuracy of the model.

5.2.2. Multi-Step Ultra-Short-Term Working Condition Prediction Performance Experiment

To verify the CL-UWCP method performance in predicting long time series, the CL-UWCP model is compared with the above prediction model. We carry out the prediction experiment of the pipeline ultra-short-term working conditions, such as one step ahead (5 min), two steps ahead (10 min), and three steps ahead (15 min). Figure 8d shows the prediction result curve of the sampling point on a certain day of pipeline C, and

Table 6. Predictive performance metrics comparison for different models with multiple steps.

Pipeline Name	Evaluation Indicators	Prediction Model
		CNN	LSTM	CNN-LSTM	CL-UWCP
Two steps ahead	MAE	0.0705	0.0623	0.0428	0.0278
	MAPE	0.1396	0.1354	0.1041	0.0756
	RMSE	0.1758	0.1689	0.1458	0.1298
Four steps ahead	MAE	0.0855	0.0812	0.0687	0.0488
	MAPE	0.1986	0.1852	0.1562	0.1125
	RMSE	0.2235	0.1989	0.1788	0.1567
Six steps ahead	MAE	0.1132	0.1065	0.0856	0.0687
	MAPE	0.2539	0.2256	0.1842	0.1565
	RMSE	0.3568	0.2598	0.1921	0.1698

Note: the test data are taken from the operation data of pipeline C in October 2021.

shows the prediction error performance indexes accumulated for 20 days comparatively.

Analyzing the experimental results, the following conclusions are given.

(1) According to Figure 8d, as the prediction step increases, the four models’ deviation degree between the predicted curve and the actual value of gradually increases, and the predictive performance gradually decreases. Compared with other models, the CL-UWPC model increases as the predicted step increases, the CL-UWCP’s deviation between the predicted curve and the actual curve is minimized, and the change trend changes with the actual data trajectory, indicating that the CL-UWPC method has the best prediction performance.

(2) According to Table 6 and Figure 8a,c, with the increase in the prediction step, the RMSE, MAE and MAPE values gradually increased. This is mainly because as the prediction step number increases, the multi-step prediction errors’ cumulative effect leads to increased prediction errors. The prediction effect of the single model is significantly lower than that of the mixed model. This is mainly because the mixed model combines the efficient feature extraction capability of CNN and the advantages of LSTM for processing long-term sequences. The results show that the hybrid model can improve the performance of the single model and improve the prediction accuracy.

(3) Compared with the CNN-LSTM model, by improving the LSTM network, increasing its gated structure and introducing the L2 norm, the CL-UWCP method’s information screening ability improves and the prediction performance also improves. Specifically, when forecasting two steps in advance, compared with the CNN-LSTM method, the RMSE and MAE of the CL-UWCP method are reduced by 0.016 and 0.15, respectively, further verifying the effectiveness of the CL-UWCP method.

5.3. Model Parameters Influence on the Prediction Precision Experiments

The CL-UWCP model parameters’ settings have a great influence on the model’s prediction accuracy. This section mainly analyzes the following parameters’ influence on the model prediction precision—a hidden layer, hidden layer neurons, and the input step size in LSTM prediction layer. In order to eliminate the random errors’ influence, the model’s RMSE, MAE and MAPE values are the average of eight experiments.

5.3.1. Hidden Layers

The hidden layers in the LSTM layers can obtain the training data’s internal rules and can realize complex mapping. The number of hidden layers determines the model’s expressive ability, but too many hidden layers will lead to model overfitting. This section explores effect of the number of hidden layers on the model’s prediction accuracy, adopting model structures with different hidden layers. The number of hidden layer neurons is eight, and the other model parameters are the same, as shown in Table 1. The values of RMSE and MAPE are shown in Figure 9a,b.

According to Figure 9a,b, when the number of hidden layers changes from one to two, the ultra-short-term working condition RMSE and MAPE of each pipeline both decrease, and the model prediction effect is improved. With the increase in the number of hidden layers, the ultra-short-term working conditions prediction error increases continuously. The experimental results show that the number of hidden layers has different effects on the ultra-short-term working conditions. When the number of hidden layers exceeds three, increasing the hidden layer will not improve the model’s prediction accuracy, but will cause model overfitting. In conclusion, for the ultra-short-term working condition prediction, when the number of hidden layers is two, the prediction effect is best, and the different pipeline MAPE values are 0.0751 and 0.0658, respectively.

5.3.2. Hidden Layer Neurons

The number of hidden layer neurons directly affects the model’s learning ability; a model with fewer hidden layer neurons cannot fully learn the data rules, and too many neurons will increase the training time, make the structure more complex, and reduce the model prediction performance. This section explores the effect of the number of hidden layer neurons on the model’s prediction accuracy. The number of hidden layer neurons is 5, 10, 15, 20, 25, 30, 35, 40, 45, and 50, respectively. The number of hidden layers is two, and the other parameters are the same. The values of RMSE and MAPE are shown in Figure 10a,b.

According to Figure 10a,b, with the change in the number of neurons, each pipeline’s ultra-short-term working condition prediction error trend is similar. When the number of neurons is 5 and 10, the prediction error is small. When the number of hidden layer neurons increases to 20, 35 and 40, the prediction error increases with the increase in the number of neurons. The experimental results show that too many neurons can improve the model’s mapping ability, but reduce the fault tolerance and the model’s prediction accuracy.

5.3.3. Input Step

The LSTM’s prediction performance depends on the memory function of the model for the input historical time series data—that is, the predicted value has a certain relationship with the model input step size. The input step represents the historical pipeline running data and features that the model needs to memorize. If the input step is too short, the relevant information in the sequence cannot be effectively extracted. If the input step is too long, redundant information in the sequence may be obtained, thereby reducing the prediction accuracy. This section explores effect of the input step on the model’s prediction accuracy. The number of input steps is 2, 5, 8, 11, 14 and 17, respectively. When the number of hidden layers is 2, the number of hidden layer neurons is 10, and the other parameters are the same. The values of RMSE and MAPE are shown in Figure 11a,b.

According to Figure 11a,b, when the input time step is five, each pipeline condition prediction error is the smallest, and the prediction errors are 0.0989 and 0.0878, respectively. This shows that the working condition data before $t-5$ has a great influence on the working condition value at time $t$; when the input step is large, the model prediction accuracy does not improve, which means that the historical data far away from the prediction period have little influence on the prediction data.

6. Conclusions

We propose a CL-UWCP method based on improved CNN-LSTM networks. This method uses the CNN network to extract the production operation data characteristics, uses the LSTM network to learn the laws between the production operation data, and outputs the working condition predicted value through the fully connected layer. We summarize the experimental results as follows.

I. The CL-UWCP method is suitable for solving the ultra-short-term working condition problem of complex and multi-dimensional production operation data. The experimental results show that the CL-UWCP method has obvious advantages with regard to its prediction accuracy, performance and stability compared with similar methods.

II. The CL-UWCP method based on the improved CNN-LSTM network combines the advantages of CNN to extract data features and LSTM to process time series, so as to realize ultra-short-term working condition fitting prediction and improve the prediction accuracy.

III. The CL-UWCP model training adopts the Bayesian method to select parameters to improve the CL-UWCP method’s prediction accuracy. The experiments show that it can effectively reduce the overfitting phenomenon.

The ultra-short-term working condition prediction method has large application potential in various fields. When our method is applied to real scenarios, some more specific problems need to be solved, such as how to add the influence of the nodes’ own attribute values and how to obtain the best weight coefficients in the LSTM network.

7. Discussion

1. The CL-UWCP prediction model is proposed in this paper to solve the problem of ultra-short-term working condition prediction. By comparing with other models, the applicability and accuracy of the CL-UWCP method is verified.

2. The prediction performance of the CL-UWCP model is significantly higher than that of the CNN-LSTM model, mainly due to the following three aspects: first, the LSTM network is improved to form a multi-level gated structure, and high-frequency and useful information can be extracted through two screening and analysis. Second, the L2 regularization LSTM network is used to enhance its anti-jamming ability and reduce the overfitting problem. The third aspect to the use of the Bayesian method to select the superparameters in the CL-UWCP model, so that it can select the optimal superparameter combination quickly, and further improve the model’s prediction performance.

3. In this study, for the processing of multi-dimensional sequential data, the quality of the dataset is relatively high. Missing values can exist in the dataset, but the missing rate of continuous data should be less than 5%.

To sum up, this study compares the effects of different prediction models in the prediction of ultra-short-term working conditions, and finds that the prediction performance of the mixed prediction model is significantly higher than that of the single prediction models. The prediction performance of the CL-UWCP method proposed by us is optimal and can be applied in different fields. In order to further improve the accuracy and applicability of the CL_UWCP method, we will continue to study how to eliminate the influence of noise in the dataset on prediction results and how to quickly obtain the best weight coefficient in LSTM networks in the future.