A Two-Delay Combination Model for Stock Price Prediction

Abstract

This paper proposes a new linear combination model to predict the closing prices on multivariate financial data sets. The new approach integrates two delays of deep learning methods called the two-delay combination model. The forecasts are derived from three different deep learning models: the multilayer perceptron (MLP), the convolutional neural network (CNN) and the long short-term memory (LSTM) network. Moreover, the weight combination of our proposed model is estimated using the differential evolution (DE) algorithm. The proposed model is built and tested for three high-frequency stock data in financial markets—Microsoft Corporation (MSFT), Johnson & Johnson (JNJ) and Pfizer Inc. (PFE). The individual and combination forecast methods are compared using the root mean square error (RMSE) and the mean absolute percentage error (MAPE). The state-of-the-art combination models used in this paper are the equal weight (EW), the inverse of RMSE (INV-RMSE) and the variance-no-covariance (VAR-NO-CORR) methods. These comparisons demonstrate that our proposed approach using DE weight’s optimization has significantly lower forecast errors than the individual model and the state-of-the-art weight combination procedures for all experiments. Consequently, combining two delay deep learning models using differential evolution weights can effectively improve the stock price prediction.

1. Introduction

Forecasting time series, mainly stock prices, is a complicated and challenging task for stock market investors. A high accuracy stock price prediction method helps investors make a profit (buying or selling a stock at a specific price) [1]. Motivated by significant profits in stock market investment, researchers and investors have focused on stock price prediction research for decades [2].

In the beginning stage, several parametric statistical models, such as autoregressive moving average (ARMA) [3], autoregressive integrated moving average (ARIMA) [4], vector autoregression (VAR) [5] and the generalized autoregressive conditional heteroskedasticity (GARCH) [6] were used to predict the stock price. Although these models have been widely applied in the prediction of time series, they have poor performance for financial time series analysis due to the limitations of parametric statistics [7]. First, they assume a linear model structure; however, the financial time series observations have a nonlinear pattern.

Secondly, they assume constant variance, while financial time series are noisy and have time-varying volatility [8]. To overcome these issues and improve the accuracy of stock price forecasting, nonlinear machine learning (ML) techniques have been applied for financial prediction. Traditional machine learning models, such as the k-nearest neighbor algorithm (KNN), artificial neural networks (ANNs), support vector machines (SVMs) and random forest (RF), are often utilized to learn the relationship between the features from the technical analysis and price movement [2].

However, in recent years, many studies have revealed that deep learning models are superior to traditional machine learning models in the financial market [9]. The vast majority of deep learning algorithms are supervised techniques, including multilayer perceptron (MLP), convolutional neural network (CNN) and long short-term memory (LSTM) networks. Multilayer perceptron (MLP) or a multilayer neural network (MLNN) is a fully connected feed-forward artificial neural network (ANN) class. The MLPs have high self-learning ability and fault tolerance; however, they have limitations in the learning process as the pattern of stock price time series has tremendous amounts of noise and high dimensions [10].

A convolutional neural network (CNN) is a class of deep learning methods. There are three types of different dimensions of CNN, one-dimensional CNN (1D-CNN), two-dimensional CNN (2D-CNN) and three-dimensional CNN (3D-CNN), which are usually used for time series data, image data and 3D image data, respectively. The advantage of CNN compared to its predecessors is that it automatically identifies the relevant features without any human supervision [11]. Using the convolution function, CNN can also extract essential and distinctive features from images. However, this method requires large memory and computation load because big data must be prepared [12].

A long short-term memory (LSTM) network is a type of recurrent neural network capable of handling long-term dependencies between data by controlling the input gate, the forget gate and the output gate. The benefit of LSTM is that it is insensitive to the input data length compared to other methods that handle series of data [9]. The weakness of LSTM is that the big data and massive processing time for training are required [13].

Deep learning techniques have been widely studied in stock price prediction in recent years. However, the nature of stock price time series is usually non-linear, non-parametric and chaotic, which is highly difficult to fit models using traditional statistics procedures. Moreover, the combination methods have become widespread in stock price prediction, and many more suitable models for stock prediction have been proposed. The linear combination method is one of the ways for scholars to predict the stock price more accurately. This technique was introduced by Bates and Granger [14]. The main objective of this approach is to combine the benefits of different single forecasting models.

Several studies have confirmed that the combination forecast has superior efficiency to the individual method [15]. However, weight selection is a significant challenge of the linear combination forecast method. In this work, the differential evolution algorithm is applied to seek the optimal weight of the combination procedure. The DE algorithm is one of the most powerful tools for global optimization. The DE algorithm has been applied to numerous areas, such as engineering, science, industry and finance, because it is easy to understand, simple to implement, widely applicable, has good performance and has good reliability [10]. To our knowledge, there is no literature on combining different delays for stock price prediction. Moreover, little work in the literature was on the delay (look-back) period [16]. This motivated us to combine two delay forecasts and estimate the weights using the differential evolution algorithm.

In this paper, we propose the two-delay combination using deep learning techniques to predict closing prices. We also present the new weights of the hybrid model to improve the forecasting accuracy using the differential evolution algorithm. The paper presents the following research contributions:

• The two-delay combination using deep learning technique is proposed to predict the closing prices to improve the accuracy of the individual deep learning procedure.
• We apply a differential evolution algorithm for the weight estimation of our proposed technique and compare it with the state-of-the-art weight estimation of the linear combination methods.
• The two-delay combination using deep learning model is applied to three real-world stock price time series. The performance of our proposed model is compared to the well-known deep learning method.

The remainder of this paper is structured as follows. Section 2 presents the idea of combining two delay deep learning forecasting techniques and their estimated weights. The data visualization is displayed in Section 3. The empirical results are represented in Section 4. The discussion of the results are detailed in Section 5. Finally, our conclusions are given in Section 6.

2. Methodology

Time series prediction has been a challenging task for over five decades. A large amount of literature has focused on the method to obtain accurate forecasts in numerous practical applications. In general, two main methods can improve the accuracy of forecasting results: (1) developing and proposing new forecasting models and (2) hybridizing existing forecasting models. Based on the advantage of the combination forecast technique from the previous section, we focus on only the hybrid model. The famous hybrid structure consists of the parallel and series hybrid structures [17]. The linear combination of parallel structures is challenging for researchers. The main aim of this area is to seek the optimal weight of each forecast. The framework of two classifications of hybrid structures is shown in Figure 1.

The delay $\left(m\right)$ in this paper refers to the network’s input length (look-back) or how many previous timesteps are used to predict the next timestep. For example, the delay is defined as four indicates that we allow our network to look back at four data to predict the next timestamp. Consequently, the predicted priced at time $t+1$ ($x_{t+1}$) is produced using past observations at time $t,t-1,t-2$ and $t-3$$(x_t,x_{t-1},x_{t-2},x_{t-3})$. In this work, each prediction model was trained to utilize two different delays for the predicted values, i.e., 43 (half-day trading delay) and 79 (one-day trading delay).

2.1. Classical Deep Learning Model

Time series forecasting using deep learning methods has been viral among researchers for more than 20 years due to the ability to deal with non-linear and multi-dimensional problems. In this paper, three well-known deep learning models—namely, the multilayer perceptron (MLP), the convolutional neural network (CNN) and the long short-term memory (LSTM) network—are applied to predict the closing stock price.

• Multilayer Perceptron (MLP)

MLP is a feedforward artificial neural network class. It consists of an input layer, one or more hidden layers and an output layer. The hyperparameters of the network are the number of neurons in each layer, the number of layers and the activation function. This model has been applied to media recording [18], stock price prediction [19], medical [20], environmental science [21] and engineering [22].

The structure of the MLP is displayed in Figure 2.

• Convolutional Neural Network (CNN)

CNN has become a trending tool in recent years, particularly in the image processing community, because it provides exceptional accuracy in this field [23]. CNNs are most effective in applications that include face recognition [24], object detection [25], environmental [26], cloud image analysis [27], traffic flow [28] and stock prediction [29].

CNN is a type of deep neural network (DNN) composed of the convolution layer and the pooling layer. In CNN architectures, there are different layers: convolutional, max-pooling and fully connected MLP layers. The number of hidden layers, the number of units in each layer, the activation functions, the learning rate, the number of epochs, the batch size (minibatch size) and the kernel size are the hyperparameters of the CNN model [12]. Figure 3 depicts the overall architecture of a CNN.

• Long Short-Term Memory (LSTM) Network

The primary objectives of LSTM are to capture long-term dependencies and determine the optimal time delay (lag) for time series problems. The hidden state of the LSTM model is divided into a short-term state and a long-term state. This method has been effectively employed in numerous studies involving time series forecasting, such as traffic flow prediction [30], wind power prediction [31], human trajectory prediction [32] and stock price prediction [8]. The LSTM model architecture consists of different numbers of layers and variant types of units in each layer. The number of hidden layers, the number of units in each layer, the regularization techniques, the activation functions, the learning rate, the number of epochs, the batch size (minibatch size) and the sequence length are the hyperparameters of the LSTM model [30]. The LSTM structure is presented in Figure 4.

2.2. Combination Forecast

To improve the single deep learning forecast accuracy, a linear combination forecast is proposed. Many researchers have studied this topic and built several combination forecast techniques to improve the forecast accuracy [15]. In this paper, we propose the two-delay combination using the deep learning method to predict the closing prices. The general two-delay combination forecast can be defined as [34]

$$\begin{array}{c}\hfill {\widehat{y}}_c=\sum _{i=1}^2{w}_i{\widehat{y}}_i,\end{array}$$

(1)

where ${\widehat{y}}_c$ is the combined forecast, $w_1,w_2$ are the combination weights, and ${\widehat{y}}_i$ are the predicted prices obtained from each delay prediction model.

The prediction model for the single DL method consists of seven parts: data collecting, data pre-processing, hyperparameter optimization, model training, model saving, model testing and prediction output. The flowchart of the hybrid deep learning process is shown in Figure 5.

The structure of the linear hybrid forecast is displayed in Figure 6.

This structure starts with obtaining the multivariate stock price time series. Then, the predicted prices of each delay are computed using the deep learning model. We then combine the forecast results using the linear combination method. Next, the weights are computed using the state-of-the-art methods and the DE algorithm. Finally, the RMSE and MAPE are calculated to compare the forecasting performance obtained from various weight estimation techniques.

The state-of-the-art of the linear combination forecast methods used in this paper are as follows:

• Equal Weights (EW)

This approach assigns equal weights to two-delay forecast. The simple average forecast [35] is

$$\begin{array}{c}\hfill w_1=w_2=0.5\end{array}$$

(2)

• Inverse of RMSE (INV-RMSE) Weights

This method was proposed with the weights proportional to their individual inverse of the RMSE. The general formula is as follows [35];

$$\begin{array}{c}\hfill w_1=\frac{RMS{E}_2}{RMS{E}_1+RMS{E}_2},\end{array}$$

(3)

$$\begin{array}{c}\hfill w_2=\frac{RMS{E}_1}{RMS{E}_1+RMS{E}_2},\end{array}$$

(4)

where $RMS{E}_1,RMS{E}_2$ are the RMSE obtained from the first and the second delay, respectively.

• Variance No Covariance (VAR-NO-CORR) Weights

The optimal weights can be obtained as [36]

$$\begin{array}{c}\hfill w_1=\frac{{\sigma }_2^2}{{\sigma }_1^2+{\sigma }_2^2},\end{array}$$

(5)

$$\begin{array}{c}\hfill w_2=1-w_1,\end{array}$$

(6)

where ${\sigma }_1^2$ and ${\sigma }_2^2$ are the variances obtained from the first and the second delay, respectively.

The pseudocode of the DE algorithm is displayed in Algorithm 1.

The Algorithm 1 starts with defining some parameters, such as the number of population size $\left(NP\right)$, the crossover ratio $\left(CR\right)$, the mutation ratio $\left(F\right)$ and the constant term ($TOL$). An initial population is generated using a uniform distribution on the interval [0, 1], where $x^L$ and $x^U$ are the lower and upper bounds for the decision parameter, respectively. In the mutation process, three individuals $(x_{r1},x_{r2},x_{r3})$ are selected in the population set of $NP$ elements, with $r1\ne r2\ne r3\ne i$. F is a user-defined constant known as the mutation factor, $F\in [0,1]$.

We then apply all values to obtain the mutant vectors $\left(v_i\right)$. The trial vectors $\left(u_i\right)$ are generated by mixing the parameters of the target vectors $\left(x_i\right)$ with the mutant vectors $\left(v_i\right)$ according to a selected crossover probability $\left(CR\right)$. The selection scheme is applied in the DE algorithm for the next step. The predicted stock prices of the test set are imported to compute the fitness of the target vectors and the fitness of trial vectors. The best solution is chosen by comparing the fitness of the trial vectors and the corresponding target vectors. We then calculate the predicted stock prices $\left(f_i\right)$ of each DL model.

Next, the observed prices are imported to compute the error measurement. The errors between the observed and predicted stock prices are calculated. The mean absolute percentage error (MAPE) is applied to seek an optimal weight $\left(x_i\right)$. The optimization process stops whenever the MAPE is less than a constant term $\left(TOL\right)$ or the number of iterations reaches the limit. Finally, the weights are applied in the hybrid predictive model Equation (1) to obtain the combined forecast.

Algorithm 1 DE Pseudocode for weight identification.

Require:$NP=10$, $CR=0.6$, $F=0.2$, $TOL=0.01$, $x^L=-5$, $x^U=5$, $MAP{E}_{min}=100$, $maxG=1000$ and $G=0$. 1: Generate the initial population 2: $x_i^G=x_i^L+rand(0,1)·(x_i^U-x_i^L),i=1,\cdots ,NP$ 3:   While ($MAP{E}_{min}>TOL$ or $G<maxG$) 4:     for $i=1$ to $NP$ 5:     Randomly select $r_1,r_2,r_3\in 1,2,\cdots ,NP$ where $r_1\ne r_2\ne r_3\ne i$     Mutation 6:       $v_i^G=x_{r1}^G+F·(x_{r2}^G-x_{r3}^G)$     Crossover 7:       if $rand(0,1)<=CR$ 8:         $u_i^G=v_i^G$ 9:         else 10:         $u_i^G=x_i^G$ 11:       end if     Selection: $sum{E}_m=0$, $sumE=0$ 12:     Import the predicted stock prices ${\left\{{\widehat{y}}_k^{\left(m_1\right)}\right\}}_{k=1}^n$, ${\left\{{\widehat{y}}_k^{\left(m_2\right)}\right\}}_{k=1}^n$ and the observed stock prices ${\left\{Y_k\right\}}_{k=1}^n$ 13:       for $k=1$ to n 14:         $f_i^m\left(k\right)=u_i^G{\widehat{y}}_k^{\left(m_1\right)}+(1-u_i^G){\widehat{y}}_k^{\left(m_2\right)}$ 15:         $f_i^m\left(k\right)=x_i^G{\widehat{y}}_k^{\left(m_1\right)}+(1-x_i^G){\widehat{y}}_k^{\left(m_2\right)}$ 16:         $e^m\left(k\right)=\left|f_i^m\left(k\right)-Y\left(k\right)\right|/Y\left(k\right)$ 17:         $e\left(k\right)=\left|f_i\left(k\right)-Y\left(k\right)\right|/Y\left(k\right)$ 18:         $sum{E}_m=sum{E}_m+e^m\left(k\right)$ 19:         $sumE=sumE+e\left(k\right)$ 20:       end for 21:       $MAP{E}_m=sum{E}_m\times 100/n$ 22:       $MAPE=sumE\times 100/n$ 23:         if $MAP{E}_m<MAPE$ 24:           $x_i^{G+1}=u_i^G$ 25:           $MAP{E}_i=MAP{E}_m$ 26:         else 27:           $x_i^{G+1}=x_i^G$ 28:           $MAP{E}_i=MAPE$ 29:         end if 30:       end for 31:     $MAP{E}_{min}=min\left(MAP{E}_i\right)$ 32:     $G=G+1$ 33:   end while 34: Compute the predicted combination forecast model using the optimal weight

In order to demonstrate the effectiveness of the proposed model, the evaluation metrics—namely, the root mean square error (RMSE) and the mean absolute percentage error (MAPE)—are employed. The definitions of the two evaluation metrics are as follows:

$$\begin{array}{c}\hfill RMSE=\frac{1}{n}\sqrt{{\displaystyle \sum _{t=1}^n}{(Y_t-{\widehat{y}}_t)}^2},\end{array}$$

(7)

$$\begin{array}{c}\hfill MAPE=\frac{1}{n}\sum _{t=1}^n\frac{\left|Y_t-{\widehat{y}}_t\right|}{Y_t}\times 100,\end{array}$$

(8)

where $Y_t$ is the actual value, $\widehat{y_t}$ is the predicted value at time t, and n is the size of the observation.

MAPE is a prevalent error measure in many works. A forecast model can be considered high, good, reasonable and inaccurate when the MAPE error is less than 10%, 11–20%, 21–50% and over 50%, respectively [37].

2.3. Hyperparameter Optimization

Deep learning algorithms have several variables, known as hyperparameters. It is challenging to choose optimal hyperparameters before the training data. Trial-and-error is one popular technique for choosing the model’s variables; however, it takes a long time. In this work, SHERPA, which is an open-source software for hyperparameter tuning of machine learning models, is applied to select the hyperparameters of each model in each stock [38]. Some articles also apply the SHERPA algorithm for hyperparameter optimization in neural network (NN) fields.

Beucler et al. [39] introduced a systematic way of enforcing nonlinear analytic constraints in neural networks via constraints in the architecture or the loss function. They implemented the three NN types and used SHERPA for hyperparameter optimization in each NN type. Ott et al. [40] introduced a software library, the Fortran–Keras Bridge (FKB), which is used for computing large-scale scientific projects and integrating with modern deep learning methods.

Before applying SHERPA to our model, it is necessary to prescribe the parameter search space. For the whole experiment, some parameters are set: the optimizer is stochastic gradient descent (SGD) and the loss function is a mean square error. For the parameter search space used in DL methods, the choice of the activation functions is linear, relu, tanh, softmax and sigmoid, the choice of batch size is 32,64,128, 256, 512 and 1024, and the learning rates fall between 0.001 and 0.1. The hyperparameter search spaces of the MLP, CNN and LSTM methods are shown in

Table 1. Hyperparameter search space of three machine learning models.

Model	Hyperparameter Search Space Values
	Name	Range	Parameter Type
MLP	Hidden unit1	[32, 256]	Discrete
	Hidden unit1	[32, 256]	Discrete
CNN	Conv1D	[10, 32, 64, 128]	Choice
	kernel size	[1, 2, 3, 4, 5]	Choice
	Hidden unit	[32, 256]	Discrete
LSTM	LSTM(layer1)	[32, 256]	Discrete
	LSTM(layer2)	[32, 256]	Discrete

.

3. Data Visualization

The intraday multivariate variables, including the opening price, the highest price, the lowest price, the closing price and the trading volume, which are recorded with the frequency of every 5 min, were extracted from the Thomson Reuters database between January 2021 and June 2022 for a total of 29,625 prices [41].

This information is called OHLCV. We selected some of the exciting stocks on the New York Stock Exchange (NYSE), including Microsoft Corporation (MSFT), Johnson & Johnson (JNJ) and Pfizer Inc. (PFE). The NYSE is an American stock exchange in the Financial District of Lower Manhattan in New York City, traded between 9.30 a.m. and 4.00 p.m. without a lunch break on weekdays. The descriptive statistics, including the minimum (Min), maximum (Max), mean, standard deviation (SD), skewness and kurtosis of the closing prices obtained from three stocks, are shown in

Table 2. Descriptive statistical analysis results of three original stock prices.

Index	Min	Max	Mean	SD	Skewness	Kurtosis
MSFT	212.20	349.31	279.12	32.57	0.0581	−0.8656
JNJ	148.49	186.45	168.15	6.77	0.3691	−0.5416
PFE	33.38	61.48	45.17	6.97	0.0720	−1.1761

.

The descriptive statistics of the closing prices obtained from three different stocks (currency in USD) were analyzed using the minimum, maximum, mean, standard deviation, skewness and kurtosis. As shown in Table 2, the mean price of MSFT was 279.12 USD. The lowest, highest and standard deviation prices were 212.20 USD, 349.31 USD and 32.57 USD. The mean of the JNJ stock and the standard deviation were 168.15 USD and 6.77 USD, respectively. The JNJ prices dropped to 148.49 USD and peaked at 186.45 USD. For Pfizer Inc. stock, 45.17 USD was the average price, and 6.97 USD was the standard deviation. The price range, skewness and kurtosis were 28.10, 0.0720 and −1.1761.

Time series plots of the observed prices along with the training set (blue line), validation set (red line) and test set (green line) are displayed in Figure 7.

It can be noticed in Figure 7 that the original closing prices of the training and validation sets for three different stocks are non-stationary. The MSFT stock shows an upward trend. JNJ displays fluctuations, while Pfizer Inc. presents an upward trend with slight fluctuations. Moreover, it is interesting to consider the JNJ and PFE stocks because they are the USA’s large pharmaceutical companies that sell medications and COVID-19 vaccines internationally.

We then focused on the data pre-processing before applying for model training and testing. After obtaining data from the Thomson Reuters database, we cleaned up any errors and filled in any missing data using a splines interpolation method. The data consists of five input variables: the opening price, highest price, lowest price, closing price and volume. In addition, we aimed to forecast the closing price at the end of every 5 min of trading. The historical financial data series were divided into three sets—namely, the training, validation and test sets.

For each stock, 19,829 data points from January to December 2021 were used for training and validating the deep learning models. We used 83% (January–October 2021) for training (learning parameters) and the remaining 17% (November–December 2021) to validate the network performance and avoid overfitting. The remaining six months (January–June 2022) were used to assess the model performance. Finally, the data were normalized using MinMaxScaler. All the experiments were conducted under Google Colaboratory (also known as Google Colab). The MinMaxScaler formula is as follows [35].

$$\begin{array}{c}\hfill x^{\prime }=\frac{x-x_{min}}{x_{max}-x_{min}},\end{array}$$

(9)

where $x^{\prime }$ is the value in the interval [0, 1], x presents the observed price, and $x_{min}$ and $x_{max}$ represent the minimum and the maximum of the set of stock price, respectively. The inverse normalization function is

$$\begin{array}{c}\hfill {\widehat{y}}_i=y_i·\left(x_{max}-x_{min}\right)+x_{min}.\end{array}$$

(10)

Here, ${\widehat{y}}_i$ is the predicted value, and $y_i$ presents the network output value.

4. Numerical Results

After the data pre-processing step, the optimal hyperparameters on the training set of three stocks obtained from the SHERPA algorithm with two delays are represented in

Table 3. The optimal hyperparameters of three deep learning models.

Model	Optimal Hyperparameters
	Half-Day Trading Delay (m = 43)			One-Day Trading Delay (m = 79)
	MSFT	JNJ	PFE	MSFT	JNJ	PFE
MLP
• Hidden unit1	150	136	58	146	82	111
• Hidden unit2	99	231	186	184	220	90
• Activation	tanh	tanh	tanh	tanh	tanh	tanh
• Learning rate	0.001493	0.012479	0.002515	0.001538	0.010000	0.001650
CNN
• Conv1D	128	32	32	32	64	64
• Kernel size	2	2	4	4	5	3
• Hidden unit1	85	76	88	111	104	72
• Activation	relu	relu	tanh	tanh	relu	relu
• Learning rate	0.021797	0.007043	0.005120	0.006477	0.009509	0.008842
LSTM
• LSTM(layer1)	243	136	171	145	196	175
• LSTM(layer2)	238	231	106	235	90	130
• Activation	tanh	relu	tanh	relu	relu	relu
• Learning rate	0.002190	0.012479	0.001176	0.001016	0.006548	0.001473

.

Table 3 presents the optimal hyperparameters obtained from the SHERPA algorithm using the training set of three stocks. All predictive experiments were performed using the Python tool, Scikit-learn and TensorFlow package. The selected CNN model architecture of JNJ stock is shown in Figure 8.

The details of the CNN architecture of JNJ stock (Figure 8) are as follows. The first layer is the input layer, which contains the shape (43,5). This indicates that we use the multivariate input, including the opening, highest, lowest, volume and closing prices of the previous 43’s stock price records. The CNN model consists of only one convolution layer that extracts 32 feature maps from the input data with a kernel of size 2. The output shape of the third layer is (21,32) because the max-pooling layer reduces the dimension of the data by a factor of 1/2. Next, a flattening operation is used for transformation of the max-pooling layer into a one-dimension array ($21\times 32=672$).

After that, the one-dimensional vector is passed through a dense layer and sent to the final output layer. In a one-day trading delay, the CNN architecture process is similar to a half-day trading delay, except the previous one-day OHLVC values (79 observations) are added to the model. The CNN model consists of only one convolution layer that extracts 64 feature maps from the input data with a kernel of size 5. After the max-pooling step, the size of the feature maps to 37. The output flatten later is (None, 2368) shows that we flatten the output of the convolutional layers to create a one-dimension vector ($37\times 64=2368$). Next, the number of neurons in the dense layer is 104. Finally, the closing prices for the next timestamp are computed using the 104 nodes at the output of the dense layer.

For deep learning forecasting, overfitting is one of the most significant problems in training neural networks. This occurs when the model performs well on training data but generalizes poorly to unseen data (the training loss is significantly lower than the validation loss) [26]. The weight regularization ($L2$-sum of the squared weights) is one of the regularization techniques added to the architectures [42] to deal with the overfitting issue. The loss curve against epochs of the selected deep learning network architecture is illustrated in Figure 9.

From Figure 9, it can be observed that the training and validation losses decrease and are stabilized around the same point (around the value $1\times {10}^{-3}$). That means our model architecture can be used for closing price prediction. We then compute and compare the evaluation metrics for other stocks’ training and validation sets. The RMSE and MAPE obtained from the MLP, CNN and LSTM models over the training and validation set are shown in

Table 4. The root mean square error of three deep learning models.

Model	Stock	Half-Day Trading Delay (m = 43)		One-Day Trading Delay (m = 79)
		Traning Set	Validation Set	Traning Set	Validation Set
MLP	MSFT	2.2676	2.1015	2.5306	3.4690
	JNJ	1.5154	1.2221	2.2584	1.7014
	PFE	0.3407	0.9737	0.3013	0.7835
CNN	MSFT	3.0918	2.1621	2.5100	2.0179
	JNJ	0.9038	0.7642	1.1192	0.9089
	PFE	1.1801	2.5591	0.7151	1.8346
LSTM	MSFT	2.7098	1.9076	4.7187	2.1386
	JNJ	1.1693	0.9688	1.5833	1.2784
	PFE	1.3875	3.2553	0.9476	2.6841

and

Table 5. The mean absolute percentage error of three deep learning models.

Model	Stock	Half-Day Trading Delay (m = 43)		One-Day Trading Delay (m = 79)
		Traning Set	Validation Set	Traning Set	Validation Set
MLP	MSFT	0.7526	0.4363	2.5306	3.4690
	JNJ	0.6842	0.5647	0.9788	0.7641
	PFE	0.6830	1.4243	0.5263	1.0548
CNN	MSFT	1.0415	0.4354	0.8397	0.4300
	JNJ	0.3978	0.3579	0.4921	0.4233
	PFE	2.5793	4.1596	1.5733	2.8311
LSTM	MSFT	0.9371	0.4308	1.6887	0.4560
	JNJ	0.5193	0.4535	0.7005	0.6004
	PFE	3.0139	5.3156	2.0781	4.4788

, respectively.

As shown from Table 4 and Table 5, the overall results of the RMSE and the MAPE obtained from the training and validation sets are almost similar, resulting in further confirmation of no overfitting [26]. It indicates that the performance of all individual DL models behaves well on both the training and validation sets and can be applied to predict out-of-sample data sets. However, the gap in the evaluation metrics obtained from the training and validation sets can be notable for PFE stock. It might be because the training and validation set patterns are quite different, as shown in Figure 7c. There is a strong upward trend on the validation set, while the training set presents upward and downward trends from June to October 2021.

After receiving the individual forecast from two different delays, the optimal weights are chosen using the state-of-the-art method and DE algorithm (Algorithm 1). We then compute the predicted closing prices obtained from our proposed model and compare them with the individual forecast for each stock. The evaluation metrics of the test set on three stocks obtained from the single and the combination forecast are shown in

Table 6. Evaluation metrics of the test set on three stocks obtained from the single and the combination forecast methods.

Model	Method	RMSE			MAPE
		MSFT	JNJ	PFE	MSFT	JNJ	PFE
MLP	Half-day delay	3.0397	1.3915	0.6508	0.8517	0.6109	1.0583
	One-day delay	3.8486	1.9552	0.5070	1.0862	0.9075	0.7439
	EW	3.4057	1.6585	0.5296	0.9591	0.7548	0.8050
	INV-RMSE	3.7075	1.6110	0.5207	1.0458	0.7297	0.7851
	VAR-NO-CORR	3.3978	1.6754	0.5336	0.9569	0.7637	0.8138
	DE	2.6579	1.0692	0.5015	0.7243	0.3829	0.7373
CNN	Half-day delay	4.5979	1.7618	1.8870	1.4079	0.7968	3.3803
	One-day delay	3.8023	2.3391	1.2475	1.0919	1.1065	2.0414
	EW	4.1268	2.0425	1.5518	1.2313	0.9505	2.6694
	INV-RMSE	4.0892	2.0019	1.4866	1.2162	0.9287	2.5308
	VAR-NO-CORR	4.1437	2.0616	1.5548	1.2380	0.9607	2.6757
	DE	3.6708	1.0717	0.8613	1.0015	0.3408	1.3758
LSTM	Half-day delay	2.9964	1.0349	2.4839	0.8510	0.3937	4.5097
	One-day delay	5.0845	2.9908	1.9764	1.5924	1.4732	3.5079
	EW	3.9791	1.9014	2.2289	1.2018	0.9035	4.0077
	INV-RMSE	3.7093	1.4252	2.2000	1.1059	0.6384	3.9507
	VAR-NO-CORR	3.9996	2.1258	2.2404	1.2091	1.0235	4.0303
	DE	2.1420	0.9125	0.6150	0.5402	0.2979	0.9034

.

Table 6 reveals the RMSE and MAPE of two individual delays of deep learning models (half-day delay and one-day delay) and combination forecasting models using the state-of-the-art and DE weights for the MLP, CNN and LSTM models. It indicates that the two-delay combination model using DE algorithm weight provides the lowest RMSE for all single deep learning models and stocks. Furthermore, our proposed model has higher forecasting accuracy because the MAPE values for all cases are lower than 10%. The prediction model of the test set obtained from the two-delay combination model for three stocks is shown in

Table 7. Prediction models of the MSFT, JNJ and PFE stocks.

Model	Stock	Two-Delay Combination Model
MLP	MSFT	${\widehat{y}}_c=-1.2046{\widehat{y}}_{(m=43)}+2.2046{\widehat{y}}_{(m=79)}$
	JNJ	${\widehat{y}}_c=-1.8013{\widehat{y}}_{(m=43)}+2.8013{\widehat{y}}_{(m=79)}$
	PFE	${\widehat{y}}_c=0.1517{\widehat{y}}_{(m=43)}+0.8483{\widehat{y}}_{(m=79)}$
CNN	MSFT	${\widehat{y}}_c=-0.7466{\widehat{y}}_{(m=43)}+1.7466{\widehat{Y}}_{(m=79)}$
	JNJ	${\widehat{y}}_c=3.0376{\widehat{y}}_{(m=43)}-2.0376{\widehat{y}}_{(m=79)}$
	PFE	${\widehat{y}}_c=-1.0764{\widehat{y}}_{(m=43)}+2.0764{\widehat{y}}_{(m=79)}$
LSTM	MSFT	${\widehat{y}}_c=1.8434{\widehat{y}}_{(m=43)}-0.8434{\widehat{y}}_{(m=79)}$
	JNJ	${\widehat{y}}_c=1.2072{\widehat{y}}_{(m=43)}-0.2072{\widehat{y}}_{(m=79)}$
	PFE	${\widehat{y}}_c=-3.5525{\widehat{y}}_{(m=43)}+4.5525{\widehat{y}}_{(m=79)}$

.

The comparison of the prediction curves between the individual delay deep learning model and the combination model and its actual price curve of three stocks obtained from the MLP, CNN and LSTM models are illustrated in Figure 10, Figure 11, Figure 12, Figure 13, Figure 14, Figure 15, Figure 16, Figure 17 and Figure 18. respectively.

Figure 10, Figure 11, Figure 12, Figure 13, Figure 14, Figure 15, Figure 16, Figure 17 and Figure 18 present the observed closing prices compared to the predicted prices obtained from the individual delay deep learning and combination models between January and June 2022. For all figures, the blue lines display the actual closing prices, and the green and magenta dashed lines represent the predicted closing prices obtained from individual deep learning models with half-day and one-day delays, respectively. The yellow, cyan and black dashed lines refer to the two-delay combination method obtained from the state-of-the-art methods—namely, the equal weights, the inverse of RMSE weights and the variance-no-covariance weight methods.

The red lines show the predicted closing prices obtained from the two-delay combination model using the DE weight-optimization algorithm. The predicted prices obtained from the single delay and two-delay combination predictive models show similar stock price patterns to the actual stock prices. Our proposed combination model via DE weight optimization performed better than the individual forecast and the state-of-the-art weight linear combination because our proposed model provides the lowest RMSE and MAPE for all stocks.

In this work, we focus on one-step-ahead prediction. Based on the results obtained from Table 6 and Figure 10, Figure 11, Figure 12, Figure 13, Figure 14, Figure 15, Figure 16, Figure 17 and Figure 18, we applied the two-delay combination model via the DE weight optimization method to predict the one-step-ahead closing price on 1 July 2022. The forecast closing prices obtained from our proposed model on the MSFT, JNJ and PFE stocks with three deep learning models are shown in

Table 8. One-step-ahead prediction of MSFT, JNJ and PFE stocks obtained from the two-delay combination model on 1 July 2022 at 9.30 a.m.

Model	Stock
	MSFT	JNJ	PFE
MLP	255.07	176.73	52.13
CNN	258.44	176.81	51.56
LSTM	258.69	177.58	52.17

.

5. Discussion of the Results

The empirical results show that the patterns of the predicted closing price obtained from our proposed method are similar to the actual prices for all three stocks. It is interesting for the Johnson & Johnson stock (Figure 11) that the predicted prices obtained from our proposed model reached 181.73 USD on 10 February 2022 because there was a steady actual price drop since 1 February 2022 and a rapid increase to 171.66 USD on 9 February 2022. However, the accuracy improved when the CNN model was applied as shown in Figure 14.

In addition, the predicted prices are matched to the actual prices when using the LSTM model because this model is capable of handling long-term dependencies. Another exciting thing is that our proposed technique via the DE weight optimization method is outstanding when the individual method performs poorly in predicting the observations. For example, in Figure 18, all individual and state-of-the-art combination methods provide low performance (underestimate) to predict the Pfizer stock.

Using the DE weight optimization in the combination procedure can improve the accuracy of the predicted prices. In summary, the evaluation metrics obtained from the DE weight optimization method are lower than the single delay and the other state-of-the-art weight combination methods for all stocks. It can be confirmed that the two-delay combination model using the differential evolution weights identification was more effective than the single DL technique and the other baseline linear combination methods.

6. Conclusions

Stock price prediction has become widespread in research. Accurately predicted price movements help investors decide to buy and sell stocks and reduce unexpected risks. This paper proposes a new combination technique to predict closing prices. This technique is called the two-delay combination model because it combines two different delays—namely, half-day and one-day—for each popular deep learning method. One challenging task of the linear combination forecast is weight identification. We utilized the differential evolution algorithm to obtain the optimal weights of the linear combination forecast procedures.

Three high-frequency multivariate data in financial markets used in this paper were collected from the Thomson Reuters database every 5 min between January 2021 and June 2022. The training (January–October 2021) and validation (November–December 2021) sets were applied to construct and validate the model, while the remaining six months (January–June 2022) were used to assess the model performance. Three deep learning models—namely, the MLP, CNN and LSTM—were tested using three stocks, including Microsoft Corporation (MSFT), Johnson & Johnson (JNJ) and Pfizer Inc. (PFE).

We evaluated the performance of our proposed model by comparing the evaluation metrics, including the RMSE and the MAPE, with the individual method. In our experiments, we compared the evaluation metrics of the proposed and single DL models. We also showed that linear combination forecast using DE weight optimization performed better than other state-of-the-art weight estimations. One benefit of our proposed technique is that applying several DL methods in the combination forecast method is unnecessary.

Based on the general idea of the combination forecast model, it requires at least two different methods to build the forecast combination. Under our proposed technique, using only one method with two different data delays can reduce the evaluation metrics. Consequently, the proposed delay combination model is a method with potentially satisfactory prediction performance. Our future research work will attempt to add further external factors (such as the company’s profit/loss, demand and supply) to improve the forecasting accuracy. We will also consider multi-step-ahead predictions on the stock price time series.