4.1. Selection of Data Sets
In order to investigate the applicability and validity of the STBL model for the forecasting of real stock indices, this part applies the STBL to the forecasting of the daily closing price of the Hong Kong Hang Seng Stock Index (HIS), with the data sample selected from the interval of 31 December 1986 to 31 December 2023 [
19]. The response variable used in the study is the daily closing price of the Hang Seng Index, whereas the influencing factors include the market factor and technical factor, totaling 29 different indicators, the details of which are listed in
Table 1. All data are sourced from the RESET Financial Database.
In this paper, the following criteria are used to select the dataset: firstly, the initial data with zero daily turnover are excluded, and all the trading data from 31 December 1986 to 31 December 2023 are selected. In order to fully assess the efficacy of the prediction model on different time scales, this paper conducts prediction analysis on the closing prices of the stock index on the 1st day (the next day), the 7th day, the 30th day, the 60th day, and the 120th day, respectively. In this study, 25% of the data are set as part of the test set, from the remaining data, 15% are classified as part of the validation set, and the remaining 60% of the data are used as part of the training set to train the model. Additionally, to optimize the training efficiency and computational accuracy of the model, this study normalized the data before model fitting by mapping all the raw data to the [0,1] interval through a linear transformation. The aim of this study is to propose the STBL model for predicting the dynamics of stock indices.
4.3. Setting of Parameter Operating Environment
The running environment for training the model was as follows: 2.4 GHz quad-core Intel Core i7 processor with 8 GB 2133 MHz LPDDR4 RAM. The development and training of the model were carried out in a Python language environment, using TensorFlow 2.2.0 as the deep learning framework.
Firstly, Adam was chosen as the optimizer in this study. Adam combines the advantages of both the adaptive gradient algorithm and root mean square propagation strategies. For example, (1) the adaptive gradient algorithm retains a learning rate for each parameter to enhance performance over sparse gradients. (2) The root-mean-square propagation adaptively reserves a learning rate for each parameter based on the mean of the nearest magnitude of the weight gradient, making it perform well when dealing with non-stationary and non-linear problems. Moreover, Adam’s algorithm is easy to implement and has high computational efficiency and low memory requirements.
When constructing the Bi-LSTM model, parameters need to be set, including the number of time steps in the window, denoted as T. This paper evaluates the short-, medium-, and long-term prediction effects of the model by choosing to calculate the model performance, respectively. The number of hidden layers in each attention module, the number of hidden layers in the encoder m, and the number of hidden layers in the decoder p are set. In this paper, we set , where the best performance is obtained on the validation set when for evaluation purposes.
The main structure of spatial–temporal-Bi-LSTM constructed in this paper consists of a fully connected layer, a spatial attention mechanism, a Bi-LSTM recursive layer, a temporal attention mechanism, and another Bi-LSTM layer. To prevent overfitting, a dropout strategy is applied during each training batch to randomly deactivate some hidden layer nodes. In this paper, the dropout rate is set to 0.2. To eliminate the effect of randomness in the training process, each model is trained 10 times, and the results are recorded and averaged to evaluate the overall performance of the model.
4.4. Analysis of Results
This paper presents a comprehensive comparison of ten models to evaluate their performance in stock index prediction. These include six baseline models: the support vector regression machine (SVR), convolutional neural network (CNN), gated recurrent unit (GRU), standard long short-term memory network (LSTM), bidirectional long short-term memory network (Bi-LSTM), and bidirectional long short-term memory network incorporating the principal component analysis (PCA-Bi-LSTM). In addition, four models that introduce attention mechanisms are examined: a bidirectional long short-term memory network incorporating a spatial attention mechanism (Spatial-Bi-LSTM), a bidirectional long short-term memory network incorporating a temporal attention mechanism (Temporal-Bi-LSTM), a standard LSTM model incorporating a spatiotemporal attention mechanism (spatial–temporal-LSTM), and a Bi-LSTM model incorporating a temporal attention mechanism (spatial–temporal-Bi-LSTM). When the step size T = 7, the RMSE and MAE values of the STBL model are 10.0881 and 260.559, respectively. Compared to the worst-performing CNN model, the RMSE and MAE of the STBL model improved by 71.19% and 72.08%, respectively. Additionally, the MAPE value of the STBL model is 1.0312%, which is a 2.5% improvement over the underperforming SVR model. When the step size is increased to 60, the performance of the STBL model remains outstanding. Regarding the number of iterations, when T = 60, the model’s performance gradually stabilizes and begins to decline after 50 iterations.
To ensure the accuracy of the evaluation results and comparability with the original data, the prediction results of all models were back-normalized.
Table 2 demonstrates the prediction performance of these models at different time steps (T = 7 and T = 60). Overall, shallow learning models such as SVR perform poorly in all time steps; in contrast, deep learning models such as CNN, GRU, and LSTM outperform the shallow learning models. In particular, the Bi-LSTM model outperforms the unidirectional LSTM in terms of accuracy due to the addition of an inverse structure to the standard LSTM, enabling it to capture spatial features and bidirectional temporal dependencies from historical data more efficiently. Similarly, the spatial–temporal-Bi-LSTM outperforms the spatial–temporal-LSTM. Moreover, models that incorporate the attention mechanism generally outperform those without it in terms of prediction accuracy, demonstrating its advantage in handling complex data features.
To comprehensively assess the effectiveness of the STBL model in stock index prediction,
Figure 8 and
Figure 9 are used in this study to show the trend of the training and validation set losses with the number of iterations at different time steps (T = 7 and T = 60). At the short time step, T = 7, the effect of the temporal attention mechanism is not significant, and the training and validation set losses show a gradual decrease regardless of whether the temporal attention mechanism is applied or not. However, the situation is different at the longer time step, T = 60. If the temporal attention mechanism is not used, the validation set loss instead shows a rising trend as the number of iterations increases; while the model using the temporal attention mechanism shows a continuous decreasing trend in the validation set loss even at longer time steps, and stabilizes after 50 iterations. This suggests that in multi-step prediction, the temporal attention mechanism can effectively capture the long-term dependency between the data, and thus the performance of the model on the training and validation sets gradually improves as the number of iterations increases.
Model complexity is primarily defined by the computational load during the forward pass and the number of parameters in the model. FLOPS (floating-point operations) serve as a crucial metric for assessing computational complexity. The STBL model has a FLOPS value of 76.9 million, indicating that it requires approximately 76.9 million floating-point operations to perform specific tasks or during forward propagation. A higher FLOPS value generally means that the model requires more computational resources, which can lead to increased running times.
The parameter count (Params) reflects the total number of learnable parameters in the model. The STBL model has 29.4 million parameters, including weights and biases. These parameters are essential elements in the learning process of the model, determining not only its learning capacity and expressive power but also its ability to generalize. While a larger number of parameters can enhance the model’s learning capabilities, it also increases the demand for memory and may lead to overfitting, especially in cases with limited data.
By comparing the complexities of different models in
Table 3, it is evident that the STBL model has a relatively low FLOPS value of 76.9 M. This typically indicates that the model operates faster and requires less computational demand. Additionally, the STBL model’s access disc value of 29.4 M is comparatively low, suggesting that it requires fewer memory resources. This not only helps reduce the computation time but also alleviates overfitting issues, making the model more suitable for use in resource-constrained environments.
Figure 10a,b shows the variations of RMSE and MAE for different models when increasing the time step. Models including BiLSTM, Spatial-BiLSTM, Temporal-BiLSTM, STBL, and PCA-BiLSTM are compared. By comparing with principal component analysis (PCA) methods, this study assesses the effectiveness of feature screening methods. The visualization of attention weights not only shows how much attention each feature receives but also helps to identify the most critical features. Comparing the PCA-BiLSTM and Spatial-BiLSTM models shows that feature screening using the spatial attention mechanism achieves lower prediction errors at any time step. In particular, the prediction accuracy of PCA-BiLSTM is lower than that of the original BiLSTM at T = 1. This may be because the PCA may cause the input data to lose some of the valid information when predicting at a single step, which in turn reduces the prediction accuracy of the model. On the contrary, through the spatial attention mechanism, the model can effectively screen and analyze the input features, extract the data structure of the input indicators more efficiently, and improve the attention to the key variables, while reducing unnecessary computation of the non-informative features during the training process of the model, which in turn reduces the risk of overfitting and high computational costs, and achieves higher prediction accuracies.
As can be seen in
Figure 10, although the prediction accuracy of all models generally decreases as the time step increases, the increase in the prediction error of Temporal-BiLSTM is smaller and decreases at T = 60 compared to T = 30. This indicates that the model with the temporal attention module has a significant advantage in performing long-term series forecasting. Models without the temporal attention mechanism have substantially higher prediction errors and significantly lower prediction accuracies when the time step increases, emphasizing the important role of the temporal attention mechanism in maintaining the long-term dependencies of the sequences.
In summary, the Bi-LSTM model based on the attention mechanism achieves better performance in stock index sequence prediction by deeply learning the spatiotemporal relationships between different attributes and sequences, providing strong interpretability for this neural network model.
Figure 11 and
Figure 12 show the prediction results of each model against the actual stock closing prices at time steps T = 7 and T = 60, respectively. It can be observed from the graphs that the fitting errors of all models increase as the time step increases. This phenomenon suggests that although the models can capture the short-term movements of stock prices better, the accuracy and reliability of the models are still challenged in terms of long-term forecasting.
By analyzing the data in
Table 4, it is evident that the STBL model achieves significant improvements in prediction accuracy compared to the Bi-LSTM and LSTM models. Specifically, the mean absolute percentage error (MAPE) of the STBL model decreases by 21.03% and 14.72% relative to the Bi-LSTM and LSTM models, respectively. Additionally, the relative performance improvements are 43.93% and 33.08%. In terms of standard deviation (SD), the STBL model is closer to the mean error compared to the BP and LSTM models, indicating that the STBL model not only has higher accuracy in predicting exchange rate fluctuations but also ensures that most predictions deviate less from the actual positions.
The Standard & Poor’s 500 Index (abbreviated as S&P 500) is a stock market index comprising 500 major publicly traded companies in the United States, covering the period from 31 December 1986, to 31 December 2023. Established by Standard & Poor’s in 1957, the index is designed to provide a reliable indicator of the broad fluctuations in the U.S. stock market. Compared to the smaller Dow Jones Industrial Average, the S&P 500 includes a greater number of companies, offering more diversified risk. Initially, the index consisted of 425 industrial stocks, 15 railroad stocks, and 60 utility stocks. As of 1 July 1976, the composition was adjusted to 400 industrial stocks, 20 transportation stocks, 40 utility stocks, and 40 financial stocks, providing more comprehensive coverage of various market sectors.
As can be seen from
Table 5, better forecasting results can also be achieved by using the STBL model for the S&P 500 index.