Optimizing Multivariate Time Series Forecasting with Data Augmentation

Abstract

The convergence of data mining and deep learning has become an invaluable tool for gaining insights into evolving events and trends. However, a persistent challenge in utilizing these techniques for forecasting lies in the limited access to comprehensive, error-free data. This challenge is particularly pronounced in financial time series datasets, which are known for their volatility. To address this issue, a novel approach to data augmentation has been introduced, specifically tailored for financial time series forecasting. This approach leverages the power of Generative Adversarial Networks to generate synthetic data that replicate the distribution of authentic data. By integrating synthetic data with real data, the proposed approach significantly improves forecasting accuracy. Tests with real datasets have proven that this method offers a marked improvement over models that rely only on real data.

1. Introduction

Understanding and analyzing economic indicators is a fundamental aspect of a country’s macroeconomic policymaking and decision-making process. In time series analysis, one of the major challenges is the scarcity of data. For example, in the field of earned value management, accurately estimating project timelines and finances depends on recorded economic data. While this information is often available from official sources, a primary issue is the infrequent reporting—typically on a monthly or annual basis. As a result, there are situations where sufficient, high-quality data on the factors influencing time series trends may not be readily accessible. This scarcity of data can significantly affect the quality of predictive models. To address the data scarcity problem in data science, various prototyping approaches have been introduced. However, these methods often fall short of improving predictive models due to their assumption of linearity in changes. As a result, the field of time series analysis has shifted toward more complex methodologies. These advanced techniques allow for a more accurate analysis of non-linear patterns within time trends.

These indicators exhibit fluctuations or trends over time, making them a type of time series data. To analyze these time series, advanced computational and algorithmic tools have been introduced, enabling the application of time series prediction methods. The fundamental concept of prediction-based models involves utilizing past data to train a model that can accurately predict the time series data that need to be monitored. Following this, the forecasted results are compared against the original time series data (Lei et al. 2023). Time series data often exhibit oscillatory patterns, which can range from simple to complex. Historically, conventional methods like Autoregressive Moving Average (ARMA), and Autoregressive Integrated Moving Average (ARIMA) were utilized to identify these patterns (Chang et al. 2023; Nazareth and Reddy 2023). However, these methods struggled to identify intricate patterns due to their simplicity. In contrast, deep learning methods have gained popularity in time series applications, as well as in fields like machine vision, owing to their versatility and effectiveness (Liu and Lin 2021; Liu et al. 2017; Ma et al. 2021; Nguyen et al. 2021). Various types of deep learning models have been introduced in the literature. By leveraging new computational tools and implementing more complex algorithms, particularly using Long Short-Term Memory network (LSTM) computing units (Nguyen et al. 2021; Somu et al. 2021; Wang et al. 2020b; Xayasouk et al. 2020), it is now possible to extract more intricate patterns from time series data, incorporating additional features and thereby enhancing the accuracy of predictions.

Traditional Recurrent Neural Networks (RNNs) encounter challenges in capturing long-term dependencies since they have a tendency to forget information from previous time steps. Addressing this issue, LSTM networks employ gates to regulate information flow within the network, enabling selective retention of pertinent data from prior time steps, a vital characteristic for effectively capturing long-term dependencies within financial data. Moreover, LSTM networks demonstrate proficiency in handling data noise and outliers, as demonstrated in recent research (Fang et al. 2023). Additionally, Bidirectional LSTM (Bi-LSTM), a variant of Recurrent Neural Networks (RNNs), exhibits the unique ability to process data in both forward and reverse directions simultaneously, setting it apart from traditional LSTM models that operate solely in the forward direction (Wang et al. 2023). Bi-LSTM networks, owing to their bidirectional time series analysis, acquire a more comprehensive comprehension of temporal data during training, yielding fewer prediction errors in comparison to other network architectures (Liu and Lin 2021; Ma et al. 2021).

Despite the growing demand for data-driven approaches in various applications, the limited availability and high cost of acquiring large datasets remain significant challenges. This is particularly true for time series data, which often exhibit complex patterns and require extensive training data. Economic characteristic analysis, due to its complexity stemming from several effective factors, can be hindered by the limited availability and low frequency of financial datasets, especially when applying deep learning models like Bi-LSTM for forecasting. To address these limitations, data augmentation has emerged as a promising solution, involving the generation of synthetic data resembling the original data. This approach expands the training dataset and enhances the generalizability of predictive models. However, traditional data augmentation techniques, such as adding noise or scaling the data, may introduce noise and inconsistencies in the generated data. This is where Generative Adversarial Networks (GANs) come into play (Goodfellow et al. 2020). GANs, capable of generating random samples, can enhance the robustness and generalization of financial models. Consequently, researching the application of GANs in prediction tasks has become an increasingly popular topic (Xu et al. 2022). GANs are a type of deep learning model capable of generating new data that closely resemble the original data. In the context of financial data, which are often limited in volume and frequency, GANs have proven to be valuable. Studies also indicate that GANs can significantly enhance the performance of Bi-LSTM for forecasting financial data, as demonstrated by the study in 2018, which showed a 10% improvement in classification accuracy when using GAN-generated synthetic data compared to traditional augmentation methods (Frid-Adar et al. 2018; Brophy et al. 2021).

To achieve more accurate forecasting, this study proposes an approach consisting of the following steps:

• Training GANs to generate synthetic time series data that closely mimic real-world datasets;
• Utilizing a Bidirectional Wasserstein Generative Adversarial Network (Bi-WGAN), an advanced model that integrates the features of Bidirectional LSTM with Wasserstein GANs;
• Leveraging the generated data to train deep learning models, including LSTM networks, for time series prediction;
• Comparing the performance of the forecasting models trained on a combination of real and synthetic data.

The proposed approach overcomes some of the limitations of traditional time series forecasting methods by harnessing the strengths of both GANs and deep learning models. The innovations introduced in this research can be categorized into two main aspects:

• Combining Bi-directional LSTM (Bi-LSTM) networks with the Wasserstein Generative Adversarial Network (WGAN) to enhance training and prevent mode collapse in real data distribution mapping;
• Conducting a comparative study on the use of WLoss and Bi-LSTM functions separately and in combination, along with a comparison of prediction errors in the predictive model.

The paper’s structure is as follows: Section 2 provides a brief review of the literature, highlighting the effectiveness of deep neural networks, particularly RNNs such as LSTM and GANs, in time series prediction applications. Section 3 explains the hybrid deep learning model that combines WGANs and Bi-LSTM networks for forecasting time series data. In Section 4, the dataset is described and implementation details are defined. Section 5 presents the experimental results and discussions, followed by the discussion and conclusion in Section 6 and Section 7.

2. Literature Review

Time series forecasting plays a pivotal role in contemporary business planning, shaping both short-term and long-term organizational goals. The pursuit of accurate and reliable forecasts has become an ongoing effort for many enterprises, resulting in substantial cost savings and efficiencies (Bandara et al. 2020). Leveraging historical and current time-series data, organizations make informed predictions about the future. These predictions span a wide spectrum of applications, ranging from weather forecasting to financial projections (Chandra 2015). In order to predict time series, effective steps have been presented by deep neural networks that offer wide applications, including energy (Somu et al. 2021; Wang et al. 2020a), financial domains (Lu et al. 2021b; Moghar and Hamiche 2020; Yadav et al. 2020), gold volatility prediction (Vidal and Kristjanpoller 2020), cryptocurrency price prediction (Patel et al. 2020), weather forecast indicators (Xayasouk et al. 2020), travel time (Liu et al. 2017), etc.

2.1. Effectiveness of Deep Neural Networks in Time Series Prediction

Deep neural networks, particularly Recurrent Neural Networks (RNNs) such as LSTM (Long Short-Term Memory), have been shown to be effective in time series prediction. These networks are able to capture the temporal dependencies in time series data, which is crucial for accurate forecasting. Recurrent Neural Networks (RNNs) are specialized architectures designed for handling sequential data. They use a vector of hidden states that act as memory, which is preserved and updated over time. Recurrent Neural Networks (RNNs) are specifically designed to extract temporal information from raw sequential data. Although RNNs were developed over two decades ago, their popularity has surged across diverse application domains only recently. This surge can be attributed to advancements in optimization techniques, computational hardware, and the availability of large-scale datasets (Tran et al. 2018; Liu et al. 2020).

RNNs have been successfully applied to a variety of time series prediction tasks, including energy consumption, financial forecasting, air pollution prediction, travel time prediction, and COVID-19 prevalence rate prediction. For example, by using recurrent networks and LSTM along with improving temporal correlations, predicting photovoltaic energy (PV) due to climate fluctuations and solar energy changes was carried out (Wang et al. 2020a). Employing a technique akin to the one utilized in a 2020 study to identify periodicity in energy consumption trends, the LSTM error (RMSE) was observed to be 64.59% lower than that of the linear and conventional ARMA method (Wang et al. 2020b). Also, in a study, the Kernel Convolutional Neural Network (kCNN-LSTM) method was introduced to predict energy consumption by receiving spatio-temporal properties (Somu et al. 2021). This method consists of a clustering algorithm and deep networks in two structures of Convolutional Neural Networks (CNNs) and LSTM. The Convolutional Neural Network (CNN) is a type of feedforward neural network specifically designed for feature extraction from data using convolutional structures. Unlike traditional feature extraction methods, CNNs eliminate the need for manual feature extraction. The architecture of CNNs draws inspiration from visual perception. Each biological neuron corresponds to an artificial neuron, while CNN kernels act as diverse receptors capable of responding to various features. Activation functions simulate the behavior where only neural electric signals surpassing a certain threshold can be transmitted to the next neuron (Li et al. 2021). Recently, CNNs have been explored as tools for feature extraction and classification in intrusion detection systems, thanks to their capability to handle complex data (Aldweesh et al. 2020).

The effectiveness of hybrid models was also explored by testing photovoltaic energy time data, revealing that reinforcing the LSTM network with a CNN network could reduce prediction errors over extended time periods (Agga et al. 2021).

2.2. Benefits of Bidirectional LSTM for Time Series Prediction

Among the applications of the financial field are the use of recurrent networks in stock market forecasting (Moghar and Hamiche 2020), optimization of LSTM networks for Indian stock market forecasting, and combination of CNN, Bi-LSTM, and Attention Mechanism (AM) for a similar purpose. Xayasouk and colleagues predicted the air pollution index by collecting datasets from feature records over three years using deep LSTM and deep autoencoder networks, and comparing the Root Mean Square Error (RMSE) of the two networks, they found that LSTM performed better (Xayasouk et al. 2020).

In forecasting travel time, LSTM had less RMSE error than other classical methods such as linear regression, logistic regression, lasso regression, and ARIMA, another example of the superiority of this method compared to linear and less complex methods. However, conventional and less complex methods can help to strengthen deep learning methods (Liu et al. 2017). To achieve this goal, researchers in 2021 combined SARIMA and LSTM to forecast tourists’ mentions of Macau SAR. SARIMA, capable of identifying seasonal patterns, halved the RMSE error compared to traditional LSTM (Wu et al. 2021). As another application, LSTM networks have been used to predict the prevalence rate of COVID-19 (Gautam 2022; Luo et al. 2021). Utilizing LSTM alongside the suggested Centralized Cluster Distribution (CCD) and Weighted Empirical Stretching (WES) loss function presents a novel method for enhancing Bitcoin price forecasting performance. This approach resulted in RMSE improvements of 11.5% and 22.5% across various label domains (Koo and Kim 2024).

Another type of LSTM, bidirectional LSTM, has improved predictive performance in some applications. A Bi-LSTM network integrates two hidden layers that process data in opposite directions before merging their outputs into a single layer. This configuration allows the network to consider both past and future states, thereby enhancing its predictive accuracy by learning from two data directions. Essentially, a Bi-LSTM divides standard LSTM neurons into a positive state (forward time direction) and a reverse state (backward time direction). At each step, the output consists of both positive and negative bidirectional LSTM, enabling the network to train on historical data from both directions and extract more valuable information (Graves and Schmidhuber 2005). In Figure 1, we illustrate the structure of a basic Bi-LSTM for clarity. For example, in a comparative study, the Bi-LSTM structure—owing to its two-way data traversal—better captures the sequential features in time series data, leading to a 37.78% improvement in model accuracy (Siami-Namini et al. 2019). In an applied example by combining the empirical Mode Decomposition approach and Bi-LSTM network structure, Beijing air quality index prediction for different time horizons was investigated, which was reported in three horizons of 3 h, 6 h, and 12 h for the lower RMSE error combination model (Zhang et al. 2021). Additionally, in another applied research study, short-term traffic was predicted using Bi-LSTM, resulting in an approximate 20% improvement in RMSE error due to the use of bidirectional layers. (Ma et al. 2021). Bi et al. (2024) proposed a hybrid prediction method called Variational Mode Decomposition, Bidirectional Input Attention, Encoder, Decoder (VBAED) to predict water-quality time series data. This method combines variational mode decomposition, bidirectional input attention, a bidirectional LSTM encoder, and a bidirectional temporal attention mechanism in a decoder (Bi et al. 2024). The method is superior to existing methods in capturing long-term dependence, noise filtering, and prediction accuracy.

2.3. Data Augmentation with Generative Adversarial Networks (GANs)

Deep neural networks often require large datasets for training. However, in many real-world applications, data availability is limited. To enrich the dataset augmentation, it is possible to turn to a productive method that has been proven in numerous articles, i.e., using generative adversarial networks or GANs. After the introduction of GAN architectures (Goodfellow et al. 2014), several pieces of research have been conducted on their application in the field of machine vision and using image datasets (Frid-Adar et al. 2018; Karras et al. 2017; Wang et al. 2018). However, these networks have also been successful with different applications in the field of time series. For example, an LSTM-based Variational Autoencoder Generative Adversarial Network (VAE-GAN) anomaly detection method is designed for equipment condition monitoring (Niu et al. 2020). Additionally, there is more research that uses GANs for anomaly detection (Lee et al. 2021; Lu et al. 2021a). To employ GANs for analyzing sequential data, a method called quant GAN was introduced in a paper in 2020. This technique enables the generation of high-quality financial time series data, including the S&P (Standard & Poor’s 500) index dataset spanning nine years (from May 2009 to December 2018) (Wiese et al. 2020). Furthermore, short-term load prediction has been investigated by combining GANs and LSTM structures to provide artificial data to better model deep networks (Zhang et al. 2020). Using Balanced Generative Adversarial Network (B-GAN) structures is an example of intrusion data generation that merges LSTM layers as a data balancing strategy. The generated samples helped in detecting intrusions in industrial control systems (Yuan et al. 2023). GAN has also been used in the time series field to help classify time series in imbalanced conditions (Deng et al. 2021). LSTM networks are widely acknowledged as highly effective sequential models for time series predictions. We assess their performance against GANs, renowned for their ability to maintain realism. GANs have demonstrated remarkable performance in generating photorealistic high-quality facial images (Quilodrán-Casas et al. 2022). Another example of the GAN application in time series is the prediction of the COVID-19 outbreak (Silva et al. 2021). Additionally, the mode collapse issue was addressed through the WGAN architecture in 2017, which minimized the distribution distance. Subsequently, this structure found application in the time series domain, as indicated by recent research (Arjovsky et al. 2017; Pfenninger et al. 2021). A group of authors in a research combined bidirectional LSTM with GANs in the time series (Wiese et al. 2020), and in another study, bidirectional LSTM was combined with GANs in order to replace and predict time series (Gupta and Beheshti 2020). Through experimentation on three dataset series, the hybrid network introduced in both applications demonstrated superior performance compared to other conventional methods. Another instance involved incorporating Bi-directional LSTM layers into a GAN structure, which was used to analyze crude oil time series data for market risk prediction (Zou et al. 2023).

2.4. Contribution of the Present Research

Based on the background of the research mentioned in the present study, deep learning methods will be used for the purpose of the prediction of effective economic indicators in the Iranian market. Due to the lack of synced data in terms of time and to reduce the effect of data deficiency, Wasserstein GANs are used to strengthen the dataset. The new dataset, a combination of synthetic and real data, is predicted in the next step using Bi-LSTM networks. The contribution of this research in two parts is to investigate the effect of data enhancement with WGAN and the performance of the WGAN–Bi-LSTM hybrid network in comparison with the methods used in recent research.

Given the diversity of methods used in time series forecasting,

Table 1. Comparison of various methods in time series forecasting.

Method	Acronym	Advantages	Disadvantages
Long Short-Term Memory	LSTM	Captures temporal dependencies	Requires large datasets
Bidirectional LSTM	Bi-LSTM	Considers both past and future data	More complex architecture
Kernel Convolutional Neural Network	kCNN-LSTM	Effective for feature extraction from spatio-temporal data	May require extensive computational resources
Variational Autoencoder GAN	VAE-GAN	Effective for anomaly detection	Sensitive to hyperparameters
Wasserstein GAN	WGAN	Addresses mode collapse issues	More complex to implement than standard GANs
Balanced GAN	B-GAN	Useful for generating balanced datasets	May not generalize well to unseen data
Generative Adversarial Network	GAN	Good at generating realistic data	Requires careful training and tuning

presents a concise comparison of various approaches, including their acronyms, along with their respective advantages and disadvantages. This information is intended to enhance the reader’s understanding of each method’s applicability and limitations, supporting more informed decision-making in the field of time series forecasting.

3. Methodology

Given that the goal of solving a prediction problem is to simultaneously analyze multiple indicators, deep networks and LSTM layers are used as the prediction methods in this paper. Since these methods require large datasets, before training the prediction algorithm, first, by deep GAN, we produce examples similar to real ones and strengthen the dataset. GAN includes two generator networks whose output is production sequences and discriminators that distinguish real examples from unrealistic ones by determining the actual/fake label (label 1 for the actual sample and label 0 for the fake sample). However, due to mode collapse problems and vanishing gradients associated with conventional GANs, this paper utilizes the Wasserstein GAN with Gradient Penalty (WGAN-GP) method. In these networks, the critic network is employed instead of discriminators. The role of the critic network is akin to identifying fake samples, but its output may not be constrained within the range of 0 to 1, as sigmoid is no longer used in the final layer of this network (Arjovsky et al. 2017). Generative Adversarial Networks (GANs) have been widely used for various applications. However, they often face issues such as vanishing gradients and instability during training. To address these challenges, we employed the Wasserstein Generative Adversarial Network (WGAN), which utilizes the Wasserstein distance as its loss function. This approach not only improves the convergence properties but also enhances the stability of training processes, particularly for high-dimensional data. we preferred the Wasserstein GAN (WGAN) over traditional GANs due to its ability to mitigate issues related to gradient vanishing and instability during the training phase. The advantages of WGAN make it particularly suitable for time-series data, allowing for more reliable results, as demonstrated in our experiments. Equations (1) and (2) are based on the standard formulations in Generative Adversarial Networks (Goodfellow et al. 2014). The difference between the GANs and the WGAN is in their loss function. In conventional GANs, the Binary Cross Entropy or BCE function is used to calculate loss, which is common in:

$${\mathrm{L}\mathrm{o}\mathrm{s}\mathrm{s}}_{\mathrm{G}}={\displaystyle \frac{1}{\mathrm{m}}}{\sum }_{\mathrm{i}=1}^{\mathrm{m}}\mathrm{l}\mathrm{o}\mathrm{g}\left(1-\mathrm{D}\left(\mathrm{G}\left({\mathrm{z}}^{\left(\mathrm{i}\right)}\right)\right)\right),$$

(1)

binary classification problems:

$${\mathrm{L}\mathrm{o}\mathrm{s}\mathrm{s}}_{\mathrm{D}}={\displaystyle \frac{1}{\mathrm{m}}}{\sum }_{\mathrm{i}=1}^{\mathrm{m}}\left[-\mathrm{l}\mathrm{o}\mathrm{g}\left(\mathrm{D}\left({\mathrm{x}}^{\left(\mathrm{i}\right)}\right)-\mathrm{l}\mathrm{o}\mathrm{g}\left(\mathrm{D}\left(\mathrm{G}\left({\mathrm{z}}^{\left(\mathrm{i}\right)}\right)\right)\right)\right)\right],$$

(2)

where D and G are symbols of discriminator and generator networks, respectively, and z is the input noise vector to the G and x networks, as well as an example of a real dataset. In this way, G(z) is the output of the generative network and is considered a fake sample, so in the Loss_D calculation, a data set including real and fake data is given to the distinguished network to update the weights, while for Loss_G, fake samples are used to calculate loss and update the weights of the productive network. Figure 2 shows the structure of a simple WGAN.

Equation (3) represents the Wasserstein loss as introduced by Arjovsky et al. (2017). However, in WGANs, to solve mode collapse problems and vanishing gradients, the cost function is defined so that the output distribution of the generated network is close to the real data distribution, and the generator can find a map of the rand space to the real data distribution space. In this way, in WGANs, the cost function (loss) or Wasserstein loss can be defined as follows:

$$\mathrm{W}\mathrm{L}\mathrm{o}\mathrm{s}\mathrm{s}=\underset{\mathrm{G}}{\mathrm{m}\mathrm{i}\mathrm{n}}\underset{\mathrm{C}}{\mathrm{m}\mathrm{a}\mathrm{x}}\underset{\mathrm{x}~{\mathbb{P}}_{\mathrm{r}}}{\mathbb{E}}\left[\mathrm{D}(\mathrm{x})\right]-\underset{\tilde{\mathrm{x}}~{\mathbb{P}}_{\mathrm{g}}}{\mathbb{E}}\left[\mathrm{D}\left(\tilde{\mathrm{x}}\right)\right],$$

(3)

where $\tilde{\mathrm{x}}$ is the output of the generated network or the same fake samples, and ${\mathbb{p}}_{\mathrm{g}}$ is the distribution of the model by the generator network as well as the ${\mathbb{P}}_{\mathrm{r}}$ of the actual data distribution. In fact, in WGAN training, the g network seeks to minimize this value, and the c network seeks to maximize WLoss. The gradient value of this function behaves better than the conventional function in such a way that it optimizes the generator with better quality (Arjovsky et al. 2017).

LSTM is an advanced variant of Recurrent Neural Networks (RNNs) that excels at capturing long-term dependencies in sequential data. Unlike conventional RNNs, LSTM tackles the vanishing gradient problem by employing specialized memory units. LSTM networks utilize two state vectors to effectively capture long-term dependencies in sequential data. The first state vector, $h_t$, stores the current state of the layer and its output at time t. The second state vector, $c_t$, incorporates more distant dependencies by continuously updating and modifying itself. These updates are regulated by four gates: input (i), output (o), forget (f), and cell candidate (g), which regulate the flow of information between the memory units.

• Input gate: The input gate $i_t$ controls the flow of new information into the cell state. It determines which parts of the current input are relevant and should be incorporated into the state representation.
• Forget gate: The forget gate $f_t$ controls the flow of old information out of the cell state. It decides which parts of the previous state should be retained or discarded.
• Cell candidate gate: The cell candidate gate $g_t$ proposes the new information that should be added to the cell state. It generates a candidate state vector that potentially contains new information from the current input.
• Output gate: The output gate $o_t$ controls the output of the LSTM cell. It determines which parts of the cell state are relevant and should be passed on as the next hidden state.

These gates work together to ensure that the LSTM cell can effectively capture long-term dependencies in the data by selectively storing and forgetting information over time. To enable the LSTM network to learn effectively, it is necessary to train the input weights, recurrent part weights, and biases stored in vectors W, R, and b, respectively. Input weights W combine information from different input nodes in the network. The recurrent part weights in R control the proportion of information that moves from one time step to the next, and the vector b contains biases. Depending on whether they belong to the forget, cell candidate, input, or output gates, the subset of weights and biases are assigned to a subvector with the subscript f, c, i, or o, respectively (Equations (4)–(12) are based on standard formulations in the literature regarding LSTM and Bi-LSTM networks (Graves and Schmidhuber 2005)).

$$\mathrm{weights}\mathrm{and}\mathrm{bias}\mathrm{matrix}=\left[W_k,R_k,b_k\right],$$

(4)

$$k_t={\sigma }_k\left(W_k{h}_{t-1}+R_k{x}_t+b_k\right),k\in \left\{i,f,o,g\right\},$$

(5)

$${\sigma }_{i,f,o}\left(x\right)=sigmoidactivationfunction\sigma ={\left(1+e^x\right)}^{-1},$$

(6)

$${\sigma }_c\left(x\right)=tanh(x)={\displaystyle \frac{e^{2x}-1}{e^{2x}+1}},$$

(7)

The t-th hidden states of Bi-LSTM for the forward and backward states are calculated using the following equations:

$$c_t=f_t\odot c_{t-1}+i_t\odot g_t,$$

(8)

$$h_t=h_{t-1}\odot tanh\left(c_t\right),$$

(9)

The traditional LSTM is limited in its ability to capture sequential information due to its unidirectional processing. This can lead to incomplete analysis of time series data. To address this issue, the Bi-LSTM was developed with a bidirectional architecture that can process input signals in both forward and backward directions. The Bi-LSTM consists of two LSTM layers that operate in parallel and in opposite directions. The forward propagation layer, represented by $h_{f(t)}$, captures information from past sequence values. The backward propagation layer, represented by $h_{b(t)}$, extracts information from future dataset values.

$$h_{f\left(t\right)}={\sigma }_c\left(W_{fh}{x}_t+R_{fh}{h}_{f\left(t-1\right)}+b_f\right),$$

(10)

$$h_{b\left(t\right)}={\sigma }_c\left(W_{bh}{x}_t+R_{bh}{h}_{b\left(t-1\right)}+b_b\right),$$

(11)

where $W_{fh}$ and $W_{bh}$ represent the forward and backward synapsis weights from the input to the internal unit weights, $R_{fh}$ and $R_{bh}$ represent the forward and backward feedback recurrent weights, and $b_f$ and $b_b$ correspond to bias information in two directions. The final output of the Bi-LSTM, $h_t$, is calculated using the following equation:

$$y_t=\sigma (W_{fhy}{h}_{f\left(t\right)}+W_{bhy}{h}_{b\left(t\right)}+b_b),$$

(12)

where $W_{fhy}$ and $W_{bhy}$ represent the forward and backward weights of the output layer, respectively, and $b_b$ denotes the output bias. The activation function of the output layer (σ) in a Bi-LSTM can be either a sigmoidal or a linear function and depends on the desired outcome.

The metrics utilized for evaluating prediction accuracy include Mean Square Error (MSE) and Root Mean Square Error (RMSE), which gauge the alignment between predicted and actual closing prices. Here, $\stackrel{\wedge }{{\mathrm{y}}_{\mathrm{i}}}(\mathrm{t})$ denotes the predicted closing price at time t, ${\mathrm{y}}_{\mathrm{i}}(\mathrm{t})$ represents the actual closing price at time t, and n denotes the total number of data points. Lower values of MSE and RMSE indicate superior model performance.

$$\mathrm{M}\mathrm{S}\mathrm{E}={\displaystyle \frac{1}{\mathrm{n}}}{\sum }_{\mathrm{i}=1}^{\mathrm{n}}{(\stackrel{\wedge }{{\mathrm{y}}_{\mathrm{i}}}\left(\mathrm{t}\right)-{\mathrm{y}}_{\mathrm{i}}(\mathrm{t}))}^2,$$

(13)

$$\mathrm{R}\mathrm{M}\mathrm{S}\mathrm{E}=\sqrt{{\displaystyle \frac{1}{\mathrm{n}}}{\sum }_{\mathrm{i}=1}^{\mathrm{n}}{\left[\stackrel{\wedge }{{\mathrm{y}}_{\mathrm{i}}}\left(\mathrm{t}\right)-{\mathrm{y}}_{\mathrm{i}}\left(\mathrm{t}\right)\right]}^2},$$

(14)

4. Implementation

In this section, we outline the step-by-step process followed throughout the research. As depicted in Figure 3, the workflow consists of several key stages, from data collection to real-world testing. Below is a detailed explanation of each step to provide a clearer understanding of the methodology.

• Data Collection (Step 1):

We collected financial data from various sources in Iran, resulting in ten distinct data categories.

• Data Preprocessing (Step 2):

During preprocessing, we encountered challenges such as differing weekend closures—some datasets closed on Thursday and Friday, while others closed on Saturday and Sunday. We resolved these inconsistencies using a noise correction method. Additionally, we observed a strong correlation between gold and coin prices, so we removed one to avoid redundancy. We also normalized the data to ensure consistency for modeling.

• Data Augmentation (Step 3):

LSTM models typically require large amounts of data, and GANs are highly effective for data generation. Therefore, we utilized a WGAN network, which is particularly well suited for time series data like ours. This technique enhanced the dataset, improving the training process.

• Improving the LSTM Model (Step 4):

We enhanced the LSTM model by incorporating Bidirectional LSTM (Bi-LSTM), which allows the model to learn in both forward and backward directions. This approach led to improved learning outcomes and more accurate predictions.

• Model Tuning (Step 5):

In time series analysis, model tuning is critical. We followed best practices and set the lambda parameter to 1/2. Detailed hyperparameters are available in

Table 2. Specifications of the generator network structure.

Generator
Hyperparameter	value
Layer type	conv, FC
Layer num	3
Dropout	0
Epoch	100
Learning rate	0.0005
Optimizer	RMSProp

and

Table 3. Specifications of the critic network structure.

Critic
Hyperparameter	value
Layer type	LSTM, FC
Layer num	4
Dropout	0
Epoch	100
Learning rate	0.0005
Optimizer	RMSProp
Critic learn frequency	5

.

• Comparison (Step 6):

We compared the performance of models with and without GAN-based data augmentation to assess its impact on the final results.

• Real-World Testing (Step 7):

Finally, we tested the model on real-world data to verify its performance in practical scenarios.

4.1. Data Collection

To explore the hybrid implementation method, we extracted and compiled a dataset containing economic parameters from Iran, sourced from TGJU—Totally Governmental Javaher Ushering. This dataset spans from 22 November 2019 to 21 December 2021 and includes eight key international and domestic economic indicators. These indicators consist of the global stock index (S&P), Bitcoin and Ethereum prices, aluminum prices, global gold prices, gold prices in IRR, coin prices, and the Iranian stock index. Figure 4 presents a visual representation of the price features within this dataset, and a summary of the dataset’s characteristics is provided in

Table 4. Description of the scraped dataset.

Series	Global stock index (S&P) Price of Bitcoin and Ethereum cryptocurrencies Price of aluminum Global price of gold Price of gold in IRR Price of coins Iranian stock index
Sequence length	20
Period	One day
Number of records	740
Test	148
Train	592

.

4.2. Data Processing

Due to the significant differences in the value ranges of various features input into the model, it is essential to normalize the data. This step is crucial to prevent the negative effects that differing numerical ranges can have on data augmentation and prediction. Normalizing the data reduces its range to between −1 and 1. The formula for this data normalization is as follows:

$${\displaystyle \frac{\mathrm{x}-\mathrm{m}\mathrm{i}\mathrm{n}\mathrm{V}\mathrm{a}\mathrm{l}\mathrm{u}\mathrm{e}}{\mathrm{m}\mathrm{a}\mathrm{x}\mathrm{V}\mathrm{a}\mathrm{l}\mathrm{u}\mathrm{e}-\mathrm{m}\mathrm{i}\mathrm{n}\mathrm{V}\mathrm{a}\mathrm{l}\mathrm{u}\mathrm{e}}}=\mathrm{y},$$

(15)

Table 4 describes the dataset used in this study, which includes various financial series. These series consist of the global stock index (S&P), the prices of Bitcoin and Ethereum cryptocurrencies, the global price of aluminum, and both the worldwide and the Iranian prices of gold. Additionally, it includes the price of coins and the Iranian stock index. The dataset is structured with a sequence length of 20, meaning that each data point is based on a sequence of 20 consecutive daily records. In total, the dataset contains 740 records, divided into 592 for training and 148 for testing. This combination of diverse financial indicators allows for comprehensive analysis and accurate forecasting across various time series.

The proposed Bi-WGAN algorithm is implemented following the process outlined in Figure 2. First, the dataset is normalized and cleaned. Next, the WGAN algorithm is trained using the parameters specified in Table 2 and Table 3. Once training is complete, the generator network produces synthetic samples to enhance the training process in the predictive network. In similar applications (Sundaram and Hulkund 2021), less than 50% of the data are artificial (between 40 and 20%), a generator will provide about 20% of the predictive algorithm training data.

Based on the experiment results, LSTM works better than other prediction methods, and GAN–LSTM works best when we do not have enough data, because LSTM needs more data and GAN helps to fill that gap. Also, Bi-GAN performs better than GAN, and among GANs, WGAN works best for time series data. The overall prediction with the WGAN tool and augmented data resulted in a lower error. Experiments were carried out in two different conditions for two different combination ratios of synthetic data, in which the LSTM and Bi-LSTM methods were compared.

Table 5. Comparative summary of the results. The MSE is reported with and without applying GAN in the prediction of time series.

		10% Generated	20% Generated
LSTM	WGAN	0.01912	0.0155
	GAN	0.01834	0.02049
	no-GAN	0.02211	0.02438
Bi-LSTM	WGAN	0.01374	0.01019
	GAN	0.01926	0.01007
	no-GAN	0.01801	0.01184

shows that the best case was using the Bi-LSTM method and network training with 20% artificial data. The error reported in this table is the MSE error (Lu et al. 2021b) or mean squared error (Zhang et al. 2020), with the help of which the predictive algorithm was used in the next step. The percentages of 10% and 20% for generated data were chosen based on findings from previous studies that showed the benefits of augmenting real-world datasets with synthetic data. Specifically, it has been shown that augmenting with less than 50% synthetic data, usually between 20% and 40%, yields optimal results. These percentages were selected to balance between enhancing the dataset without causing overfitting or introducing excessive noise. The values of 10% and 20% were used as sample points to observe how the model performance changes as the proportion of synthetic data increases. While this study focused on 10% and 20% synthetic data, further exploration of other percentages could provide additional insights. Due to resource and time constraints, only these two percentages were tested. In this study, we performed extensive hyperparameter tuning to optimize the performance of our models. One of the key parameters we tuned was the learning rate. We adjusted the learning rate logarithmically through several stages. Initially, we set the learning rate to 0.1. Then, we progressively reduced it as follows: first to 0.05, then to 0.01, followed by 0.005, 0.001, and, finally, 0.0005. After testing each value, we found that the best-performing learning rate was the one mentioned in the results section. This iterative process helped to ensure that the models were trained efficiently, avoiding both underfitting and overfitting.

5. Results and Discussion

The following section describes the performance of the proposed method and presents relevant analyses for the dataset of economic indicators. As mentioned earlier, the use of GANs comes with challenges, including the lack of a suitable method for quantitatively evaluating their results. Since this approach is an unsupervised learning method, there is no ground truth available, making it difficult to define a stopping condition for the learning process or to effectively evaluate the algorithm’s progress and output. For this purpose, the evaluation of the WGAN’s performance involves utilizing the visualization of the generated distribution, similar to prior research (Pfenninger et al. 2021).

Figure 5 illustrates the learning processes of both the generator and the critic networks. The higher number of epochs for the critic network is due to the repeated training cycles within each learning loop specific to this network. Additionally, batch size impacts the number of weight update loops in the network. With a batch size of 64 and 720 learning samples, approximately 10 weight updates occur per epoch. Considering 100 epochs, the total number of weight updates is around 1000.

In Figure 6, the diagram on the left illustrates the model’s improvement process over time as it learns, typically showing performance metrics such as error or accuracy during each training period. The diagram on the right demonstrates how closely the model’s predictions align with the actual data values. By examining this graph, one can assess the model’s prediction accuracy and observe the differences between predicted and actual values.

After training at ten epochs, samples of the signals generated by the network are visible (Figure 7). These synthetic signals’ range and manner of oscillation are similar to real signals and have similar characteristics, such as being non-stationary. The range of values of the generated numbers also indicates that similarities to real signals can be observed.

This heatmap in Figure 8 visualizes the Mean Squared Error (MSE) for the predictions made by a Support Vector Regression (SVR) model. With dimensions of (1, 148), each row corresponds to a sample, and each column shows the respective MSE. The depicted results represent the different stages of evaluating the SVR using MSE, covering input analysis, model training, MSE computation, and heatmap creation.

Nevertheless, to assess the network’s effectiveness, the distribution of generated samples is examined. As depicted in Figure 9, the log return distributions of actual and predicted data for oil, gold, aluminum, and Bitcoin are plotted against production and real datasets, showing considerable similarity. This suggests that WGAN has effectively learned to generate synthetic samples.

6. Discussion

When selecting a model for forecasting, it is vital to balance the trade-offs between accuracy, complexity, and computational cost. Traditional models like ARMA and ARIMA may be preferable for simpler, linear time series with limited resources, while neural network models like LSTM and Bi-LSTM offer powerful tools for capturing long-term dependencies in complex data. GAN-based models, although computationally intensive, provide a unique advantage by generating synthetic data and are particularly valuable in cases where data are scarce or imbalanced.

Table 6. Comparative summary of the forecasting methods.

Methods	Pros	Cons
ARMA	Simple to implement; effective for stationary time series; good for short-term forecasting	Limited to linear patterns; not suitable for non-stationary data; struggles with complex dependencies
ARIMA	Handles non-stationary data with differencing; well suited for data with seasonality or trends	Computationally expensive; requires manual parameter tuning; assumes linearity
RNN	Capable of handling sequential data; captures temporal dependencies	Prone to vanishing/exploding gradient issues; struggles with long-term dependencies
LSTM	Good at capturing long-term dependencies; reduces vanishing gradient problem; suitable for time series forecasting	High computational cost; longer training time; requires a large amount of data
Bi-LSTM	Better at capturing context from both past and future sequences; enhanced accuracy for sequential data	Increased computational complexity; higher resource demand; slower training times
GAN	Excellent at generating synthetic data; captures complex patterns in data	Difficult to train; risk of mode collapse; requires careful tuning of hyperparameters
Bi-WGAN	Improved stability over traditional GAN; can generate high-quality synthetic data; good for imbalanced datasets	High computational cost; complex architecture; requires a significant amount of data and tuning

summarizes the advantages and limitations of various forecasting models commonly used in time series analysis and prediction tasks. The methods range from traditional statistical models like ARMA and ARIMA to more complex neural networks such as RNNs, LSTM, Bi-LSTM, GANs, and Bi-WGANs.

The ARMA and ARIMA models are favored for their simplicity and effectiveness with linear and stationary data. However, they struggle with non-linear patterns and require careful parameter tuning, especially ARIMA, which is also computationally intensive due to differencing steps needed for non-stationary data.

In contrast, neural networks like RNNs and LSTM are well suited to handle complex, sequential data. LSTM in particular excels in capturing long-term dependencies in time series data, although it comes with higher computational costs and demands more data for accurate predictions. Bi-LSTM further enhances this by capturing context from both past and future sequences, but at the cost of even greater computational requirements.

Generative Adversarial Networks (GANs) offer an advanced approach by generating synthetic data and capturing complex patterns, making them useful for tasks involving data augmentation. However, GANs are challenging to train and require substantial tuning to prevent issues like mode collapse. The Bi-WGAN variant improves on GANs by adding stability and is particularly useful for generating high-quality synthetic data for imbalanced datasets, albeit with a significant increase in computational demand.

The choice of model ultimately depends on the specific requirements of the forecasting task, including the nature of the data, the need for accuracy, and available computational resources. For applications that require high precision in capturing both short-term and long-term dependencies, LSTM and Bi-LSTM are strong contenders. In cases where synthetic data are needed to boost model performance, GANs or Bi-WGANs may be the optimal choice, despite their training complexity.

7. Conclusions

In this paper, our primary objective was to enhance multivariate time series prediction using data sourced from economic indicators affecting the Iranian market. To achieve this goal, we implemented data reinforcement through the Bi-WGAN generative algorithm. Our analysis of the results demonstrates a notable enhancement in predictive accuracy. Specifically, when we integrated 20% artificial data generated by the generator network into the dataset, we observed a remarkable 53% reduction in Mean Squared Error (MSE) compared to the baseline data state, and a substantial 34% improvement when contrasted with the performance of the conventional LSTM model in comparison to the Bi-LSTM. It is worth noting that while the network’s performance exhibits significant progress, there remains room for optimization, primarily attributed to the absence of a universally recognized and reliable measurement benchmark. Nonetheless, a qualitative evaluation of the generated data, along with an examination of the log return distribution within signals related to economic features, reveals that the data distribution demonstrates a satisfactory degree of alignment with actual data. Moreover, the oscillation patterns in the generated signals closely resemble those observed in real-world signals. Additionally, our RMSE error was 0.098. In conclusion, both qualitative and quantitative analyses, as indicated by the MSE error reduction, affirm the effectiveness of our proposed approach in strengthening the dataset for improved time series prediction.