Augmentation of Deep Learning Models for Multistep Traffic Speed Prediction

Abstract

Traffic speed prediction is a vital part of the intelligent transportation system (ITS). Predicting accurate traffic speed is becoming an important and challenging task with the rapid development of deep learning and increasing traffic data size. In this study, we present a deep-learning-based architecture for network-wide traffic speed prediction. We propose a deep-learning-based model consisting of a fully convolutional neural network, bidirectional long short-term memory, and attention mechanism. Our design aims to consider both backward and forward dependencies of traffic data to predict multistep network-wide traffic speed. Thus, we propose a model named AttBDLTSM-FCN for multistep traffic speed prediction. We augmented the attention-based bidirectional long short-term memory recurrent neural network with the fully convolutional network to predict the network-wide traffic speed. In traffic speed prediction, this is the first time that augmentation of AttBDLSTM and FCN have been exploited to measure the backward dependency of traffic data, as a building block for a deep architecture model. We conducted comprehensive experiments, and the experimental evaluations illustrated that the proposed architecture achieved better performance compared to state-of-the-art models when considering the short and long traffic speed prediction, e.g., 15, 30, and 60 min, in multistep traffic speed prediction.

1. Introduction

The intelligent transportation system (ITS) is an advanced tool designed to provide services related to traffic management and different modes of transportation, also enabling people to use the transportation network more intelligently and efficiently. The ITS is reflected as a key to the transport system. Traffic optimization control, network traffic planning, and accurate and real-time short-term traffic prediction in the road network play a vital role in analyzing the better traffic conditions of the road network. Several methods have been developed to enhance the efficiency and accuracy of the traffic prediction, e.g., parametric and nonparametric approaches [1].

It is challenging to model the interaction between vehicles in large-scale transportation traffic in highly populated urban road networks, and collecting traffic data is also a difficult task. Existing short-term traffic prediction methods mostly use sensor location or corridor level, which cannot achieve network-wide prediction accuracy unless N models were trained for N nodes traffic networks [2]. For time series data, existing LSTM-based prediction models use only one hidden layer and, thus, relatively shallow structure [2,3,4]. Along with a corridor location, future traffic speed values are influenced by the past speed of downstream and upstream locations [5,6,7]. The predictive performance shall be enhanced by backward and forward temporal perspectives, particularly for recurring traffic patterns, by investigating the periodicity of time series data [6]. We cannot ignore the impact of downstream and upstream speed in each network location while predicting network-wide traffic speed instead of speed at one location. The learned features will be more widespread because of both backward as well as forward dependencies of spatiotemporal data. This research focuses on predicting network-wide traffic speed to better understand traffic states in a network instead of a single road link.

We propose an AttBDLSTM-FCN model for the multistep traffic speed prediction over network-wide traffic data to overcome the aforementioned challenges. From experimental results, we come to know that our proposed model achieves better performance to predict network-wide traffic speed. In summing up, our contribution is as follows: we examined the traffic forecasting network-wide instead of at a specific location or many nearby locations. We proposed a deep architecture considering both forward and backward dependencies using AttBDLSTM-FCN to enrich the feature extraction and feature learning from time series spatial data. We examined the attention layer behavior in our model and concluded that using attention mechanism can enhance the model performance accordingly.

The rest of the paper is as follows: Section 2 describes the literature review. Section 3 describes the methodology in great detail. Section 4 describes the results and discussion. Some interesting conclusions are drawn in Section 5.

2. Literature Review

Only a few studies in short-term traffic prediction from the last few decades elaborated network-wide traffic prediction. Most of the approaches engrossed arterial or corridor levels. To increase the accuracy and efficiency of traffic forecasting, numerous studies have been proposed.

On the whole, these approaches are grouped as parametric and nonparametric approaches. In many traffic prediction practices, parametric approaches leverage counts in time series and linear regressive models. To identify the primary travel time pattern, Fei et al. [8] proposed a Bayesian dynamic linear model to predict accurate travel time. To characterize the temporal travel time evolution, many entropy-based measures were proposed by Oh and Park [9]. Ahmed et al. [10] envisaged expressway traffic flow by applying the ARIMA model. When the traffic conditions are regular, parametric approaches are very appropriate to use. However, as traffic data involve nonlinear and stochastic properties, the performance of parametric approaches becomes undesirable. Therefore, applying the techniques mentioned above make traffic prediction unreliable.

Regarding nonparametric approaches, also known as data-driven models, during the traffic speed prediction, Cai et al. [11] proposed the KNN model to depict the spatiotemporal correlation. MingHeng et al. [12] proposed a short-term traffic flow model based on SVM to predict multistep traffic flow prediction. When traffic conditions are large in scale and complex with intersections and loops, upstream and downstream both refer to relative positions, and two arbitrarily locations can be downstream and upstream. The upstream and downstream are defined with respect to space, while forward and backward dependencies are defined with respect to time.

When traffic conditions become large in scale and complex, the parametric and nonparametric approaches do not work efficiently. Deep-learning-based approaches are significantly used in traffic prediction due to their great ability to deal with spatiotemporal patterns. For traffic forecasting problems, many neural-network-based methods, e.g., feedforward NN [13], fuzzy NN [14], recurrent NN [15], hybrid NN [16], and Gaussian process (GP) [17] have been adopted. Kim et al. [18] proposed a composite road network Caps-Net architecture, which switched the max pooling task of CNN. To simultaneously predict weather conditions and traffic flow, decision level data fusion and deep belief networks were incorporated by Koesdwiady et al. [19]. Transportation has a dynamic nature, which is why RNNs models are very suitable to deal with the temporal evolution of traffic status. In traffic speed prediction, long-term temporal characteristics were captured using LSTMs by Ma et al. [3]. To predict travel demand in the large-scale transportation network, using stacked long short-term memory, Cheng et al. [20] proposed the feature-level data fusion model. Zhang et al. [21] incorporated a deep spatiotemporal residual network for the prediction of the citywide crowd flow by considering the period, temporal closeness, and the trend properties of human mobility. Zhang et al. [22] proposed a multitask model to predict network-wide traffic speed. Cui et al. [7] proposed a bidirectional and unidirectional RNN-based model for the prediction of network wide traffic speed. Adnan et al. [23] proposed a stacked bidirectional attention-based gated recurrent unit model for large-scale traffic speed prediction.

3. Methodology

In this section, architecture and its components are discussed in great detail.

3.1. Input Data

The prediction of traffic speed on a single location usually requires the series of the speed values associating with ‘n’ historical time steps for input data. We have the input data with p locations $[l_1,l_2,\cdots ,l_n]$, where each location $l_i$$[t_1,t_2,\cdots ,t]$ represents the traffic speed data at p location, with ‘t’ historical time steps (frames) from the sensor location $l_i$, that can be illustrated with the vector as shown in the following equation.

$$Y_T=\left[Y_{\left(i,t-1\right)},Y_{\left(i,t-2\right)},\dots ,Y_{\left(i,t-n\right)}\right]$$

(1)

where $Y_{i,t-1}$ indicates the speed at ith location and $t-1$ time. When traffic jams or traffic congestion propagate in a traffic network, it may affect the speed at nearby as well as faraway locations. Thus, considering these influences, network-wide speed data are taken as input data for the proposed and compared models. We aim to envisage traffic speed at ‘T’ time using ‘t’ historical time frames from the traffic network consisting of ‘P’ locations. The speed data matrix can be characterized as input.

$$Y_{P,T}=\left[\begin{array}{ccccc}{Y}_{T-n}^1& Y_{T-n+1}^1& \cdots & Y_{T-2}^1& Y_{T-1}^1\\ Y_{T-n}^2& Y_{T-n+1}^2& \ddots & Y_{T-2}^2& Y_{T-1}^2\\ ⋮& ⋮& & ⋮& ⋮\\ Y_{T-n}^P& Y_{T-n+1}^P& \cdots & Y_{T-2}^P& Y_{T-1}^P\end{array}\right]$$

(2)

Equation (2) shows every component $Y_{P,T}$ is the speed at the $p_{t}h$ location along with the tth time steps. To imply temporal features of speed statistics and rationalize the representation of the equation, the vector $Y_{P,T}=\left[Y_{i,t-1},Y_{i,t-2},\dots ,Y_{i,t-n}\right]$ denotes the speed matrix in which every element indicates ‘P’ locations’ speed values. The prediction speed output can be represented as

$${\widehat{Y}}_{P,T}=\left[\begin{array}{ccccc}{Y}_{T-n+1}^1& Y_{T-2}^1& \cdots & Y_{T-1}^1& Y_T^1\\ Y_{T-n+1}^2& Y_{T-2}^2& \ddots & Y_{T-1}^2& Y_T^2\\ ⋮& ⋮& & ⋮& \\ Y_{T-n+1}^P& Y_{T-2}^P& \cdots & Y_{T-1}^P& Y_T^P\end{array}\right]$$

(3)

3.2. Bidirectional LSTMs

The bidirectional RNN [24] deals with the sequential data in a backward and forward direction in two different hidden layers. The thought of using bidirectional LSTMs came from a bidirectional recurrent neural network. In various fields, the BDLSTMs show better performance than LSTMs, e.g., phoneme classification and speech recognition [25,26]. Figure 1 demonstrates the structure of the BDLSTM.

$$h_t={\sigma }_h\left(W_{xh}{x}_t+W_{hh}{h}_{t-1}+b_h\right)$$

(4)

$${\widehat{y}}_t={\sigma }_h\left(W_{hy}{h}_t+b_y\right)$$

(5)

$$f_t={\sigma }_g\left(W_f{x}_t+U_f{h}_{t-1}+b_f\right)$$

(6)

$$i_t={\sigma }_g\left(W_i{x}_t+U_i{h}_{t-1}+b_i\right)$$

(7)

$$o_t={\sigma }_g\left(W_o{x}_t+U_o{h}_{t-1}+b_o\right)$$

(8)

$${\tilde{C}}_t=\tan h\left(W_C{x}_t+U_C{h}_{t-1}+b_C\right)$$

(9)

The weight matrices $W_f$, $W_i$, $W_o$, and $W_C$ draw the hidden layer input to the input cell state and gates, whereas the weight matrices that join the input cell state and three gates to the output of the previous cell state are $U_f$, $U_i$, $U_o$, and $U_C$. The gate activation function is ${\sigma }_g$, technically which can be a sigmoid function. The bias vectors are $b_f$, $b_i$, $b_o$, and $b_C$, and $\tan h$ is a hyperbolic tangent function. Figure 2 depicts the structure of the AttBDLSTM architecture.

$${\widehat{y}}_t=\sigma \left({\leftharpoonup }_h,{\rightharpoonup }_h\right)$$

(10)

${\leftharpoonup }_h$ and ${\rightharpoonup }_h$ are backward and forward layer output sequences computed by utilizing positive sequence inputs from time $T-n$ to $T-1$ and vice versa, respectively. Equations (4)–(9) compute both backward and forward layers outputs by updating these LSTM equations. ${\widehat{Y}}_T$ is an output vector, where to combine the two output sequences, $\sigma$ function is used. The final output of the BDLSTM is also demonstrated by a vector the same as the LSTM output, ${\widehat{Y}}_T=\left[y_T,\dots ,y_{T+n}\right]$, where the last element $y_T$ is the predicted speed of the next time iteration, and so on.

3.3. Attention

Meanwhile, the research is aimed at the attention mechanism, we tend to study the self-attention mechanism. As the hidden state of the RNN enfolds the summary of the data in the sequence, it processes a sequence as it outputs its hidden states $h_i.$

We strengthen the contribution of the self-attention mechanism [27]. In various sequence transduction tasks, including machine translation [27], image captioning [28], mining bug repository [29] and speech emotion recognition [30], attention-based RNNs have proven very proficient. The magnitude of weights can signify the importance of hidden states $\widehat{{\propto }_i}$ learned by the network. The context vector $\widehat{r}$ is computed by Equation (11).

$$\widehat{r}=\sum _{i=1}^N{\widehat{\propto }}_i{\widehat{h}}_i$$

(11)

The input of the attention layer is ${\widehat{h}}_i$; at every time step, the attention weight $\widehat{w_i}$ is calculated as

$${\widehat{w}}_i=\tan h\left({\widehat{h}}_i\right)$$

(12)

$$\widehat{b}=a^T{m}_i+b$$

(13)

$$\widehat{{\propto }_i}=\frac{exp(\widehat{b})}{{\sum }_{k}exp(\widehat{b})}$$

(14)

The output of the attention layer at ith time step is computed as shown in Equation (15).

$$\widehat{r_i}={\widehat{\propto }}_i{\widehat{h}}_i$$

(15)

3.4. Fully Convolutional Network (FCN)

The fully convolutional network has been applied successfully in many domains such as semantic segmentation [31,32,33], breast cancer detection using historical image data [34], time series classification [35,36,37], model-free prediction of multilane traffic scenes [38], etc. We were inspired by CNN models used for time series prediction and based on the literature review. We applied a fully convolutional network to the traffic prediction problem. The model’s architecture consisted of a Conv1D, also known as temporal convolution, used for spatial/temporal input. We used temporal convolutional networks (Conv1D) in the FCN branch, which means we applied a 1D filter. A convolutional block consisted of a convolution layer, batch normalization (BN), and a sigmoid activation function.

$$y=W\ast x+b$$

(16)

$$\widehat{y}=BN+\left(y\right)$$

(17)

$$h=Sigmoid+\left(\widehat{y}\right)$$

(18)

where ∗ is the convolutional operator. We applied causal convolutional architecture provided by the TensorFlow implementation by adding the padding parameter as ‘causal’ results in causal convolutions, which means no information leakage from the future to the past in the network. The causal shift of the filter in the right direction did not depend on future inputs, as shown in Figure 3. Finally, after the convolutional block, learned features fed into the global average pooling layer rather than a fully connected layer, which vastly decreased the model’s number of parameters and weights.

3.5. AttBDLSTM-FCN

The proposed model, AttBDLSTM-FCN, augments an attention-based bidirectional LSTM with a fully convolutional network to predict multistep network-wide traffic speed. To encapsulate the bidirectional spatial features and temporal dependencies, augmentation of AttBDLSTM and FCN are exploited. In traffic speed prediction, to compute the backward dependency of data, this is the first time that augmentations of AttBDLSTM and FCN are used as a building block for a deep architecture model. Figure 4 describes the architecture of the proposed methodology. Several hidden layers of deep LSTM architectures are built up and thus work more effectively than described in existing studies [25,26,39]. AttBDLTSM-FCN comprises two blocks, as depicted in Figure 4. The FCN block consists of temporal convolutions that are applied for feature extraction generally. The global average pooling [40] is used to diminish the number of parameters.

The FCN block augments through the AttBDLSTM block. The FC block contains a temporal block having a filter size of 128. The structure of the FC block is described as a temporal convolutional layer accompanied by batch normalization [40] and followed by a sigmoid activation function. Finally, we used the global average pooling after the convolutional block.

We used the self-attention mechanism in the BDLSTM module as attention-based RNNs are very efficient in many fields, e.g., machine translation, image captioning, speech emotion recognition, etc. Thus, we strengthened the contribution of the self-attention mechanism in our study. The AttBDLSTM and the FCN block are concatenated and fed into the sigmoid layer. We used the sigmoid function in our proposed model as the last layer of the model. Further, it also serves as an activation function in machine learning. The AttBDLTSM-FCN inputs spatial time series data and predicts the expected future values for the multistep traffic speed prediction. When the time series spatial–temporal traffic data are fed to the AttBDLSTMs, the spatial correlation of the speed at different locations and the temporal dependencies of the speed values are captured during the process of feature learning, as bidirectional LSTMs can utilize both backward and forward dependencies, so the BDLSTM is an entirely reasonable model to take in more valuable information from spatial time series data. In the FCN model, there is no fully connected layer, so learning filters are everywhere. It is faster than CNN. An FCN learns representations and makes decisions based on local spatial input. The proposed model, AttBDLSTM-FCN, uses the augmentation of AttBDLSTM and FCN. It is evident that when applying FCN and AttBDLSTM blocks parallelly, the blocks augment and force each other to find a set of features when combined, resulting in a model with better overall performance. Our proposed model significantly enhances the performance of the FCN network with minimal increase in the model size and requires minimal processing of the dataset. One of the major advantages of using the attention BDLSTM cell is that it provides a visual representation of the attention vector. Utilization of the attention mechanism allows one to visualize the decision process of the LSTM cell. A major advantage of the proposed model is that it does not require heavy preprocessing or feature engineering.

The mechanism used to make short–long-term traffic speed predictions in the present study is depicted in Figure 4. The one-time-step prediction model property is shown in Figure 5. Therefore, we transform the univariate time series data of traffic speed prediction into $Y_T$ as predicted variable and ${\widehat{Y}}_{T+1}$ as a response variable. Afterward, to predict the $Y_{T+2}$, ${\widehat{Y}}_{T+1}$ is appended to the training data. In Figure 6, the model predicts future multistep traffic speed prediction at a time. Thus, it predicts multiple future steps from ${\widehat{Y}}_{T+1}$ to ${\widehat{Y}}_{T+n}$ at once, where n is the number of future time step predictions.

3.6. Dataset Description

The dataset used in this study is publicly available [41]. The dataset covers four connected freeways, I-405, I-5, SR-520, and I-90. There are 323 sensor stations located on these connected freeways to collect the data every 5 min from the Seattle area. The dataset covers the whole year of 2015.

3.7. Experimental Setup

Each sample of the input data $Y_{P,T}$ is a 2-dimensional vector, and the output is a 1-dimensional vector with 323 components. Based on the model description, [$N,n,P$] = [$N,n,323$] is the dimension of input data. We used an early stopping mechanism to overcome the problem of overfitting. Before feeding input to the model, all samples were divided and randomized by the ratio of 7:2:1 as training, validation, and testing set, respectively.

Table 1. Hyperparameters.

Time lags	3, 6, and 12, respectively
Total time steps	105,120
Activation function	Sigmoid
Training process	Mini-batch gradient descent
Optimizer	RMSProp
Loss	MSE
Patience	20
Training sample	75,679
Validation samples	18,920
Number of trainable parameters	3,491,173
Number of nontrainable parameters	256
Total parameters	3,491,429

shows the hyperparameters of the training process. To determine the performance of the different speed prediction algorithms, we used mean absolute percentage error ($MAPE$) and mean absolute error ($MAE$) as evaluation metrics from the following equations:

$$MAE=\frac{1}{n}\sum _{i=1}^n\left|h_i-{\hslash }_i\right|$$

(19)

$$MAPE=\frac{100}{n}\sum _{i=1}^n\left|\frac{h_i-{\hslash }_i}{h_i}\right|$$

(20)

where, $h_i$ is actual traffic speed, and ${\hslash }_i$ is observed speed. The proposed AttBDLSTM-FCN model’s results are examined and compared with different models.

4. Results Analysis and Discussion

Different baseline models were used in traffic forecasting problems, such as ARIMA and Kalman filter. In this study, ARIMA and Kalman filter methods are not compared, as the performances of these models are far behind, based on our literature review [42]. In this study, we compared and evaluated the performance of our proposed model, AttBDLSTM-FCN, with different state-of-the-art baseline models, e.g., fully convolutional network (FCN) [38], LSTM [3], GRU NN [4], LSTM with fully connected deep neural network layer (LSTM-DNN) [43,44], LSTM with fully convolutional network (LSTM-FCN) [35,45], and BDLSTM [46]. We also examined and evaluated the performance of baseline models with a different number of time lags (time frames) studied in this paper, such as time lags of 3, 6, and 12, respectively. After conducting comprehensive research, we learned that the compared models performed well on small time lags as the time lags (time frames) increased, and their performance decreased. The GRU model showed its best performance when it used six time lags together. The MAE and MAPE were 1.80 and 3.69%, respectively. As the time lags increased onward, the performance of the GRU model decreased accordingly. In the case of LSTM, when the time lag was set to greater than six, its performance started deceasing, accordingly. The best results of LSTM models were 1.74 and 3.90% MAE and MAPE, respectively, when it used six time lags as input. By adding a DNN layer in the LSTM model, the performance of the model was enhanced, and the error was reduced to 1.86 and 4.45% MAE and MAPE, respectively. LSTM-DNN models also outperformed when the time lag was set as three. By increasing the time lags from three onward, the model performance decreased, and the error rate increased. In nonhybrid models, the BDLSTM model outperformed other baseline models such as GRU, LSTM, and LSTM-DNN. The BDLSTM model also showed consistent behavior with the LSTM-DNN model discussed above. It had its best performance when the time lag was set as three. The MAE and MAPE were reduced to 1.40 and 2.99%, respectively. In hybrid models, the LSTM-FCN showed the best performance when it used six time lags together as input; the MAE and MAPE reduced to 1.45 and 3.40%, respectively. In the proposed model, when the time lags were set to three, the MAE and MAPE were 1.17 and 2.73, but as the number of time lags increased from three onward, the performance behavior of the proposed model was not decreased as it was in the baseline models. The proposed model achieved its best performance when time lag was set as three, the MAE and MAPE reduced to 1.17 and 2.73%, respectively. As the number of time lags increased from three onward, the performance of the proposed model did not decrease as did the baseline models. The experimental results are shown in

Table 2. Comparison with baseline models(MAE).

Time Lags	Models	Error	T + 1	T + 3	T + 6	T + 12
3	FCN	MAE	6.53	5.03	5.61	7.60
	GRU	MAE	1.89	2.45	3.43	3.30
	LSTM	MAE	1.94	2.64	2.83	2.67
	LSTM-DNN	MAE	1.86	2.32	2.37	2.75
	LSTM-FCN	MAE	1.54	2.21	2.24	2.31
	BDLSTM	MAE	1.40	2.18	2.22	2.26
	PROPOSED-MODEL	MAE	1.17	2.10	2.16	2.11
6	FCN	MAE	6.58	5.28	5.58	8.27
	GRU	MAE	1.80	2.66	3.80	3.53
	LSTM	MAE	1.74	2.87	3.14	2.69
	LSTM-DNN	MAE	2.09	2.74	2.80	2.61
	LSTM-FCN	MAE	1.45	2.82	2.66	2.50
	BDLSTM	MAE	1.42	2.62	2.71	2.61
	PROPOSED-MODEL	MAE	1.19	2.25	2.56	2.44
12	FCN	MAE	8.09	5.29	7.70	12.06
	GRU	MAE	2.17	3.36	3.77	3.52
	LSTM	MAE	2.29	3.51	3.54	3.11
	LSTM-DNN	MAE	2.62	3.45	3.38	3.09
	LSTM-FCN	MAE	1.71	3.43	3.33	2.71
	BDLSTM	MAE	1.54	3.20	3.31	2.75
	PROPOSED-MODEL	MAE	1.30	3.06	3.21	2.67

and

Table 3. Comparison with baseline models(MAPE).

Time Lags	Models	Error	T + 1	T + 3	T + 6	T + 12
3	FCN	MAPE	18.10	16.04	16.72	20.30
	GRU	MAPE	3.82	5.48	7.52	7.24
	LSTM	MAPE	4.22	5.87	6.35	6.23
	LSTM-DNN	MAPE	4.45	5.59	5.56	6.97
	LSTM-FCN	MAPE	3.62	5.18	5.29	5.59
	BDLSTM	MAPE	2.99	4.99	5.26	5.15
	PROPOSED-MODEL	MAPE	2.73	4.95	5.15	5.05
6	FCN	MAPE	18.99	16.26	17.14	21.87
	GRU	MAPE	3.69	6.52	8.88	8.64
	LSTM	MAPE	3.90	6.96	7.53	6.28
	LSTM-DNN	MAPE	5.0	7.09	7.08	6.33
	LSTM-FCN	MAPE	3.40	6.91	6.81	5.96
	BDLSTM	MAPE	3.11	6.60	6.74	6.09
	PROPOSED-MODEL	MAPE	2.75	5.39	6.66	5.80
12	FCN	MAPE	20.48	17.01	21.18	27.89
	GRU	MAPE	4.37	9.06	10.04	8.53
	LSTM	MAPE	5.13	9.36	9.45	7.53
	LSTM-DNN	MAPE	6.63	10.07	9.52	7.80
	LSTM-FCN	MAPE	4.02	9.78	9.52	6.98
	BDLSTM	MAPE	3.41	8.93	9.25	7.20
	PROPOSED-MODEL	MAPE	2.99	7.52	9.05	6.80

. Figure 7 shows the comparison of different models’ training times per epoch. The graphical representation of the results are shown in Figure 8 and Figure 9. The ground truth speed values and predicted speed values of the proposed model at the sensor station 167.56 milepost, from a randomly selected day, while predicting multistep future traffic speed prediction on 15, 30, and 60 min time frames are shown in Figure 10. We used the same sensor location to clearly show the performance of the proposed model with different time steps without having biased or ambiguous results.

Ablation Study

We performed an ablation study of the proposed model to investigate the causality of the model. The results are shown in

Table 4. Ablation study.

Time Lag	Error	FCN	BDLSTM	AttBDLSTM	Proposed Model
3	MAE	6.73	1.40	1.29	1.17
	MAPE	18.10%	2.99%	2.77%	2.73%
6	MAE	6.58	1.42	1.34	1.19
	MAPE	18.9%	3.11%	2.99%	2.75%
12	MAE	8.09	1.54	1.39	1.30
	MAPE	20.48%	3.41%	3.11%	2.99%

. The experimental evaluations demonstrated that the combination of the AttBDLSTM block features and the FCN block features improved the model’s performance. We performed an ablation study on three different time lags, e.g., 3, 6, and 12 (15, 30, and 60 min, respectively), and we obtained surprising results by augmenting two models. When the time lag was set as 3 (e.g., 15 min), FCN, BDLSTM, AttBDLSTM, and the proposed model had MAE and MAPE values of 6.73, 18.10%, 1.40, 2.99%, 1.29, 2.77%, and 1.17, 2.73%, respectively. When the time lag was set as 6 (e.g., 30 min), the MAE and MAPE of the FCN, BDLSTM, AttBDLSTM, and the proposed model were 6.58, 18.99%, 1.42, 3.11%, 1.34, 2.99%, and 1.19, 2.75%, respectively. When the time lag was set as 12 (e.g., 60 min), we obtained the MAE and MAPE of the compared models as follows: 8.09, 20.48%, 1.54, 3.41%, 1.39, 3.11%, and 1.30, 2.99%, respectively. After the ablation study, based on our experimental findings, we can see that the attention mechanism along with BDLSTM can help to improve the performance of the proposed model. Moreover, we can conclude that the FCN block augmented with the AttBDLSTM block statistically helped improve the overall model performance.

5. Conclusions and Future Work

In this study, we proposed an attention-based bidirectional long short-term memory fully convolutional network to predict multistep network-wide traffic speed. The contributions of this study are as follows:

• We proposed a deep-stacked model studying both forward and backward dependencies of the traffic data. Firstly, it integrates both attention-based BDLSTMs and FCN as an introductory module to capture the spatiotemporal correlation in the network-wide traffic data. We examined the attention layer behavior in our model and concluded that using attention mechanisms can enhance the model performance accordingly.
• The proposed model exploited the benefits of deep-learning-based architectures, e.g., bidirectional long short-term memory and fully convolutional neural networks, to improve prediction performance in this study.
• Our study demonstrated that AttBDLSTM-FCN achieved better performance in network-wide traffic speed prediction when predicting traffic for 15, 30, and 60 min, respectively. Moreover, the proposed model is also competent for multistep network-wide traffic speed prediction in the future.

Nevertheless, as the results indicate, we know that deep-learning-based models for predicting network-wide traffic speed still need further study. In the future we will extend this work to improve the models by combining evolutionary computation solutions to automatically optimize deep neural network hyperparameters. The other possible future direction is to extend this work of speed prediction in between different levels, which is still missing.