Deep Learning Architecture for Flight Flow Spatiotemporal Prediction in Airport Network

Abstract

Traffic flow prediction is a significant component for the new generation intelligent transportation. In the field of air transportation, accurate prediction of airport flight flow can help airlines schedule flights and provide a decision-making basis for airport resource allocation. With the help of Deep Learning technology, this paper focuses on the characteristics of flight flow easily disturbed by environmental factors, studies the spatiotemporal dependence between flight flows, and predicts the spatiotemporal distribution of flight flows from the airport network level. We proposed a deep learning architecture named ATFSTNP, which combining the residual neural network (ResNet), graph convolutional network (GCN), and long short-term memory (LSTM). Based big data analytics of air traffic management, this method takes the spatiotemporal causal relationship between weather impact and flight flow as the core, and deeply mines the complex spatiotemporal relationship of flight flow. The model’s methodologies are improved from the practical application level, and extensive experiments conducted on the China’s flight operation dataset. The results illustrate that the improved model has significant advantages in predicting the flight flow under weather affect. Even in the complex and variable external environment, the model can still accurately predict the spatiotemporal distribution of the airport network flight flow, with strong robustness.

1. Introduction

When flight delays or cancelations (irregularities) occur, the real-time flight flow in the airport network is difficult to predict, especially when there are uncertainties in the external environment. If we can accurately predict the spatiotemporal distribution of flight flows in the airport network, it will be helpful for the formulation of proactive flight schedule recovery plans during irregular operations. The current flight schedule recovery plan of disrupted flight is mainly aimed at the short-term local traffic congestion. After the delay occurs, the schedule disrupted flight is handled passively according to personal experience, which often ignores the long-term impact caused by the flight delay spreading in the airport network. This makes the overall operation of the transportation system inefficient. If an active recovery system is constructed for schedule disrupted flights from the micro perspective of supply and demand balance, it can effectively avoid the occurrence of large-scale flight delays, and improve the punctuality rate of flight operations and the operating efficiency of airline flights.

The airport network flight flow time series mainly has two signal manifestations: (1) the regularity of airport flight operations, and (2) the uncertainty affected by environmental factors. The traditional prediction method of predicting the future flight flow directly from the historical flow big data can only capture the periodic characteristics of flight flow, but cannot reflect the random changes of the flow influenced by interference factors from the perspective of flight scheduling. Therefore, we treats the flight flow signal as a random flight residual flow signal, and only considers the superposition results of multiple characteristics caused by environmental factors on the flight flow. It includes the nonlinearity inherent in the complex structure of the airport network, the time-varying of flight plans due to meeting the demands of passengers, the uncertainty that flight operations are susceptible to external random factors, as well as the long-range correlation of flight flows caused by delays propagating further through the network [1,2].

This paper intends to build a deep learning model framework Air Traffic Flow Spatial-Temporal Network Prediction Model (ATFSTNP) based on data-driven technology, and establishes a machine learning model for China’s major airport networks. The model solves the flight flow prediction problem from airport network level by predicting and analysing the airport network flight flow performance patterns. The flow of airport flights refers to the number of inbound and outbound flights operating at the airport per unit time, which should be equal to the planned flow of airport flights under ideal operating conditions. However, in actual operation, the flight plan made in advance will be affected by uncertain factors, resulting in discrepancies between the actual flight schedule and the plan flight schedule. This paper combines the airport network topology, flight flow redistribution decision, and the impact of external environmental factors to construct a flight flow residual sequence to reflect the impact of flight flow, and design a deep learning framework based on flight flow characteristics including:

(1)

Spatial dependence: Since the flight of the aircraft follows the flight plan, there is an interaction between the flight flow of the local airport and the flight flow of the navigable airport. In this paper, branch 1 of the model is used to reflect the influence of the topology of the airport network on the flight flow. By constructing a weighted adjacency matrix as the spatial matrix of the network, the improved ResNet-GCN technology is used to deeply mine the abstract spatial dependence of the flight flow residual sequence.

(2)

Temporal dependence: The change of airport flight flow has obvious periodicity and trend. Within a day, the traffic peak of the airport will appear at a specific time period according to the demand of airport passengers, usually between 12:00–14:00 and 17:00–19:00, which is time-sensitive. In addition, the flight flow in the current period may also affect the flight flow in the future period. For example, the flight in the current period is delayed, which will move part of the current flow to the future period, resulting in the redistribution of the flight flow. In this paper, branch 2 is used to reflect the influence of environmental factors on flight flow in the time dimension, and LSTM is introduced to deal with the temporal dependence of flight flow.

(3)

Influence of external environmental factors: Airport flight flow is easily affected by environmental factors, resulting in inaccurate predictions. The interference of external environmental factors such as weather impact will cause the real-time capacity of the airport to be lower than the operating capacity of the airport on which the flight plan is based. This will directly cause some flight delays or cancellations. It is worth noting that one flight string, formed by a sequence of flight legs for one aircraft in daily scheduled operations, is related to multiple time nodes at multiple airports. In an airport network composed of multiple flight strings, weather phenomena can affect the airport capacity distribution, which in turn affects the spatiotemporal distribution of airport flight flows, and increase the difficulty of forecasting. Hence, the sequence of the influence degree of the airport flight flow by the random weather conditions is constructed as the input of branch 2. The information of weather phenomena is processed through backbone fusion, and the attention mechanism based on network learning is added to dynamically capture the spatiotemporal dependence mechanism of airport network flight flow.

The rest of the paper is organized as follows. Section 2 reviews the current research status in the field of traffic flow forecasting, based on which the methodological contributions of the paper are stated. In Section 3, the research problem is described and defined, and the framework structure and learning method of the ATFSTNP model are introduced. In Section 4, the proposed deep learning framework is applied to the flight operation data prediction of 67 airports in China. The model parameter configuration and accuracy evaluation indicators are introduced, and the prediction results are analysed and discussed by comparing the prediction performance of the ATFTNSP model with several advanced models. Finally, Section 5 summarizes the research conclusions and identifies future research directions.

2. Literature Review

Traffic flow prediction has always been a hot research topic in academia because of its practical application. Although there are many methods for predicting traffic flow, they are mainly used in the traffic flow prediction in the field of ground traffic, and those in the field of air traffic also focus on the prediction of aircraft flight trajectories in the air [3]. Traffic flow prediction methods can be roughly divided into three categories: statistical theory [4,5,6], traditional machine learning [7,8] and deep learning. Among them, statistical theory and traditional machine learning often assume that the predicted traffic flow data has the same characteristics as the historical flow data, which is limited to the acquisition of time information and ignores the impact of the spatial structure of the airport network on the airport flight flow [9]. Chen et al. [10] used dynamic networks to describe the structure of airway and airspace, and used the continuity equation in fluid mechanics to describe the continuous behaviour of airway traffic. But they only focused on the flow distribution of aircraft operating space, and did not consider the time dependence of historical flow conditions.

In recent years, deep learning has achieved great success in capturing spatiotemporal, topological and many other information. Many scholars apply it to the problem of predicting the spatiotemporal distribution of traffic flow. Liu et al. [11] and Lin et al. [12] studied the spatial dependence of adjacent airspaces and the temporal dependence of historical traffic in a given airspace based on the ConvLSTM model. However, the spatial distribution of the airport is in a network topology relationship, which is not the same as the distribution of the airspace grid format. To more effectively integrate the spatiotemporal dependencies of traffic flow, recent research introduces Graph Convolutional Networks (GCNs) to learn the spatial topology of traffic network sites. Li et al. [13] and Han et al. [14] used GCN to extract the correlations between rail transit stations. Zhao et al. [15] proposed the T-GCN model, which used the gating mechanism to study the problem of traffic flow prediction in urban road networks from the spatiotemporal dimension, and proved the superiority of the model in traffic prediction. Liu et al. [16] used an improved residual networks to capture the bus traffic flow spatiotemporal correlation to predict bus traffic flow and found that it was effective for the prediction of scheduled bus lines. Han and Gong [17] embed the LSTM model into the GCN parameters. Hu, Shao and Sun [18] proposed a graph space–time network (GSTNCNI) incorporated complex network feature information, is proposed to predict future highway traffic flow time series. Their models greatly reduce the amount of computation and make a good use of the temporal and spatial information of the California highway data.

Air transportation is more susceptible to traffic congestion due to variable weather than ground transportation, and ground transportation is fundamentally different from air transportation in dealing with congestion problems. So that the selection of traffic flow characteristics and the acquisition of the topology relationship of the traffic network are not universal for ground traffic and air traffic. For example, ground transportation vehicles can stop or queue slowly if they encounter traffic congestion. To move forward, the aircrafts running at high speed need to consume fuel at all times to maintain power when they stay in the air, and the limited airspace cannot accommodate too many aircrafts running at high speed at the same time. Therefore, the ground holding decision [19,20] is the main means to adjust and control the air traffic flow, that is, to make the aircraft park at the airport and wait for take-off [21]. This means that the ground traffic flow forecast is more concerned with the forecast of traffic OD (Origin-Destination) demand, while in air transportation, since the flight plan is formulated in advance, predicting the actual flight flow at the airport needs to consider the dynamic impact of weather phenomena on flight plans, which is the propagation characteristics of flight delays referred to in the field of air transportation research.

Because flights need to operate according to a pre-established flight plan and are easily affected by weather conditions, the spatiotemporal dependencies of airport flight flows are extremely complex. In air transport activities, the delay of one flight may affect the normal operation of other flights with which it shares certain resources vertically or horizontally. The vertical direction can be understood as a chain of related flights that share aircraft or crews on the time axis [22,23], while the horizontal direction can be understood as other flights that share airport or airspace resources in the same time period in terms of spatial distribution [24]. This makes the airport flight flow highly non-linear in spatiotemporal correlation—the flight flow at an airport may affect neighbouring airports or distant airports, and the flight flow at the current moment may also affect the flight flow at future moments. At present, the impact of airport network structure [25] on flight delay and propagation is mostly studied from a static perspective, without considering the dynamic impact of random weather factors and the dynamic changes of the affected transportation network. Most of the research on airport flight flow forecasting concentrates on computing the information entropy of the airport from the macro level, and predicts the airport operation situation quantitatively as a whole [26,27]. The prediction accuracy of this method is not high. Because it cannot achieve real-time prediction of airport flight flow from the perspective of flight planning, and also ignores the influence of external uncertain factors on flight operation.

At present, the deep learning methods are mostly used to predict ground traffic flow problems, and there are few studies on predicting flight flow, especially airport flight flow. While, the current research on airport flight flow prediction mainly focuses on a single airport, and do not consider the flight delay behaviour and its propagation characteristics in the airport network. It is a micro-scale research from the perspective of the network, and cannot reflect the spatiotemporal dependence of airport network flight flow and the basic characteristics affected by external environmental factors. Yan et al. [28] constructed a deep-learning-based model that considered the influence of the topological airport network, but failed to consider the impact of weather on airport flight flow. Therefore, this paper links the single airport to an airport network by flight string, and incorporates flight delay spatiotemporal propagation characteristics to solve the flight flow prediction problem at airport network level. This is fundamentally different from the problem of flight flow prediction for a single airport or a single flight string, and this is the core methodological contribution for this paper.

Furthermore, the design of the deep learning model framework ATFSTNP in this paper is mainly to solve the practical problems of air transportation. From the perspective of the flight strings, ATFSTNP combining GCN, Attention-LSTM and ResNet to conduct spatiotemporal prediction on the airport network scale. In addition to the topological relationships between airports, the spatiotemporal correlations between airports’ flight flow, and airport dynamic weather conditions are all incorporated into ATFSTNP to determine how such factors affect flight flow. The flight flow residual sequence is used for the first time to accurately predict the airport flight flow. The advantage is that it can capture the operational differences of flight flows at different airports and their effects on the entire network, and amplify the impact of each airport’s flight planning flow in the spatiotemporal dimension of the airport network through the flight residual flow. Then we synthesize the direct influence of weather phenomena on the airport flight plan, superimpose the influence of space and time on the influence of random factors, and restore the residual. The flight flow prediction results of the network can better reflect the randomness of variable weather interference airport flow and improve the prediction accuracy of the model. The ATFSTNP provides data support for the next step to proactively optimize flight schedules before flight delays occur. It also evaluates airport operating conditions and flight delays in the air transportation system, and provides a predictive predictive model for airlines to proactively manage and control flight flows across airport networks.

3. Model Development

3.1. Definition

(1)

Airport Network: The airport network is defined as an undirected topological graph $G=\left(V, E, A\right)$, where each airport is a node, $V=\left\{v_1,v_2,\dots ,v_N\right\}$ is the set of N nodes, E is a set of edges, A is a weighted adjacent matrix of G, indicating the physical property of an airport connected to an airport by flight.

(2)

Flight Flow Feature: Define the characteristic matrix of the real-time flight operation flow of each airport in the network as $X^{N\times m}$, where m is the total length of historical time series, $X_t,Y_t,Z_t\in R^{N_k}$ represent the flight operation flow, planned flow and real-time capacity of each airport in the kth time series, respectively.

(3)

Input Feature: The airport flight flow residual subsequence is constructed as $Inpu{t}_t=Outpu{t}_t=Y_t-X_t$, which is used as the input of branch 1 of the ATFSTNP model. $Impac{t}_t=max\left\{0,Y_t-Z_t\right\}$ is to quantify the influence degree of the random weather impacts on the airport flight flow, as the input of branch 2.

Therefore, in the ATFSTNP model, the problem of spatiotemporal flight traffic forecasting in the airport network can be regarded as learning the mapping function f, as shown in Equation (1):

$$\left[Outpu{t}_{t+1},\dots ,Outpu{t}_{t+T}\right]=f\left(G; \left(Inpu{t}_{t-n}, \dots ,Inpu{t}_t,Impac{t}_{t-n},\dots ,Impac{t}_t\right)\right)$$

(1)

where T is the length of the time series needed to be predicted and n is the length of historical time series.

3.2. Overview of Model Framework

The architecture of the ATFSTNP model is mainly based on ResNet, GCN and attention LSTM methodologies. As shown in Figure 1, it includes two branches of mutual causality, which respectively consider the correlation between changes in airport flight flow (effect) in the spatial dimension, and the impact of weather conditions (cause) on airport flight flow in the temporal dimension.

The model predicts the flow residuals from t + 1 to t + T in real time by inputting the residual sequence of airport flight flow and the degree of weather impact from t-n to t online. Firstly, the model converts the two-dimensional feature matrix reflecting flight flow residuals in branch 1 into three-dimensional tensors according to the time prediction step size and the number of airport nodes, and uses the ResNet-GCN module that captures spatial features to perform regularization operations and convolution operations on graph data. Then, the influence of weather conditions reflected in branch 2 on the planned flight flow is spatially expanded through LSTM processing. The spatial state features of each time period of the two branches are fused into the attention LSTM module. The prediction features with spatiotemporal information are obtained by comprehensive scoring based on the influence degree of input on output in rolling time domain through the attention LSTM module. Finally, the predicted flight flow is obtained by using the planned flow data to restore the residual flow of each airport. Detailed model architecture descriptions are given below.

3.3. Branch 1: Airport Traffic Flow Distribution

The spatial structure of airport networks and historical operational flow data have important implications for predicting airport network flows. Hence, the branch 1 of the ATFSTNP model uses the improved ResNet-GCN technology to deeply mine the spatial dependence of the residual flow sequence. In previous studies on airport traffic flow, inflow and outflow are always considered separately through independent models. However, the flight is the transportation behaviour to complete the displacement activity according to the plan. It’s departure and arrival exist at the departure airport and the arrival airport at the same time. For any airport, there is an interaction between the inflow and outflow of flights at the same time. For example, in the case of limited capacity, airports usually guarantee the normal operation of arriving flights first. So we propose a method that considers both inflow and outflow to study the spatial dependence of air transport flow network topological structure.

(1)

GCN

The traditional Convolutional Neural Network (CNN) treats the traffic flow network as grid matrices, which cannot reflect the complex topology of the airport network, so it cannot accurately capture the spatial dependencies. However, GCN can construct filters in the Fourier domain, act on nodes and their adjacent (related) nodes, encode the network topological structure, and obtain spatially dependent features between nodes, as shown in Figure 2.

We use the airport as a node to construct a multi-layer GCN to capture the spatial topological dependencies between the central airport and its neighbouring airports, and to realize the information transfer of multi-order neighbourhoods by stacking convolutional layers to capture the impact of flight flows of other airports on the flight flow of central airport. The transformation rules are expressed as:

$$H^{\left(l+1\right)}=f\left(H^{\left(l\right)}, A\right)=\sigma \left({\tilde{D}}^{-\frac{1}{2}}\tilde{A}{\tilde{D}}^{-\frac{1}{2}}{H}^{\left(l\right)}{W}^{\left(l\right)}\right)$$

(2)

where, $W^{\left(l\right)}$ is the weight matrix of the lth layer of the neural network, σ(·) is an activation function, $H^{\left(l\right)}$ is the output of the lth layer, and $H^{\left(0\right)}=Input\in R^{N\times m}$ is the feature matrix composed of the flight flow residual subsequence of each airport in the airport network. A is the adjacency matrix, $I_N$ is the identity matrix, $\tilde{A}=A+I_N$, $\tilde{D}={{\displaystyle \sum }}_j{\tilde{A}}_{ij}$ is the diagonal node-degree matrix of $\tilde{A}$.

(2)

Airport Network Adjacency Matrix

In air transport activities, flight is a one-time location transfer process. It only connects departure and arrival airports at the time of operation, and has no subsequent connection impact after the task is completed. In addition, the number of operating flights of each route is different, and the traditional 0–1 adjacency matrix is not enough to reflect the relationship between airport flight flows. Therefore, this paper further considers the influence of the degree value setting of each node in the network on the prediction accuracy, defines the airport network as an undirected topology weighted graph. For a flight plan of a certain duration (such as one month), we set (A) the Pearson correlation coefficient of airport traffic, and (B) the airport navigability rate, as the weight of the adjacency matrix $W^{\left(A\right)}=\left\{0, a_{ij}\right\}\in R^{N\times N}, i,j\in V$ (the calculation formula is shown in

Table 1. The setting of airport network adjacency matrix.

Code	Description	Function
O	Airport connectivity	$a_{ij}=\left\{0,1\right\}$
A	Airport flight flow correlation	$a_{ij}=\frac{{\sigma }_{ij}}{{\sigma }_i·{\sigma }_j}=\frac{\sum \left(x_k-\overline{x}\right)\left(y_k-\overline{y}\right)}{\sqrt{\sum {\left(x_k-\overline{x}\right)}^2{\left(y_k-\overline{y}\right)}^2}}$
B	Airport navigability rate	$a_{ij}=\frac{The number of time slots with flight between airports i and j}{Total number of time slots}$

), and compare the results with the prediction results using the traditional adjacency matrix. $\widehat{A}=W^{\left(A\right)}\circ \tilde{A}$ is used to describe the degree of mutual influence of the flight flow residuals between airports. By performing the regularization operation ${\tilde{D}}^{-\frac{1}{2}}\widehat{A}{\tilde{D}}^{-\frac{1}{2}}$ on the weighted adjacency matrix $\widehat{A}$, the spatial distribution information of the feature matrix H can be preserved in the process of information transfer.

Therefore, the above definition can be used to superimpose the departure and arrival flight flow matrices of each airport into a two-channel image-like tensor to obtain the airport network topology in a specific time zone. Taking the data in June as an example, Figure 3 visually shows the network topology of 67 airports. There is a particularly obvious difference between the spatial matrix considering weights and those without weights. The darker the colour, the greater the spatiotemporal relationship of flight flows between airports. Actually, considering the airport flight flow correlation and airport navigability rate is more complicated than the adjacency matrix considering only airport connectivity, which will have an impact on the prediction of the spatiotemporal correlation of airport network flight flow under different operating conditions.

(3)

ResNet

Past studies have found that stacking multiple GCN layers not only increases the complexity of backpropagation, but also produces gradient dispersion or gradient explosion, which degrades the performance of deeper GCNs. The ResNet directly connects the high-level neural network to the low-level neural network through a shortcut connection, which effectively prevents the gradient vanishing during backpropagation, and solves the problem that the deep network is not easy to fit the identity map [29]. Considering the real-time nature of traffic flow prediction, this paper introduces the improved residual block proposed in article [30] (as shown in Figure 4b). We extend the traditional GCN into ResNet-GCN to improve the prediction accuracy and speed of the model, treat the input feature matrix as a graph signal, and process it according to the following formula:

$$H^{\prime }={\tilde{D}}^{-\frac{1}{2}}\widehat{A}{\tilde{D}}^{-\frac{1}{2}}H$$

(3)

Then build the residual block by setting the skip connection, skip the identity mapping layer $H_{l+1}=X_l+F\left(H_l\right)$, and directly fit the residual mapping $F\left(H_l\right)=0$. In this way, the feature extraction process of multiple GCN layers is optimized, and the model prediction speed and accuracy are improved. Where ${\tilde{D}}^{-\frac{1}{2}}\widehat{A}{\tilde{D}}^{-\frac{1}{2}}$ is the regularized Laplacian matrix, $H\in R^{N\times n}$ is the feature matrix input containing graph features, where N is the number of airports, n is the length of historical time, and $H^{\prime }$ is a feature matrix with the same size, which includes the airport network and the spatial dependency information between airport flight flows.

In branch 1, the traffic characteristics and network topological relationship of each airport are input into the ResNet-GCN consisting of two residual blocks (see Figure 1 for details). The graph convolution sets 32 filters, the first residual block sets 32 filters, the second residual block sets 64 filters, the convolution kernel size of the convolution layer is set to 3 × 3, the stride Set to 1 × 1, and fill rule to SAME. The data is then flattened and fully linked with 67 neurons, which finally feed the output data of branch 1 to the feature fusion layer.

3.4. Branch 2: Airport Capacity Impact

Although flight operations are easily affected by environmental factors such as weather, there are few studies that incorporate this into the consideration of flight flow as an influencing factor. However, the reduction of airport capacity due to weather changes is the essential reason that affects airport flight flow, such as flight delays or cancellations due to capacity constraints. The branch 2 of the ATFSTNP model uses multi-layer LSTM to mine the direct influence of weather phenomena on airport flight flow changes in the time dimension, and then input the weather impact into the feature fusion layer from the spatial dimension.

Compared with the traditional RNN which simply superimposes the memory, LSTM adds a gating mechanism [31] to limit the transmission state of information and learn the dynamic changes of the affected airport flight traffic data in the time dimension. It is more suitable for traffic data prediction problems that require ‘long-term memory’. As can be seen from Figure 5, three ‘gates’ are added to LSTM. The forget gate f_t controls how much the information in the previous memory state is retained. The input gate i_t controls the degree to which the current calculated new state is updated to the memory state. The state transition between the internal memory state C_t is jointly determined by the i_t and the f_t. The output gate o_t controls how much the current output depends on the current memory state. x_t and h_t are the input and output states at time t, respectively. Except for the use of tanh as the activation function when calculating candidate memory and hidden states, all other gates use sigmoid as the activation function. LSTM completes the acquisition and transmission of data information through the gating mechanism. The specific transformation is shown in Equations (4)–(9), where W is the weight matrix in the gated cyclic unit, b is the bias, σ(·) and t tanh(·) are activation functions, and ‘∘’ represents the Hadamard product.

$$f_t=\sigma \left(W_f·\left[h_{t-1},x_t\right]+b_f\right)$$

(4)

$$i_t=\sigma \left(W_i·\left[h_{t-1},x_t\right]+b_i\right)$$

(5)

$$\widetilde{C_t}=tanh\left(W_C·\left[h_{t-1},x_t\right]+b_C\right)$$

(6)

$$C_t=f_t\circ C_{t-1}+i_t\circ \widetilde{C_t}$$

(7)

$$o_t=\sigma \left(W_o·\left[h_{t-1},x_t\right]+b_o\right)$$

(8)

$$h_t=o_t\circ tanh\left(C_t\right)$$

(9)

As shown in Figure 1, branch 2 adds the input flattening to the fully connected layer to obtain weighted indicators, and mines features through the first layer of LSTM with an output dimension of 128, and updates and resets information through the control gate of LSTM. Then, the second layer of LSTM is used to spatialize the data with time information according to the airport network structure, output the data with spatiotemporal characteristics and the characteristics of flight flow, and input it to the backbone fusion layer together with the output data of branch one.

3.5. Backbone: Spatial-Temporal Fusion

(1)

Spatial-Temporal Feature Fusion

Because the changes of airport flight flow are periodic and trendy, the outputs of branch 1 (effect) and branch 2 (cause) are weighted and fused in the backbone part of the model through the fully connected layer. The formula is as follows:

$$Fusion=W_1\circ Inpu{t}^{*}+W_2\circ Impact$$

(10)

where $W_i\in R^{N\times T}$ is the weight vector of the input tensor i, where N is the number of airports, T is the length of the predicted time series, and ‘∘’ is the Hadamard product. The weight vector W is randomly initialized before training and can be updated during neural network backpropagation. Then the LSTM layer based on the attention mechanism is used to capture the time dependence of the airport flight flow, and finally the Dense layer is connected with 67 neurons to generate the final output $\left[Outpu{t}_{t+1},\dots ,Outpu{t}_{t+T}\right]$.

(2)

Attention Mechanism Based on Network Learning

Since the attention mechanism was proposed, it has been widely used in various deep learning models based on the RNN framework, and achieved very good results [32]. Since civil aviation transportation is affected by weather, flight delay propagation and the complex spatiotemporal influence of nonlinear network structure, the traditional attention mechanism based on distance assignment weight or function score is not enough to reflect. Therefore, based on the previous research by Wu et al. [33], this paper uses the fully connected network to learn the weights, uses the influence of the input on the output in the rolling time domain to comprehensively score, and uses back propagation to correct the network weights.

$$Attention=f\left(W^U\circ U+b\right)$$

(11)

$$U^{\prime }=Attention\circ U$$

(12)

Let the matrix $U\in R^{T\times N}$ be the output of LSTM, where T is the length of the predicted time series, and N is the number of features. Attention is the weight matrix with the same size as U, ‘∘’ is the Hadamard product, f(·) is the fully connected layer whose activation function is linear, $W^U$ is the weight of the fully connected network, and b is the bias. The attention-based output $U^{\prime }$ is obtained by formulas (11) and (12), and its architecture is shown in Figure 6, where u and a represent the input elements and the corresponding scoring coefficients of the elements, respectively.

By combining the capture of the spatial characteristics of flight flow and the temporal characteristics of external environmental factors in branches 1 and 2, the change characteristics of the airport network flight flow at the spatial-temporal level are further captured, and the dynamic time-space flight flow change rules are obtained. From the practical application and theoretical level, it can better capture the spatiotemporal correlation mechanism of airport network flight flow, and achieve more accurate and effective prediction.

4. Case Study Results and Discussion

Based on the model framework proposed above, this paper takes China civil aviation flight operation data as an example to design and apply baseline models and ablation models to evaluate the specific prediction performance of ATFSTNP. This section describes the dataset, gives detailed model configuration and evaluation metrics, and discusses and analyses the prediction results.

4.1. Data Description

The research data in this paper comes from Umetrip, which is an application created by TravelSky Technology Limited (Beijing, China) for providing civil aviation travel information. We selected the operation data of all domestic flights in the first 30 unit days of China Civil Aviation in June with good weather conditions and July with more thunderstorms in 2019. A total of 702,922 plus 798,593 planned flights and 683,554 plus 769,959 actual flight transportation activities were involved in June and July. It covers a total of 2888 routes and 227 domestic airports. Since 90% of flight transportation activities only involve 67 major domestic airports, this paper only performs pre-processing and statistics on the airport network flight flow composed of these 67 airports to reduce the computational complexity of the GCN network in the model. Figure 7 shows the spatiotemporal distribution of flight flow at 12 major airports in Eastern China from 25–30 June.

Time granularity is the basic unit of system management time. Generally, the smaller the time granularity, the more refined the system management, but the corresponding management burden will be increased. The time granularity setting of the proactive flight control system needs to be based on various operating time specifications of civil aviation transportation. Based on the previous research results of predicting the real-time capacity of the airport considering the influence of weather, and considering the unit time limitation of weather data collection and airport capacity-flow data statistics (such as the airport time slot is 3–5 min, the flight delay standard is 15–30 min, the weather collection frequency is half an hour to two hours, etc.), we finally take 30 min as the time granularity of the active flight control system.

4.2. Model Configuration

The model used the Python language to complete data processing and was implemented using Pandas, Numpy, TensorFlow, and Keras. The data of the first 24 days of the 30 days was used as the training set, and the data of the last 6 days was used as the test set. The ratio of training and testing data is 8:2. In order to avoid improper parameter initialization, the model training adopts the early stopping technique to avoid overfitting. The model hyperparameters are determined by experiments: the learning rate is 0.001, the number of training rounds is 1000, and the batch size is 32. For other parameters, see Section 3. In order to balance the model training time and prediction accuracy, this paper used the first 2, 3, 4, 6 and 8 h of flight traffic at airports in the entire network to predict the flight traffic at each airport in the next half an hour. It was found that the model prediction using the flight flow information of the previous 6 h as the historical input feature performs best. As shown in Figure 8, during model training, its training loss and validation loss can be quickly fitted in the first 3–5 iterations. The training loss and validation loss of the model only slightly oscillate before the iteration stops. The model stops within 80 times and the training speed is fast.

4.3. Loss Function and Evaluation Metrics

The experiment uses Mean Square Error (MSE) as the loss function, and the optimizer is ‘Adam’. Root Mean Square Error (RMSE), Mean Absolute Error (MAE), and Weighted Mean Absolute Percentage Error (WMAPE) were used to evaluate the predictive performance of the model. The formulas are as follows.

$$Loss=MSE=1/n{{\displaystyle \sum }}_{i=1}^n{\left(y_i-{\widehat{y}}_i\right)}^2, \in \left[0,+\infty \right)$$

(13)

$$RMSE=\sqrt{1/n{{\displaystyle \sum }}_{i=1}^n{\left(y_i-{\widehat{y}}_i\right)}^2} , \in \left[0,+\infty \right)$$

(14)

$$MAE=1/n{{\displaystyle \sum }}_{i=1}^n\left|y_i-{\widehat{y}}_i\right| , \in \left[0,+\infty \right)$$

(15)

$$WMAPE={{\displaystyle \sum }}_{i=1}^n\left(\frac{y_i}{{{\displaystyle \sum }}_{i=1}^n{y}_i}\left|\frac{y_i-{\widehat{y}}_i}{y_i}\right|\right), \in \left[0, 1\right]$$

(16)

where, $y_i$ is the actual value, ${\widehat{y}}_i$ is the predicted value, n is the number of samples, and i is the sample.

4.4. Results and Discussion

4.4.1. Comparison of Airport Network Flight Flow Prediction Results of Different Models

In order to verify the prediction performance of the ATFSTNP model, we selects three types of models based on mathematical statistics, traditional machine learning and depth graph convolution methods as the benchmark models for the experiments to verify the prediction performance of ATFSTNP. Then, through the Ablation Study, the influence of the spatial dependence, temporal dependence and environmental interference factors in the ATFSTNP model on the accurate prediction of flight flow value was verified from the model structure level. We analyzed the influence of different adjacency matrix weight settings on the prediction accuracy of the model from the perspective of aviation network topology, and further studied the performance of neural networks in depth to understand network behavior.

(1)

Baseline Models

We set up Seasonal-ARIMA [34], RBF-SVR, and traditional two-layer GCNs as benchmark models for predicting airport flight traffic. Except for ARIMA, all models took the time series of airport flight traffic as input, and obtained the overall prediction results of 67 airports by training a single model.

(2)

Ablation Models

(a)

Model Structure: We ablated the proposed ATFSTNP deep learning framework, analyzed the influence of each part of the model on the performance of airport traffic forecasting through the control variable method, and used the airport flight traffic residual subsequence constructed in this paper as the input of the model. (1) Only the Attention LSTM model is built without considering the correlation of the spatial dimension and the influence of weather phenomena; (2) Only the Rest-GCN model is built without considering the correlation of the time dimension and weather impact; (3) Do not consider the dynamic impact of changes in airport capacity caused by weather changes, only build the Rest-LSTM model, and delete branch 2; (4) Set up a complete ATFSTNP model that considers time, space and weather impact for comparison.

(b)

Adjacency Matrix: We used the control variable method to verify the prediction performance of the weighted adjacency matrices O, A and B, representing the connectivity, flow correlation and navigability between airports, respectively, in models that consider the airway transportation network topology.

Table 2. Comparison of prediction accuracy of various models for flight flow in 67 airport networks.

Category	Model	June				July
		RMSE	MAE	WMAPE/%	Time/s	RMSE	MAE	WMAPE/%	Time/s
Baseline Models	ARIMA	4.025	3.087	30.576	Avg (489.6) × 67	4.463	2.075	37.138	Avg (361.3) × 67
	SVR	6.750	2.805	35.917	190.8	7.868	6.857	81.827	199.3
	GCNs	3.250	2.301	29.209	447.3	3.706	2.823	34.092	489.8
Ablation Models	A-LSTM	3.826	2.772	34.454	287.5	3.642	2.807	35.988	214.4
	Rest-GCN-O	2.766	1.888	25.664	292.6	3.098	2.448	27.152	383.1
	Rest-GCN-A	2.948	2.188	27.922	111.9	2.726	1.989	25.105	136.2
	Rest-GCN-B	2.744	1.870	25.382	219.9	3.230	2.361	29.652	122.3
	Rest-LSTM-O	2.330	1.622	23.213	266.5	2.722	1.977	24.496	246.1
	Rest-LSTM-A	2.318	1.654	21.692	262.3	2.713	1.931	24.443	296.1
	Rest-LSTM-B	2.444	1.749	21.301	339.7	2.614	1.842	24.092	244.8
	ATFSTNP-O	2.518	1.859	23.290	344.5	2.679	1.849	24.102	321.1
	ATFSTNP-A	2.251	1.556	19.632	332.4	2.619	1.845	23.829	368.7
	ATFSTNP-B	2.270	1.595	20.751	346.5	2.585	1.803	22.790	290.3

shows the prediction accuracy of each model for the June and July flight traffic data of 67 major airports in China in 2019. In most cases, the prediction performance based on the ATFSTNP framework and using the flight flow residual subsequence as input significantly outperforms the baseline models with flight flow as input feature. Deep learning models that use GCN to capture transportation network topology, or use attention mechanism LSTM to extract temporal dependence also outperform learning models based on mathematical statistics and traditional machine learning. The improved Rest-GCN significantly improves the prediction accuracy of GCN, and effectively reduces the time spent on model training. It proves that applying ResNet to the ATFSTNP model framework can optimize the overall model structure and improve the prediction reliability.

In addition, the ARIMA model has certain limitations for dealing with flight flow sequences affected by complex factors. This method combines autoregressive and moving average components for time series modelling, ignoring the spatial dependence of traffic flow. Due to the lack of multiple captures of the temporal or spatial dimension features of flight flows in the airport network, the prediction accuracy of the SVR, GCN, A-LSTM and Rest-GCN models is lower than that of the Rest-LSTM model which considers the spatiotemporal relationship. The prediction performance of ATFSTNP considering weather impact is higher than all other models, and its optimal WMAPE error percentage is lower than other models by nearly 2% to 15%. It also proves that the prediction accuracy of the airport network flight flow data can be further improved by introducing the weather impact dataset.

Compared with the traditional prediction model, the ATFSTNP model proposed in the paper can effectively improve the prediction accuracy and robustness of airport flight flow.

4.4.2. Analysis of Airport Network Flight Flow Prediction Results in Different Months by ATFTNSP Model Using Different Adjacency Matrices

In this paper, the weight value of the GCN adjacency matrix reflects the spatiotemporal influence relationship of flight flow in the airport network. Among them, the weight calculation of the adjacency matrix is mainly based on the planned flight flow data of the airport network, and the weather impact is the main reason for the difference between the plan and the actual flight schedule. For the data of different months (see Figure 9), the prediction accuracy of each model is generally higher in June with better weather than in July with more complicated weather. But the prediction performance of models using different adjacency matrices and different model structures is different.

In general, the prediction accuracy of the weighted adjacency matrix A and B is higher than that of the traditional adjacency matrix O, and it will also affect the training speed of the model. For the June data that is less affected by weather factors, the prediction accuracy of the correlation matrix A is the highest, the error is about 2 sorties, and the training speed is the fastest. For the more affected July data, the overall forecast performance of the navigable rate matrix B is better than that of matrices O and A.

Combining with Figure 3 of the airport network topology relationship, it can be seen that the correlation of the airport traffic itself can better describe the topology relationship of the traffic network from the time and space dimensions when the weather impact is small. When the weather conditions seriously affect the flow of upstream airports in the flight string, the impact will be transmitted to downstream airports along the network through flight delays. This creates the phenomenon of flight flow hysteresis.

Hence, for the flight flow sequence affected by uncertain weather phenomena, applying the airport navigability rate to describe the spatiotemporal relationship can better grasp the flight flow characteristics of the airport network and reduce the impact of flight delay propagation on the flow prediction accuracy.

4.4.3. Analysis of Flight Flow Prediction Results of Different Airports Using Different Adjacency Matrices in ATFTNSP Model

There are certain differences in the prediction results of the ATFTNSP model using different adjacency matrices. In view of the model prediction performance of a single airport, we chose three typical airports in the East China airport group to analyse the real-time traffic forecast of the airport. (1) Shanghai Pudong International Airport (PVG), which is a large international hub airport with an annual throughput of over 120 million passengers; (2) Nanjing Lukou Airport (NKG) is a typical regional hub airport with an annual throughput of over 30 million passengers; (3) Hefei Xinqiao Airport (HFE), a major airport with an annual throughput of over 10 million passengers. This paper takes the air traffic data set from 25–30 June 2019 as an example, and the ATFTNSP with different adjacency matrices predicts the flight flow of three typical airports as follows (see Figure 10):

PVG, located in the international financial centre—Shanghai, is an important part of Shanghai’s comprehensive transportation hub and the base airport of major domestic airlines. NKG is a regional hub airport in the East China Airport Group, located in Nanjing City, Jiangsu Province. From Figure 10, it can be seen that the PVG flight traffic peaks in the morning and evening are obvious, showing the ‘M-shaped’ double peak characteristics in one day. The traffic peak of NKG appears briefly in the morning, and the traffic peak in the afternoon lasts for a long time.

The flight flow data of the two hub airports have obvious periodic regularity and relatively stable changes from the 25th to the 27th. Comparing the prediction results of the ATFTNSP model using the other two adjacency matrices, the prediction using the traffic correlation matrix A performs the best on the 25th to 27th. However, due to the influence of weather factors (thunderstorms), the flight flow data of PVG and NKG on the 28th to 30th are chaotic and lack regularity in the time dimension. While, the ATFTNSP model can still accurately predict the flow distribution of each time period, and the prediction effect of using the navigable rate matrix B is better than that of using the matrices A and O. It can be considered that the ATFTNSP model can not only accurately predict the time series with periodic regularity, but also has excellent prediction performance for the flight flow data affected by uncertain weather conditions and spatial remote effects at the same time, which reflects the strong robustness of the ATFTNSP model.

Different from the hub airport, the flight flow peak of HFE mostly occurs at noon, and the time regularity is low, and the data fluctuation is more messy. But the ATFTNSP model can still capture the changing trend of flight flow independent of peak or off-peak period of airport traffic. From the prediction results, for the HFE that is greatly affected by the flight operation status of the hub airport, the airport flow results predicted by using the navigable rate matrix to describe the spatiotemporal relationship between the airport network flows are closer to the true value. It shows that ATFTNSP can well learn the spatiotemporal distribution law of flight flow in airport network affected by uncertain weather phenomena, and has the ability to predict the spatiotemporal distribution of traffic from the airport network level.

5. Conclusions

In conclusion, airport flight flow has the characteristics of non-linearity, time-variability, strong coupling and uncertainty, so it is difficult to build an accurate mathematical model. Based on GCN, Attention LSTM and ResNet technology. This paper constructs an ATFSTNP deep learning framework, and applies China’s civil aviation transportation big data to accurately predict the spatiotemporal distribution of domestic airport network flight flows. It is a practical application of deep learning technology in big data analytics of air traffic management. The main contributions and fundings are summarized as follows:

(1)

Different from the previous studies, this paper considers the uncertain influence of weather phenomena in the airport network and the spatiotemporal correlation of flight flow from the perspective of flight operation schedule at the same time. Using GCN to capture the topological structure of the airport network, combined with the LSTM of the attention mechanism, the spatiotemporal correlation of flight flows in the airport network per unit time is studied. We also added the ResNet module to the model, which effectively solved the gradient explosion problem of GCNs, improved the prediction effect of the model, and made it have obvious advantages over other prediction models.

(2)

In this paper, the flight operation big data of 67 airports in China is used to construct a real-time flight flow prediction model for predicting China’s civil aviation transportation, and an ablation experiment is designed for the proposed ATFTNSP model. Based on the experimental results, the composition of the model framework and the action mechanism of the core parts are discussed. The effects of the spatial topology of the airport network, the temporal dimension of the flight flow and the external environmental factors (weather impacts) on the spatiotemporal distribution of the airport flight flow are further analysed. It proves the prediction reliability and robustness of the model proposed in this paper, and provides a prediction model with practical application significance for flight flow control.

(3)

Flight operations are susceptible to disruptions that lead to changes on both the supply and demand sides. While, the supply side and the demand side maintain a dynamic balance. The bad weather conditions will reduce the supply side, and have a greater impact on the prediction accuracy. Hence, this paper studies the spatiotemporal correlation of flight flow by quantifying the impact of weather on airport network by airport capacity. With the help of Rest-GCN technology, the input sequence is weighted by spatial correlation. The weighted sequence is input into the LSTM to realize the joint mining and prediction of the spatiotemporal characteristics of flight flow. Later, by introducing the attention mechanism, the importance of each time step in the historical data is obtained in a rolling manner, which further captures the changing characteristics of the time series and improves the prediction accuracy of the model.

(4)

The key to real-time quantitative prediction of flight flow is to analyse flight delay behaviour and its propagation characteristics. In this paper, the ATFTNSP model, based on the deep learning framework, comprehensively considers the chain conduction caused by the execution of multiple cross-domain flight string by one aircraft and the actual needs of flight operation management. It has significant advantages in real-time prediction of the flight flow with periodic changes.

(5)

This paper represents the airport network as a weighted graph based on flight schedules. We took the traditional adjacency matrix, flow correlation and navigability rate as the spatiotemporal influence weights of the airport flight flow, and used the edge weights to describe the mutual influence and effect of flight flow values between airports from the spatiotemporal dimension, so as to realize the visual analysis of the spatiotemporal influence relationship of flight flows in the airport network.

For the air transportation system with nonlinear and time-varying uncertainty, the method of dynamic real-time prediction from the network level can timely compensate for the uncertainty of flight flow caused by the interference of environmental factors, and essentially improve the dynamic performance of flight flow prediction. The prediction of the spatiotemporal distribution of flight flows at each airport in the airport network is not only helpful for airline operation management, but also provides proactive flight scheduling decisions for air traffic flow control. It is one of the key core technologies for constructing a proactive recovery system for schedule disrupted flights, and it is also a significant way to realize the digital transformation of the aviation industry. The model algorithm proposed in this paper can perform real-time calculation on online big data of air transportation. By predicting the spatiotemporal distribution of flight flows in the airport network, the active recovery of schedule interrupted flights can be realized. At the same time, it improves flight operation and control efficiency, and the economic benefits of passengers, airlines and airports. However, deep learning algorithms cannot make unbiased estimates of the regularity of airport network flight flow performance, and depend on the quality of training data and computer hardware configuration.

Except for the weather impact considered in this paper, other external environmental factors are also worth investigating and systematically studying in the future, such as aircraft failures, air traffic control information, and military activity. In addition, human intervention and traffic control decisions will also affect the spatiotemporal distribution of airports flight flow. In order to effectively control the impact of delays on airport congestion and flight delays and achieve precise management, it is necessary to further discuss the performance law and evolution mechanism of airport network flight flow under human control.