Exploring Delay Propagation Causality in Various Airport Networks with Attention-Weighted Recurrent Graph Convolution Method

Abstract

Exploring the delay causality between airports and comparing the delay propagation patterns across different airport networks is critical to better understand delay propagation mechanisms and provide effective delay mitigation strategies. A novel attention-based recurrent graph convolutional neural network is proposed to identify the hidden delay causality relationship among airports in three different airport networks of China. The selected three airport networks show great heterogeneities in topological characteristics, such as average intensity, modularity and eigenvector centrality. The modeling results indicate that the identified delay causality networks of three airport networks are greatly varied in terms of complexity, delay propagation distance and efficiency. Moreover, the delay state of each airport is categorized into three levels, and the delay state transition of the three networks is explored. The results indicate that delay state transition in the North China Control Area exhibits an obvious bidirectional transition form that mainly propagates between the large-degree airports and small-degree airports, while severe delays of some hub airports account for a relatively large proportion in the other two networks. The results of this study could better reveal the delay propagation mechanism among airports and help airport operators develop effective strategies to alleviate flight delays and improve airport operation efficiency.

1. Introduction

Over the past few decades, the rapid development of the civil aviation industry has greatly promoted air transportation demand and imbalance between airspace and traffic flow, resulting in serious flight delays and traffic congestion within the air traffic network [1,2]. The air transportation system is essentially a large-scale complex network and is usually involved with complicated interconnectivities and great heterogeneities so that a local flight delay will propagate to other airports and even induce large-scale flight delays in the air transportation network [3]. Airport flight delays are caused by many uncertain factors, such as adverse weather, late passengers and additional maintenance duration [4], and usually exhibit different characteristics across various air traffic networks, making the delay propagation mechanism complex and heterogeneous [5,6]. Thus, exploring the delay propagation characteristics across various airport networks is important and challenging, which could help airport operators develop effective flight management strategies to alleviate flight delays and improve airport operation efficiency.

To date, many researchers have developed numerous methods to explore the airport delay propagation mechanism, including the Approximate Network Delay (AND) model, epidemic process-based model and graph-based model. For example, Malone proposed the classical Approximate Network Delay (AND) model to analyze the delay propagation between three typical airports [7]. Then, Pyrgiotis et al. refined the AND model to investigate the complex delay propagation patterns and mitigation strategies within an extensive airport network [8]. Baspinar and Koyunce attempted to develop a Susceptible–Infected–Recovered (SIR) model to simulate the delay propagation process of airport networks under the spread of infectious disease [9]. Li et al. proposed an integrated airport-based Susceptible–Infected–Recovered–Susceptible (ASIRS) epidemic model to simulate the delay propagation in airport networks from a network-level perspective [10]. Barrat et al. modeled the airport network as a graph, with an airport as the node and a flight as the link, then investigated the delay propagation mechanism from the topological analysis of graph characteristics [11]. Zeng et al. further applied the complex network theory to explore the delay characteristics and propagation strength in the study of airport networks [12].

More recently, to fully reveal the network-level delay propagation characteristics, some researchers have started to discover the hidden delay causality relationship between each airport pair and develop the delay causality network according to the identified causality links. The characteristic analysis of a delay causality network can reveal the potential delay propagation directions and infection paths, which could intuitively illustrate the delay correlations between two airports and provide suggestions for airport operators to reallocate the time slots and adjust the buffer time for delay mitigation. The critical component for developing a delay causality network is to determine the delay causality relationship between airports, which mainly fall under three types of approach: the Granger causality test, information entropy-based method and data-driven method. Runge et al. used the Granger causality test to explore the delay causality links between airports and examine the characteristics of a delay causality network [13]. Since the Granger causality test only determines the linear delay causality dependencies between two airports, Zhang et al. applied Transfer Entropy to identify the delay causal links between two airports and quantitatively measure the relevant causality probability [14]. To further capture the nonstationary time series feature in the delay sequential data of each airport in the network, Tan et al. developed an attention-based deep convolution neural network (CNN) for mining airport delay propagation causality [15]. The result indicates that each airport in China is affected by an average of six airports, and airports with small delays are more likely to be affected by other airports.

Although the data-driven method has been applied in dynamic and nonlinear delay causality link identification, the widely used deep learning framework mainly depends on a CNN-based approach, which is skilled at learning embedded patterns from Euclidean structured data, such as image, video and grip-based maps [16,17]. However, the delay time of airports in airport networks is time series sequential data distributed over the topology graph, which is essentially non-Euclidean structured data, making the performance of CNN-based models very poor [18]. Moreover, the temporal dependencies between delays in various time steps are also prominent and should be incorporated into the analysis model. In addition, different airport networks show different network topological characteristics and geographical economics, leading to great heterogeneities in delay propagation patterns and delay state transitions.

To fill the above research gaps, this study develops a novel attention-based recurrent graph convolutional neural network (ARGC-Net) for identifying the delay causality relationships and exploring the delay propagation mechanism in various airport networks. The developed ARGC model employs an end-to-end deep learning framework, which integrates a Spatial Block and a Temporal Block. The Spatial Block employs a graph convolution network (GCN) and attention mechanism to fully capture the spatial dependencies and causality linkage of each airport pair. Compared with a CNN, the GCN can generalize the convolution on the input of airport network delay time series, better characterize the delay state of the entire airport network and uncover the hidden heterogeneous pairwise correlations of flight delays between airports. The Temporal Block employs Long Short-Term Memory (LSTM) to capture the temporal dependencies among time steps. The primary contributions of this study can be summarized as follows:

• The proposed ARGC-Net can uncover both the spatial and temporal dependencies in delay time across different airports simultaneously, and the embedded attention mechanism can indicate the delay causality links between any two airports.
• The developed model shows great transferability when applied to different airport networks. Additionally, the delay propagation networks can be automatically built through testing from the identified attention scores.
• The delay propagation mechanism of three airport networks with various topological characteristics are compared based on the complex network theory.
• The airport delays are grouped into three states, and the delay state transition network is developed and compared in three different ATC areas.

The rest of the paper is organized as follows. Section 2 discusses the structure of the developed ARGC-Net; describes the methodology of the graph convolutional neural network, long short-term memory neural network, the attention mechanism and explains how to identify the causality relationships from the attention scores and t-test. Section 3 introduces the data source and data processing. Section 4 presents the characteristics of the airport network in each region, discusses the predictive performance of the proposed model, compares the topological characteristics in each delay causality network, and analyses the delay state transition in each network. Finally, conclusions are drawn in Section 5.

2. Methodology

2.1. Problem Formulation

The primary objective of the delay causality analysis is to identify the embedded delay influence relationship between airports and understand the impact of delay occurred in one airport that may cause delays in another airport. For example, the delay causality between two airports is established when the delay in the former airport leads to the delay in the latter airport, and accordingly the delay state of the latter airport will change with the change in delay state of the former airport. In this condition, the former airport is considered to have a causal effect on the latter airport. In some previous studies, delay causality between airports is built when the delay information of one specific airport contributes to the delay explanation of another airport after some predefined time steps [19,20,21].

Figure 1 illustrates four types of delay causality between airports: No Causation, Direct Causation, Indirect Causation, and Both Direct and Indirect Causation. No Causation means the delay at airport A will not affect airport B (See Figure 1a), while Direct Causation (See Figure 1b) means the delay at airport A directly propagates to airport B without intermediary airports. Figure 1c depicts Indirect Causation between airport A and airport B with an intermediate, airport C; note that there may be multiple indirect causal paths between the two airports. Moreover, both Direct Causation and Indirect Causation may simultaneously exist between airport A and airport B, as shown in Figure 1d. Thus, the Direct Causation involves only two airports, whereas the Indirect Causation may involve three or more airports. In this study, only Direct Causation is considered in the causality analysis mainly because the direct causality links between airports result in the ATC local authority better understanding the delay propagation process.

Furthermore, a weighted directed graph, also named as the Delay Causality Network (DCN), is utilized to represent the delay propagation process and is based on all the identified direct causality links within the entire airport network. For example, a delay network that consists of N airports is denoted as G = (V, E, D), where V = {v_i}_{i = 1:N} indicates the set of airports in the study network, and v_i represents the feature vector of airport i; E = {e_ij}_{i,j = 1:N} represents the edges set; if the delay in airport i directly causes the delay in airport j, the airport i has an edge pointing to j, and e_ij= 1; otherwise, e_ij = 0. D = {d_ij}_{i,j = 1:N} represents the weight set of edges, where d_ij indicates the degree of delay in airport i that affects airport j. Figure 2 shows an example of the delay propagation network, where the arrows denote the direction of delay propagation.

2.2. The Framework of Proposed ARGC Model

In this study, a novel attention-based recurrent graph convolutional neural network (ARGC-Net) is developed for delay causality relationship identification in the airport network. Figure 3 illustrates the structure of the developed model, which integrates a Spatial Block and a Temporal Block into an end-to-end deep learning architecture. More specifically, the input of the model is a series of network-shaped delay time data from airports. Then, the spatial block employs a graph convolution network (GCN) to fully capture the spatial dependencies among airports in the input network, and the attention mechanism is utilized to capture the causality linkage of each airport pair in the form of computed attention scores. In addition, a Temporal Block employs Long Short-Term Memory (LSTM) to capture the temporal dependencies among different time steps of the delay sequences. Finally, multiple fully connected (FC) layers are further utilized to generate the predicted delay output (See Figure 3). Each component of the developed model will be detailed and explained as follows.

2.3. Graph Convolutional Network

The graph convolutional network (GCN) is a deep learning framework for modeling data on irregular or non-Euclidean domains [21,22,23], and has achieved greater performance than traditional CNN architecture in the fields of citation network analysis [24], flight delay prediction [25] and metro ridership forecasting [18]. Previously, two types of GCNs have been widely explored, including spatial-based GCNs that utilize local graph convolutions to extract feature information from adjacent nodes [26], and spectral-based GCNs that transform an original graph signal into a parameterized Fourier domain [27]. A spectral-based GCN performs with high efficiency and with the input dataset having same size; thus, this study employs the spectral-based approach to develop a basic GCN to process regular delay time data [28].

In this study, for each input time step, the delay time of airports can be viewed as a graph G = (V, E, A), where V is airports set; E represents the set of edges that connect each pair of airports and $A\in {ℝ}^{N\times N}$ indicates the adjacency matrix (See Figure 4). Then, a normalized graph Laplacian matrix is defined as follows:

$$L=I_N{D}^{-1/2}A{D}^{-1/2}$$

(1)

where I_N represents the identity matrix and $D\in {ℝ}^{N\times N}$ indicates a diagonal degree matrix with $D_{ii}={\sum }_j{A}_{ij}$.

L is a symmetric positive semidefinite matrix, which can be further broken down as:

$$L=U\Lambda U^T$$

(2)

where $U$ = [u₀, u₁, …, u_N−1] and Λ = diag ([λ₀, λ₁, …, λ_N−1]]); λ₀, λ₁, …, λ_N−1 indicates the eigenvalues of L, and u₀, u₁, …, u_N−1 indicates the relevant set of orthonormal eigenvectors.

Accordingly, a spectral convolution on the graph is defined as follows:

$$g_{\theta }\ast x=U{g}_{\theta }{U}^{T}x$$

(3)

where $g_{\theta }$ indicates the function of the eigenvalues of L. To improve the computation efficiency of the eigen decomposition operation on largescale graphs, Defferrard et al. suggested a form of Chebyshev polynomial expansion (up to Kth order) to approximate the operation of $g_{\theta }$ [21], such as:

$$g_{\theta }\left(\Lambda \right)\approx {\displaystyle \sum }_{k=0}^K{\theta }_k{\Lambda }^T$$

(4)

Accordingly, the spectral convolutional of a signal × with filter $g_{\theta }$ can be further written as:

$$g_{\theta }\ast x\approx {\displaystyle \sum }_{k=0}^K{\theta }_k{L}^{T}x$$

(5)

Furthermore, Kipf and Welling further simplified the spectral filter to a first-order Chebyshev polynomial to avoid the potential overfitting issue on graphs with wide node distribution and further improve the computation efficiency of the spectral convolution operation [24], which has the following form:

$$g_{\theta }\ast x\approx \theta \left(D^{-\frac{1}{2}}A{D}^{-\frac{1}{2}}\right)x$$

(6)

Therefore, the GCN layers propagate from the input to the output in the following ways:

$$H_{l+1}=\sigma \left(D^{-\frac{1}{2}}A{D}^{-\frac{1}{2}}{H}_l{W}_l\right)$$

(7)

where H_l+1 and H_l represent the outputs of the lth and l + 1st hidden layer; $\sigma \left(·\right)$ represents the activation function. W_l represents the trainable weight matrix in the lth graph convolution hidden layer. Note that the dimensions of the tensors in Equation (7) are: $A\in {ℝ}^{N\times N}$, $H_l\in {ℝ}^{N\times I}$, $W_l\in {ℝ}^{I\times O}$ and $H_{l+1}\in {ℝ}^{N\times O}$; where N, I, O indicate the number of nodes, input features and output features, respectively.

2.4. Attention Mechanism

The output of the basic GCN is usually encoded into a fixed-size context vector through allocating equal weights to all the airports in the network, which neglects the varied contributions of other airports for the predicted airport and may lead to a poor prediction performance and inaccurate causality relationship identification. Recently, Bahdanau et al. proposed the concept of the attention mechanism, which aims to allocate varying attention weights to each node in networks during the learning process [29]. In this study, the attention mechanism is embedded into the GCN layer to capture the influence of delay in connected source airports i on target airports j. For any target airport, the input vector is X = {x_ij}_{i = 1:N} at time t, where x_ij = {x_ij^T}_{T = t-timestep:t} represents the delay time of airport i, which is connecting with airport j, in a time range of t-timestep to t. The metrics V are used to extract delay information in each airport, and the metrics Q and metrics K are used to exchange delay information between different airport [30]; these metrics are calculated based on Equation (8):

$$\left\{\begin{array}{c}Q=X⨀W_Q\\ K=X⨀W_K\\ V=X⨀W_V\end{array}\right.$$

(8)

where W_Q, W_K and W_V are the trainable weight metrics. Additionally, the attention score Z₀ of each connected airport is obtained using Equation (9):

$$Z_0=\sigma \left(\frac{Q{K}^T}{\sqrt{d_k}}\right)V$$

(9)

where $\sigma \left(·\right)$ is the activation function and d_k is the dimensions of metrics K, which ensure a steady gradient decent during training. The final outputs of the delay time in the target airport are computed as follows:

$$Z=Z_0⨀W_0$$

(10)

where W₀ is the trainable weight metrics. The computation process of the attention mechanism is shown in Figure 5.

2.5. Long Short-Term Memory Neural Network

The Spatial Block fully captures the spatial dependencies of the delay times among various airports in the network. However, the delay times of airports also exhibit significant temporal correlations between historical and future time steps. To address this issue, a Recurrent Neural Network (RNN) has been employed in the deep learning architecture to capture the temporal dependencies in various fields of traffic flow prediction, crash risk prediction and traffic speed prediction [17,31]. However, the traditional RNN usually suffers from the problem of gradient vanishing or gradient exploding when the prediction time steps become larger, resulting in the occurrence of an enhanced RNN-based architecture, named the Long Short-Term Memory Neural Network (LSTM) [32].

Figure 6a illustrates the structure of the LSTM neural network; the time steps of the input dataset and output dataset are T_step and T_{output_step}, respectively. In the LSTM neural network, the input vector X = {X_t-Tstep, X_t-Tstep+1, …, X_t−1} represents the preprocessing information of airports, where X_t−1 represents the preprocessing information at time t − 1. The LSTM contains three gates, the input gate, forget gate and output gate, which are used to decide whether to add or remove historical information to a cell state (See Figure 6b). For each time step, the three gates will be iteratively calculated using Equations (11)–(15):

$$i_t=\sigma \left(W_{xi}{x}_t+W_{hi}{h}_{t-1}+W_{ci}⨀c_{t-1}+b_i\right)$$

(11)

$$f_t=\sigma \left(W_{xf}{x}_t+W_{hf}{h}_{t-1}+W_{cf}⨀c_{t-1}+b_f\right)$$

(12)

$$c_t=f_t⨀c_{t-1}+i_t⨀tanh\left(W_{xc}{x}_t+W_{hc}{h}_{t-1}+b_c\right)$$

(13)

$$o_t=\sigma \left(W_{xo}{x}_t+W_{ho}{h}_{t-1}+W_{co}⨀c_t+b_o\right)$$

(14)

$$h_t=o_t⨀tanh\left(c_t\right)$$

(15)

where W_xi indicates the weight matrix between the input variables and the output of the input gate, and b_i indicates the bias of input gate. Similarly, W_hi, W_ci, W_xf, W_hf, W_cf, W_xc, W_hc, W_xo, W_ho and W_co indicate the weights matrices which conduct a linear transformation from the vector of the first subscript to the second subscript, while b_f, b_c and b_o indicate the associated biases. $⨀$ indicates the Hadamard product, which calculates the element-wise products of two vectors, matrices, or tensors with the same dimensions. σ (∙) and tanh (∙) are the two commonly used nonlinear active functions [33].

2.6. Objective Function

The objective of the developed ARGC model is to minimize the mean squared error between the predicted delay time and real delay time of the target airport. The objective function is given as follows:

$$\begin{array}{c}min\\ W,b\end{array}{\displaystyle \sum }_i{\displaystyle \sum }_t{Y}_t^i-X_t^i\begin{array}{c}2\\ 2\end{array}+\alpha Y_t^i\begin{array}{c}2\\ 2\end{array}$$

(16)

where Y_j^t and X_j^t in the first term represent the estimated delay time of airport j at time step t and its ground truth, respectively. W, b indicates all the trainable weights and biases in the whole model architecture. The second term represents the L2-norm regularization term, which is utilized to avoid the potential overfitting problem during the training process. α indicates the regularization rate, which is greater than zero. The whole process of the training algorithm of the developed ARGC model is given as Algorithm 1:

Algorithm 1 ARGC training algorithm

Input: Airports number i∈V, Historical delay time of all airports in the network {xi}, i∈V, Time steps of input dataset Tstep, Time steps of output dataset Toutput_step, Delay time graph G = (V, E, A) Output: Delay time of airports, ARGC model with attention scores of the well-trained parameters // Prepare the training dataset   Initialize a null set: Ɗ = Ø;   for time interval t (1 ≤ t ≤ T) do     Obtain the historical delay time of all airports at each previous Tstep:     Xinput = {xijt-Tstep, …, xijt−1, xijt}, i∈V     Obtain the historical delay time of all airports at each predicted time step:     Xoutput = {xijt+1, xijt+2, …, xijt+Toutput_step}, i∈V     Put the training sample into the dataset: (Xinput, Xoutput) → Ɗ;   end for // Training ARGC model   Initialize the hidden status, all weights and bias parameters;   Concatenate the graphs at Tstep: [A1, A2, …, ATstep] → A;   for n = 0 → number of epochs do     Randomly select a batch of sample Ɗb from Ɗ as input, where b = 1, 2, …, B;     Obtain the output through hidden GCN layers, followed by Equations (1)–(7);     Obtain the attention score through Spatial Block, followed by Equations (8)–(10);     Flatten the output into a latent vector;     Obtain the output of LSTM by passing the flattened vector through Temporal Block, followed by Equations (11)–(15);     Estimate the predicted delay time for each output time step: Yjt= σ (WFC • LSTM +bFC);     Optimize W, b by minimizing the loss function defined in Equation (16);   end for

2.7. True Causality Relationship Test

From the developed ARGC model, the impact of the source-connected airports on the target airport can be evaluated using the attention score values in the final trained model. To further determine the true direct causal relationship from the output attention scores, this study proposes a two-step method. In the first step, a filter threshold of attention weights w₀ should be defined. However, no specific rules are built for setting this threshold value. If the threshold value w₀ is set too small, more invalid links will be selected as causal links, improving the complexity of the delay propagation network. If the threshold value w₀ is set too large, some critical causal relationships will be missed in the delay propagation network. Previous studies usually subjectively determine the value of w₀ when the number of identified causal links will not significantly change, and this study follows the same method [15].

In the second step, for any filtered candidate causality x_i → x_j, the order of the delay time series of airport i will be shuffled and then input into the trained model. If the delay time prediction error of airport j is significantly changed, the identified causal effect between airport i and j can be considered as a robust and true causality relationship. A t-test is accordingly used to evaluate the significance of difference between the original delay time series and shuffled delay time series [34], assuming the model output of the original delay time series is D^o = {D₁, D₂, D₃, …, D_N}, and the model output of the shuffled delay time series is D^s = {D₁, D₂, D₃, …, D_N}. The t-value is calculated as follows:

$$t=\frac{{\overline{X}}_1-{\overline{X}}_2}{\sqrt{\frac{(n_1-1)S_1^2+(n_2-1)S_2^2}{n_1+n_2-2}\left(\frac{1}{n_1}+\frac{1}{n_2}\right)}}$$

(17)

where S₁² and S₂² are the variance of D^o and D^s; ${\overline{X}}_1$ and ${\overline{X}}_2$ are the average value of D^o and D^s; and n₁ = n₂ = n are the size of D^o and D^s. Then, the critical value z is obtained based on the freedom degree n − 1 and the significance level $\sigma$. If t > z, the t-value falls in the rejection domain, meaning the difference between D^o and D^s is significantly larger, and the connected airports have a true effect on the target airport.

3. Data Source

This study uses the data collected from Civil Aviation Administration of China (CAAC) to illustrate the procedure for delay propagation network analysis with the developed ARGC model. To explore the differences in the delay propagation mechanism across different airport networks, this study selects three typical airport networks in the North China Control Area, East China Control Area and Central and South Control Area, respectively. Figure 7 illustrates the spatial distribution of the three selected airport networks. The three selected airport networks show great diversities in their traffic flow patterns, regional economic levels and weather characteristics, which can better reveal the differences in the delay propagation mechanism among different ATC areas and validate the spatial transferability of our proposed ARGC model.

The selected flight operation data for model development cover three complete months from 1 July to 30 September 2019, which mainly covers the demand peak of air transportation in China. Each flight trip record contains the following information: Flight Callsign, Departure Airport, Arrival Airport, Estimate Departure Time, Actual Departure Time, etc. Then, the Flight Delay Time could be calculated based on the difference between the Estimate Departure Time and Actual Departure Time. Notably, in this study, the delay time is recorded as 0 min once the flight Actual Departure Time is earlier than the Estimate Departure Time, and cancelled flights are removed from the database since no delay propagation is caused by them. The delay duration is aggregated into one hour to calculate the Average Delay Time for each time step.

Figure 8a depicts the frequency distribution of the delay times in the three selected airport networks, where airport network I, airport network II and airport network III represent the airport networks in the North China Control Area, East China Control Area and Central and South Control Area, respectively. It can be found that airport network II shows a relatively larger frequency than the other two airport networks, and the delay times of most airports are less than 60 min. Figure 8b further illustrates the temporal distribution of the average daily delay time in the three networks. It indicates that different types of airport networks exhibit quite different delay patterns. Specifically, the delay pattern of the airport network in the North China Control Area is similar to that of the airport network in Central and South Control Area, which both exhibit a morning peak and an evening peak. The delay pattern of the airport network in the East China Control Area fluctuates more and shows an obvious peak around 2pm. Moreover, the late-night delay times are also very large in the airport networks in the North China Control Area and Central and South Control Area but drop significantly in the airport network in the East China Control Area. Furthermore, the airport network in the East China Control Area has a relatively larger number of flights but exhibits a lower average delay time, which indicates that the flights operate with higher efficiency in this network. The different patterns of delay times in the three airport networks created great challenges for the developed causality relationship identification model.

4. Results of Data Analysis

4.1. Characteristics of Airport Network in Each Region

In this study, three airport networks located in different air traffic control areas of China are selected. Figure 9 depicts the airline connections in the local regions of three selected airport networks. A total of 34 airports are selected in the North China Control Area (see Figure 9a), while 44 airports and 35 airports are selected in the East China Control Area (see Figure 9b) and the Central and South Control Area (see Figure 9c), respectively. The size of each node is proportional to the degree of each airport, and the thickness of each edge is proportional to the air traffic flow between the two airports.

Table 1. The topological characteristics of the three selected airport networks.

Indicators	Airport Network I	Airport Network II	Airport Network III	Mean	STD
Degree	34	44	35	39	7.071
Edge	213	497	354	354.667	142.001
Average Degree	6.265	11.295	10.144	9.235	2.635
Average Intensity	18.277	29.006	31.463	26.249	7.012
Density	0.19	0.263	0.297	0.25	0.055
Number of Communities	3	4	5	4	1
Modularity	0.199	0.117	0.11	0.142	0.049
Average Clustering Coefficient	0.57	0.69	0.616	0.625	0.061
Average Path Length	2.059	1.807	1.759	1.875	0.161
Eigenvector Centrality	4.482 × 10−6	3.179 × 10−6	1.225 × 10−6	2.962 × 10−6	1.639 × 10−6

shows the results of some critical topological characteristics of the three selected airport networks.

From Table 1, it can be seen that airport network II exhibits the largest values for degree and edge due to the large traffic of the East China area. The Average Degree and Average Intensity indicators can partially reflect the complexity of the airport network. Compared with the other two airport networks, the traffic flows of the airport network in the North China Control Area mainly concentrate in several hub airports, such as ZBAA and ZBTJ. In addition, the Density and Modularity indicators are also selected to evaluate the connectivity of the airport network. The density of the three airport networks is generally similar. The Modularity indicator measures the degree to which the components of a large airport network may be separated and recombined. In general, airport networks with a high modularity value show dense connectivity between airports within the same community and sparse connectivity between airports within different communities. It can be observed that airport network I can be divided into three sub-communities with higher modularity values than the other two airport networks, which indicates that the delays are more likely to be dispersed among three sub-regions of this airport network.

The Average Clustering Coefficient indicator evaluates the local connection characteristics of the airport networks and is measured by the degree to which nodes in a graph tend to cluster together. More specifically, the local clustering coefficient for an airport is given by a proportion of the number of links between the airport within its neighborhoods divided by the number of links that could possibly exist between them. For example, the local coefficients of airport A in the network of Figure 10a–c are 1, 2/3 and 1/3, respectively. Then, the Average Clustering Coefficient of a specific airport network is defined by the average of the local clustering coefficients for all airports in the network. The Average Clustering Coefficient indicator quantifies how close the airport and its neighbor airports are with a complete graph, and is commonly employed in previous studies to determine whether a graph is a small-world network [35]. Accordingly, if the delays are generated in hub airports that are usually involved with a high local clustering coefficient value, then a high probability of a cascading effect of delay propagation across the whole airport network may exist. From Table 1, the delays generated by the hub airports in airport network II are more likely to cause severe effects on the whole network, rather than just other networks.

Moreover, the Average Path Length indicator is selected to represent the average distance between any pair of airports in the selected network. To normalize the distance metrics in different airport networks, the Path Length of the closest node is set to 1. As is shown in Table 1, airport network I exhibits a larger value for average path length than the other two airport networks, which indicates that air traffic delays will take more time to propagate across the entire network. In addition, the Eigenvector Centrality indicator is further employed to evaluate the diversity of airport importance in the network. The airport’s importance mainly depends on its node degree value. The higher the Eigenvector Centrality is, the higher the variance of airport importance in the network will be. Since airports with higher importance are more likely to propagate the delay to the airports with lower importance, airport networks I and II with higher values of Eigenvector Centrality may generate more delay propagation.

Figure 11 overviews the characteristics of the airport networks with the proposed eight indicators in the form of a radar chart. In general, the comparative analysis results of the selected airport characteristic indicators reveal that different airport networks show great diversities in their topological patterns, which will greatly affect how delay generation and propagation arises. For example, the airport network in the North China Control Area may have more delay propagation duration and localization. The airport network in the East China Control Area is involved with more complex network structures and more serious delay cascading effects. Thus, constructing a delay prorogation network for each airport network could provide insightful suggestions to each ATC local administration for understanding the delay causes and developing more targeted countermeasures to reduce air traffic delays.

4.2. Causality Analysis Using the Proposed ARGC Model

In this study, the ARGC model is developed for exploring the causality relationship between the delays of airports in the study network. The model consists of one GCN layer for spatial feature extraction, one LSTM layer for temporal feature extraction and one FC layer for model output. Prior to training, a series of hyper-parameters needs to be set. Note that the developed ARGC model is conducted on each airport in the network successively to fully explore the causal relationships between any airport pair. Thus, to reduce the computational burden, the models of the same airport network are designed to share the same hyper-parameters. A grid search method is employed to determine the optimal value of each hyper-parameter, and the results of optimal hyper-parameters for each airport network are shown in

Table 2. The main hyper-parameters of the model.

Hyper-Parameter	Description	Hyper-Parameters in Airport Network I	Hyper-Parameters in Airport Network II	Hyper-Parameters in Airport Network III
Tstep	The time steps of input dataset	5	4	4
Toutput_step	The time steps of output dataset	1	1	1
HGCN	The number of hidden units in each GCN layer	1	1	1
activationGCN	The activation function of the GCN cell	Relu	Relu	Relu
HLSTM	The number of hidden units in each LSTM layer	8	8	8
activationLSTM	The activation function of the LSTM cell	Relu	Relu	Relu
HFC	The number of hidden units in each fully connected layer	1	1	1
ActivationFC	The activation function of the FC cell	Relu	Relu	Relu
Optimizer	Implemented optimizer during the training process	Adam	Adam	Adam
$\alpha$	Learning rate	0.005	0.01	0.01
B	Batch size	16	16	16
d	The dropout rate	0.05	0.1	0.05

.

During the training process, the mean squared error (MSE) is used as the loss metric for optimization. All of the input dataset is processed through Z-score normalization before model training. A Dropout Layer and Early Stopping Mechanism are also employed to avoid the issue of model overfitting [36]. The proposed model is implemented with the deep learning libraries of TensorFlow, Keras and TF_geometric [37]. All the experiments are conducted using Python 3.8.13 on a computing environment consisting of the MacOS Ventura 13.1 system, 16 GB RAM and Apple M1 Pro Silicon.

The average prediction performance of the proposed ARGC models for each airport network is shown in Figure 12a. It can be observed that the developed model exhibits the lowest performance on airport network II in terms of highest RMSE and MAE values, which is potentially due to the relatively large throughput of the airports in this network. Moreover, Figure 12b further depicts the loss functions during the training process for some typical airports in the three selected networks. The developed model achieves decent prediction performance for each of the six typical airports after nearly 50 training epochs, and the ZGSZ, ZBTJ and ZBAA airports show a relatively high prediction accuracy.

Figure 13a shows the attention score generated by the models of the selected six typical airports. The result generally indicates that the delay causality relationship between the selected typical airports and their neighboring airports are quite different, which represents that the delay propagation impact varies by different airports. Figure 13b further depicts the relationship between the attention score threshold value w₀ and the related number of identified causal links. In general, if the threshold value w₀ is set too small, more invalid links will be selected as causal links. If the threshold value w₀ is set too large, some critical causal relationships will be missed. Previous studies usually subjectively determine the value of w₀ when the number of identified causal links will not significantly change. This study follows the same method, and the final threshold values w₀ of each airport network are set as 0.3, 0.3 and 0.2, respectively.

Table 3. Changed number of candidate airports.

Airport Network	Original Candidate Number	Candidate Number after Attention Score Selection	Candidate Number after t-test * Selection
Airport Network I	221	$57 \left(\downarrow 74.2\%\right)$	$44 \left(\downarrow 22.8\%\right)$
Airport Network II	510	$186 \left(\downarrow 63.5\%\right)$	$70 \left(\downarrow 62.3\%\right)$
Airport Network III	362	$153 \left(\downarrow 57.7\%\right)$	$61 \left(\downarrow 60.1\%\right)$

* The significance level of the t-test is 0.05.

shows the result of the t-test. It can be observed from the attention score that a relatively high proportion of identified causal links has been kept in airport network I after the test.

4.3. Topological Analysis of the Delay Causality Network

On the basis of the final identified causal links, the DCNs are developed for each airport network, respectively. Figure 14 illustrates the DCNs, and the three DCNs exhibit quite different topological structures intuitively. Many previous studies usually employ Complex Network Metrics to explore the topological characteristics of DCNs, including network indicators such as Diameter, Network Efficiency, Density, Assortativity and Modularity [12]. The indicators of the DCNs are presented in

Table 4. Results of Complex Network Metrics.

Indicators	Delay Causality Network I	Delay Causality Network II	Delay Causality Network III	Mean	STD
Node	34	44	33	37	4.966
Edge	53	88	65	68.667	14.522
Average Degree	1.853	4.681	4.515	3.683	1.295
Diameter (KM)	1137.51	1479.253	1849.669	1488.81	290.816
Network Efficiency	1.025 × 10−4	2.341 × 10−4	2.483 × 10−4	1.949 × 10−4	6.564 × 10−5
Density	0.056	0.108	0.141	0.101	0.034
Assortativity	−0.065	−0.033	−0.032	−0.043	0.015
Modularity	0.023	0.005	0.033	0.02	0.014

and Figure 15 for topological analysis and airport network comparison.

The Average Degree value of the DCN indicates the average number of potentially infected airports caused by the source airport. From Figure 15, DCN II and DCN III are involved with 4.681 and 4.515 infected airports on average, which are higher than DCN I. This indicates that delays occurred in airports of the East China Control Area and Central and South Control Area are more likely to generate intense effects. Moreover, DCN III shows the largest Diameter value, indicating that air traffic delays in the Central and South Control Area will propagate farther and accordingly may cause a more far-reaching impact to the airport network. In Table 4, the Density indicates the centrality and complexity of a network. DCN I shows a relatively low Density, which represents that the delay propagation mechanism of the North China Control Area is less complicated than that of the other two areas. The Network Efficiency indicator further explores the delay propagation efficiency. The higher the value, the more accessible and faster the delay will propagate. It can be seen that the East China Control Area and Central and South Control Area are involved with higher delay propagation efficiencies that are mainly due to the fact that the average path length in these two areas is shorter.

The Modularity indicator is used to indicate the degree of the network that could be divided into sub-parts. The airports belonging to the same sub-regions tend to exhibit similar delay propagation characteristics. From Table 4, DCN I and DCN III have a higher Modularity value, which indicates that the North China Control Area and Central and South Control Area are involved with more diverse sub-regions and inconsistent delay propagation patterns, leading to great challenges for local ATC authorities in developing effective policies and flow control strategies for relieving air traffic delays. The Assortativity indicator is used to measure the correlation between nodes in a network. In this study, the Assortativity in all three DCNs is found to be negative, indicating that airports with high degrees of connectivity are more likely to propagate delays to airports with lower degrees of connectivity. Notably, the absolute value of Assortativity in DCN I is higher than that of other two networks, which suggests that in the North China Control Area, airports with a large number of airlines are more likely to transfer delays to airports with a small number of airlines.

In summary, the comparative analysis of three DCNs based on Complex Network Metrics reveals quite different topological characteristics and delay propagation features in different ATC areas. Specifically, the delay propagated in the The orth China Control Area is involved with a specific delay propagation direction. Delays in East China Control Area and Central South Control Area are more complex but propagate with higher efficiency. The results again validate the fact that different ATC areas exhibit different delay propagation patterns, and the comparative analysis provides insightful suggestions for local ATC authorities to develop more targeted strategies for reducing air traffic delays.

4.4. Delay State Transition Analysis

Based on the above analysis, the delay causality relationship between airport pairs in each network can be identified. However, the severity of delay causality is still unclear. For example, a normal delay in airport A might cause a serious delay in airport B. Thus, in this section, the delay state transition in each identified causality link will be further explored. Followed by the definition by the Federal Aviation Administration (FAA), the delay times are categorized into three levels, including Delay Level I (delay time less than 15 min), Delay Level II (delay time between 15 min and 30 min) and Delay Level III (delay time more than 30 min). Note that the delay propagation time window between each causality link is mainly determined by the flight time between the related airport pairs [38]. Accordingly, the delay state transition from airport A to airport B in each time step t is denoted as the delay level at time t of airport A to the delay level of airport B after the flight time window between the two airports. Aggregated by all the time steps, the delay causality link between the two airports can be represented as a mixture of various delay state transitions. As is shown in Figure 16, the delay causality from ZBAA to ZBAL can be further indicated as a mixture of nine possible delay state transitions. To make an easy and intuitive visualization, only the delay state transition that accounts for the largest proportion (Delay Level III → Delay Level I) is plotted.

Figure 17 visualizes the delay state transitions of the three DCNs. In general, the delay state transitions are quite different in the three airport networks. The delay state transition of the airport network in the North China Control Area is mainly related with the ZBAA and ZBTJ airports and mainly manifests in two types of transitions: Delay Level III → Delay Level I and Delay Level I → Delay Level III. The delay state transition in DCN I exhibits an obvious bidirectional transition form which indicates that the delays mainly propagate between the large-degree airports and small-degree airports. The results indicates that delays in the airport network in North China Control Area propagate locally in such a way that serious air traffic delays generated from hub airports cannot be effectively absorbed and relieved during the propagation process. Compared with airport network I, the other two networks show more complex and diverse delay state transition. Specifically, most of the delay state transitions in the airport network in the East China Control Area are related with the airports of ZSPD, ZSSS and ZSAM, and the transition of Delay Level III → Delay Level III accounts for a relatively large proportion, indicating that severe delays in some hub airports, e.g., those caused by some adverse weather, seem difficult to reduce and tend to also cause severe delays in their neighboring airports within the East China region. The delay state transitions in the airport network in the Central and South Control Area are more evenly distributed. Each type of delay state transition accounts for a similar proportion. For some hub airports, such as ZGGG, ZGSZ and ZJHK, Delay Level III → Delay Level I is a prominent transition; while for some airports, such as ZHHH and ZHCC, Delay Level III → Delay Level III is a prominent transition. The evenly distributed delay state transitions in the airport network in the Central and South Control Area provide great challenges for local ATC authority in developing some unified and generic air traffic flow management strategy for reducing airport network delays.

5. Conclusions

The primary objective of this study was to explore the delay causality between airports and compare the delay propagation patterns across various airport networks. Three complete months of flight operation data ranging from 1 July to 30 September 2019 were collected from CAAC to illustrate the procedure. An attention-based recurrent graph convolutional neural network (ARGC-Net) was proposed to identify the delay causalities between airports in three typical airport networks of China. The learned attention score values identified a causal relationship between different airports and the t-test-validated true delay causality, then the causality links among airports in each network were obtained, and were accordingly used to develop the delay causality networks. Some critical indicators were selected to compare the topological characteristics of different obtained DCNs; in general, the analysis results of the selected delay characteristics indicators revealed that different airport networks showed great diversities in their delay propagation patterns. Furthermore, the delay state of each airport was categorized into three levels, and the delay state transition of the three networks was explored. The results indicate that the delay state transition in the airport network in the North China Control Area exhibits an obvious bidirectional transition form that mainly propagates between the large-degree airports and small-degree airports, while severe delays of some hub airports account for a relatively large proportion in the East China Control Area and Central and South Control Area.

The results of this study have provided a new perspective for ATC to understand traffic delays in air transportation systems. The findings reveal that different ATC areas exhibit quite different delay propagation patterns, and the comparative analysis of different airport networks could help local ATC authorities to develop more effective and targeted strategies to reduce air traffic delays and improve airport operation efficiency. However, several limitations of this study should be further explored. First, this study only considered limited factors that may cause the delays between different airports, but other factors may also affect the delay propagation, such as adverse weather and controller intervention. Second, only some typical network topological indicators have been introduced to evaluate the topological characteristics of networks in this study. Third, the attention mechanism is an embedded block in the developed deep learning architecture, and the attention score is a trainable weighted metric and updated during each training step in this study. Thus, it is hard to obtain a functional model in explainable mathematical form based on the learned weights and transition function.

Future studies could focus on incorporating more contributing factors into the delay propagation causality. For example, the adverse weather conditions in hub airports have great influence on delays in the whole network, a fact which could be incorporated into the delay causality identification model. More detailed and comprehensive network topological indicators could reveal the complex characteristics of networks; further studies could introduce more network topological indicators to explore the relationship between topological structures of airport network and the related delay propagation patterns, which could provide insightful suggestions for route planning and new airport localization. In addition, the attention score in this study is an unexplainable training weighted metric; further work could focus on how to make this trainable weight explainable through some mathematical functions.