A Novel Aircraft Trajectory Generation Method Embedded with Data Mining

Abstract

Data mining has achieved great success in air traffic management as a technology for learning knowledge from historical data that benefits people. However, data mining can rarely be embedded into the trajectory optimization process since regular optimization algorithms cannot utilize the functional and implicit knowledge extracted from historical data in a general paradigm. To tackle this issue, this research proposes a novel data mining-based trajectory generation method that is compatible with existing optimization algorithms. Firstly, the proposed method generates trajectories by combining various maneuvers learned from operation data instead of reconstructing trajectories with generative models. In such a manner, data mining-based trajectory optimization can be achieved by solving a combinatorial optimization problem. Secondly, the proposed method introduces a majorization–minimization-based adversarial training paradigm to train the generation model with more general loss functions, including non-differentiable flight performance constraints. A case study on Guangzhou Baiyun International Airport was conducted to validate the proposed method. The results illustrate that the trajectory generation model can generate trajectories with high fidelity, diversity, and flyability.

1. Introduction

1.1. Motivation

Society has entered a data-driven era in which huge amounts of data are being generated every day. In the last decade, more and more researchers have focused on mining useful information from historical data to create benefits for people. Accordingly, various data-based applications have been developed, including traffic-related mining [1,2], autonomous path planning [3,4,5], and popular route discovery [6]. In the air traffic management domain, trajectory data have been used to identify traffic patterns [7,8,9], estimate the loss of separation probability [10], and gain operation experience [11,12].

However, the knowledge learned from historical data can rarely be utilized in optimization since it belongs to functional space. To tackle this issue, this paper proposes a novel trajectory generation method embedded with data mining that can be compatible with optimization algorithms. Compared with the other data-driven trajectory generation methods mentioned in Section 1.2, the proposed method can automatically mine helpful information (all kinds of pilot maneuvers) from historical data to improve generation performance.

1.2. Related Work

This study defines three requirements that the trajectory generation method should meet: firstly, fidelity—synthetic trajectories should follow the characteristics of actual trajectories [13]; secondly, diversity—the proposed method could reproduce some trajectories with a small probability of occurrence; and thirdly, flyability—synthetic trajectories should meet the flight performance constraints.

The current research on trajectory generation can be divided into model-driven and data-driven methods. Model-driven methods [14,15] generate aircraft trajectories based on kinetic and kinematic models with several parameters, such as initial aircraft states, aircraft performance coefficients [16,17], weather conditions, and aircraft intents. Since these methods depend on the model and parameters of aircraft, the trajectories generated by these models can undoubtedly meet the requirement of flyability. To ensure the generation of trajectories with good diversity, some model-driven methods [18,19,20] adopt the free-flight concept [21,22] and do not fly along the published flight routes. However, such an approach induces poor fidelity of trajectories since they have different distributions from the actual ones. To address the poor-fidelity problem, other model-driven methods [23,24,25,26] search for optimal flight paths among predefined routes, such as Standard Terminal Arrival Routes and some of their related deviation patterns. However, due to the commonly existing long-tail effect, these predefined routes are limited and not enough to describe all the potential routes in real operation, leading to the homogenization of generated trajectories (poor diversity).

Incorporating controllers’ experience from historical data while generating trajectories can be a promising approach to improve fidelity and diversity. Data-driven methods have been intensively explored to generate trajectories using machine learning algorithms in the ground transportation field [27,28,29,30,31]. Some trajectory generation models have exploited Multilayer Perceptron [32], Long Short-Term Memory (LSTM) [33], and convolutional neural networks [34] to extract spatial–temporal features from actual trajectories. Other trajectory generation models have taken advantage of Variational Auto-Encoders (VAEs) [35], Generative Adversarial Networks [36], and their variants (GANs) [37,38] to create new trajectories. With the success of data-driven data generation in ground transportation, several researchers have extended this concept to generate aircraft trajectories with data-driven methods. Existing solutions include VampPrior TCVAE [39], Conv1D-GAN [40], Gaussian mixture model-based generators [41], Principal Component Analysis (PCA) [42], and functional PCA [43].

However, existing data-driven trajectory generation methods have the following drawbacks:

• For most trajectory generation models, one cannot tell what a generated trajectory looks like before obtaining the final results. To tackle this issue, Ref. [43] generated trajectories via functional PCA and controlled the shape of the trajectories by modifying the mean and principal component functions. Analogously, Krauth et al. [39] controlled the shape of trajectories by sampling points in certain areas in latent space. Although these efforts partially solved the problem and enable one to roughly determine the shape of trajectories, uncertainties still remain in the generation process. Thus, these data-driven methods can rarely cooperate with optimization algorithms to generate optimal trajectories.
• Existing methods prefer to exploit a distribution to capture the flight path deviations. However, deviations are not only caused by uncertainties such as weather, aircraft performance, and human factors, but are also caused by controllers’ instructions. Unlike uncertainties in actual operations, the controller’s instructions must always meet certain rules and cannot be modeled through a distribution. To imitate controllers’ instructions, Jarry et al. [43] additionally introduced several modification operators to change the shape of trajectories. However, such a method requires some fine-tuning by experts and cannot automatically learn controllers’ experience (instructions).
• Existing data-driven generation models can only be trained with differentiable loss functions, whereas flight performance constraints are often non-differentiable. Thus, the generated trajectories may not be able to stay flyable. The key to solving this problem is to build a neural network with more general loss functions. To this end, some researchers have adopted a strategy that approximates non-differentiable constraints with differentiable functions [44,45,46,47,48,49].

To overcome all the issues mentioned above, this paper develops a novel data-driven method called CTG-MMAT to generate trajectories. Firstly, a connection-based trajectory generation (CTG) framework is proposed to automatically mine helpful information (all kinds of maneuvers) from historical trajectories. Secondly, a majorization–minimization-based adversarial training (MMAT) paradigm is adopted to train the trajectory generation model with flight performance constraints. The main contributions of this paper are summarized as follows:

• The proposed CTG-MMAT is compatible with combinatorial optimization algorithms to build a data-driven trajectory optimization methodology by combining several maneuvers.
• The proposed CTG framework enables the trajectory generation model to capture more controllers’ experiences from historical data. As a result, the generated trajectories are closer to the ones observed in the operations.
• The proposed MMAT technique enables the trajectory generation model to be trained with a more general loss function with non-differentiable flight performance constraints. As a result, the generated trajectories are more likely to stay flyable.

The remainder of this paper is organized as follows: Section 2 introduces the proposed CTG framework in detail and discusses why the proposed method has the potential to be used in trajectory optimization. Section 3 presents the MMAT, which is a non-differentiable constraint approximation technique. Section 4 provides details about the experiment. The results of Section 5 are divided into three parts. Section 5.1 analyzes the flyability of the generated trajectories by comparing CTG-MMAT with existing methods. Section 5.2 highlights how CTG-MMAT incorporates operation rules while generating trajectories. Section 5.3 highlights how CTG-MMAT generates trajectories with the controllers’ experience. Section 5.4 demonstrates how the CTG framework reduces training costs while maintaining generation performance. Section 6 concludes this work and describes the direction for future research.

2. Connection-Based Trajectory Generation Framework

To fully utilize the controller’s experience while generating trajectories, this paper presents a novel trajectory generation framework. Compared with existing data-driven trajectory generation methods, the proposed framework generates a new trajectory by combining several maneuvers.

As shown in Figure 1, instead of generating a complete trajectory at a time, the proposed CTG generates a trajectory by connecting several sub-trajectories. Figure 1a shows the trajectory segmentation process: a complete trajectory is divided into backward, connecting, and forward parts. Figure 1b shows the trajectory connection process: given the forward part of trajectory $\alpha$ and the backward part of trajectory $\beta$, a connecting part is generated by a generation model to connect the backward and forward parts.

The key to achieving CTG is generating a proper connecting part for the forward and backward parts. To this end, this paper trains a connecting part generation model via supervised learning. The input data include the forward and backward parts, and the output data include the connecting part.

The training data preparation process is given in Figure 2. Firstly, the historical trajectories are resampled with equal time intervals. Each trajectory point includes latitude, longitude, and altitude information. The time interval is set to 10 s. Then, the resampled trajectories are normalized to make training more straightforward. Finally, the training data are collected through three sliding windows: backward, connecting, and forward windows. Among them, the backward and forward windows are used to obtain the features (input data) of the supervised generation model, and the connecting window is used to obtain labels (output data). According to the sensitivity analysis given in Appendix A, the length of these three windows should be set to 24, 12, and 24, respectively.

According to Figure 2, both the features and labels for training the model are matrixes composed of several trajectory points. Each row of the matrix represents a timestamp, and each column represents a dimension of a 3D trajectory point. The size of a feature is $48\times 3$, and the size of a label is $12\times 3$.

This study intentionally introduces data leakage while collecting training data. As shown in Figure 2, the connecting window partially overlaps with the others. In such a manner, the overlapping parts simultaneously exist in the features and labels, and the supervised model can easily learn to generate a connecting part to connect the backward and forward parts seamlessly.

Since the proposed CTG takes the best advantage of the controller’s experience extracted from historical data, its benefits are obvious. On the one hand, the fidelity of the generated trajectories is high since most parts of a synthetic trajectory are from real ones. On the other hand, the diversity of the generated trajectories is high since the proposed framework enables us to modularize trajectories and create new ones by connecting sub-trajectories with various flight intentions.

Although the CTG framework can generate trajectories with high diversity and fidelity, it has a critical problem that needs to be solved before being applied. The problem is finding proper forward trajectories for a given backward trajectory to ensure the generated trajectories are flyable. On the one hand, if the backward and forward parts are far from each other, it is impossible to find a proper connecting part. On the other hand, even if the backward and forward parts are close enough, the CTG cannot guarantee that the generated connecting parts meet the flight performance constraints. To solve these problems, this paper employs majorization–minimization (MM) to add the non-differentiable flight performance constraints to the supervised generation model and propose a two-round flyable trajectory filtering mechanism to eliminate unfeasible trajectories. Section 3 describes this process in detail.

Compatibility with Optimization Algorithms

This is a short sub-section showing how the CTG could be used in an optimization loop. Considering that the proposed CTG provides an efficient way to modify a trajectory, it can be plugged into an optimization algorithm to achieve trajectory optimization in the future.

As shown in Figure 3, the CTG can be applied in an optimization loop to update the trajectory. Firstly, the loss of the original trajectory can be computed to represent the goodness of the current trajectory. Then, some optimization algorithms, such as genetic algorithms and simulated annealing, can be used to compute decision variables, which are used to update the original trajectory via the proposed CTG. Finally, if the stop criteria are satisfied, the optimized trajectory is obtained; otherwise, we proceed to the next iteration.

3. Majorization–Minimization-Based Adversarial Training

Developing a trajectory generation model that creates synthetic trajectories that meet flight performance constraints is not trivial work. Conventional neural networks should be trained with differentiable loss functions, which allows the use of gradient-based optimization methods, e.g., SGD. However, the flight performance constraints in actual operation are always non-differentiable and cannot be added to the loss functions of the neural networks. As a result, many previous data-driven trajectory generation models cannot consider flight performance constraints in the training process. To tackle this challenge, this paper exploits the MM algorithm [50] to approximate non-differentiable flight performance constraints with a differentiable neural network.

As shown in Figure 4, the proposed MMAT includes two steps. In the majorization step, the MMAT works by finding a surrogate network that could approximate the non-differentiable objective function (flight performance constraints). Then, in the minimization step, the MMAT works by finding a generator that could minimize the surrogate network. MMAT can indirectly train the generator with the non-differentiable flight performance constraints by iteratively performing the majorization and minimization steps.

The convergence proof of the MMAT is presented in Appendix B. According to Appendix B, a Negative Experience Replay (NER) technique is required to ensure the convergence of the generator. Specifically, the NER technique argues that all the mistakes (connecting parts that cannot meet the flight performance constraints) made by the generator should be collected and reused while training the surrogate network. In other words, NER ensures that the generator will not make the same mistakes again.

In this paper, the surrogate network and generator are trainable neural networks, denoted as $S_{\varphi }$ and $G_{\theta }$, respectively. The subscripts $\varphi$ and $\theta$ represent the surrogate network’s and generator’s trainable parameters, respectively. Section 3.1 introduces the majorization step and the loss function of the neural network $S_{\varphi }$. Section 3.2 introduces the minimization step and the loss function of the neural network $G_{\theta }$. Section 3.3 presents the pseudo-code for iteratively training $S_{\varphi }$ and $G_{\theta }$.

3.1. Majorization Step

This paper trained a neural network $S_{\varphi }$ as a surrogate network to approximate the non-differentiable constraints. A schematic diagram of the majorization step is shown in Figure 5. The parameters of the surrogate network $S_{\varphi }$ are trainable, and the parameters of the generator $G_{\theta }$ are frozen.

Firstly, a batch of forward and backward parts are input into the generator $G_{\theta }$, and the output of $G_{\theta }$ can be regarded as the predicted connecting parts. Then, these connecting parts are labeled with 0 or 1 according to whether these connecting parts can meet the flight performance constraints. For example, if a connecting part exceeds the flight performance limits, it will be labeled with 1; otherwise, it will be labeled with 0. Thirdly, the surrogate network $S_{\varphi }$ is trained based on previously labeled data. Specifically, according to the instruction of NER, this paper uses all the labeled connecting parts in this iteration and all the connecting parts that cannot meet the constraints in previous iterations to train the surrogate network.

As shown in Equation (1), conventional cross-entropy is used as the training loss of $S_{\varphi }$ because the surrogate network $S_{\varphi }$ is a binary classification model.

$$Loss\_S=-\sum _{i=1}^B\left\{s_{i}log{s}_i^{*}+(1-s_i)log(1-s_i^{*})\right\}$$

(1)

where B is the batch size, and $s_i$ and $s_i^{*}$ are the outputs of the flight performance constraints P and surrogate network $S_{\varphi }$, respectively.

3.2. Minimization Step

To generate connecting parts that meet the flight performance constraints, this paper trained a neural network $G_{\theta }$ as a generator to minimize the outputs of the surrogate network $S_{\varphi }$. A schematic diagram of the minimization step is shown in Figure 6. The parameters of the generator $G_{\theta }$ are trainable, and the parameters of the surrogate network $S_{\varphi }$ are frozen.

Firstly, a batch of forward and backward parts are input into the generator $G_{\theta }$ to obtain the predicted connecting parts. According to Equation (2), the reconstruction error R can be calculated based on the predicted and real connecting parts. According to Equation (3), the approximated flight performance constraint value P can be calculated via the surrogate network $S_{\varphi }$. Finally, the total loss for the generator can be calculated based on Equation (4). Minimizing R encourages the predicted connecting parts to look more natural in geometry, and minimizing P encourages the predicted connecting parts to meet the flight performance constraints.

$$R={\displaystyle \frac{1}{B}}\sum _{i=1}^B\sum _{n=1}^{N}w\left(n\right)·{(t_{i,n}-t_{i,n}^{*})}^2$$

(2)

$$P={\displaystyle \frac{1}{B}}\sum _{i=1}^B{S}_{\varphi }\left(t_i^{*}\right)$$

(3)

$$Loss\_G=R+\lambda ·P$$

(4)

where B is the batch size, N is the length of a connecting part, $w\left(n\right)$ is the weight of the $n^{th}$ point of the connecting part, and $t_{i,n}$ and $t_{i,n}^{*}$ are the real and predicted connecting parts, respectively. $S_{\varphi }$ is the surrogate network whose parameters are not trainable in the minimization step, and $\lambda$ is a non-negative coefficient.

As shown in Figure 7, the weights for both sides of a connecting part are higher than the others to ensure the connecting part can smoothly connect with the backward and forward parts.

3.3. Implementation Details

In practice, to improve convergence, this paper separately pre-trains the generator $G_{\theta }$ and the surrogate network $S_{\varphi }$ until both $G_{\theta }$ and $S_{\varphi }$ are stable. Then, this paper iteratively fine-tunes $G_{\theta }$ and $S_{\varphi }$. Algorithm 1 describes the pseudo-code of the MMAT training process in more detail.

However, MMAT cannot guarantee that the generated connecting parts meet the flight performance constraints. On the one hand, the generator has no strict flight performance guarantee, but only approximately satisfies those constraints by minimizing the output of the surrogate network $S_{\varphi }$. On the other hand, not all the backward and forward parts can be connected properly. For example, if a backward part and a forward part are far from each other, it is impossible to combine those two to obtain a new trajectory.

To ensure that all the generated trajectories can meet the flight performance constraints, this paper additionally introduced two-round flyable trajectory filtering to eliminate all the trajectories that cannot meet the constraints. Algorithm 2 describes the two-round flyable trajectory filtering process in more detail. Firstly, the surrogate network $S_{\varphi }$ is exploited to quickly and roughly eliminate connecting parts that cannot be achieved. Secondly, the flight performance constraint P is used to eliminate the unflyable connecting parts.

Algorithm 1: Pseudo-code of the training process

Algorithm 2: Trajectory Generation Implementation

4. Experiment

4.1. Experimental Setup

Air traffic operation in the terminal area (TMA) is much more complex than in other airspaces. Arriving aircraft need to follow the instructions delivered by controllers instead of standard procedures to establish landing sequences and maintain safe separation in the TMA. As a result, arrival trajectories always have a higher diversity than departure and en-route trajectories. Thus, this paper uses the proposed CTG-MMAT to generate arrival trajectories to validate its efficiency. It is worth noting that the trajectory generation model proposed in this paper can also be used to generate departure or en-route trajectories.

In this paper, we collected trajectory data (from 1 September 2019 to 31 September 2019) from the TMA of Guangzhou Baiyun International Airport (ZGGG) to conduct our experiments. According to the statistics, there are 20,121 northbound arrival trajectories and 357 southbound arrival trajectories. As shown in Figure 8, arrival trajectories enter the TMA from six entry points: ATAGA, IGONO, P270, IDUMA, GYA, and LUPVU.

As shown in Figure 8, arrival trajectories adopt specific maneuvers in specific areas. For example, trajectories entering from GYA prefer to take short-cut maneuvers before joining the downwind leg of the traffic pattern, and trajectories entering from ATAGA prefer to take parallel-offset maneuvers before joining the final leg of the traffic pattern. Figure 8 shows that some operation rules are contained in the collected 20,478 arrival trajectories, and CTG-MMAT should learn these rules to be considered successful. Furthermore, CTG-MMAT should also generate trajectories that meet the flight performance constraints mentioned in Section 4.2.

4.2. Flight Performance Constraints for Arriving Aircraft

In this paper, the flight performance constraints include six indicators: speed, altitude, acceleration, rate of climb, bank angle, and turning rate. Two assumptions are made when calculating these indicators:

• Due to the lack of reliable atmospheric data, this paper assumes that the wind speed is zero. According to such an assumption, the value of ground speed equals the value of true airspeed.
• To simplify computing, this paper assumes that the magnetic deviation is zero. Therefore, the value of the aircraft’s true heading is equal to the value of the aircraft’s magnetic heading.

The details of six related indicators are summarized as follows:

• Speed refers to the aircraft’s ground speed. Acceleration refers to the difference in the aircraft’s ground speed over time.
• Altitude refers to the aircraft’s pressure altitude. The rate of climb refers to the difference in the aircraft’s pressure altitude with time.
• The bank angle refers to the amount the aircraft rolls. The turning rate refers to the difference in the aircraft’s true heading with time.
• The dependencies between indicators are considered in this paper. As shown in Figure 9, speed and altitude are coupled, and the bank angle and the turning rate are also coupled.

If a generated connecting part is always within the feasible area, the generated trajectory’s flight performance penalty is 0. Otherwise, the generated trajectory’s flight performance penalty is 1.

According to [51,52], aircraft can be divided into four weight turbulence categories: L, M, H, and J. As shown in Figure 9, an aircraft’s flight performance depends on its weight turbulence category. Due to the lack of trajectories belonging to the L-category in the collected data, only the other three categories of aircraft (M, H, and J) are considered in Figure 9.

Although trajectories of different aircraft categories have various flight performances, as shown in Figure 9, the feasible area for various aircraft categories is the same. This phenomenon is probably because, in most cases, the standard procedures and controllers’ instructions should be suitable for any aircraft and not vary from aircraft to aircraft to reduce the associated workload. According to this finding, this paper does not specify the aircraft categories used to train the model in the following experiment.

4.3. Settings of Parameters

The computer used in this experiment was an HP desktop with an Intel Core i7-7700 at a CPU frequency of 3.60 GHz. All the code was implemented in Python 3.7.

Firstly, this paper initialized the generator by pre-training the model for 600,000 epochs by minimizing the $Loss\_G$ while $\lambda =0$ using an Adam [53] optimizer with a batch size of 200,000, and a learning rate of ${10}^{-4}$. The adapted architecture of the generator is shown in

Table 1. Adapted architecture of generator.

Layer Type	Layer Parameters	Output Size
Input layer	-	$48\times 3$
Flatten layer	-	144
Dense layer	units = 100, activation = sigmoid	100
Dense layer	units = 100, activation = sigmoid	100
Dense layer	units = 36, activation = sigmoid	36
Reshape layer	target shape = (12, 3)	$12\times 3$

.

Secondly, this paper initialized the surrogate network by pre-training the model for 1,000,000 epochs by minimizing the $Loss\_S$ using an Adam optimizer with a batch size of 100,000 and a learning rate of $2\times {10}^{-4}$. The adapted architecture of the discriminator is shown in

Table 2. Adapted architecture of discriminator.

Layer Type	Layer Parameters	Output Size
Input layer	-	$12\times 3$
Feature construction layer 1	-	$(12+11+10)\times 3$
Flatten layer	-	99
Dense layer	units = 330, activation = sigmoid	330
Dense layer	units = 1, activation = sigmoid	1

¹ The feature construction layer was designed to calculate the first-order and second-order differentiation of the input data in the time dimension and stack the results behind the input data.

.

Thirdly, this paper trained the generator and surrogate network for 300,000 epochs while $\lambda =5\times {10}^{-2}$. Specifically, the generator was trained using an Adam optimizer with a batch size of 200,000 and a learning rate of ${10}^{-5}$, and the surrogate network was trained using an Adam optimizer with a batch size of 100,000 and a learning rate of $2\times {10}^{-4}$.

4.4. Benchmark Model

To further demonstrate that the trajectories generated via CTG-MMAT are more likely to stay flyable, this paper compared CTG-MMAT with the state-of-the-art trajectory generation model VampPrior TCVAE proposed by [39]. VampPrior TCVAE was chosen for two reasons. First, it outperforms other models, such as the Gaussian mixture model presented in [41]; second, it is good at generating arrival trajectories.

As shown in Figure 10, VampPrior TCVAE can be regarded as a VAE architecture embedded with some improvements. Firstly, a temporal convolutional network was used to extract spatial–temporal features from trajectories; secondly, a variational mixture of posteriors was used to replace the simplistic prior (standard Gaussian).

5. Results and Discussion

This section evaluates the proposed trajectory generation method to demonstrate that CTG-MMAT can be embedded with more domain knowledge. First, this paper evaluated whether the proposed MMAT technique could train the trajectory generator with non-differentiable flight performance constraints; second, this paper evaluated whether the proposed CTG framework could generate trajectories with higher diversity and fidelity by incorporating more operation experience and maneuvering rules.

5.1. Generating Trajectories with Flight Performance Constraints

To ensure flyability, the generated trajectories should meet the flight performance constraints. Therefore, this paper proposed an MMAT technique to train a generator with non-differentiable flight performance constraints.

A controlled experiment is presented in this sub-section to explore whether the proposed MMAT technique enables the generator to learn more about flight performance constraints. The experiment group is the generator trained with the MMAT technique, and the control group is the generator trained without the MMAT technique. Firstly, this paper pre-trained the experiment and control groups for 600,000 epochs without the MMAT technique. Then, this paper trained the experiment group for 300,000 epochs with the MMAT technique and the control group for 300,000 epochs without the MMAT technique. The generated trajectories’ flyability changes throughout the training epochs are shown in Figure 11.

Figure 11 shows that after introducing the MMAT technique, the flyable rate of the experimental group first sharply declines and then rapidly rebounds, and finally exceeds the control group. The decrease in the flyable rate is because the surrogate network was trained with biased data at the beginning of the introduction of MMAT. Fortunately, by iteratively training the generator and surrogate network, the bias of the surrogate network continues to decrease, and the MMAT technique finally shows its positive effects. The flyability changes in Figure 11 strongly suggest that the MMAT technique plays a key role in requiring the generator to consider flight performance constraints.

Once a well-trained generator and surrogate network are obtained, the feasible trajectories can be generated in three steps. For the first step, as shown in Figure 12a, a large number of synthetic trajectories are generated by combining the backward trajectory with randomly selected forward trajectories via the CTG framework. For the second step, as shown in Figure 12b, the well-trained surrogate network is used to conduct the first-round flyable trajectory filtering to quickly and roughly eliminate connecting parts that cannot be achieved. For the third step, as shown in Figure 12c, the flight performance constraints are used to conduct the second-round flyable trajectory filtering to precisely eliminate unflyable connecting parts. Furthermore, due to the fact that CTG-MMAT is trained offline and used online, as shown in Figure 12, generating candidate trajectories for a given backward trajectory takes about 0.053 + 0.064 + 0.002 = 0.119 s.

In this paper, we let CTG-MMAT and VampPrior TCVAE generate 20,000 trajectories each and calculated the flight performance parameters of those generated trajectories. It is worth noting that, according to the findings mentioned in Section 4.2, the differences in aircraft categories are ignored in this experiment. The distribution of flight performance parameters is shown in Figure 13.

Figure 13e,f shows that although VampPrior TCVAE can generate high-fidelity and -diversity trajectories, the generated trajectories are often out of the feasible area. In other words, VampPrior TCVAE cannot consider flight performance constraints while generating trajectories. For example, as shown in Figure 13e, a large portion of the generated trajectories would fly at an abnormally low speed at a high altitude or at high speed at a low altitude. Compared with real trajectories’ flight performance distribution in Figure 13a–d, the distribution of trajectories generated via VampPrior TCVAE in Figure 13e,f is almost not feasible.

Figure 13e,f shows that CTG-MMAT can consider flight performance constraints since the generated trajectories are always within the feasible area. Only a few trajectory points are out of the feasible area. There are two reasons for those outliers that are out of the feasible area. First, considering that the new trajectories are generated by connecting different trajectories, those trajectory points out of the feasible area may come from the historical trajectories instead of the connecting part generated via CTG-MMAT. Second, trajectory points out of the feasible area may happen at the joint part because of an unsmooth connection. Despite these exceptions, CTG-MMAT outperforms the state-of-the-art trajectory generation model.

To summarize, with the help of the proposed MMAT technique, the proposed trajectory generation model successfully learns non-differentiable flight performance constraints and can generate trajectories that are more likely to stay flyable.

5.2. Generating Trajectories with Operation Rules

To demonstrate that CTG-MMAT can learn more operation experience, this paper compares CTG-MMAT with a state-of-the-art trajectory generation model, VampPrior TCVAE. Figure 14 shows the real historical trajectories, those generated via VampPrior TCVAE, and those generated via CTG-MMAT.

The real historical trajectories are shown in Figure 14a. It is easy to observe that most arriving aircraft flew over the published waypoints, denoted as red points in Figure 14, and only took necessary maneuvers in some regions. For example, arriving aircraft entered the TMA from only a few entry points rather than from any point to maintain safe separation from departure aircraft. As another example, an aircraft entering the TMA from ATAGA or IGONO tends to make a dog-leg maneuver in the yellow area in Figure 14a to postpone its landing time to establish landing sequences and maintain safe separation.

The trajectories generated via VampPrior TCVAE are shown in Figure 14b. It seems that those generated trajectories that have a similar distribution to the real ones fail to utilize operation rules even though the fidelity and diversity of these trajectories are high. For example, compared with the trajectories in Figure 14a, the generated trajectories in Figure 14b deviate from the published waypoints and occupy almost all the area of the TMA, which is strictly forbidden in the conditions of arrival and departure separation. Moreover, compared with the classic flight intentions, such as short-cut, dog-leg, hold-on, etc., the flight intentions in Figure 14b are hard to implement because the heading of these trajectories changes continuously rather than changing in steps. As a result, a pilot must constantly adjust the aircraft’s attitude to fly along such a trajectory.

In contrast, CTG-MMAT can capture operation rules quite well. Figure 14c shows trajectories generated via CTG-MMAT across the published waypoints and that have classic and easy-to-implement flight intentions, such as short-cut, dog-leg, hold-on, etc. As a result, the diversity and fidelity of trajectories generated via CTG-MMAT are high enough to be mistaken for real historical trajectories.

To summarize, the proposed CTG framework helps the trajectory generation model successfully gain more operation experience.

5.3. Generating Trajectories with Controllers’ Experience

Instead of mimicking the distribution of actual trajectories, CTG generates trajectories by connecting different trajectories. As shown in Figure 15, by connecting trajectory $\alpha$ and trajectory $\beta$, the CTG framework can create trajectories with common flight intentions such as parallel-offset, hold-on, short-cut, and dog-leg.

In addition, Figure 15 shows that the connecting parts partially overlap with the backward and forward parts. These overlaps are intentionally designed, as mentioned in Section 2. They can introduce data leakage while training the generator to ensure that both ends of a connecting part can seamlessly connect with the backward and forward parts. When combining these three parts to obtain a new trajectory, the overlaps should be eliminated from the connecting part.

Figure 16 dynamically demonstrates the flight intention-based trajectory generation process from the perspective of pilots and controllers.

As shown in Figure 16, the CTG can provide alternative arrival routes for an arriving aircraft by connecting the aircraft’s already flown trajectory with candidate historical trajectories. This connection is achieved through flight intentions such as dog-leg and short-cut. This process is close to the actual operation where a controller plans a rerouting path for an aircraft. In such cases, the CTG framework maximizes the actual operation experience while generating trajectories, since most of the content comes from historical data. Please refer to Appendix C for more examples similar to Figure 16 to show how the CTG framework recommends alternative trajectories for aircraft from different entry points.

To summarize, with the help of the proposed CTG framework, the proposed trajectory generation model can generate trajectories similar to the ones used in operations.

5.4. Lower Training Costs and Better Outcomes

The adapted architecture of the proposed generator—including only three dense layers—is quite simple and may raise concerns about their capability. Hence, more complicated architectures are considered in this subsection to analyze the effects of different architectures.

Recurrent neural networks, such as LSTM, Gated Recurrent Unit (GRU), and Bi-directional LSTM (BiLSTM), were developed to replace dense layers because these layers can effectively leverage the temporal relation information in the trajectory. In addition, the temporal convolutional network (TCN) used in [39] was also considered since it exhibits longer memory than recurrent architectures.

For a fair comparison, as shown in Figure 17, these models have a close number of trainable variables. However, their floating-point operations (FLOPs) differ. Specifically, the TCN’s FLOPs are about 38 times those of the other models. The higher the FLOPs, the more complex the model. In addition, as shown in Figure 18, the training time of CTG-MMAT is significantly lower than that of the other models. This is because the computations in dense layers can happen in parallel, while in the other models, they need to be processed sequentially. The simplicity of the generator matters. This explains why CTG-MMAT can be trained nearly a million times in a short time with a regular office computer. This simple neural network structure avoids dependency on high-performance deep learning workstations during the training process and lowers the bar when applying a trajectory generation model.

Simplifying the generator’s architecture does not always degrade its generation performance. As shown in Figure 19, the reconstruction error of CTG-MMAT is close to the best one (TCN) and outperforms the others (GRU, LSTM, BiLSTM). Although the TCN has the lowest reconstruction error, its robustness is worse than that of the other models. As shown in Figure 19, when the TCN is trained for more than 300 epochs, its reconstruction error fluctuates strongly.

Figure 20 shows the increase in the number of flyable trajectories during training. Compared with the results given in Figure 19, there is no clear relationship between the models’ reconstruction error and their ability to generate flyable trajectories: the performance of the TCN and BiLSTM differs significantly in reducing the reconstruction error, but they are similar in generating flyable trajectories. In other words, minimizing a trajectory generation model’s reconstruction error cannot guarantee that the generated trajectories are flyable, and the flight performance constraints need to be additionally considered. CTG-MMAT, which was trained to minimize both reconstruction error and flight performance penalty, is the best example. As shown in Figure 20, CTG-MMAT outperforms the other models in generating flyable trajectories.

6. Conclusions

This paper proposed a novel trajectory generation method to incorporate more domain knowledge while generating trajectories, such as operation rules, controllers’ experience, and flight performance constraints. Such a generative model is of great interest in traffic-related mining and simulation since it can provide large-scale trajectory data when only a limited amount of observation is available. By incorporating the proposed CTG framework and MMAT technique, the model fulfills all expectations mentioned in the introduction:

• This paper adds non-differentiable flight performance constraints into the trajectory generation model. Considering that the non-differentiable flight performance constraints cannot be added to the trajectory model directly, the proposed model incorporates the MMAT technique to approximate non-differentiable constraints with differentiable functions to train CTG-MMAT with flight performance constraints. As a result, the trajectories generated via CTG-MMAT are more likely to be flyable than those generated using previous models.
• Unlike previous trajectory generation methods focusing on generating a whole trajectory, the proposed CTG-MMAT generates trajectories by connecting several sub-trajectories. This way, CTG-MMAT can not only fully utilize the operation rules and controllers’ experience, but can also be plugged into a combinatorial optimization algorithm to achieve trajectory optimization in the future.
• With the help of the CTG framework, the synthetic trajectory generation task becomes far easier. Accordingly, the generator composed of several dense layers is able to generate high-diversity and -fidelity trajectories. The training costs can be reduced significantly, reducing the threshold of development and application for future trajectory generation-related tasks.

Although the proposed trajectory generation method was validated by generating arrival trajectories for the ZGGG airport, it can also be directly applied to other types of trajectories. In the future, we will analyze the trajectory generation model’s robustness and integrate the proposed model into traffic-related mining and simulation tasks to improve the performance of these related algorithms.