An Unsupervised Learning Approach for Analyzing Unsafe Pilot Operations Based on Flight Data

Abstract

Flight safety is a hot topic in the aviation industry. Statistics show that safety incidents during landing are closely related to the flare phase because this critical period requires extensive pilot operations. Many airlines require that pilots should avoid performing any forward stick inputs during the flare. However, our statistical results from about 86,504 flights show that this unsafe pilot operation occasionally happens. Although several case studies were conducted previously, systematic research, especially based on a large volume of flight data, is still missing. This paper aims to fill this gap and provide more insights into the issue of pilots’ unsafe stick operations during the flare phase. Specifically, our work is based on the Quick Access Recorder (QAR) data, which consist of multivariate time-series data from various flight parameters. The raw data were carefully preprocessed, then key features were extracted based on flight expert experience, and a K-means clustering algorithm was utilized to divide the unsafe pilot operations into four categories. Based on the clustering results, we conducted an in-depth analysis to uncover the reasons for different types of unsafe pilot stick operations. In addition, extensive experiments were conducted to further investigate how these unsafe operations are correlated with different factors, including airlines, airports, and pilots. To the best of our knowledge, this is the first systematic study analyzing pilots’ unsafe forward stick operations based on a large volume of flight data. The findings can be used by airlines to design more targeted pilot training programs in the future.

1. Introduction

Flight safety is one of the most important topics in the aviation industry [1]. According to the 2020 EASA (European Union Aviation Safety Agency) annual safety review (EASA Annual Safety Review: 2020, https://www.easa.europa.eu/document-library/general-publications/annual-safety-review-2020, accessed on 22 November 2022), the final approach and landing phases are the most prone to flight safety accidents. As shown in Figure 1, during 2009–2018 and 2019, the total accidents and serious incidents rate accounted for 68 and 74%, respectively, in the approach and landing phases, even though these two phases only occupy 4% of the entire flight time. During these two phases, extensive pilot operations are required to make sure the aircraft is landing steadily, and any pilot misjudgment or inappropriate operation may lead to adverse consequences, such as flight safety incidents or accidents, especially when the weather condition is not ideal [2], or the aircraft is landing at high altitude airports [3].

In general, although the overall occurrence of serious flight safety accidents in the aviation industry is very rare, the possibility of adverse events (e.g., flight exceedances) that may affect flight safety and further lead to severe accidents cannot be ignored. For example, it is not uncommon for the vertical acceleration to be too large at the touchdown moment, which is called the hard landing incident [4,5], and this incident may cause severe damage to the landing gears. Excessive vertical overload not only gives passengers a bad flight experience but also largely increases the airlines’ maintenance costs. Severe incidents may even threaten the lives of passengers. According to [6], the most important reason for the increase in the vertical overload is due to the pilot’s inappropriate applying of forward stick inputs which makes the aircraft nose down before it touches the ground. This operation usually reduces the pitch angle of the aircraft, which will further reduce the lift force the aircraft can gain. If this happens in the flare phase [7], i.e., the few seconds before touchdown, then with a high probability the aircraft will touch ground with an excessive vertical overload, which causes the hard landing safety incident [8,9]. Therefore, we define the event in which a pilot performs this forward stick operation as an adverse event. Actually, from the Airbus A320 Flight Crew Techniques Manual (FCTM), a pilot should avoid applying any nose down inputs during the flare phase to avoid the hard landing or bounced landing risk. However, our statistical results from about 86,504 flights show that this unsafe pilot operation occasionally happens. In view of the above phenomenon and given that all the pilots have undergone professional training, why would they perform such unsafe stick operations in the flare phase? Although the Airbus company conducted two case studies on this topic (A Focus on the Landing Flare, https://safetyfirst.airbus.com/a-focus-on-the-landing-flare/, accessed on 22 November 2022), systematic research, especially based on a large volume of flight data, so as to give more insights and a comprehensive overview of this phenomenon, is still missing.

The Quick Access Recorder (QAR) [10,11] is an airborne flight recorder designed to provide quick and easy access to raw flight data, which can record multivariate time-series flight parameters during the entire flight. Compared to other data recording systems like the DFDR (Digital Flight Data Recorder), black box, etc., the QAR has the advantages of a higher sampling rate, more parameters, and faster data transmission. Recently, the QAR devices have been widely adopted by airlines to improve flight safety [4,12,13,14]. The flight parameters collected by the QAR devices include the state parameters of the aircraft, such as the speed, pitch angle, radio altitude, acceleration, etc. They also contain the parameters of the pilot operations, such as the pitch control, roll control, throttle lever position, etc. Some external environmental parameters, including the wind speed, wind direction, and temperature, are also collected by the QAR device.

Our work is based on a QAR dataset with 86,504 flights from the domestic airlines in China. We mainly aim to fill the aforementioned research gap, give more insights, and establish a comprehensive overview about the unsafe pilot stick operations during the flare phase. Specifically, our work will answer the following key questions: (1) Are there any typical reasons among the pilots who applied forward stick inputs during the flare? (2) What are the key contributing factors related to this unsafe stick operation? and (3) How is this phenomenon related to different airlines, airports, and pilots? To this end, we first carefully preprocessed the raw flight data through data cleaning, parameter transformation, etc. After that, we selected key features that are helpful for explaining this operation based on the experience of flight experts, and then clustered these features through the K-means algorithm [15,16]. From the clustering results, four main categories are summarized, corresponding to four influencing factors, i.e., the headwind influence, high pitch influence, long flare influence, and pilot personal influence. In addition, we also investigate how the above four classes correlate with different airlines, airports, and pilots. From the experimental results, we find that for different airports and pilots, there are significant differences in the occurrence probabilities of different classes. Specifically, for flights related to the headwind influence, coastal airports and inland airports show significant differences. Different pilots show a significant difference in the long flare influence and personal influence, which is consistent with our analysis.

The main contributions of our study are summarized as follows:

• To the best of our knowledge, this work is the first systematic study analyzing pilots’ unsafe forward stick operations based on a large volume of flight data. The findings from this work can be used by airlines to design more targeted pilot training programs in the future, which has great practical importance for aviation safety.
• The key features are extracted based on the experience of flight experts, and then the K-means clustering method is used to uncover the reasons for unsafe pilot stick operations. A benefit to the flight expert experience, the obtained results show good explainability.
• Extensive experiments are conducted to investigate how different classes of the adverse event are correlated with different airlines, airports, and pilots. The results provide new insights into the understanding of unsafe pilot operations during landing.

The rest of this paper is as follows: We review the related works about aviation safety and K-means clustering in Section 2, followed by the methodology illustration in Section 3. Then, Section 4 will show the experimental results with a detailed discussion about the different categories. Finally, this paper is concluded in Section 5.

2. Related Work

Recently, many scholars have utilized QAR data to study aviation safety incidents. The main studies can be divided into two groups: safety incident prediction and flight safety analysis.

2.1. Safety Incident Prediction

The research of flight safety incident prediction mainly aims to establish forecasting models which can be utilized as a warning before a safety incident occurs. For a hard landing, Cao et al. [17] and Hu et al. [8] took advantage of a BP neural network and an SVM to predict the incident, respectively. Because both of them are relatively early works, their prediction accuracy is unsatisfactory. Then, Qiao et al. [18] tried to use the RBF neural network and K-means clustering algorithm to predict a hard landing. With the rise of deep learning and in order to capture time-series features, Tong et al. [9] proposed a model based on the Long Short-Term Memory (LSTM) network to predict a hard landing. The same model was also used to address the landing speed prediction problem [19] and the tail strike risk prediction problem [20]. Kang et al. [21] further proposed a deep sequence-to-sequence model based on LSTM and an attention mechanism to improve the landing speed prediction accuracy. These deep learning-based methods not only take advantage of the information at the feature level but also capture the temporal information from these time-series flight parameters. Hence, the LSTM-based methods have achieved a good prediction performance. Similarly, for the long landing incident, Wang et al. [22] investigated the correlation between different QAR parameters and a long landing through the analysis of variance method and utilized the logical regression and linear regression models for the long landing risk prediction. Recently, Kang et al. [23] utilized a deep sequence-to-sequence model for long landing prediction, which further improved the prediction accuracy by incorporating an attention mechanism. Predicting aviation safety incidents can enable proactive warnings before safety incidents occur, but these methods cannot help uncover the reasons for safety incidents.

2.2. Flight Safety Analysis

Recently, many scholars have tried to explore the reasons for or risks of safety incidents so that the findings can be used to provide more targeted pilot training and improve the safety level of airlines. Specifically, Wang et al. [10] used QAR data to divide the safety incident risk space through the golden section method to find the high-risk subspace where safety incidents occur. Subsequently, they proposed a new algorithm based on the rough set theory and a particle swarm multi-objective optimization algorithm [24] to analyze the flight safety risk. Some scholars divided the safety incident subspace according to the value of each parameter in the QAR and then constructed the state transition function of risk based on the Markov model [25]. Liu et al. [26] studied the risk assessment model of a safety incident and defined the risk as the probability of occurrence and the severity of the incident. Then, they developed a pilot operation quality assessment system based on this model. Moreover, for a hard landing, Li et al. [27] recognized hard landing patterns based on a curve clustering method. In their following work [4], they further validated the proposed method on a larger QAR dataset and gave a corresponding risk analysis model.

To analyze and explain safety incidents, Wang et al. [28] studied the relationship between risk perception and safety incidents. In [29], Lv et al. divided the aircraft landing phase into four sub-phases and extracted the mean, variance, and maximum values of the QAR parameters as featured in each phase, based on which the connection between the overrun risk and these features was investigated. Janakiraman [30] proposed an algorithm named DT-MIL, which combines multiple-instance learning and a recurrent neural network with a GRU (Gated Recurrent Unit) to find abnormal time points leading to safety incidents. In [31], the authors took advantage of reinforcement learning to construct a Markov decision process, with which adverse state transition points were identified as precursors to a safety incident. In addition, Ayra et al. [32] analyzed the contributing factors of runway overrun accidents and made operational recommendations.

2.3. Application of K-Means Clustering

K-means clustering is a classical unsupervised learning method, and it has been widely used in a variety of applications. Recently, Ran et al. [33] investigated the urban road planning problem and proposed a K-means clustering algorithm based on a noise algorithm to capture urban hotspots. Gu et al. [34] investigated the mixed-layer depth (MLD) estimation problem which is important in the area of ocean dynamics and global climate change. They proposed a hybrid approach by combining the K-means clustering algorithm and an artificial neural network (ANN) model and evaluated their approach through a case study of the Indian Ocean data. Abernathy et al. [35] investigated the color quantization problem commonly used in image processing and proposed a partitional color quantization algorithm based on a binary splitting formulation of MacQueen’s online K-means algorithm. Richardo et al. [36] proposed a neutrosophic K-means algorithm by combining a classic K-means method with neutrosophy to analyze the earthquake data in Ecuador. Hutagalung et al. [37] used the K-means clustering algorithm to analyze COVID-19 cases and deaths in Southeast Asia. Ikotun et al. [38] considered automatic K-means clustering where the specification of a cluster number is not required and comprehensively reviewed recently proposed studies related to the improvements in the K-means clustering algorithm with nature-inspired optimization techniques.

2.4. Summary of Existing Studies

For the existing studies related to flight safety incident prediction and risk analysis, most of them only provide a parameter-level explanation, such as locating the high-risk moments of abnormal parameter values. However, they can hardly explain the safety incidents from the physical or operational aspect. As a result, the traditional methods lack practical significance for airlines and pilots. Therefore, this paper aims to fill this gap by conducting comprehensive research on a large volume of flight data and providing explainable results to uncover the reasons for unsafe pilot stick operations during the flare.

3. Methodology

3.1. Problem Statement

There are three key phases during landing of a flight, i.e., final approach phase, flare phase, and landing phase. In detail, when the aircraft descends to an altitude of about 50 feet above the ground, the pilot will apply back stick inputs to increase the aircraft pitch angle and reduce the vertical speed, to ensure a steady touchdown. After this back stick operation is completed, the aircraft enters the flare phase. Usually, at the beginning of the flare, the aircraft speed is still very high, and it does not touch ground immediately. Therefore, the pilot has to maintain the pitch attitude to make the speed continue to decrease in this phase, so that the aircraft will touch the ground at a relatively small vertical speed, as shown in Figure 2. The phase when the aircraft touches the ground and the subsequent movement on the runway until the aircraft ground speed reduces to specific level is called the landing phase.

Adverse event: In this paper, the adverse event is specifically referred to as the events in which the pilot applies forward stick inputs during the flare phase. In this phase, both airspeed and vertical speed of the aircraft gradually decrease, and the lift continues to decrease. In order to make the lift of the aircraft approximately equal to gravity and allow the aircraft to slowly descend close to the ground, the pilot should maintain the pitch attitude of aircraft, help increase the lift to reduce the aircraft’s vertical speed, and finally touch the ground smoothly [39]. It is generally required by airlines that pilots should not apply any forward stick inputs during the flare. However, our statistical results from a large number of flights show that this unsafe pilot stick operation occasionally happens. Specifically, from the QAR data of 86,504 flights covering A320 and A321 models at domestic airports in China from May 2017 to December 2018, we observe a total number of 11,385 flight samples with the adverse event. The above unsafe pilot operation may eventually result in an excessive vertical overload at touchdown and cause serious damage to the aircraft’s landing gears.

In order to uncover the typical reasons for the adverse event, an innovative approach based on unsupervised K-means clustering algorithm is established in this paper. Our approach is divided into three parts. Specifically, we first preprocess the flight data and select key features based on expert experience. Then, the processed data are input into K-means algorithm to automatically classify them into different clusters. Finally, we investigate the classification results to reveal different reasons for the adverse events. We also analyze the performance of different types of adverse events from different airlines, airports, and pilots.

3.2. Data Preprocessing and Feature Selection

In the QAR dataset, the sampling frequency varies from 1 to 8 Hz with respect to different parameters.

Table 1. Description of commonly used QAR parameters.

No.	Parameter	Description	Frequency (Hz)
1	ALT_QNH	Altitude	1
2	ALT_STD	Standard altitude corrected	1
3	RADIO_LH	Left radio height	4
4	RADIO_RH	Right radio height	4
5	LDGL	Left landing gear state	4
6	LDGR	Right landing gear state	4
7	LDGNOS	Nose landing gear state	4
8	IAS	Indicated airspeed	1
9	VAPP	Landing reference speed	1
10	GS	Ground speed	1
11	VRTG	Vertical acceleration	8
12	IVV	Vertical speed	1
13	PITCH	Pitch angle	4
14	PITCH_CPT	Captain pitch control	8
15	PITCH_FO	Deputy captain pitch control	8
16	GW	Aircraft gross weight	1
17	ROLL	Roll angel	2
18	ROLL_CPT	Captain roll control	8
19	ROLL_FO	Deputy captain roll control	8
20	HEAD_MAG	Magnetic heading direction	1
21	WIN_DIR	Wind direction	1
22	WIN_SPD	Wind speed	1
23	RUDD	Rudder position	2
24	N11	Engine 1 speed ratio	1
25	N12	Engine 2 speed ratio	1
26	TLA1	Throttle lever 1 position	1
27	TLA2	Throttle lever 2 position	1
28	FLAP_PL	Left flap actual angle	1
29	FLAP_PR	Right flap actual angle	1
30	DME1	DME 1 distance	1
31	DME2	DME 2 distance	1

shows 31 commonly used QAR parameters and their sampling frequencies. For airlines, they can use QAR data to find abnormalities in pilot operations, engine conditions, and aircraft performance in time. For researchers, QAR data can be used to investigate safety events to ensure aviation safety. In this paper, the QAR dataset is processed in the following way.

First, all original data need to be uniformly scaled, i.e., features with small values are scaled up while those with large values are scaled down, so that the contributions of different features to the final results are roughly equal. Next, by consulting with flight experts, including experienced pilots, QAR data decoding experts, and airline managers, we choose the following parameters as the basic features: wind speed (WIN_SPD parameter), wind direction (WIN_DIR), height (RADIO_LH) (RADIO_LH and RADIO_RH show very similar values, so we only use one of them), the magnetic heading (HEAD_MAG) of aircraft, and pitch angle (PITCH). Then, these basic features are further processed based on prior knowledge and expert experience.

From flight expert experience, we know that the pitch angle of an aircraft is largely impacted by the headwind it encounters. However, because the headwind is not included in the original flight parameters, we should combine the aircraft’s magnetic heading (i.e., the flying direction of the aircraft relative to the north), wind speed, and wind direction to calculate the effect of headwind on the aircraft, as shown in Figure 3. We mainly use the longitudinal component of the wind as our feature, and the calculation formulas are as follows:

$$\theta =\alpha -\beta$$

(1)

$$WI{N}_{alg}=WI{N}_{spd}\times cos\theta$$

(2)

where $\alpha$ represents the wind direction, and $beta$ represents the magnetic heading of aircraft, so $\theta =\alpha -\beta$ represents the wind direction relative to the aircraft. $WI{N}_{spd}$ denotes the absolute wind speed, so the longitudinal component of the wind speed with respect to the aircraft is represented by $WI{N}_{alg}$.

On the other hand, the aircraft pitch control is also closely related to how long the pitch angle keeps changing in the same direction (increasing or decreasing) and how far it has gone. Therefore, we extract cumulative change values of pitch angle and the maximum pitch angle over a period of time, which are calculated as:

$$\Delta Pitc{h}_t=Pitc{h}_{t+1}-Pitc{h}_t$$

(3)

$$Pitc{h}_{TotalChange}=\sum _{t=0}^{n-1}|\Delta Pitc{h}_t|$$

(4)

Finally, we select the radio height values from one second around the moment when the pilot applied forward stick inputs during the flare as features to see whether the aircraft has kept flaring (flying at a relatively fixed and low height) for a while before touchdown. Because the radio height parameter is sampled with 4Hz frequency, four consecutive height values (H1, H2, H3, and H4) will be obtained.

3.3. K-Means Clustering

Clustering is a traditional unsupervised learning method, which divides data samples into subgroups, thus enabling us to uncover and explain the reasons for different types of adverse events. Among the various clustering methods, K-means clustering is most widely used due to its simplicity and high-efficiency characteristics, though it highly relies on the parameter k and cannot handle data with varying densities. According to the characteristics of the QAR data and following our previous work [4], we use the Euclidean distance metric-based K-means clustering algorithm. Its basic idea is to divide the sample set into k clusters according to the distance among samples, so that points within the same cluster are closely connected while the distances between different clusters are as far as possible. Currently, K-means clustering algorithm has been widely used in a variety of applications [34,38], such as urban hotspots detection [33], earthquake analysis [36], COVID-19 disease analysis [37], etc.

Let the QAR data be represented by X with dimensions $(N,d)$, where N and d denote the number of flight records and features, respectively. Let record i be expressed as $x_i\in {\mathbb{R}}^d,i=1,2,\cdots ,N$. Firstly, the algorithm randomly generates k cluster centers, and the j-th center is represented as $u_j(0\le j\le k)$, which represents one category. For each $x_i$, we calculate the class that $x_i$ should belong to based on the Euclidean distance between $x_i$ and the cluster centers. Then, $x_i$ is assigned to the closest cluster $c_i$, i.e.,

$$c_i={arg}_{j}min\left|\right|x_i-u_j{\left|\right|}^2$$

(5)

Then, for each class j, the cluster center is updated in the following way:

$$u_j=\frac{{\sum }_{i=1}^{N}1\{c_i=j\}{x}_i}{{\sum }_{i=1}^{N}1\{c_i=j\}},j=1,2,\dots ,k$$

(6)

Repeat the above process until the following objective reaches the minimum.

$$J=\sum _{j=1}^k\sum _{i=1}^N\left|\right|x_i-u_j{\left|\right|}^2$$

(7)

where J represents sum of squared distances of the clusters.

Moreover, we use the silhouette coefficient [40] to choose the most suitable value of k and evaluate the algorithm. The silhouette coefficient combines the cohesion and separation characteristics of the clusters. Let $d_i$ denote the average distance from record i to other records in the same cluster, $d_{ij}$ be the average distance from record i to all records of other cluster j, then the silhouette coefficient $s_i$ of record i is defined as:

$$s_i=\frac{d_{ij}-d_i}{max(d_i,d_{ij})},$$

(8)

Finally, the silhouette coefficients of all records are averaged to obtain the total silhouette coefficient of the clustering result, which is represented as $S=\frac{{\sum }_{i=1}^N{s}_i}{N}$, and it satisfies $-1\le S\le 1$. Higher silhouette coefficient means better performance.

4. Experiment

In this section, we conduct the experiments based on the QAR dataset to classify the adverse events and analyze their reasons.

4.1. Dataset

The dataset used contains QAR data of 86,504 flights covering A320 and A321 models at domestic airports in China from May 2017 to December 2018. The QAR data include 42 aircraft parameters during entire flight process, covering flights of 73 airports, 2 airlines, 2 aircraft models, and 837 pilots in total (due to privacy concern, the specific airline and pilot information is not released). From the dataset, we find a total number of 11,385 flight samples with the adverse event.

In our experiment, we use the methods as described in Section 3.2 to extract features from the period when the aircraft’s radio altitude is between 10 feet and touchdown, and this phase is the last stage of flare phase with important research significance. The finally obtained features are listed in

Table 2. Features extracted from the QAR dataset for K-means clustering.

Feature	Description
PITCH_C	The cumulative change in pitch angle
PITCH_M	The maximum pitch angle in the flare phase
WIND	The average headwind encountered by the aircraft
H1, H2, H3, H4	Four consecutive height values in one second

.

4.2. Model Hyperparameters

For K-means clustering model, the total number of iterations is 300, the termination parameter is $ϵ=1\times {10}^{-4}$, i.e., when the error is less than $ϵ$, the algorithm will terminate. Finally, the number of clusters is set to $k=4$, because it yields the maximum silhouette coefficient, as shown in Figure 4.

4.3. Experiment Results and Interpretation

Through the K-means algorithm, the adverse events caused by pilots’ forward stick inputs are divided into four types, and the results are shown in Figure 5, where the x-axis indicates the feature index while the y-axis represents the corresponding feature values. From the clustering results, the reasons for different types of adverse events can be interpreted as follows:

• The first type (headwind influence): This type is represented by yellow lines. The characteristic of this type is that its headwind parameter values are significantly higher than other types. When the aircraft encounters heavy headwinds during the flare, the wind will have an effect of increasing the aircraft pitch angle in a short period of time. If the pilot is not prepared for this situation, then he may apply the forward stick operation to counteract the wind effect. For this type, the pilot should pay significant attention to the wind conditions during the landing stage.
• The second type (high pitch influence): This type is represented by red lines. The characteristic of this type is that the cumulative value of the pitch angle change (PITCH_C) and maximum pitch angle of the aircraft (PITCH_M) is very large before the pilot applies the forward stick inputs. Meanwhile, the height is relatively high, and the influence from the wind is insignificant. So, it can be concluded that the aircraft has endured a continuous increasing in the pitch angle, and its pitch angle finally becomes too large. This is usually caused by the pilot’s unawareness of the aircraft status and attitude, i.e., keeping a stable pitch angle with a relatively small vertical speed. As a result, the pilot applies the stick forward operation to directly reduce the pitch angle of the aircraft in order to avoid the tail strike risk. For this type, the pilot should be aware of the pitch attitude of the aircraft, especially when it is close to the ground. If the pitch angle becomes too high and it was not likely to stabilize in time, then the pilot should initiate a go-around.
• The third type (long flare influence): This type is represented by blue lines, from which we observe that the height parameter almost keeps unchanged at a very low attitude during the one second. In this type, although there are some tail winds (WIND < 0), its impact is insignificant. The result indicates that the aircraft keeps flaring at a relatively low height. Most frequently, this is caused by the pilots excessive applying of back stick inputs, which will quickly reduce the aircraft vertical speed. When the aircraft keeps flaring at a relatively low height due to the pilot’s excessive reduction in the vertical speed, then they may further perform the forward stick operation to make the aircraft touch the ground as soon as possible so as to avoid the runway overrun [21,32] risk. For this type, the pilot should be aware of the vertical speed of the aircraft and avoid reducing the vertical speed too much before entering the flare.
• The fourth type: This type is represented by green lines. This type of flight does not show a significant low height or large pitch angle, which means the aircraft was not enduring an abnormal situation. For this type, the pilot should get more flight training to improve their landing skills.

4.4. Further Analysis on Impacting Factors

In general, the overall proportions of these four types are shown in Figure 6. It can be seen that among all the types, the third type of adverse event has the largest proportion, and the lowest proportion is the first type. In addition, we also investigate how the different types are correlated with different airlines, airports, and pilots.

Airline impact:Figure 7 shows the distribution of the adverse events with respect to different airlines, from which we see that airline A accounts for about 60% of the overall occurrences. Given that airline A has a total number of 23,445 flights, while airline B has 63,059 flights in total, it is interesting to observe that the occurrence probability of airline A (6807/23,445 ≈ 29%) is about four times that of airline B (4578/63,059 $\approx 7.3\%$). This result is mainly due to the different pilot training programs of the two airlines. From the results, it can be seen that airline B has more rigorous restrictions on the pilots’ forward stick operations during the flare. We also analyzed the distribution of the four types of adverse events in each of the two airlines. As we can see, airline A has a higher proportion of the third type of events, while the proportion of the first type is relatively lower. According to these results, airlines can train pilots to avoid these adverse events in a more targeted manner.

Airport impact: We investigate the adverse event distribution with respect to different airports, and the results indicate a significant difference between the coastal and inland airports, as shown in Figure 8a,b. The left one is an airport from Guangzhou (CAN), a coastal city of China, while the right one is an airport from Changchun (CGQ), a northeastern inland city of China (the difference between coastal and inland airports is widely observed; we only show two representative airports due to the space limit). It can be clearly seen that the inland airport has a significantly higher proportion of the first type of adverse events (caused by the wind influence) than the coastal airport. We think the result is mainly due to the different wind conditions of these two airports. To validate our assumption, we further investigated the monthly wind statistics data of the two airports, which are available from https://www.windfinder.com/windstatistics/, accessed on 22 November 2022. From the statistics, we found that these two airports exhibit significantly different characteristics. Specifically, for the Guangzhou airport, the proportion of east winds is significantly higher than the west winds, while for the Changchun airport, the southwest winds dominate the wind directions. Moreover, the average wind speed of the Changchun airport is stronger than the Guangzhou airport.

Pilot impact: Finally, we analyze the results from different pilots. Because there are hundreds of pilots in our dataset and it is impractical to show all of them, we only chose two representatives, as shown in Figure 9a,b. The most significant difference between the two pilots is observed on the third and fourth adverse event types. The percentage of the fourth type of adverse event for pilot 1 is as high as 51.2%, while that value for pilot 2 is only 5%. The results indicate that for pilot 1, many of the adverse events were due to personal factors, e.g., operational habits. Based on the results, the airlines can train their pilots in a more personalized way.

5. Conclusions

Flight safety plays a vital role in the aviation industry. In this paper, we addressed this issue by investigating pilots’ inappropriate stick operation during the flare phase. Specifically, we extracted key features from flight parameters with expert knowledge and took advantage of the K-means clustering algorithm to uncover the reasons for this adverse event. Based on the clustering results, we summarized the reasons into four types, including the headwind influence, high pitch influence, height influence, and pilot personal influence. In addition, we further analyzed the characteristics of the four types of reasons from different airlines, airports, and pilots. The results in this paper can provide researchers with new insights into this problem.

This study is not immune from its limitations. Firstly, our datasets were only for two different airlines. In the future, we will try to incorporate more airlines in our research. Secondly, the methodology in this paper is based on K-means clustering, which has its own limitations. In the future, we will investigate new clustering algorithms to improve the method performance. Lastly, in this paper, we only conducted a data-level analysis with limited domain knowledge. In the future, we will conduct more in-depth research and incorporate more expert knowledge to increase the generality of our results.

Another important direction worth investigating in the future is applying advanced optimization algorithms to address the flight safety problem, such as the online-learning-based evolutionary many-objective algorithm [41], polyploid memetic algorithm [42], island-based metaheuristic algorithm [43], etc. These algorithms, which are mainly based on heuristics or metaheuristics, have been widely used for challenging decision problems in a variety of domains, such as vehicle routing [44], berth scheduling [43,45], ambulance routing in a disaster response [46], etc.