A Runway Overrun Risk Assessment Model for Civil Aircraft Based on Quick Access Recorder Data
Abstract
:1. Introduction
2. Related Work
2.1. Runway Overrun
2.2. Other QAR-Based Flight Safety Studies
3. Landing Trajectory Data Correction
3.1. Data Preprocessing
3.2. Landing Interval Extraction
3.3. Landing Trajectory Correction
3.3.1. Trajectory Correction Based on Projection
3.3.2. Trajectory Correction Based on Projection and Ground Speed
4. Dynamic Runway Overrun Risk Model
4.1. Basic Idea
4.2. Automatic Extraction of the Deceleration Rate
4.3. Establishment of the Risk Assessment Model
5. Classification and Regression of the Runway Overrun Risk
5.1. Regression Analysis of the Runway Overrun Risk
5.1.1. Feature Extraction
5.1.2. Regression Model and Evaluation Metrics
5.1.3. Regression Results
5.2. The Classification of Flights with Runway Overrun Risk
5.2.1. Classification Model and Evaluation Metrics
5.2.2. Classification Results
5.3. Further Discussion of the Results
6. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
Abbreviations
MDPI | Multidisciplinary Digital Publishing Institute |
QAR | Quick Access Recorder |
LSTM | Long Short-Term Memory |
IATA | International Air Transport Association |
LR | Linear Regression |
NN | Neural Network |
DT | Decision Tree |
RF | Random Forest |
SVR | Support Vector Regression |
SVM | Support Vector Machine |
MAE | Mean Absolute Error |
RMSE | Root Mean Square Error |
MAPE | Mean Absolute Percentage Error |
References
- Lu, X.; Chen, D.; Zhang, F. Grey correlation analysis of hard landing based on QAR data. In Proceedings of the 2019 IEEE 1st International Conference on Civil Aviation Safety and Information Technology (ICCASIT), Kunming, China, 17–19 October 2019; IEEE: Piscataway, NJ, USA, 2019; pp. 79–83. [Google Scholar]
- Xie, J.; Sun, H.; Jiao, Y.; Lu, B. Association rules mining with QAR data: An analysis on unstable approaches. In Proceedings of the 2019 IEEE 1st International Conference on Civil Aviation Safety and Information Technology (ICCASIT), Kunming, China, 17–19 October 2019; IEEE: Piscataway, NJ, USA, 2019; pp. 603–606. [Google Scholar]
- Boeing, C.A. Statistical Summary of Commercial Jet Airplane Accidents-Worldwide Operations 1959–2018. 2019. Available online: http://www.boeing.com/news/techissues (accessed on 1 October 2021).
- IATA. IATA Runway Safety Accident Analysis Report 2010–2014; IATA: Montreal, QC, Canada, 2015. [Google Scholar]
- Memarzadeh, M.; Matthews, B.; Avrekh, I. Unsupervised anomaly detection in flight data using convolutional variational auto-encoder. Aerospace 2020, 7, 115. [Google Scholar] [CrossRef]
- Oehling, J.; Barry, D.J. Using machine learning methods in airline flight data monitoring to generate new operational safety knowledge from existing data. Saf. Sci. 2019, 114, 89–104. [Google Scholar] [CrossRef]
- Memarzadeh, M.; Matthews, B.; Templin, T. Multiclass Anomaly Detection in Flight Data Using Semi-Supervised Explainable Deep Learning Model. J. Aerosp. Inf. Syst. 2022, 19, 83–97. [Google Scholar] [CrossRef]
- Kirkland, I.; Caves, R.; Hirst, M.; Pitfield, D. The normalisation of aircraft overrun accident data. J. Air Transp. Manag. 2003, 9, 333–341. [Google Scholar] [CrossRef]
- Kirkland, I.; Caves, R.; Humphreys, I.; Pitfield, D. An improved methodology for assessing risk in aircraft operations at airports, applied to runway overruns. Saf. Sci. 2004, 42, 891–905. [Google Scholar] [CrossRef]
- Valdés, R.M.A.; Comendador, F.G.; Gordún, L.M.; Nieto, F.J.S. The development of probabilistic models to estimate accident risk (due to runway overrun and landing undershoot) applicable to the design and construction of runway safety areas. Saf. Sci. 2011, 49, 633–650. [Google Scholar] [CrossRef]
- Ayres, M., Jr.; Shirazi, H.; Carvalho, R.; Hall, J.; Speir, R.; Arambula, E.; David, R.; Gadzinski, J.; Caves, R.; Wong, D.; et al. Modelling the location and consequences of aircraft accidents. Saf. Sci. 2013, 51, 178–186. [Google Scholar] [CrossRef]
- Wagner, D.C.; Barker, K. Statistical methods for modeling the risk of runway excursions. J. Risk Res. 2014, 17, 885–901. [Google Scholar] [CrossRef]
- Wang, L.; Wu, C.; Sun, R. Pilot operating characteristics analysis of long landing based on flight QAR data. In Proceedings of the Engineering Psychology and Cognitive Ergonomics. Applications and Services: 10th International Conference, EPCE 2013, Held as Part of HCI International 2013, Las Vegas, NV, USA, 21–26 July 2013; Proceedings, Part II 10. Springer: Berlin/Heidelberg, Germany, 2013; pp. 157–166. [Google Scholar]
- Wang, L.; Wu, C.; Sun, R. An analysis of flight Quick Access Recorder (QAR) data and its applications in preventing landing incidents. Reliab. Eng. Syst. Saf. 2014, 127, 86–96. [Google Scholar] [CrossRef]
- Wang, L.; Ren, Y.; Sun, H.; Dong, C. A landing operation performance evaluation system based on flight data. In Proceedings of the Engineering Psychology and Cognitive Ergonomics: Cognition and Design: 14th International Conference, EPCE 2017, Held as Part of HCI International 2017, Vancouver, BC, Canada, 9–14 July 2017; Proceedings, Part II 14. Springer: Berlin/Heidelberg, Germany, 2017; pp. 297–305. [Google Scholar]
- Wang, L.; Zhang, J.; Dong, C.; Sun, H.; Ren, Y. A method of applying flight data to evaluate landing operation performance. Ergonomics 2019, 62, 171–180. [Google Scholar] [CrossRef]
- Kang, Z.; Shang, J.; Feng, Y.; Zheng, L.; Liu, D.; Qiang, B.; Wei, R. A Deep Sequence-to-Sequence Method for Aircraft Landing Speed Prediction Based on QAR Data. In Proceedings of the International Conference on Web Information Systems Engineering, Biarritz, France, 31 October–3 November 2020; pp. 516–530. [Google Scholar]
- Kang, Z.; Shang, J.; Feng, Y.; Zheng, L.; Wang, Q.; Sun, H.; Qiang, B.; Liu, Z. A deep sequence-to-sequence method for accurate long landing prediction based on flight data. IET Intell. Transp. Syst. 2021, 15, 1028–1042. [Google Scholar] [CrossRef]
- Lv, H.; Yu, J.; Zhu, T. A novel method of overrun risk measurement and assessment using large scale QAR data. In Proceedings of the 2018 IEEE Fourth International Conference on Big Data Computing Service and Applications (BigDataService), Bamberg, Germany, 26–29 March 2018; IEEE: Piscataway, NJ, USA, 2018; pp. 213–220. [Google Scholar]
- Ayra, E.S.; Ríos Insua, D.; Cano, J. Bayesian network for managing runway overruns in aviation safety. J. Aerosp. Inf. Syst. 2019, 16, 546–558. [Google Scholar] [CrossRef]
- Koppitz, P.; Wang, C.; Höhndorf, L.; Sembiring, J.; Wang, X.; Holzapfel, F. From raw operational flight data to incident probabilities using subset simulation and a complex thrust model. In Proceedings of the AIAA Scitech 2019 Forum, San Diego, CA, USA, 6–9 January 2019; p. 2233. [Google Scholar]
- Xueyan, S.; Mingliang, Q.; Mingang, G. Safety risk analysis in flight operations quality assurance. Syst. Eng. Procedia 2012, 5, 81–86. [Google Scholar] [CrossRef]
- Liu, S.; Zhang, Y.; Chen, J. A system for evaluating pilot performance based on flight data. In Proceedings of the Engineering Psychology and Cognitive Ergonomics: 15th International Conference, EPCE 2018, Held as Part of HCI International 2018, Las Vegas, NV, USA, 15–20 July 2018; Proceedings 15. Springer: Berlin/Heidelberg, Germany, 2018; pp. 605–614. [Google Scholar]
- Wang, L.; Zhang, J.; Sun, H.; Ren, Y. Risk cognition variables and flight exceedance behaviors of airline transport pilots. In Proceedings of the Engineering Psychology and Cognitive Ergonomics: 15th International Conference, EPCE 2018, Held as Part of HCI International 2018, Las Vegas, NV, USA, 15–20 July 2018; Proceedings 15. Springer: Berlin/Heidelberg, Germany, 2018; pp. 725–737. [Google Scholar]
- Li, X.; Qian, Y.; Chen, H.; Zheng, L.; Wang, Q.; Shang, J. An Unsupervised Learning Approach for Analyzing Unsafe Pilot Operations Based on Flight Data. Appl. Sci. 2022, 12, 12789. [Google Scholar] [CrossRef]
- Hu, C.; Zhou, S.H.; Xie, Y.; Chang, W.B. The study on hard landing prediction model with optimized parameter SVM method. In Proceedings of the 2016 35th Chinese Control Conference (CCC), Chengdu, China, 27–29 July 2016; IEEE: Piscataway, NJ, USA, 2016; pp. 4283–4287. [Google Scholar]
- Qiao, X.; Chang, W.; Zhou, S.; Lu, X. A prediction model of hard landing based on RBF neural network with K-means clustering algorithm. In Proceedings of the 2016 IEEE International Conference on Industrial Engineering and Engineering Management (IEEM), Bali, Indonesia, 4–7 December 2016; IEEE: Piscataway, NJ, USA, 2016; pp. 462–465. [Google Scholar]
- Tong, C.; Yin, X.; Li, J.; Zhu, T.; Lv, R.; Sun, L.; Rodrigues, J.J. An innovative deep architecture for aircraft hard landing prediction based on time-series sensor data. Appl. Soft Comput. 2018, 73, 344–349. [Google Scholar] [CrossRef]
- Tong, C.; Yin, X.; Wang, S.; Zheng, Z. A novel deep learning method for aircraft landing speed prediction based on cloud-based sensor data. Future Gener. Comput. Syst. 2018, 88, 552–558. [Google Scholar] [CrossRef]
- Chen, R.; Jin, J. Detection of Influential Features of Hard Landing Based on QAR Data. In Proceedings of the 2021 IEEE 3rd International Conference on Civil Aviation Safety and Information Technology (ICCASIT), Dali, China, 11–13 October 2021; IEEE: Piscataway, NJ, USA, 2021; pp. 526–529. [Google Scholar]
- Li, X.; Shang, J.; Zheng, L.; Liu, D.; Qi, L.; Liu, L. CurveCluster: Automated recognition of hard landing patterns based on QAR curve clustering. In Proceedings of the IEEE International Conference on Ubiquitous Intelligence and Computing, Leicester, UK, 19–23 August 2019; IEEE: Piscataway, NJ, USA, 2019; pp. 602–609. [Google Scholar]
- Li, X.; Shang, J.; Zheng, L.; Wang, Q.; Sun, H.; Qi, L. Curvecluster+: Curve clustering for hard landing pattern recognition and risk evaluation based on flight data. IEEE Trans. Intell. Transp. Syst. 2022, 23, 12811–12821. [Google Scholar] [CrossRef]
- Chen, H.; Shang, J.; Zheng, L.; Li, X.; Liu, X.; Sun, H.; Zhao, X.; Yu, L. SDTAN: Scalable Deep Time-Aware Attention Network for Interpretable Hard Landing Prediction. IEEE Trans. Intell. Transp. Syst. 2023. [Google Scholar] [CrossRef]
- Jin, J.; Yan, J.; Chen, K. Transfer learning with quantile regression. arXiv 2022, arXiv:2212.06693. [Google Scholar]
- Li, L.; Gariel, M.; Hansman, R.J.; Palacios, R. Anomaly detection in onboard-recorded flight data using cluster analysis. In Proceedings of the 2011 IEEE/AIAA 30th Digital Avionics Systems Conference, Seattle, WA, USA, 16–20 October 2011; IEEE: Piscataway, NJ, USA, 2011; pp. 4A4-1–4A4-11. [Google Scholar]
- Das, S.; Li, L.; Srivastava, A.; Hansman, R.J. Comparison of algorithms for anomaly detection in flight recorder data of airline operations. In Proceedings of the 12th AIAA Aviation Technology, Integration, and Operations (ATIO) Conference and 14th AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference, Indianapolis, IN, USA, 17–19 September 2012; p. 5593. [Google Scholar]
- Li, L.; Das, S.; John Hansman, R.; Palacios, R.; Srivastava, A.N. Analysis of flight data using clustering techniques for detecting abnormal operations. J. Aerosp. Inf. Syst. 2015, 12, 587–598. [Google Scholar] [CrossRef]
- Li, L.; Hansman, R.J.; Palacios, R.; Welsch, R. Anomaly detection via a Gaussian Mixture Model for flight operation and safety monitoring. Transp. Res. Part C Emerg. Technol. 2016, 64, 45–57. [Google Scholar] [CrossRef]
No. | Name | Physical Meaning | Correlation Coefficient |
---|---|---|---|
1 | LEAVE_MANNER | Leave from the middle or end of runway | −0.71 |
2 | ABK_RATE | Automatic deceleration rate | −0.65 |
3 | MBK_POS | Manual brake position | 0.48 |
4 | IS_EXIST_HMBK | Is the emergency brake present? | −0.38 |
5 | HMBK_RE_DIS | Remaining runway distance when emergency braking | −0.44 |
6 | TD_POS | Touchdown position | 0.33 |
7 | 50FT_KE | Kinetic energy at 50 feet | 0.34 |
8 | GS_LEAVE_10 | Maximum ground speed in the last 10 s before reaching 20 knots | −0.36 |
9 | FLARE_TIME | Time from 50 feet to touchdown | 0.35 |
No. | Regression Model | MAE | RMSE | MAPE |
---|---|---|---|---|
1 | LR | 0.0295 ± 0.0025 | 0.0416 ± 0.0036 | 0.1712 ± 0.0242 |
2 | NN | 0.0206 ± 0.0024 | 0.0279 ± 0.0039 | 0.1129 ± 0.0217 |
3 | DT | 0.0401 ± 0.0053 | 0.0585 ± 0.0081 | 0.1908 ± 0.0356 |
4 | RF | 0.0298 ± 0.0036 | 0.0426 ± 0.0065 | 0.155 ± 0.0253 |
5 | SVR | 0.0464 ± 0.0063 | 0.0588 ± 0.0097 | 0.2853 ± 0.0509 |
No. | Classification Model | Accuracy | Precision | Recall | F1-Score |
---|---|---|---|---|---|
1 | LR | 0.8326 ± 0.0376 | 0.7536 ± 0.0824 | 0.7625 ± 0.085 | 0.7522 ± 0.0513 |
2 | NN | 0.899 ± 0.0331 | 0.8671 ± 0.0637 | 0.8312 ± 0.0971 | 0.8439 ± 0.0552 |
3 | DT | 0.8056 ± 0.0444 | 0.7153 ± 0.0836 | 0.7135 ± 0.0802 | 0.71 ± 0.0614 |
4 | RF | 0.8372 ± 0.0425 | 0.8103 ± 0.0877 | 0.6792 ± 0.0916 | 0.7341 ± 0.07 |
5 | SVR | 0.8576 ± 0.0451 | 0.8018 ± 0.0904 | 0.7833 ± 0.0773 | 0.787 ± 0.0562 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Li, X.; Zhang, L.; Shang, J.; Li, X.; Qian, Y.; Zheng, L. A Runway Overrun Risk Assessment Model for Civil Aircraft Based on Quick Access Recorder Data. Appl. Sci. 2023, 13, 9828. https://doi.org/10.3390/app13179828
Li X, Zhang L, Shang J, Li X, Qian Y, Zheng L. A Runway Overrun Risk Assessment Model for Civil Aircraft Based on Quick Access Recorder Data. Applied Sciences. 2023; 13(17):9828. https://doi.org/10.3390/app13179828
Chicago/Turabian StyleLi, Xiuyi, Lin Zhang, Jiaxing Shang, Xiaoquan Li, Yu Qian, and Linjiang Zheng. 2023. "A Runway Overrun Risk Assessment Model for Civil Aircraft Based on Quick Access Recorder Data" Applied Sciences 13, no. 17: 9828. https://doi.org/10.3390/app13179828