Cockpit crew safety performance prediction based on the integrated machine learning multi-class classification models and Markov chain
Abstract
The main tool of cockpit crew performance evaluation is the recorded flight data used for flight operations safety improvement since all certified airlines require implementation of a safety and quality management system. The safety performance of a flight has been a challenging issue in the aviation industry and plays an important role to acquire competitive benefits. In this study, an integrated multi-class classification machine learning models and Markov chain were developed for cockpit crew performance evaluation during their flights. At the outset, the main features related to a flight are identified based on the literature review, flight operations expert’s statements, and the case study dataset (as numerical example). Afterwards, the flights’ performance is evaluated as a target column based on four multi-class classification models (Decision Tree, Support Vector Machine, Neural Network, and Random Forest). The results showed that the random forest classifier has the greatest value in all evaluation metrics (i.e., accuracy = 0.90, precision = 0.91, recall = 0.97, and F1-score = 0.93). Therefore, this model can be used by the airline companies to predict flight crew performance before the flight in order to prevent or decrease flight safety risks.
Keyword : cockpit crew performance, flight evaluation prediction, multi-class classification, Markov chain, safety management system
This work is licensed under a Creative Commons Attribution 4.0 International License.
References
Ashiku, L., Al-Amin, M., Madria, S., & Dagli, C. (2021). Machine learning models and big data tools for evaluating kidney acceptance. Procedia Computer Science, 185(June), 177–184. https://doi.org/10.1016/j.procs.2021.05.019
Castillo-Botón, C., Casillas-Pérez, D., Casanova-Mateo, C., Ghimire, S., Cerro-Prada, E., Gutierrez, P. A., Deo, R. C., & Salcedo-Sanz, S. (2022). Machine learning regression and classification methods for fog events prediction. Atmospheric Research, 272, 106157. https://doi.org/10.1016/j.atmosres.2022.106157
Corker, K. M., & Pisanich, G. M. (1995). Analysis and modeling of flight crew performance in automated air traffic management systems. IFAC Proceedings Volumes, 28(15), 547–552. https://doi.org/10.1016/S1474-6670(17)45289-X
Delgado, F., Trincado, R., & Pagnoncelli, B. K. (2019). A multistage stochastic programming model for the network air cargo allocation under capacity uncertainty. Transportation Research Part E: Logistics and Transportation Review, 131(November), 292–307. https://doi.org/10.1016/j.tre.2019.09.011
EASA. (2019). AMC1 ORO.AOC.130 – Annex III. https://www.easa.europa.eu/
European Aviation Safety Agency (EASA). (2016). Developing Standardised Fdm-Based Indicators Focus (2 December, pp. 1–55). https://www.easa.europa.eu/sites/default/files/dfu/EAFDM__standardised_FDM-based_indicators_v2_Ed2017.pdf
Filippone, A. (2008). Comprehensive analysis of transport aircraft flight performance. Progress in Aerospace Sciences, 44(3), 192–236. https://doi.org/10.1016/j.paerosci.2007.10.005
Fodeh, S. J., & Tiwari, A. (2018). Exploiting MEDLINE for gene molecular function prediction via NMF based multi-label classification. Journal of Biomedical Informatics, 86(August 2017), 160–166. https://doi.org/10.1016/j.jbi.2018.08.009
Gharaibeh, A., Shaamala, A., Obeidat, R., & Al-Kofahi, S. (2020). Improving land-use change modeling by integrating ANN with Cellular Automata-Markov Chain model. Heliyon, 6(9), e05092. https://doi.org/10.1016/j.heliyon.2020.e05092
Güven, İ., & Şimşir, F. (2020). Demand forecasting with color parameter in retail apparel industry using artificial neural networks (ANN) and support vector machines (SVM) methods. Computers and Industrial Engineering, 147. https://doi.org/10.1016/j.cie.2020.106678
Hon, K. K., Ng, C. W., & Chan, P. W. (2020). Machine learning based multi-index prediction of aviation turbulence over the Asia-Pacific. Machine Learning with Applications, 2, 100008. https://doi.org/10.1016/j.mlwa.2020.100008
International Civil Aviation Organization. (2013). Operation of Aircraft – Fatigue. Excerpts of Fatigue Management Realted Provisions from: Annex 6 to the Convention of International Civil Aviation, February. https://www.icao.int/safety/fatiguemanagement/FRMS%20Tools/Amendment%2037%20for%20FRMS%20SARPS%20%28en%29.pdf
Kulkarni, V. G. (2011). Brownian motion. In Introduction to modeling and analysis of stochastic systems (pp. 247–280). Springer. https://doi.org/10.1007/978-1-4419-1772-0_7
Lan, C. E., Wu, K., & Yu, J. (2012). Flight characteristics analysis based on QAR data of a jet transport during landing at a high-altitude airport. Chinese Journal of Aeronautics, 25(1), 13–24. https://doi.org/10.1016/S1000-9361(11)60357-9
Li, H., Wang, W., Fan, L., Li, Q., & Chen, X. (2020). A novel hybrid MCDM model for machine tool selection using fuzzy DEMATEL, entropy weighting and later defuzzification VIKOR. Applied Soft Computing, 91, 106207. https://doi.org/10.1016/j.asoc.2020.106207
Lyu, Y., & Liem, R. P. (2020). Flight performance analysis with data-driven mission parameterization: Mapping flight operational data to aircraft performance analysis. Transportation Engineering, 2(September), 100035. https://doi.org/10.1016/j.treng.2020.100035
Martini, G., Scotti, D., & Volta, N. (2013). Including local air pollution in airport efficiency assessment: A hyperbolic-stochastic approach. Transportation Research Part D: Transport and Environment, 24(2007), 27–36. https://doi.org/10.1016/j.trd.2013.05.002
Mokhtarimousavi, S., & Mehrabi, A. (2022). Flight delay causality: Machine learning technique in conjunction with random parameter statistical analysis. International Journal of Transportation Science and Technology, 12(1), 230–244. https://doi.org/10.1016/j.ijtst.2022.01.007
Moral-García, S., Mantas, C. J., Castellano, J. G., & Abellán, J. (2020). Non-parametric predictive inference for solving multi-label classification. Applied Soft Computing Journal, 88. https://doi.org/10.1016/j.asoc.2019.106011
Moshkov, M. (2021). On the depth of decision trees over infinite 1-homogeneous binary information systems. Array, 10(March), 100060. https://doi.org/10.1016/j.array.2021.100060
Nguyen, T. P., & Lin, Y. K. (2021). Reliability assessment of a stochastic air transport network with late arrivals. Computers and Industrial Engineering, 151(January). https://doi.org/10.1016/j.cie.2020.106956
Okwuashi, O., & Ndehedehe, C. E. (2021). Integrating machine learning with Markov chain and cellular automata models for modelling urban land use change. Remote Sensing Applications: Society and Environment, 21(January). https://doi.org/10.1016/j.rsase.2020.100461
Onan, A., Korukoğlu, S., & Bulut, H. (2016a). Ensemble of keyword extraction methods and classifiers in text classification. Expert Systems with Applications, 57, 232–247. https://doi.org/10.1016/j.eswa.2016.03.045
Onan, A., Bal, V., & Yanar Bayam, B. (2016b). The use of data mining for strategic management: A case study on mining association rules in student information system. Croatian Journal of Education: Hrvatski časopis za odgoj i obrazovanje, 18(1), 41–70. https://doi.org/10.15516/cje.v18i1.1471
Onan, A. (2019). Two-stage topic extraction model for bibliometric data analysis based on word embeddings and clustering. IEEE Access, 7, 145614–145633. https://doi.org/10.1109/ACCESS.2019.2945911
Onan, A., & Korukoğlu, S. (2017). A feature selection model based on genetic rank aggregation for text sentiment classification. Journal of Information Science, 43(1), 25–38. https://doi.org/10.1177/0165551515613226
Onan, A. (2021). Sentiment analysis on product reviews based on weighted word embeddings and deep neural networks. Concurrency and Computation: Practice and Experience, 33(23), e5909. https://doi.org/10.1002/cpe.5909
Onan, A. (2015). A fuzzy-rough nearest neighbor classifier combined with consistency-based subset evaluation and instance selection for automated diagnosis of breast cancer. Expert Systems with Applications, 42(20), 6844–6852. https://doi.org/10.1016/j.eswa.2015.05.006
Onan, A. (2020). Mining opinions from instructor evaluation reviews: A deep learning approach. Computer Applications in Engineering Education, 28(1), 117–138. https://doi.org/10.1002/cae.22179
Onan, A. (2016). Classifier and feature set ensembles for web page classification. Journal of Information Science, 42(2), 150–165. https://doi.org/10.1177/0165551515591724
Onan, A. (2018a). An ensemble scheme based on language function analysis and feature engineering for text genre classification. Journal of Information Science, 44(1), 28–47. https://doi.org/10.1177/0165551516677911
Onan, A. (2018b). Biomedical text categorization based on ensemble pruning and optimized topic modelling. Computational and Mathematical Methods in Medicine, 2018. https://doi.org/10.1155/2018/2497471
Onan, A. (2019a). Consensus clustering-based undersampling approach to imbalanced learning. Scientific Programming, 2019. https://doi.org/10.1155/2019/5901087
Onan, A. (2019b). Topic-enriched word embeddings for sarcasm identification. In R. Silhavy, Software Engineering Methods in Intelligent Algorithms: Proceedings of 8th Computer Science On-line Conference 2019 (Vol. 984, pp. 293–304). Springer International Publishing. https://doi.org/10.1007/978-3-030-19807-7_29
Onan, A., & Toçoğlu, M. A. (2021). A term weighted neural language model and stacked bidirectional LSTM based framework for sarcasm identification. IEEE Access, 9, 7701–7722. https://doi.org/10.1109/ACCESS.2021.3049734
Oreschko, B., Kunze, T., Schultz, M., Fricke, H., Kumar, V., & Sherry, L. (2012). Turnaround prediction with stochastic process times and airport specific delay pattern airport delays. In The 5th International Conference on Research in Air Transportation, 9. ResearchGate.
Papadopoulos, C. T., Li, J., & O’Kelly, M. E. J. (2019). A classification and review of timed Markov models of manufacturing systems. Computers and Industrial Engineering, 128(November 2018), 219–244. https://doi.org/10.1016/j.cie.2018.12.019
Qian, W., Xiong, C., & Wang, Y. (2021). A ranking-based feature selection for multi-label classification with fuzzy relative discernibility. Applied Soft Computing, 102. https://doi.org/10.1016/j.asoc.2020.106995
Rey, M., Aloise, D., Soumis, F., & Pieugueu, R. (2021). A data-driven model for safety risk identification from flight data analysis. Transportation Engineering, 5, 100087. https://doi.org/10.1016/j.treng.2021.100087
Ross, S. M. (2014). Introduction to probability models. Academic Press. https://doi.org/10.1016/B978-0-12-407948-9.00001-3
Samaee, S., & Kobravi, H. R. (2020). Predicting the occurrence of wrist tremor based on electromyography using a hidden Markov model and entropy based learning algorithm. Biomedical Signal Processing and Control, 57(March). https://doi.org/10.1016/j.bspc.2019.101739
Shone, R., Glazebrook, K., & Zografos, K. G. (2021). Applications of stochastic modeling in air traffic management: Methods, challenges and opportunities for solving air traffic problems under uncertainty. European Journal of Operational Research, 292(1), 1–26. https://doi.org/10.1016/j.ejor.2020.10.039
Toçoğlu, M. A., & Onan, A. (2020, July). Sentiment analysis on students’ evaluation of higher educational institutions. In International Conference on Intelligent and Fuzzy Systems (pp. 1693–1700). INFUS 2020: Intelligent and Fuzzy Techniques: Smart and Innovative Solutions. Springer. https://doi.org/10.1007/978-3-030-51156-2_197
Utami, N. A., Maharani, W., & Atastina, I. (2021). Personality classification of Facebook users according to big five personality using SVM (Support Vector Machine) method. Procedia Computer Science, 179(2020), 177–184. https://doi.org/10.1016/j.procs.2020.12.023
Van Giffen, B., Herhausen, D., & Fahse, T. (2022). Overcoming the pitfalls and perils of algorithms: A classification of machine learning biases and mitigation methods. Journal of Business Research, 144, 93–106. https://doi.org/10.1016/j.jbusres.2022.01.076
Wang, L., Wu, C., & Sun, R. (2014). An analysis of flight Quick Access Recorder (QAR) data and its applications in preventing landing incidents. Reliability Engineering and System Safety, 127, 86–96. https://doi.org/10.1016/j.ress.2014.03.013
Yaakoubi, Y., Soumis, F., & Lacoste-Julien, S. (2020). Machine learning in airline crew pairing to construct initial clusters for dynamic constraint aggregation. EURO Journal on Transportation and Logistics, 9(4), 100020. https://doi.org/10.1016/j.ejtl.2020.100020
Yamaguchi, S., Nakashima, H., Moriwaki, Y., Terada, T., & Shimizu, K. (2022). Prediction of protein mononucleotide binding sites using AlphaFold2 and machine learning. Computational Biology and Chemistry, 107744. https://doi.org/10.1016/j.compbiolchem.2022.107744
Yan, S., & Tang, C.-H. (2007). A heuristic approach for airport gate assignments for stochastic flight delays. European Journal of Operational Research, 180(2), 547–567. https://doi.org/10.1016/j.ejor.2006.05.002
Yang, C., Yin, T., Zhao, W., Huang, D., & Fu, S. (2014). Human factors quantification via boundary identification of flight performance margin. Chinese Journal of Aeronautics, 27(4), 977–985. https://doi.org/10.1016/j.cja.2014.03.016
Zhou, Y., Liu, Y., Wang, D., Liu, X., & Wang, Y. (2021). A review on global solar radiation prediction with machine learning models in a comprehensive perspective. Energy Conversion and Management, 235(13), 113960. https://doi.org/10.1016/j.enconman.2021.113960