Machine Learning Predictions and Identifying Key Predictors for Safer Intubation: A Study on Video Laryngoscopy Views
Abstract
:1. Introduction
2. Materials and Methods
2.1. Study Design
2.2. Ethical Approval/Informed Consent
2.3. Definition of Airway-Associated Factors and Data Collection
- Class I: Soft palate, uvula, fauces, and pillars visible.
- Class II: Soft palate, major part of uvula, and fauces visible.
- Class III: Soft palate and base of uvula visible.
- Class IV: Only the hard palate visible.
2.4. Factors Considered and Addressed in This Study
- Patient Anatomy: Anatomical variations, such as neck circumference, mouth opening, thyromental height, thyromental distance, and sternomental distance, and the presence of abnormalities can significantly affect laryngeal visualization. To address this issue, we included airway-related anatomical characteristics as predictors in our analysis. Patients with airway-related anatomical abnormalities were excluded.
- Operator Experience and Skill: Operator proficiency is crucial to achieving optimal visibility during video laryngoscopy. To enhance the reliability, the intubator did not take part in the assessment of POGO scores.
- Presence of Blood, Secretions, or Debris: Blood, secretions, or debris can obstruct the camera lens and impair visualization. To mitigate this, patients with potential obstructive factors were excluded, intubation was performed under optimal conditions, and an optimal laryngoscopic view was preserved.
- Use of Stylets: Assistive devices, such as stylets, may affect tube insertion ease and laryngeal visibility. For standardization, a stylet was used in all cases to maintain an optimal intubation environment.
- Hemodynamic Status: Hemodynamic stability is crucial for airway patency and visibility. Therefore, only electively stable surgical patients were included in this study.
- Patient Position During Intubation: Patient positioning affects airway alignment and visualization. Hence, for consistency, all patients were placed in a supine position with the same pillow configuration.
- Use of Sedatives or Muscle Relaxants: Sedatives and muscle relaxants can alter airway dynamics and affect visualization. Therefore, sedatives were withheld before anesthesia, and only rocuronium was used as a muscle relaxant. Intubation was performed when neuromuscular monitoring indicated complete muscle relaxation with a train-of-four count of 0.
2.5. Data Preprocessing, Transformation, and Feature Engineering
2.6. Model Development and Validation
2.7. Augmentation and Robustness Testing
2.8. Interpretability with SHAP Values
3. Results
3.1. Training and Testing Datasets
3.2. Performance of Predicting Models
3.3. Average Impact Magnitude of Features on Model Output: Comparative Analysis of Mean Absolute SHAP Values across Different Models
3.4. Analysis of SHAP Values: Evaluating the Impact on Model Outputs across Various Machine Learning Models
3.5. Univariate Regression Analysis Using Age
- MSE: 503.255
- RMSE: 22.433
- MAE: 16.805
- R2: 0.1836
4. Discussion
- Sample Diversity: The study was limited by the relatively small number of participants, which may restrict the generalizability of the findings. Additionally, the sample lacked diversity in terms of ethnicity, sex, and health status, which potentially limits the applicability of the results to a broader population. Additionally, despite the implementation of data augmentation techniques, our models exhibited challenges in accurately predicting lower POGO scores. This limitation primarily stems from the under-representation of cases with low POGO values in both the training and test datasets. Video laryngoscopy inherently yielded lower POGO scores than direct laryngoscopy. This discrepancy likely contributes to the models’ decreased ability to effectively predict such outcomes, thus affecting the overall performance of our predictive models. Future studies should focus on enhancing data collection strategies to better represent lower POGO score scenarios, potentially improving model accuracy and generalizability across various clinical conditions.
- Technological Variations: Variability in the equipment used, such as different types of laryngoscopes and variations in video quality, could have affected the visibility of the glottic opening. These variations may have influenced the accuracy of the POGO score assessments.
- Generalizability of the Findings: The findings were derived from specific clinical settings, including emergency rooms, emergency surgeries, and intensive care units. Consequently, these results may not be directly applicable to other clinical environments or routine surgical settings. Additionally, it is important to note that the POGO scores included in our study were recorded prior to performing any maneuvers, such as BURP or lifting the epiglottis. These maneuvers can significantly alter the POGO scores, and their exclusion ensures consistency in our data but may limit the applicability of the findings to scenarios where such techniques are employed. All patients were positioned supine using a standardized pillow designed to provide consistent support and alignment. However, this standardized pillow does not guarantee an exact angle of head positioning. This is critical for reproducibility and understanding of the influence of age on POGO scores. Limited head mobility in older patients may contribute to lower POGO scores, highlighting the importance of standardized head positioning in future studies.
- Temporal Factors: The relevance of the study may diminish over time owing to technological advancements and changes in clinical practice. Such changes could alter the techniques used in video laryngoscopy, thereby affecting the validity of the conclusions of future studies.
- Limitations of POGO Score Selection: We used the POGO score as the primary outcome measure to assess airway visibility during intubation. Although the POGO score offers detailed visibility metrics and quantifies the visual access to the glottis, it has not been widely adopted in clinical practice and may not fully address the clinical nuances associated with airway management. Specifically, the POGO score, by focusing solely on the percentage of glottic opening visible, might not adequately distinguish between the varying degrees of difficulty encountered in challenging intubations typically classified as Cormack–Lehane grades 3 or 4. Critically, as noted by the reviewer and evident in the practices of many airway experts, a POGO score greater than 0 generally indicates a relatively easier airway compared to a score of 0, often corresponding to Cormack–Lehane grades 1, 2a, or 2b. However, this classification highlights a potential limitation in the ability of the POGO score to provide actionable clinical insights for more challenging scenarios, where detailed differentiation between more nuanced grades of visibility could inform better management decisions. To address these concerns, further research is needed to evaluate the effectiveness of POGO scores in predicting truly difficult intubations and explore how they might be integrated or adapted for broader clinical use.
- Weak Performance: Our model exhibits weak performance, which likely indicates the presence of other factors or variables not included in the model that significantly influence the outcomes. In practice, the low explanatory power of the model limits its utility in clinical applications. Collaboration with clinical practitioners is essential for validating and refining a model based on real-world data and feedback. Continuous research on better predictive models is necessary to enhance predictive accuracy and clinical relevance. Despite its limitations, our model can still provide valuable insights into the factors affecting difficult intubation. It can serve as an educational tool in training scenarios to highlight potential issues and considerations in airway management, emphasizing the importance of preparing for various challenges that may arise during intubation.
5. Conclusions
Supplementary Materials
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
Appendix A
Appendix B
- Best hyperparameters for RandomForest: {‘max_depth’: 20, ‘n_estimators’: 200}
- Best hyperparameters for LGBM: {‘learning_rate’: 0.1, ‘max_depth’: −1, ‘n_estimators’: 100}
- Best hyperparameters for KNN: {‘n_neighbors’: 7, ‘weights’: ‘distance’}
- Best hyperparameters for SVR: {‘C’: 1, ‘gamma’: ‘scale,’ ‘kernel’: ‘linear’}
- Best hyperparameters for Ridge: {‘alpha’: 10}
- Best hyperparameters for Lasso: {‘alpha’: 0.01}
References
- Alvarado, A.C.; Panakos, P. Endotracheal tube intubation techniques. In StatPearls; StatPearls Publishing: Treasure Island, FL, USA, 2020. [Google Scholar]
- Abdelgadir, I.S.; Phillips, R.S.; Singh, D.; Moncreiff, M.P.; Lumsden, J.L. Videolaryngoscopy versus direct laryngoscopy for tracheal intubation in children (excluding neonates). Cochrane Database Syst. Rev. 2017, 5, Cd011413. [Google Scholar] [CrossRef]
- Hansel, J.; Rogers, A.M.; Lewis, S.R.; Cook, T.M.; Smith, A.F. Videolaryngoscopy versus direct laryngoscopy for adults undergoing tracheal intubation. Cochrane Database Syst. Rev. 2022, 4, Cd011136. [Google Scholar] [CrossRef]
- Kim, J.G.; Ahn, C.; Kim, W.; Lim, T.-H.; Jang, B.-H.; Cho, Y.; Shin, H.; Lee, H.; Lee, J.; Choi, K.-S.; et al. Comparison of video laryngoscopy with direct laryngoscopy for intubation success in critically ill patients: A systematic review and Bayesian network meta-analysis. Front. Med. 2023, 10, 1193514. [Google Scholar] [CrossRef]
- Silverberg, M.J.; Li, N.; Acquah, S.O.; Kory, P.D. Comparison of video laryngoscopy versus direct laryngoscopy during urgent endotracheal intubation: A randomized controlled trial. Crit. Care Med. 2015, 43, 636–641. [Google Scholar] [CrossRef]
- Alimian, M.; Zaman, B.; Seyed Siamdoust, S.A.; Nikoubakht, N.; Rounasi, R. Comparison of RAMP and New Modified RAMP Positioning in Laryngoscopic View During Intubation in Patients with Morbid Obesity: A Randomized Clinical Trial. Anesth. Pain Med. 2021, 11, e114508. [Google Scholar] [CrossRef]
- Kim, W.H.; Ahn, H.J.; Lee, C.J.; Shin, B.S.; Ko, J.S.; Choi, S.J.; Ryu, S.A. Neck circumference to thyromental distance ratio: A new predictor of difficult intubation in obese patients. Br. J. Anaesth. 2011, 106, 743–748. [Google Scholar] [CrossRef]
- Tamire, T.; Demelash, H.; Admasu, W. Predictive Values of Preoperative Tests for Difficult Laryngoscopy and Intubation in Adult Patients at Tikur Anbessa Specialized Hospital. Anesthesiol. Res. Pract. 2019, 2019, 1790413. [Google Scholar] [CrossRef]
- SHAP. SHAP (SHapley Additive exPlanations). Available online: https://shap.readthedocs.io/en/latest/index.html (accessed on 10 May 2024).
- Kim, J.-H.; Cheon, B.-R.; Kim, H.; Hwang, S.-M.; Lee, J.-J.; Kwon, Y.-S. Influence of Curved Video Laryngoscope Blade Sizes and Patient Heights on Video Laryngoscopic Views: A Randomized Controlled Trial. J. Pers. Med. 2024, 14, 209. [Google Scholar] [CrossRef]
- Levitan, R.M.; Ochroch, E.A.; Kush, S.; Shofer, F.S.; Hollander, J.E. Assessment of airway visualization: Validation of the percentage of glottic opening (POGO) scale. Acad. Emerg. Med. 1998, 5, 919–923. [Google Scholar] [CrossRef]
- Stutz, E.W.; Rondeau, B. Mallampati score. In StatPearls; StatPearls Publishing: Treasure Island, FL, USA, 2024. [Google Scholar]
- Prakash, S.; Mullick, P.; Singh, R. Evaluation of thyromental height as a predictor of difficult laryngoscopy and difficult intubation: A cross-sectional observational study. Braz. J. Anesthesiol. (Engl. Ed.) 2022, 72, 742–748. [Google Scholar] [CrossRef]
- Tripathi, M.; Pandey, M. Short Thyromental Distance: A Predictor of Difficult Intubation or an Indicator for Small Blade Selection? Anesthesiology 2006, 104, 1131–1136. [Google Scholar] [CrossRef]
- Prakash, S.; Mullick, P.; Bhandari, S.; Kumar, A.; Gogia, A.R.; Singh, R. Sternomental distance and sternomental displacement as predictors of difficult laryngoscopy and intubation in adult patients. Saudi J. Anaesth. 2017, 11, 273–278. [Google Scholar] [CrossRef]
- Riad, W.; Vaez, M.N.; Raveendran, R.; Tam, A.D.; Quereshy, F.A.; Chung, F.; Wong, D.T. Neck circumference as a predictor of difficult intubation and difficult mask ventilation in morbidly obese patients: A prospective observational study. Eur. J. Anaesthesiol. 2016, 33, 244–249. [Google Scholar] [CrossRef]
- Gholamy, A.; Kreinovich, V.; Kosheleva, O. Why 70/30 or 80/20 relation between training and testing sets: A pedagogical explanation. Int. J. Intell. Technol. Appl. Stat 2018, 11, 105–111. [Google Scholar]
- GridSearchCV. Available online: https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.GridSearchCV.html (accessed on 8 May 2024).
- Lee, M.C.; Tseng, K.Y.; Shen, Y.C.; Lin, C.H.; Hsu, C.W.; Hsu, H.J.; Lu, I.C.; Cheng, K.I. Nasotracheal intubation in patients with limited mouth opening: A comparison between fibreoptic intubation and the Trachway®. Anaesthesia 2016, 71, 31–38. [Google Scholar] [CrossRef]
- Gorgy, A.; Ahmed, A.; Atef, M.; Mekawy, N.; Sami, W.; Nagy, H. Sternomental displacement and neck circumference: A new look for the neck as a difficult airway predictor in obese surgical patients—A cohort study. Ain-Shams J. Anesthesiol. 2023, 15, 58. [Google Scholar] [CrossRef]
- Ahmed, A.M.; Zaky, M.N.; El-Mekawy, N.M.; Ollaek, M.A.; Sami, W.M.; Mohamed, D.M. Evaluation of thyromental height test in prediction of difficult airway in obese surgical patients: An observational study. Indian J. Anaesth. 2021, 65, 880–885. [Google Scholar] [CrossRef]
- Ollaek, M.; Abo Elela, S.; Ahmed, A.; Abdel Rahman, N.; ElKholy, G.; Gorgy, A.; Reda, I.; Mohamed, D. Thyromental height test as a predictor of difficult airway. Single test versus multivariate predictive models. A cohort study. Egypt. J. Anaesth. 2022, 38, 622–628. [Google Scholar] [CrossRef]
- Neligan, P.J.; Porter, S.; Max, B.; Malhotra, G.; Greenblatt, E.P.; Ochroch, E.A. Obstructive sleep apnea is not a risk factor for difficult intubation in morbidly obese patients. Anesth. Analg. 2009, 109, 1182–1186. [Google Scholar] [CrossRef]
- Kim, J.H.; Choi, J.W.; Kwon, Y.S.; Kang, S.S. Predictive model for difficult laryngoscopy using machine learning: Retrospective cohort study. Braz. J. Anesthesiol. (Engl. Ed.) 2022, 72, 622–628. [Google Scholar] [CrossRef]
- Kim, J.H.; Kim, H.; Jang, J.S.; Hwang, S.M.; Lim, S.Y.; Lee, J.J.; Kwon, Y.S. Development and validation of a difficult laryngoscopy prediction model using machine learning of neck circumference and thyromental height. BMC Anesthesiol. 2021, 21, 125. [Google Scholar] [CrossRef]
- Wang, G.; Li, C.; Tang, F.; Wang, Y.; Wu, S.; Zhi, H.; Zhang, F.; Wang, M.; Zhang, J. A fully-automatic semi-supervised deep learning model for difficult airway assessment. Heliyon 2023, 9, e15629. [Google Scholar] [CrossRef]
- Xia, M.; Jin, C.; Zheng, Y.; Wang, J.; Zhao, M.; Cao, S.; Xu, T.; Pei, B.; Irwin, M.G.; Lin, Z.; et al. Deep learning-based facial analysis for predicting difficult videolaryngoscopy: A feasibility study. Anaesthesia 2024, 79, 399–409. [Google Scholar] [CrossRef]
- Yamanaka, S.; Goto, T.; Morikawa, K.; Watase, H.; Okamoto, H.; Hagiwara, Y.; Hasegawa, K. Machine Learning Approaches for Predicting Difficult Airway and First-Pass Success in the Emergency Department: Multicenter Prospective Observational Study. Interact. J. Med. Res. 2022, 11, e28366. [Google Scholar] [CrossRef]
- Hayasaka, T.; Kawano, K.; Kurihara, K.; Suzuki, H.; Nakane, M.; Kawamae, K. Creation of an artificial intelligence model for intubation difficulty classification by deep learning (convolutional neural network) using face images: An observational study. J. Intensive Care 2021, 9, 38. [Google Scholar] [CrossRef]
- El-Ganzouri, A.R.; McCarthy, R.J.; Tuman, K.J.; Tanck, E.N.; Ivankovich, A.D. Preoperative airway assessment: Predictive value of a multivariate risk index. Anesth. Analg. 1996, 82, 1197–1204. [Google Scholar]
- L’Hermite, J.; Nouvellon, E.; Cuvillon, P.; Fabbro-Peray, P.; Langeron, O.; Ripart, J. The Simplified Predictive Intubation Difficulty Score: A new weighted score for difficult airway assessment. Eur. J. Anaesthesiol.|EJA 2009, 26, 1003–1009. [Google Scholar] [CrossRef]
- Shiga, T.; Wajima, Z.i.; Inoue, T.; Sakamoto, A. Predicting difficult intubation in apparently normal patients: A meta-analysis of bedside screening test performance. J. Am. Soc. Anesthesiol. 2005, 103, 429–437. [Google Scholar] [CrossRef]
- Nørskov, A.K.; Wetterslev, J.; Rosenstock, C.V.; Afshari, A.; Astrup, G.; Jakobsen, J.C.; Thomsen, J.L.; Bøttger, M.; Ellekvist, M.; Schousboe, B.M.; et al. Effects of using the simplified airway risk index vs usual airway assessment on unanticipated difficult tracheal intubation—A cluster randomized trial with 64,273 participants. Br. J. Anaesth. 2016, 116, 680–689. [Google Scholar] [CrossRef]
- Koh, W.; Kim, H.; Kim, K.; Ro, Y.-J.; Yang, H.-S. Encountering unexpected difficult airway: Relationship with the intubation difficulty scale. Korean J. Anesthesiol. 2016, 69, 244. [Google Scholar] [CrossRef]
- Pacheco-Lopez, P.C.; Berkow, L.C.; Hillel, A.T.; Akst, L.M. Complications of airway ManagementDiscussion. Respir. Care 2014, 59, 1006–1021. [Google Scholar] [CrossRef]
- Russotto, V.; Myatra, S.N.; Laffey, J.G.; Tassistro, E.; Antolini, L.; Bauer, P.; Lascarrou, J.B.; Szułdrzyński, K.; Camporota, L.; Pelosi, P.; et al. Intubation Practices and Adverse Peri-intubation Events in Critically Ill Patients From 29 Countries. JAMA 2021, 325, 1164–1172. [Google Scholar] [CrossRef]
- Linardatos, P.; Papastefanopoulos, V.; Kotsiantis, S. Explainable AI: A Review of Machine Learning Interpretability Methods. Entropy 2021, 23, 18. [Google Scholar] [CrossRef]
- Mohamed, E.; Sirlantzis, K.; Howells, G. A review of visualisation-as-explanation techniques for convolutional neural networks and their evaluation. Displays 2022, 73, 102239. [Google Scholar] [CrossRef]
- Merrick, L.; Taly, A. The explanation game: Explaining machine learning models using shapley values. In Proceedings of the Machine Learning and Knowledge Extraction: 4th IFIP TC 5, TC 12, WG 8.4, WG 8.9, WG 12.9 International Cross-Domain Conference, CD-MAKE 2020, Dublin, Ireland, 25–28 August 2020; pp. 17–38. [Google Scholar]
- Serikawa, M.; Ambe, K.; Usami, A. Histological observations of age-related changes in the epiglottis associated with decreased deglutition function in older adults. Anat. Cell Biol. 2023, 56, 374–381. [Google Scholar] [CrossRef]
- Fakhoury, J.; Dowling, T.J. Cervical degenerative disc disease. In StatPearls; StatPearls Publishing: Treasure Island, FL, USA, 2020. [Google Scholar]
- Maeda, M.; Chaki, T.; Kawaguchi, R.; Kimijima, T.; Yamakage, M. Difficult airway management of a patient with limited range of motion in the temporomandibular joint and cervical extension caused by psoriatic arthritis: A case report. JA Clin. Rep. 2020, 6, 44. [Google Scholar] [CrossRef]
Features | Train Set (n = 169) | Test Set (n = 43) | |
---|---|---|---|
Age (years) | 44 (32 to 58) | 46 (32 to 60) | |
Female | 8 (4.73) | 1 (2.3) | |
Thyromental height (cm) | 5.1 (4.5 to 5.8) | 50.0 (4.3 to 5.3) | |
Thyromental distance (cm) | 9.8 (9.0 to 10.5) | 9.5 (8.75 to 10.5) | |
Sternomental distance (cm) | 17.0 (15.5 to 18.0) | 16.5 (15.4 to 17.8) | |
Mouth opening (cm) | 5.8 (5.2 to 6.2) | 5.8 (5.4 to 6.1) | |
Neck circumference (cm) | 40.0 (38.5 to 42.0) | 40.0 (38.1 to 41.6) | |
Height (cm) | 174.4 (172.4 to 178.0) | 174.0 (172.2 to 176.7) | |
Weight (kg) | 79.4 (72.9 to 90.0) | 80.8 (68.5 to 88.8) | |
Body mass index (kg/m2) | 25.7 (23.6 to 28.7) | 24.9 (22.8 to 28.7) | |
Mallampati class | Grade 1 | 102 (60.4) | 26 (60.5) |
Grade 2 | 46 (27.2) | 8 (18.6) | |
Grade 3 | 17 (10.1) | 7 (16.3) | |
Grade 4 | 4 (2.4) | 2 (4.7) | |
Percentage of glottic open score | 91.8 (71.9 to 100.0) | 87.8 (69.9 to 100) |
Algorithm | RMSE | MSE | MAE | R2 |
---|---|---|---|---|
Random Forest | 21.606 | 466.82 | 16.409 | 0.1215 |
Light Gradient Boosting Machine | 20.985 | 440.39 | 15.677 | 0.1713 |
K-Nearest Neighbors | 21.905 | 479.82 | 16.616 | 0.0971 |
Support Vector Regression | 20.632 | 425.68 | 15.25 | 0.199 |
Ridge Regression | 20.453 | 418.32 | 15.57 | 0.2128 |
Lasso Regression | 20.551 | 422.34 | 15.572 | 0.2052 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Kim, J.-H.; Han, S.-W.; Hwang, S.-M.; Lee, J.-J.; Kwon, Y.-S. Machine Learning Predictions and Identifying Key Predictors for Safer Intubation: A Study on Video Laryngoscopy Views. J. Pers. Med. 2024, 14, 902. https://doi.org/10.3390/jpm14090902
Kim J-H, Han S-W, Hwang S-M, Lee J-J, Kwon Y-S. Machine Learning Predictions and Identifying Key Predictors for Safer Intubation: A Study on Video Laryngoscopy Views. Journal of Personalized Medicine. 2024; 14(9):902. https://doi.org/10.3390/jpm14090902
Chicago/Turabian StyleKim, Jong-Ho, Sung-Woo Han, Sung-Mi Hwang, Jae-Jun Lee, and Young-Suk Kwon. 2024. "Machine Learning Predictions and Identifying Key Predictors for Safer Intubation: A Study on Video Laryngoscopy Views" Journal of Personalized Medicine 14, no. 9: 902. https://doi.org/10.3390/jpm14090902