Assessment of Visual Attention in Teams with or without Dedicated Team Leaders: A Neonatal Simulation-Based Pilot Randomised Cross-Over Trial Utilising Low-Cost Eye-Tracking Technology
Abstract
:1. Background
2. Methods
2.1. Settings
2.2. Participants
2.3. Simulation
2.4. Study Algorithm
2.5. Visual Attention (Situation Awareness)
2.6. Team Performance and Behavioural Skills
2.7. Workload
2.8. Sample Size and Statistics
3. Results
4. Discussion
5. Conclusions
Supplementary Materials
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Newborn Life Support; Resuscitation Council: London, UK, 2021.
- Mduma, E.; Ersdal, H.; Svensen, E.; Kidanto, H.; Auestad, B.; Perlman, J. Frequent brief on-site simulation training and reduction in 24-h neonatal mortality—An educational intervention study. Resuscitation 2015, 93, 1–7. [Google Scholar] [CrossRef]
- Aziz, K.; Lee, C.H.C.; Escobedo, M.B.; Hoover, A.V.; Kamath-Rayne, B.D.; Kapadia, V.S.; Magid, D.J.; Niermeyer, S.; Schmölzer, G.M.; Szyld, E.; et al. Part 5: Neonatal Resuscitation 2020 American Heart Association Guidelines for Cardiopulmonary Resuscitation and Emergency Cardiovascular Care. Pediatrics 2021, 147, e2020038505E. [Google Scholar] [CrossRef] [PubMed]
- Yamada, N.K.; Yaeger, K.A.; Halamek, L.P. Analysis and classification of errors made by teams during neonatal resuscitation. Resuscitation 2015, 96, 109–113. [Google Scholar] [CrossRef]
- Madar, J.; Roehr, C.C.; Ainsworth, S.; Ersdal, H.; Morley, C.; Rüdiger, M.; Skåre, C.; Szczapa, T.; Te Pas, A.; Trevisanuto, D.; et al. European Resuscitation Council Guidelines 2021: Newborn resuscitation and support of transition of infants at birth. Resuscitation 2021, 161, 291–326. [Google Scholar] [CrossRef] [PubMed]
- England, N. Human Factors in Healthcare: A Concordat from the National Quality Board (2013). 2016. Available online: https://www.england.nhs.uk/wp-content/uploads/2013/11/nqb-hum-fact-concord.pdf (accessed on 23 March 2023).
- American Academy of Pediatrics. Available online: https://services.aap.org/en/learning/neonatal-resuscitation-program/nrp-research-grant-and-young-investigator-award-program/ (accessed on 23 March 2023).
- Brennan, P.A.; Holden, C.; Shaw, G.; Morris, S.; Oeppen, R.S. Leading article: What can we do to improve individual and team situational awareness to benefit patient safety? Br. J. Oral. Maxillofac. Surg. 2020, 58, 404–408. [Google Scholar] [CrossRef]
- Law, B.H.Y.; Cheung, P.Y.; Wagner, M.; van Os, S.; Zheng, B.; Schmölzer, G. Analysis of neonatal resuscitation using eye tracking: A pilot study. Arch. Dis. Child. Fetal Neonatal Ed. 2018, 103, F82–F84. [Google Scholar] [CrossRef]
- Law, B.H.Y.; Schmölzer, G.M. Analysis of visual attention and team communications during neonatal endotracheal intubations using eye-tracking: An observational study. Resuscitation 2020, 153, 176–182. [Google Scholar] [CrossRef] [PubMed]
- Wagner, M.; Gröpel, P.; Bibl, K.; Olischar, M.; Auerbach, M.A.; Gross, I.T. Eye-tracking during simulation-based neonatal airway management. Pediatr. Res. 2020, 87, 518–522. [Google Scholar] [CrossRef]
- Herrick, H.; Weinberg, D.; Cecarelli, C.; Fishman, C.E.; Newman, H.; den Boer, M.C.; Martherus, T.; Katz, T.A.; Nadkarni, V.; Te Pas, A.B.; et al. Provider visual attention on a respiratory function monitor during neonatal resuscitation. Arch. Dis. Child. Fetal Neonatal Ed. 2020, 105, 666–668. [Google Scholar] [CrossRef]
- Katz, T.A.; Weinberg, D.D.; Fishman, C.E.; Nadkarni, V.; Tremoulet, P.; Te Pas, A.B.; Sarcevic, A.; Foglia, E.E. Visual attention on a respiratory function monitor during simulated neonatal resuscitation: An eye-tracking study. Arch. Dis. Child. Fetal Neonatal Ed. 2019, 104, F259–F264. [Google Scholar] [CrossRef]
- Wagner, M.; Gröpel, P.; Eibensteiner, F.; Kessler, L.; Bibl, K.; Gross, I.T.; Berger, A.; Cardona, F.S. Visual attention during pediatric resuscitation with feedback devices: A randomized simulation study. Pediatr. Res. 2022, 91, 1762–1768. [Google Scholar] [CrossRef] [PubMed]
- Ashraf, H.; Sodergren, M.H.; Merali, N.; Mylonas, G.; Singh, H.; Darzi, A. Eye-tracking technology in medical education: A systematic review. Med. Teach. 2018, 40, 62–69. [Google Scholar] [CrossRef] [PubMed]
- Stuart, S.; Alcock, L.; Godfrey, A.; Lord, S.; Rochester, L.; Galna, B. Accuracy and re-test reliability of mobile eye-tracking in Parkinson’s disease and older adults. Med. Eng. Phys. 2016, 38, 308–315. [Google Scholar] [CrossRef] [PubMed]
- Merali, N.; Veeramootoo, D.; Singh, S. Eye-Tracking Technology in Surgical Training. J. Investig. Surg. 2019, 32, 587–593. [Google Scholar] [CrossRef]
- Henneman, E.A.; Marquard, J.L.; Fisher, D.L.; Gawlinski, A. Eye Tracking: A Novel Approach for Evaluating and Improving the Safety of Healthcare Processes in the Simulated Setting. Simul. Healthcare 2017, 12, 51–56. [Google Scholar] [CrossRef]
- Anzalone, A.J.; Blueitt, D.; Case, T.; McGuffin, T.; Pollard, K.; Garrison, J.C.; Jones, M.T.; Pavur, R.; Turner, S.; Oliver, J.M. A Positive Vestibular/Ocular Motor Screening (VOMS) Is Associated with Increased Recovery Time After Sports-Related Concussion in Youth and Adolescent Athletes. Am. J. Sports Med. 2017, 45, 474–479. [Google Scholar] [CrossRef]
- Stuart, S.; Hickey, A.; Vitorio, R.; Welman, K.; Foo, S.; Keen, D.; Godfrey, A. Eye-tracker algorithms to detect saccades during static and dynamic tasks: A structured review. Physiol. Meas. 2019, 40, 02TR01. [Google Scholar] [CrossRef]
- Skåre, C.; Calisch, T.E.; Saeter, E.; Rajka, T.; Boldingh, A.M.; Nakstad, B.; Niles, D.E.; Kramer-Johansen, J.; Olasveengen, T.M. Implementation and effectiveness of a video-based debriefing programme for neonatal resuscitation. Acta Anaesthesiol. Scand. 2018, 62, 394–403. [Google Scholar] [CrossRef]
- Sawyer, T.; Leonard, D.; Sierocka-Castaneda, A.; Chan, D.; Thompson, M. Correlations between technical skills and behavioral skills in simulated neonatal resuscitations. J. Perinatol. 2014, 34, 781–786. [Google Scholar] [CrossRef]
- Litke-Wager, C.; Delaney, H.; Mu, T.; Sawyer, T. Impact of Task-Oriented Role Assignment on Neonatal Resuscitation Performance: A Simulation-Based Randomized Controlled Trial. Am. J. Perinatol. 2021, 38, 914–921. [Google Scholar] [CrossRef]
- Hart, S. Development of NASA-TLX: Results of empirical and theoretical research. Adv. Psychol. 1988, 52, 139–183. [Google Scholar]
- Hoonakker, P.; Carayon, P.; Gurses, A.; Brown, R.; McGuire, K.; Khunlertkit, A.; Walker, J.M. Measuring workload of icu nurses with a questionnaire survey: The NASA task load index (TLX). IIE Trans. Heal. Syst. Eng. 2011, 1, 131–143. [Google Scholar] [CrossRef] [PubMed]
- Hart, S.G. NASA-task load index (NASA-TLX); 20 years later. Proc. Hum. Factors Ergon. Soc. Annu. Meet. 2006, 50, 904–908. [Google Scholar] [CrossRef]
- Said, S.; Gozdzik, M.; Roche, T.R.; Braun, J.; Rössler, J.; Kaserer, A.; Spahn, D.R.; Nöthiger, C.B.; Tscholl, D.W. Validation of the Raw National Aeronautics and Space Administration Task Load Index (NASA-TLX) Questionnaire to Assess Perceived Workload in Patient Monitoring Tasks: Pooled Analysis Study Using Mixed Models. J. Med. Internet Res. 2020, 22, e19472. [Google Scholar] [CrossRef] [PubMed]
- Currie, J.; Bond, R.R.; McCullagh, P.; Black, P.; Finlay, D.D.; Peace, A. Eye Tracking the Visual Attention of Nurses Interpreting Simulated Vital Signs Scenarios: Mining Metrics to Discriminate Between Performance Level. IEEE Trans. Hum.-Mach. Syst. 2018, 48, 113–124. [Google Scholar] [CrossRef]
- Iyasere, C.A.; Wing, J.; Martel, J.N.; Healy, M.G.; Park, Y.S.; Finn, K.M. Effect of Increased Interprofessional Familiarity on Team Performance, Communication, and Psychological Safety on Inpatient Medical Teams: A Randomized Clinical Trial. JAMA Intern. Med. 2022, 182, 1190–1198. [Google Scholar] [CrossRef]
- Neveln, N.K.; Khattab, M.; Hagan, J.L.; Fortunov, R.M.; Sundgren, N.C. A recorder/time coach decreases time errors during neonatal resuscitation: A randomized, simulation-based clinical trial. Resusc. Plus 2023, 15, 100411. [Google Scholar] [CrossRef] [PubMed]
- Henneman, E.A.; Cunningham, H.; Fisher, D.L.; Plotkin, K.; Nathanson, B.H.; Roche, J.P.; Marquard, J.L.; Reilly, C.A.; Henneman, P.L. Eye tracking as a debriefing mechanism in the simulated setting improves patient safety practices. Dimens. Crit. Care Nurs. 2014, 33, 129–135. [Google Scholar] [CrossRef]
- Kessler, L.; Gröpel, P.; Aichner, H.; Aspalter, G.; Kuster, L.; Schmölzer, G.M.; Berger, A.; Wagner, M.; Simma, B. Eye-tracking during simulated endotracheal newborn intubation: A prospective, observational multi-center study. Pediatr. Res. 2023, 94, 443–449. [Google Scholar] [CrossRef]
Training Level/Nurse Band (Level) | Neonatal Experience (Years) | Times Completed NLS Course (Number) | Completed NLS within Last 1 Year (Yes/No) | Prior Neonatal Simulation Experiences (Number) | Prior Neonatal Resuscitation Real Life Experiences (Number) |
---|---|---|---|---|---|
Senior trainee-1 | 1.5 | 2 | No | 6 | >30 |
Senior trainee-2 | 3 | 1 | No | 2 | >30 |
Senior trainee-3 | 3 | 2 | No | 2 | >30 |
Senior trainee-4 | 2 | 2 | No | 6 | >30 |
Senior trainee-5 | 2.5 | 2 | Yes | 6 | >30 |
Senior Nurse practioners-1 | 20 | 5 | No | >5 | >30 |
Junior trainee-1 | 1 | 1 | No | 1 | >20 |
Junior Nurse practioners-1 | 2 | 3 | No | <5 | 3 |
Junior trainee-2 | <1 year | 1 | No | 2 | 0 |
Senior Nurse-1 | 11 | 2 | No | 0 | 4 |
Senior Nurse-2 | 7 | 2 | Yes | <5 | >10 |
Senior Nurse-3 | 9 | 2 | No | <10 | 20–30 |
Participant 1 | Participant 2 | Participant 3 | |||||
---|---|---|---|---|---|---|---|
No Team Leader | Team Leader | No Team Leader | Team Leader | No Team Leader | Team Leader | Wilcoxon Sign Rank Test (p Value) | |
Number of Fixations | |||||||
| 51 | 27 | 43 | 23 | 39 | 30 | 0.109 |
| 23 | 20 | 19 | 10 | 20 | 12 | 0.109 |
| 38 | 34 | 29 | 16 | 42 | 39 | 0.109 |
Fixation Time in seconds on specific AOI (=dwell time) | |||||||
| 227 | 151 | 244 | 354 | 218 | 155 | 1.000 |
| 42 | 34 | 50 | 18 | 53 | 15 | 0.109 |
| 104 | 279 | 110 | 22 | 291 | 188 | 1.000 |
Assessor-1 | Assessor-2 | |||||
---|---|---|---|---|---|---|
Team Leader | No Team Leader | Comparison within Groups p Value (Paired t Test) | Team Leader | No Team Leader | Comparison within Groups p Value (Paired t Test) | |
NRPE Decision | 100 ± 0 | 100 ± 0 | NA | 81.9 ± 3.290 | 100 ± 0 | 0.010 |
NRPE Technique scores | 95.3 ± 4.04 | 93.33 ± 0.577 | 0.48 | 98 ± 3.46 | 90.33 ± 2.3 | 0.148 |
NRPE total scores | 96.67 ± 2.89 | 95 ± 0 | 0.423 | 94 ± 1.73 | 93 ± 1.73 | 0.667 |
BAT Scores | 40.33 ± 3.21 | 39.66 ± 3.05 | 0.85 | 34.67 ± 8.144 | 34.66 ± 5.03 | 1.00 |
All the values are reported in Mean ± standard deviation |
Team Leader | No Team Leader | Wilcoxon Sign Rank Test (p Value) | |
---|---|---|---|
All Participants (TL = 12; No TL = 9) | |||
Mental Demand | 12.5 (9.5–16.5) | 15 (14–17) | 0.211 |
Physical demand | 5 (2.5–9) | 10 (5–12) | 0.039 |
Temporal demand | 9.5 (5–15.5) | 14 (5–16) | 0.898 |
Performance | 5 (4–8) | 7 (5–10) | 0.543 |
Effort | 10.5 (5–15) | 13 (5–15) | 0.637 |
Frustration level | 3.5 (1.5–7.5) | 5 (4–12) | 0.508 |
Nurse (TL = 3; No TL = 3) | |||
Mental Demand | 10 (10–16) | 15 (14–18) | 0.500 |
Physical demand | 9 (2–10) | 10 (3–16) | 0.500 |
Temporal demand | 16 (4–16) | 16 (14–16) | 1.000 |
Performance | 13 (5–14) | 9 (5–14) | 1.000 |
Effort | 11 (10–13) | 15 (13–16) | 0.250 |
Frustration level | 15 (3–16) | 9 (5–12) | 0.750 |
Junior trainee (TL = 5; No TL = 3) | |||
Mental Demand | 14 (9–14) | 16 (13–17) | 0.75 |
Physical demand | 5 (2–5) | 9 (4–12) | 0.250 |
Temporal demand | 8 (5–9) | 5 (3–16) | 1.000 |
Performance | 5 (2–6) | 3 (2–7) | 0.750 |
Effort | 9 (4–16) | 5 (2–12) | 1.000 |
Frustration level | 2 (1–4) | 4 (3–17) | 0.500 |
Senior trainee (TL = 4; No TL = 3) | |||
Mental Demand | 14 (8–17.5) | 15 (9–17) | 1.00 |
Physical demand | 5 (4–7) | 11 (5–15) | 0.500 |
Temporal demand | 12 (7.5–14.5) | 9 (5–15) | 0.750 |
Performance | 4.5 (4–5) | 10 (5–16) | 0.250 |
Effort | 10 (5–15) | 14 (5–16) | 1.000 |
Frustration level | 3.5 (1.5–7.5) | 4 (1–17) | 0.750 |
All the domains were score from 0–20, with 20 being the worst and ‘0’ being the best | |||
All the data presented in Median with Inter-quartile range. |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Kannan Loganathan, P.; Garg, A.; McNicol, R.; Wall, C.; Pointon, M.; McMeekin, P.; Godfrey, A.; Wagner, M.; Roehr, C.C. Assessment of Visual Attention in Teams with or without Dedicated Team Leaders: A Neonatal Simulation-Based Pilot Randomised Cross-Over Trial Utilising Low-Cost Eye-Tracking Technology. Children 2024, 11, 1023. https://doi.org/10.3390/children11081023
Kannan Loganathan P, Garg A, McNicol R, Wall C, Pointon M, McMeekin P, Godfrey A, Wagner M, Roehr CC. Assessment of Visual Attention in Teams with or without Dedicated Team Leaders: A Neonatal Simulation-Based Pilot Randomised Cross-Over Trial Utilising Low-Cost Eye-Tracking Technology. Children. 2024; 11(8):1023. https://doi.org/10.3390/children11081023
Chicago/Turabian StyleKannan Loganathan, Prakash, Anip Garg, Robert McNicol, Conor Wall, Matthew Pointon, Peter McMeekin, Alan Godfrey, Michael Wagner, and Charles Christoph Roehr. 2024. "Assessment of Visual Attention in Teams with or without Dedicated Team Leaders: A Neonatal Simulation-Based Pilot Randomised Cross-Over Trial Utilising Low-Cost Eye-Tracking Technology" Children 11, no. 8: 1023. https://doi.org/10.3390/children11081023