Quantifying the Remote Driver’s Interaction with 5G-Enabled Level 4 Automated Vehicles: A Real-World Study
Abstract
:1. Introduction
1.1. Related Work and Research Gaps
- There has been limited research quantifying remote driver behaviour and performance in real-world scenarios, limiting the understanding of how remote drivers interact with automated vehicles in practical situations in the real-world settings;
- There has been limited research exploring the factors potentially affecting remote drivers’ performance and behaviour, such as distractions and the implications of out-of-loop driving; understanding how these factors impact remote operation is crucial for developing effective teleoperation systems.
1.2. Research Aim
- To propose a new methodology to quantify and assess remote drivers’ performance and behaviour when operating automated vehicles remotely in real-world settings;
- To explore remote drivers’ attention and behaviour when interacting with the 5G enabled Level 4 automated vehicles in real-world conditions, with a particular focus on investigating the effect of mental disengagement on remote drivers’ takeover performance and behaviour in Level 4 automated vehicles.
2. Materials and Methods
2.1. Level 4 Automated Vehicle
2.2. The Trail Route and Outline of the Trial
2.3. Study Overview
2.4. Experimental Design
- Baseline Condition–Monitoring driving (constantly monitoring the automated system driving);
- Experimental Condition–Disengaged (disengaged from monitoring the automated system driving).
Measurements
3. Results
3.1. Motor Readiness Time
3.2. Decision-Making Time
3.3. Remote Driver’s Visual Attention While Interacting with the 5G CAL
4. Discussion
5. Conclusions and Future Work
- Compared to constantly monitoring driving, mental disengagement (achieved by distraction via a reading task on a tablet) leads to slowed motor readiness time from the remote driver when required by the Level 4 automated driving system to step in, with a difference of 5.309 s, which highlights the risks associated with divided attention during remote driving;
- Compared to constantly monitoring driving, mental disengagement leads to slowed decision-making time from the remote driver when required by the Level 4 automated driving system to step in and make decision, with a difference of 4.232 s, emphasising the detrimental effect of distraction on the timely intervention of remote drivers;
- Compared to constantly monitoring driving, mental disengagement was found to shift the remote driver’s attention away from the road, the diminished focus from which compromises the driver’s ability to maintain situational awareness;
- Compared to constantly monitoring driving, mental disengagement leads to increased cognitive workload for the remote driver;
- When the remote driver is controlling the vehicle remotely via the teleportation system, it resulted in higher cognitive workload compared to the “monitoring” and “disengagement” conditions.
Author Contributions
Funding
Institutional Review Board Statement
Data Availability Statement
Conflicts of Interest
References
- HM Government. Connected & Automated Mobility 2025: Realising the Benefits of Self-Driving Vehicles in the UK. 2022. Available online: https://assets.publishing.service.gov.uk/media/62ff438c8fa8f504cdec92df/cam-2025-realising-benefits-self-driving-vehicles.pdf (accessed on 1 October 2024).
- Tyers, R. Automated Vehicles Bill [HL] 2023–24. Parliament. 2024. Available online: https://commonslibrary.parliament.uk/research-briefings/cbp-9973/ (accessed on 1 October 2024).
- Dichabeng, P.; Merat, N.; Markkula, G. Research Journeys to Net Zero; Routledge: London, UK, 2024; pp. 148–159. [Google Scholar]
- Bhardwaj, C.; Axsen, J.; Crawford, C. Simulating long-term emissions from private automated vehicles under climate policies. Transp. Res. D Trans. Environ. 2023, 118, 103665. [Google Scholar] [CrossRef]
- Taiebat, M.; Brown, A.L.; Safford, H.R.; Qu, S.; Xu, M. A review on energy, environmental, and sustainability implications of connected and automated vehicles. Environ. Sci. Technol. 2018, 52, 11449–11465. [Google Scholar] [CrossRef]
- Jiang, H.; Hu, J.; An, S.; Wang, M.; Park, B.B. Eco approaching at an isolated signalized intersection under partially connected and automated vehicles environment. Transp. Res. Part C Emerg. Technol. 2017, 79, 290–307. [Google Scholar] [CrossRef]
- Tsakalidis, A.; Gkoumas, K.; Pekár, F. Digital transformation supporting transport decarbonisation: Technological developments in EU-funded research and innovation. Sustainability 2020, 12, 3762. [Google Scholar] [CrossRef]
- Campbell, M.; Egerstedt, M.; How, J.P.; Murray, R.M. Autonomous driving in urban environments: Approaches, lessons and challenges. Philos. Trans. R. Soc. A Math. Phys. Eng. Sci. 2010, 368, 4649–4672. [Google Scholar] [CrossRef]
- Lee, D.; Hess, D.J. Regulations for on-road testing of connected and automated vehicles: Assessing the potential for global safety harmonization. Transp. Res. A Policy Pract. 2020, 136, 85–98. [Google Scholar] [CrossRef]
- Sohrabi, S.; Khodadadi, A.; Mousavi, S.M.; Dadashova, B.; Lord, D. Quantifying the automated vehicle safety performance: A scoping review of the literature, evaluation of methods, and directions for future research. Accid. Anal. Prev. 2021, 152, 106003. [Google Scholar] [CrossRef]
- Haghi, A.; Ketabi, D.; Ghanbari, M.; Rajabi, H. Assessment of human errors in driving accidents; analysis of the causes based on aberrant behaviors. Life Sci. J. 2014, 11, 414–420. [Google Scholar]
- Wang, L.; Zhong, H.; Ma, W.; Abdel-Aty, M.; Park, J. How many crashes can connected vehicle and automated vehicle technologies prevent: A meta-analysis. Accid. Anal. Prev. 2020, 136, 105299. [Google Scholar] [CrossRef]
- Khayyam, H.; Javadi, B.; Jalili, M.; Jazar, R.N. Artificial intelligence and internet of things for autonomous vehicles. In Nonlinear Approaches in Engineering Applications: Automotive Applications of Engineering Problems; Springer: Berlin/Heidelberg, Germany, 2020; pp. 39–68. [Google Scholar]
- Bathla, G.; Bhadane, K.; Singh, R.K.; Kumar, R.; Aluvalu, R.; Krishnamurthi, R.; Kumar, A.; Thakur, R.N.; Basheer, S. Autonomous vehicles and intelligent automation: Applications, challenges, and opportunities. Mob. Inf. Syst. 2022, 2022, 7632892. [Google Scholar] [CrossRef]
- Fu, Y.; Li, C.; Yu, F.R.; Luan, T.H.; Zhang, Y. A survey of driving safety with sensing, vehicular communications, and artificial intelligence-based collision avoidance. IEEE Trans. Intell. Transp. Syst. 2021, 23, 6142–6163. [Google Scholar] [CrossRef]
- Aria, E.; Olstam, J.; Schwietering, C. Investigation of automated vehicle effects on driver’s behavior and traffic performance. Transp. Res. Proc. 2016, 15, 761–770. [Google Scholar] [CrossRef]
- Festag, A. Cooperative intelligent transport systems standards in Europe. IEEE Commun. Mag. 2014, 52, 166–172. [Google Scholar] [CrossRef]
- Cohen, T.; Cavoli, C. Automated vehicles: Exploring possible consequences of government (non) intervention for congestion and accessibility. Transp. Rev. 2019, 39, 129–151. [Google Scholar] [CrossRef]
- Gruyer, D.; Orfila, O.; Glaser, S.; Hedhli, A.; Hautière, N.; Rakotonirainy, A. Are connected and automated vehicles the silver bullet for future transportation challenges? Benefits and weaknesses on safety, consumption, and traffic congestion. Front. Sustain. Cities 2021, 2, 607054. [Google Scholar] [CrossRef]
- Li, S.; Blythe, P.; Zhang, Y.; Edwards, S.; Xing, J.; Guo, W.; Ji, Y.; Goodman, P.; Namdeo, A. Should older people be considered a homogeneous group when interacting with level 3 automated vehicles? Transp. Res. Part F Traffic Psychol. Behav. 2021, 78, 446–465. [Google Scholar] [CrossRef]
- Li, S.; Blythe, P.; Guo, W.; Namdeo, A.; Edwards, S.; Goodman, P.; Hill, G. Evaluation of the effects of age-friendly human-machine interfaces on the driver’s takeover performance in highly automated vehicles. Transp. Res. Part F Traffic Psychol. Behav. 2019, 67, 78–100. [Google Scholar] [CrossRef]
- Voinescu, A.; Morgan, P.L.; Alford, C.; Caleb-Solly, P. The utility of psychological measures in evaluating perceived usability of automated vehicle interfaces–A study with older adults. Transp. Res. Part F Traffic Psychol. Behav. 2020, 72, 244–263. [Google Scholar] [CrossRef]
- Robertson, R.D.; Woods-Fry, H.; Vanlaar, W.G.; Hing, M.M. Automated vehicles and older drivers in Canada. J. Saf. Res. 2019, 70, 193–199. [Google Scholar] [CrossRef]
- Young, K.L.; Koppel, S.; Charlton, J.L. Toward best practice in Human Machine Interface design for older drivers: A review of current design guidelines. Accid. Anal. Prev. 2017, 106, 460–467. [Google Scholar] [CrossRef]
- Li, S.; Blythe, P.; Guo, W.; Namdeo, A. Investigation of older drivers’ requirements of the human-machine interaction in highly automated vehicles. Transp. Res. Part F Traffic Psychol. Behav. 2019, 62, 546–563. [Google Scholar] [CrossRef]
- Li, S.; Zhang, Y.; Blythe, P.; Edwards, S.; Ji, Y. Remote driving as the Failsafe: Qualitative investigation of Users’ perceptions and requirements towards the 5G-enabled Level 4 automated vehicles. Transp. Res. Part F Traffic Psychol. Behav. 2024, 100, 211–230. [Google Scholar] [CrossRef]
- Li, S.; Zhang, Y.; Edwards, S.; Blythe, P.T. Exploration into the needs and requirements of the remote driver when teleoperating the 5G-enabled level 4 automated vehicle in the real world—A case study of 5G connected and automated logistics. Sensors 2023, 23, 820. [Google Scholar] [CrossRef]
- J3016_202104; Taxonomy and Definitions for Terms Related to Driving Automation Systems for On Road Motor Vehicles. Society of Automotive Engineers: Warrendale, PA, USA, 2016.
- Shladover, S.E. Connected and automated vehicle systems: Introduction and overview. J. Intell. Transp. Syst. 2018, 22, 190–200. [Google Scholar] [CrossRef]
- Soteropoulos, A.; Mitteregger, M.; Berger, M.; Zwirchmayr, J. Automated drivability: Toward an assessment of the spatial deployment of level 4 automated vehicles. Transp. Res. A Policy Pract. 2020, 136, 64–84. [Google Scholar] [CrossRef]
- Li, S.; Blythe, P.; Guo, W.; Namdeo, A. Investigation of older driver’s takeover performance in highly automated vehicles in adverse weather conditions. IET Intell. Transp. Syst. 2018, 12, 1157–1165. [Google Scholar] [CrossRef]
- Vom Dorff, S.; Böddeker, B.; Kneissl, M.; Fränzle, M. A fail-safe architecture for automated driving. In Proceedings of the Design, Automation & Test in Europe Conference & Exhibition, Grenoble, France, 9–13 March 2020; IEEE: New York, NY, USA, 2020; pp. 828–833. [Google Scholar]
- Kettwich, C.; Schrank, A.; Oehl, M. Teleoperation of highly automated vehicles in public transport: User-centered design of a human-machine interface for remote-operation and its expert usability evaluation. Multimodal Technol. Interact. 2021, 5, 26. [Google Scholar] [CrossRef]
- Goodall, N. Non-technological challenges for the remote operation of automated vehicles. Transp. Res. A Policy Pract. 2020, 142, 14–26. [Google Scholar] [CrossRef]
- Majstorović, D.; Hoffmann, S.; Pfab, F.; Schimpe, A.; Wolf, M.-M.; Diermeyer, F. Survey on Teleoperation Concepts for Automated Vehicles. In Proceedings of the IEEE International Conference on Systems, Man, and Cybernetics (SMC), Prague, Czech Republic, 9–12 October 2022; pp. 1290–1296. [Google Scholar]
- Bogdoll, D.; Orf, S.; Töttel, L.; Zöllner, J.M. Taxonomy and Survey on Remote Human Input Systems for Driving Automation Systems. In Proceedings of the Future of Information and Communication Conference, San Francisco, CA, USA, 3–4 March 2022; pp. 94–108. [Google Scholar]
- Zulqarnain, S.Q.; Lee, S. Selecting Remote Driving Locations for Latency Sensitive Reliable Tele-Operation. Appl. Sci. 2021, 11, 21. [Google Scholar] [CrossRef]
- Charness, G.; Gneezy, U.; Kuhn, M.A. Experimental methods: Between-subject and within-subject design. J. Econ. Behav. Organ. 2012, 81, 1–8. [Google Scholar] [CrossRef]
- Li, S.; Blythe, P.; Zhang, Y.; Edwards, S.; Guo, W.; Ji, Y.; Goodman, P.; Hill, G.; Namdeo, A. Analysing the effect of gender on the human–machine interaction in level 3 automated vehicles. Sci. Rep. 2022, 12, 11645. [Google Scholar] [CrossRef] [PubMed]
- Stevens, A.; Quimby, A.; Board, A.; Kersloot, T.; Burns, P. Design Guidelines for Safety of In-Vehicle Information Systems; Project report PA3721/01; TRL Limited: Berks, UK, 2002. [Google Scholar]
- Gold, C.; Happee, R.; Bengler, K. Modeling take-over performance in level 3 conditionally automated vehicles. Accid. Anal. Prev. 2018, 116, 3–13. [Google Scholar] [CrossRef]
- Zhang, B.; de Winter, J.; Varotto, S.; Happee, R.; Martens, M. Determinants of take-over time from automated driving: A meta-analysis of 129 studies. Transp. Res. Part F Traffic Psychol. Behav. 2019, 64, 285–307. [Google Scholar] [CrossRef]
- Zeeb, K.; Buchner, A.; Schrauf, M. Is take-over time all that matters? The impact of visual-cognitive load on driver take-over quality after conditionally automated driving. Accid. Anal. Prev. 2016, 92, 230–239. [Google Scholar] [CrossRef]
- Körber, M.; Gold, C.; Lechner, D.; Bengler, K. The influence of age on the take-over of vehicle control in highly automated driving. Transp. Res. Part F Traffic Psychol. Behav. 2016, 39, 19–32. [Google Scholar] [CrossRef]
- Niehorster, D.C.; Hessels, R.S.; Benjamins, J.S. GlassesViewer: Open-source software for viewing and analyzing data from the Tobii Pro Glasses 2 eye tracker. Behav. Res. Methods 2020, 52, 1244–1253. [Google Scholar] [CrossRef]
- Luan, X.; Zhao, B.; Liu, D.; Liu, X.; Tong, X.; Li, K. A lightweight heatmap-based eye tracking system. In Proceedings of the International Conference on Computer Communications and Networks (ICCCN), Athens, Greece, 19–22 July 2021; pp. 1–9. [Google Scholar]
- Menges, R.; Kramer, S.; Hill, S.; Nisslmueller, M.; Kumar, C.; Staab, S. A visualization tool for eye tracking data analysis in the web. In Proceedings of the ACM Symposium on Eye Tracking Research and Applications, Palm Beach Gardens, FL, USA, 6–8 November 2000; pp. 1–5. [Google Scholar]
- Skaramagkas, V.; Giannakakis, G.; Ktistakis, E.; Manousos, D.; Karatzanis, I.; Tachos, N.; Tripoliti, E.E.; Marias, K.; Fotiadis, D.I.; Tsiknakis, M. Review of eye tracking metrics involved in emotional and cognitive processes. IEEE Rev. Biomed. Eng. 2021, 16, 260–277. [Google Scholar] [CrossRef]
- Marquart, G.; Cabrall, C.; De Winter, J. Review of eye-related measures of drivers’ mental workload. Procedia Manuf. 2015, 3, 2854–2861. [Google Scholar] [CrossRef]
- Lohani, M.; Payne, B.R.; Strayer, D.L. Review of psychophysiological measures to assess cognitive states in real-world driving. Front. Hum. Neurosci. 2019, 13, 57. [Google Scholar] [CrossRef]
- Underwood, G. Visual attention and the transition from novice to advanced driver. Ergonomics 2007, 50, 1235–1249. [Google Scholar] [CrossRef]
- Lazaro, M.J.; Yun, M.H.; Kim, S. Stress-level and attentional functions of experienced and novice young adult drivers in intersection-related hazard situations. Int. J. Ind. Ergon. 2022, 90, 103315. [Google Scholar] [CrossRef]
- Radlmayr, J.; Gold, C.; Lorenz, L.; Farid, M.; Bengler, K. How traffic situations and non-driving related tasks affect the take-over quality in highly automated driving. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting, Adelaide, Australia, 17–19 November 2014; Volume 58, pp. 2063–2067. [Google Scholar]
- Eriksson, A.; Stanton, N.A. Takeover time in highly automated vehicles: Noncritical transitions to and from manual control. Hum. Factors 2017, 59, 689–705. [Google Scholar] [CrossRef]
- Li, S.; Blythe, P.; Guo, W.; Namdeo, A. Investigating the effects of age and disengagement in driving on driver’s takeover control performance in highly automated vehicles. Transp. Plan. Technol. 2019, 42, 470–497. [Google Scholar] [CrossRef]
- Stanton, N.A.; Chambers, P.R.; Piggott, J. Situational awareness and safety. Saf. Sci. 2001, 39, 189–204. [Google Scholar] [CrossRef]
- Endsley, M.R. Measurement of situation awareness in dynamic systems. Hum. Factors 1995, 37, 65–84. [Google Scholar] [CrossRef]
- Gugerty, L.J. Situation awareness during driving: Explicit and implicit knowledge in dynamic spatial memory. J. Exp. Psychol. Appl. 1997, 3, 42. [Google Scholar] [CrossRef]
- Cunningham, M.L.; Regan, M.A. Driver distraction and inattention in the realm of automated driving. IET Intell. Transp. Syst. 2018, 12, 407–413. [Google Scholar] [CrossRef]
- Horberry, T.; Anderson, J.; Regan, M.A.; Triggs, T.J.; Brown, J. Driver distraction: The effects of concurrent in-vehicle tasks, road environment complexity and age on driving performance. Accid. Anal. Prev. 2006, 38, 185–191. [Google Scholar] [CrossRef]
- Bellet, T.; Cunneen, M.; Mullins, M.; Murphy, F.; Pütz, F.; Spickermann, F.; Braendle, C.; Baumann, M.F. From semi to fully autonomous vehicles: New emerging risks and ethico-legal challenges for human-machine interactions. Transp. Res. Part F Traffic Psychol. Behav. 2019, 63, 153–164. [Google Scholar] [CrossRef]
- Endsley, M.R. Toward a theory of situation awareness in dynamic systems. Hum. Factors 1995, 37, 32–64. [Google Scholar] [CrossRef]
- Strayer, D.L.; Watson, J.M.; Drews, F.A. Cognitive distraction while multitasking in the automobile. Psychol. Learn. Motiv. 2011, 54, 29–58. [Google Scholar] [CrossRef]
- Zhang, Y.; Li, S.; Blythe, P.; Edwards, S.; Guo, W.; Ji, Y.; Xing, J.; Goodman, P.; Hill, G. Attention Pedestrians Ahead: Evaluating User Acceptance and Perceptions of a Cooperative Intelligent Transportation System-Warning System for Pedestrians. Sustainability 2022, 14, 2787. [Google Scholar] [CrossRef]
- Wiseman, Y. Autonomous vehicles will spur moving budget from railroads to roads. Int. J. Intell. Unmanned Syst. 2024, 12, 19–31. [Google Scholar] [CrossRef]
- Nijboer, M.; Borst, J.P.; Van Rijn, H.; Taatgen, N.A. Driving and multitasking: The good, the bad, and the dangerous. Front. Psychol. 2016, 7, 1718. [Google Scholar] [CrossRef] [PubMed]
- Neyens, D.M.; Boyle, L.N. The effect of distractions on the crash types of teenage drivers. Accid. Anal. Prev. 2007, 39, 206–212. [Google Scholar] [CrossRef] [PubMed]
- Goh, K.N.; Chen, Y.; Lai, F.; Daud, S.; Sivaji, A.; Soo, S. A Comparison of Usability Testing Methods for an E-Commerce Website: A Case Study on a Malaysia Online Gift Shop. In Proceedings of the 2013 10th International Conference on Information Technology: New Generations, Las Vegas, NV, USA, 15–17 April 2013; pp. 143–150. [Google Scholar]
- Negi, S.; Mitra, R. Fixation duration and the learning process: An eye tracking study with subtitled videos. J. Eye Mov. Res. 2020, 13. [Google Scholar] [CrossRef]
- Naeeri, S.; Kang, Z.; Mandal, S.; Kim, K. Multimodal analysis of eye movements and fatigue in a simulated glass cockpit environment. Aerospace 2021, 8, 283. [Google Scholar] [CrossRef]
- Vreeswijk, J.; Habibovic, A.; Madland, O.; Hooft, F. Remote Support for Automated Vehicle Operations. In Road Vehicle Automation 9; Meyer, G., Beiker, S., Eds.; Springer International Publishing: New York, NY, USA, 2022; pp. 149–164. [Google Scholar]
- Zhang, T. Toward automated vehicle teleoperation: Vision, opportunities, and challenges. IEEE Internet Things J. 2020, 7, 11347–11354. [Google Scholar] [CrossRef]
- GOV.UK. Policy Paper UK Wireless Infrastructure Strategy, Department for Science, Innovation & Technology. 2023. Available online: https://www.gov.uk/government/publications/uk-wireless-infrastructure-strategy/uk-wireless-infrastructure-strategy (accessed on 1 October 2024).
- Naujoks, F.; Mai, C.; Neukum, A. The effect of urgency of take-over requests during highly automated driving under distraction conditions. Adv. Hum. Asp. Transp. 2014, 7, 431. [Google Scholar] [CrossRef]
- Liang, N.; Yang, J.; Yu, D.; Prakah-Asante, K.O.; Curry, R.; Blommer, M.; Swaminathan, R.; Pitts, B.J. Using eye-tracking to investigate the effects of pre-takeover visual engagement on situation awareness during automated driving. Accid. Anal. Prev. 2021, 157, 106143. [Google Scholar] [CrossRef]
Measurements | Data Type | Unit |
---|---|---|
Motor readiness time | Continuous | Seconds |
Decision-making time | Continuous | Seconds |
Vitalization of gaze behaviour | Nominal | N/A |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Li, S.; Zhang, Y.; Edwards, S.; Blythe, P. Quantifying the Remote Driver’s Interaction with 5G-Enabled Level 4 Automated Vehicles: A Real-World Study. Electronics 2024, 13, 4366. https://doi.org/10.3390/electronics13224366
Li S, Zhang Y, Edwards S, Blythe P. Quantifying the Remote Driver’s Interaction with 5G-Enabled Level 4 Automated Vehicles: A Real-World Study. Electronics. 2024; 13(22):4366. https://doi.org/10.3390/electronics13224366
Chicago/Turabian StyleLi, Shuo, Yanghanzi Zhang, Simon Edwards, and Phil Blythe. 2024. "Quantifying the Remote Driver’s Interaction with 5G-Enabled Level 4 Automated Vehicles: A Real-World Study" Electronics 13, no. 22: 4366. https://doi.org/10.3390/electronics13224366