HMD-Based VR Tool for Traffic Psychological Examination: Conceptualization and Design Proposition

Abstract

In the present theoretical paper, the current body of knowledge regarding the use of wearable virtual reality (VR) technologies for traffic psychological examination (TPE) is introduced, critically discussed and a specific application is suggested. The combination of wearable head-mounted displays for VR with an interactive and cost-effective haptic driving interface is emphasized as a valid and viable platform for a driving skills psychological assessment, which is in several aspects superior to standard TPE as well as driving simulators. For this purpose, existing psychological examination methods and psychological phenomena relevant in the process of driving are discussed together with VR technology’s properties and options. Special focus is dedicated to situation awareness as a crucial, but currently hardly measurable construct, where VR in combination with embedded eye-tracking (ET) technology represents a promising solution. Furthermore, the suitability and possibilities of these VR tools for valid traffic psychological examination are analyzed and discussed. Additionally, potentially desirable measures for driving assessment based on recent advances of VR are outlined and practical applications are suggested. The aim of this article is to bring together recent advances in TPE, VR and ET; revise previous relevant studies in the field; and to propose concept of the cost effective, mobile and expandable HMD-based driving simulator, which can be suitable for an ecologically valid driving assessment and follow-up TPE in common practice.

1. Introduction

With the increasing density of traffic infrastructure, effective driver examinations represent an important issue for the public as well as the business sphere. Current examination tools, usually psychological, provide satisfactory but not an exhaustive nor automated source of predictions about real driving skills of an individual. Additionally, they are limited in several ways, including time and cost efficiency, reliability or limited application for specific populations, e.g., aging people. Real traffic examinations are often problematic due to considerable risks, necessary approvals, high costs for test sites and unpredictable conditions such as traffic density, etc. Regarding this, driving simulators are being used as an alternative for driver skills assessment. High-fidelity driving simulators are usually expensive and immobile so the potential of cost-effective immersive virtual reality solutions for various issues in the traffic industry is being discussed, studied and tested [1]. Mobile/wearable virtual-reality-based driving simulations represent strongly established tools in traffic research [1,2], which have the potential to study complex activity such as driving, while securing an ecologically valid context. With respect to dynamic development of the automotive industry and rapid and continuous development of immersive VR technologies, the possibilities of using VR technologies in driving simulations with the goal to promote cost-effective as well as reliable diagnostics about driving performance have emerged. Sportillo et al. indicated VR as a strategic tool for automotive industry development, including training of automated vehicles’ driving [3]. In this review we focus primarily on head-mounted displays (HMD) and immersive virtual technology as tools for driving assessment within traffic psychological examinations. The HMD is a helmet with glasses displaying a 3-dimensional picture of the virtual environment and is usually complemented with a head tracking device. Since current HMDs may be equipped with sensors for the tracking of eye movements [4,5], the additional measures indicating drivers cognitive processing or emotional and physiological states can be easily recorded and analyzed.

Using a psychological theoretical background, the goal of this article is to summarize relevant existing research and create a viable concept of an affordable, effective and valid VR assessment tool that will be able to identify drivers who possess limited abilities to perform safe driving in real traffic. We outline and discuss the combination of wearable HMDs with an interactive car cockpit interface including auditory (spatialized sound) and haptic feedback as a promising platform for driving skills within traffic psychological examination. For this purpose, existing psychological phenomena relevant for driving skills are discussed and the potential and suitability of VR technology for valid traffic psychological examination is analyzed and further discussed. Additionally, we outline potentially affordable measures for driving assessment based on recent advances of low-cost virtual technologies. The motivation for designing and developing an effective VR-based examination platform is especially centered around the attainment of reliability, time effectiveness, cost-effectiveness, the possibility to take the HMD everywhere and to automate the driver examination process, but also the ecological validity of the standard traffic psychological examination, which is necessary and frequently disputed.

1.1. Traffic Psychological Examination of Drivers

Driving a motor vehicle represents a complex activity which is influenced by various factors such as environmental aspects, specific social context and the mental state of the driver [6]. It means that successful driving requires several psychological preconditions which play an important role for the prevention of road accidents. It was demonstrated that 90% of all road accidents are the result of human error [7]. Regarding this, psychological assessment involving measurements of abilities and attitudes relevant for road safety provide valuable predictions about the driver’s fitness to become or remain a road user, or to identify ineligible road users. Therefore, professional drivers (before starting work and in 5-year periods after reaching the age of 50) and drivers with a problematic course of driving experience (drivers who have reached at least 12 registration points and drivers with a driving ban) in the Czech Republic are subject to a legal obligation to undergo the traffic-psychological examination (TPE). Within the standard TPE, the crucial psychological measures important for adequate driving performance are tested, which are primarily represented in personal factors, as shown in the figure below (Figure 1). A specific focus is dedicated to standard mental capacities related to the driving process such as intellect, attention (concentration, distribution and capacity), sensorimotor reactivity and coordination (mainly the speed and accuracy of sensorimotor reactions to a series of visual or acoustic stimuli in a time-pressing situation), decision-making (particularly when it comes to speed and dependability in a time pressing situation), memory (mainly visual) and other mental functions [8]. An examination of personal characteristics, including mainly the examination of emotional stability, liability, impulsiveness, adaptive behavior, tendency to take risks, resistance to stress, psycho-pathological symptomatology and other personal characteristics, is also included [8]. The need for TPE can be also determined by a physician, usually for the population of seniors and people with health problems or after injuries.

The TPE diagnostic methods are generally used with respect to their relative ease of use, availability and cost-effectiveness. These methods, however, possess several limitations that play a significant role when assessing the real level of the driving skill in drivers. These include dependence on subjective interpretation of behaviors, which in some cases includes non-standardized procedures and a few ecologically valid measures [9]. The validity of standard TPEs is insufficient in many aspects since these psychological methods usually measure individual performance in different than real settings which may prioritize (or on the other hand handicap) specific driver profiles [10].

Driving represents a complex sensorimotor, cognitive, behavior and socio-cultural activity [11]. The psychological measures aspire to predict driver’s skills based on (cor)relations between measurements of human mental constructs and driving abilities demonstrated in a real-world diving. In standard TPE, however, the individual’s mental skills are measured separately so their mutual effects are captured only in a limited way and the predicted performance in the real world may considerably lag behind, as real driving is a complex, coordinated and environment-dependent process [6,11]. Therefore, since the 1970s, methods have been created for rides in real traffic [12], which compensate for the shortcomings of standard TPE implemented by the traditional method of administration of psychological tests in a written or computer form. In the context of Central Europe, the Wiener Fahrprobe methodology is considered the most elaborate and reliable technique for assessing driving skills [10]. With respect to the diagnosis of the real ability to drive motor vehicles, Wiener Fahrprobe methodology involving rides in actual traffic, which provides maximal validity of the driving evaluation. Chaloupka and Risser support the claim that the results of driving in actual traffic are strongly related to the police record of traffic accidents [10]. The real driving assessment via Wiener Fahrprobe (driving test in real traffic) represents a valid but also expensive and risky method with limited control options. Additionally, the main problem is that the driver who is indicated for TPE is often subject to a driving ban and driving in real traffic is not possible. In addition, implementation of actual rides is subject to strict methodology including long-term and comprehensive training of the evaluator, which in turn raises the time and money costs [10]. Drivers are thus forced to undergo standard TPE anyway (excluding real driving), which may disadvantage or disqualify specific populations who benefit mainly from driving experience. Especially the population with cognitive impairments or the elderly may be disqualified by using traditional psychological assessment methods since their preconditions for driving measured by a standard TPE (such as the level of attention) are already declining [9]. VR-based simulations may be perceived as an economical, valid and safe method in comparison to the on-road tests, not only for these populations but also for drivers in general. [13]. Furthermore, real traffic is hardly predictable and controllable (density, risks), which downplays assessment fairness and comparability. VR-based systems also allow for capturing hardly measurable variables such as situation awareness (discussed below) and can be easily customized as well as scaled for commercial use. Regarding this, in combination with its rapid development and decreasing prices, VR tools represent promising technology capable of effectively complementing or directly replacing standard TPE.

1.2. Potential of VR Technologies for TPE

VR technologies, following the development of computer graphics and increased computational power have only recently become common. VR can efficiently generate immersive virtual worlds within user interfaces involving human-centered interactivity and sensory feedback [14]. The significant potential of VR simulations was anticipated long before VR technologies were available to the general public and was increasingly and successfully applied in many expert areas such as laparoscopic surgery [15], industry and maintenance [16], geoinformatics [17] and other fields. Driving simulation and VR technologies are closely related from their very beginning in the 1960s [14], including especially computer-generated imagery (CGI) and physical and mental immersion into VR content. From the 1960s onwards, virtual environments have grown to become a promising trend for various simulations since they offer safe and economic opportunities to reliably explore situations happening in the real world [18]. Generally, virtual simulations help psychologists and researchers collect valid data on human behavior, sensorimotor activity and cognitive processes, because simulations provide a reality-like environment where cognitive phenomena are considered equivalent to reality and can be directly transferable to practice [19]. For more than 50 years, high-fidelity driving simulators were designed to help professions such as pilots or drivers assess and train their necessary skills. However, traditional big simulators possess several practical and conceptual limitations, which have not been successfully resolved. The most problematic for TPE is the immobility of simulators, cost-inefficiency, inconsistent visualization and fidelity aspects [20,21], space requirements, HW and SW technical limitations (e.g., licensing and support) and the necessity of continuous technical maintenance. Conceptually, regarding their specific setting and form, simulators lose much of their ability to generate a sense of immersion and presence compared to reality [2]. The subjective immersion may refer to a perceived level of danger affecting for example sleepiness or other phenomena [22,23], which are highly relevant for driving activity. The research on driving simulators generally provided supportive evidence for validity of driving simulators [24,25], but the relatively small number of participants in these studies and obsolescence of the technologies used speak for the necessity of further empirical tests. The abovementioned limitations prevented driving simulators from spreading into widespread diagnostic practice and driving skill assessment remained complemented by the standard TPE.

From several relevant perspectives the use of HMD-based VR technologies for driving skill assessment can be seen as the best path towards widespread practice of using simulations for this very purpose. VR-based assessment platforms allow psychologists to design critical scenarios in which the driving skill may be demonstrated. It also concurrently offers strict experimental control as well as a wide range of additional measures including physiology and sensorimotor activity of the driver, which may indicate features of a driver undetectable by means of traditional examination methods. It represents the leap from the abstract laboratory examination into dynamic and complex simulations, where users are perceptually surrounded by a virtual world preserving a high level of ecological validity, while maintaining experimental control. The extent to which the simulator engages potential users regarding the quality and principles of cognitive activities as they happen in the real-world task, is labeled as cognitive fidelity [26]. Head-mounted virtual displays with their function of isolating users from their physical reality [27], while securing an adequate level of visual and cognitive fidelity [28], generate psychological manifestations that can be classified as immersion into the virtual world [29,30]. It is the cognitive fidelity that represents a key factor for effective simulations as well as the potential training transfer [30]. These immersive virtual environments (iVEs) provide a strong sense of presence [31], which is defined and studied as a subjective feeling of being there in the virtual world. Once the key perceptual–cognitive components of simulation are attained, such methods are expected to evaluate and potentially promote performance in the actual real-world task [32]. The usual goal of simulations is the training—i.e., enhancement of the potential performance in the real-world tasks [30]. However, the virtual technology also has excellent research and diagnostic potential for studying drivers, as it can lead a person to a well-controlled and ecologically valid experimental environment [33], with controllable levels of interactivity [34] and activity logging systems [18]. VR technologies, especially those based on HMD, have the potential to transcend the boundaries of place and time, providing unlimited opportunities to create any specific content and deliver it wherever it is needed, as well as incorporating gamification principles to increase their attractiveness. At the same time, the easy accessibility and dissemination in comparison to standard big simulators promotes the option of gathering valid data following for example the big data paradigm. Still, we can only dispute to which extent simulation really provokes reality-like behavior since the aspects of the different specific technologies used vary considerably. General empirical evidence on the equivalence of cognitive and behavioral processes demonstrated in virtual environments compared to reality has so far been documented only to a limited extent and usually only in technologically limited facilities or from thematically narrowed perspectives [35]. In this article we need to discuss the extent to which the current VR technology may provide precise predictions about human skills in driving scenarios. To adopt the VR technology as a reliable methodology for driving research we need to establish its external validity.

1.3. VR-Based Simulation Validity

The general question regarding VR simulations is whether behavioral, cognitive, emotional and psychophysiological responses in VR are equivalent to those happening in real world situations [36]. In the last two decades, studies have demonstrated the validity of driving simulators for driving behavior [37] and the ability of virtual environments to elicit adequate emotional responses [38]. At the physiological level, interactive 3D iVEs have been found to evoke responses that are more similar to responses in real conditions than iVEs with low interactivity [39], which emphasizes the importance of interactive features and the specific haptic interface type. On the other hand, in goal-oriented cognitive navigational tasks people tend to be less time-efficient in virtual simulations [40], which questions the VR tools’ reliability for behavioral measures. This notion was also identified in other studies, where the virtual environment promoted worse performance than the physical environment [19]. Based on available research, this fact can be attributed to the limits of the used technology (e.g., image quality, field of view or photorealism), but, above all, to a form of movement metaphor (i.e., UI control), which may not always be easy or intuitive for the user [41]. The virtual simulation should thus always be complemented by the corresponding haptic interface (as discussed below). From a psychological point of view, lower realism in VR usually decreases the realism of mental reactions [39]. The need for a photorealistic level of quality will be soon resolved by the ever-increasing quality of iVR technologies, where better tools are expected to lead to ever-increasing physical and cognitive fidelity and to deeper levels of immersion. Regarding recent advances in VR technologies, the increasing level of immersion corresponds positively to the subjective feeling of presence [42], so the particular type of virtual technology device can affect the level of psychological response. Regarding this, it is generally assumed that the cognitive and behavioral processes demonstrated in immersive virtual simulations as opposed to other forms of simulations (e.g., desktop applications) correspond more truly to the real behavior of an individual in the actual environment [43]. However, summarizing evidence on this issue is still absent and discussion on this topic persists. Generally, VR technologies provide unique potential for the study of natural phenomena through simulations where people can be observed in natural-like conditions [44] since VR technologies support the activation of cognitive mechanisms like those that occur in the real world [45] and may be therefore considered full-fledged substitutes for real-world scenarios [46], including driving behavior.

Another strong advantage of VR technologies is the possibility of data triangulation–the possibility to easily combine it with additional measures such as sensorimotor activity or psycho-physiological indicators. In the sense of mixed research design, we speak about concurrent triangulation [47]. Possibility to combine various measuring instruments (complementary measures) represents a viable trend that can benefit from virtual technologies [48]. Since the VR interface can be held indoors, within a controlled environment, the interface can be complemented with different types of measures such as electroencephalogram (EEG) [49]; functional near-infrared spectroscopy (fNIRs); electrocardiogram (ECG); electromyogram (EMG); mechanomyogram (MMG); galvanic skin response (GSR), blood pressure measures (pulse transit time; PTT); photoplethysmogram (PPG); and others. Especially valuable indicators are provided by the human eye movement activity. This non-invasive technique using corneal reflections can be recorded by eye-trackers, which have lately become an embedded part of HMD technology such as with the HTC VIVE pro eye. The eye movements recording, and analysis represent an important factor for understanding the car driving processes [50]. Special focus is usually placed on the phenomenon of conspicuity, the probability of perceiving an object in the visual field and the factors that determine it, which can be attributed to the ability to maintain situational insight. Regarding this, specific methods of oculographic examination have already been reported, and specific criteria for optimal assessment of the test apparatus for drivers were suggested. Additionally, other phenomena such as fatigue or distractibility can be effectively measured (as discussed below). To summarize, with the help of VR technologies we can build visually authentic, dynamic and complex stimuli with the necessary control options, so reliable measurement options persist simultaneously with the measurement control and necessary environmental validity. The proposed scenarios can be repeated, and human activity can be accurately monitored. At the pragmatic level, the VR headsets are affordable and mobile, so it can also be used outside the traffic-psychological laboratory, which further promotes its application potential. With respect to declining prices and increasing quality of VR technologies, the novel ways of TPE are overcoming standard TPEs as well as high-fidelity driving simulators.

2. Cognitive Measures Crucial for Driving

As discussed above, crucial psychological measures important for adequate driving performance such as intellect, attention, reactivity or personality traits are tested within the standard TPE. The standard tests such as the Vienna test system distributed by official providers (e.g., Schuhfried) include assessment of logical reasoning (Adaptive matrices tes–AMT, Concentration (COG), stress tolerance/resilience (DT), reactivity (RT), attention performance/obtaining an overview (ATAVT), peripheral perception (PP-R), orientation ability (LVT) and driving related personality questionnaire (IVPE-R). All mentioned measures are usually captured as separate variables, which questions the validity of real performance of the driver within the driving process, where individual skills mix and complement each other [51]. Furthermore, there are several complex and dynamic cognitive constructs that are currently very hard to reliably assess in drivers. The main reason they cannot be easily measured is their complexity or the technological difficulty of the requested measurement tool. These concepts usually reflect the complexity within the process of car driving, raising the importance of predictions based on a driver’s performance in a real-context scenario.

A crucial construct, which has been identified as a significant predictor of successful driving [52], is situational insight or situation awareness [53], in which both the assumptions of a standard TPE (such as intelligence or attention) and driving experience are involved. SA is commonly discussed in highly specialized areas such as air traffic control [54], military activities [55,56], remotely controlled vehicles operation [57] and it is also directly studied in the context of virtual car driving [58,59]. Situational awareness represents the capability to perceive elements in the surrounding within a specific period of time and volume of space. However, it also expresses the ability to comprehend the meaning of these elements and ability to project their status into the near future—i.e., reliably anticipate events. Some researchers label situational awareness as an ability to dynamically diagnose the real-world features [60], where more than simple memory mechanisms are used within the complex and organizational construct standing behind the mental model of the world [61]. Endsley includes three abilities in situational awareness: elements of perception of a given situation, their understanding and prediction of future events [53]. These three abilities enable decision-making during the driving process. In other words, situational awareness is a mental process against the background of successful responses and interactions in traffic situations. Various aspects are emphasized in this mental process. For example, Moray emphasizes the processes of perception [62] and Bell and Lyon emphasize the processes of memory [63]. Smith and Hancock emphasize that both the processes of perception and the processes of memory and situational awareness are externally focused consciousness that creates knowledge about the current situation [64]. Situational insight thus represents a key category within TPE since previous studies suggested that a driver’s SA is crucial for decision-making and behavior in driving [52]. Concurrently, they emphasized that SA is influenced by several basic factors, which were, actually, usually only studied separately. Regarding this, they identified crucial factors and significant effects for these factors affecting the level of drivers SA (see Figure 2). Specifically, SA was found to be affected by cognitive abilities (0.500), the driver’s emotional and fatigue states (0.360), the drivers’ age (0.277), driving experience (0.198) and gender (0.156). Additionally, SA was negatively affected by distracting elements (−0.253) and road characteristics (−0.213). This notion emphasizes the key role of SA in the driving process and calls for development of effective measurement tools, since a previous study by Yang et al. primarily employed the questionnaire method, which possesses many limitations regarding, e.g., subjectively reported values or limited ecological validity [52]. From this perspective, with respect to the above discussed properties, VR represents a strategic technology for follow-up research of driving as well as promising TPE tools for driving assessment.

The existing research does not provide conclusive evidence on gender effects in SA. However, since there are studies demonstrating significant gender differences in driving behavior [57,65], gender factors should be taken into account and its current inconclusiveness requires future research.

2.1. Drivers Measures Using Eye-Tracking

Testing complex constructs such as SA is hard for implementation and there is no simple method to study individual SA abilities during a traditional driving assessment. Standard methods (such as Schufried’s ATAVT) do not correspond to the definition of SA as a dynamic process changing with time. Other SA measurement methods like the Situation Present Assessment Method [66] or the Situation Awareness Global Assessment Technique [53] might be potentially used in virtual environments, but due to the continuous development within technology, now the driver’s level of SA can be researched with the precise eye movements analysis. Eye-tracking, a technology for capturing and recording eye movements [67], is currently compatible with HMD-based virtual technologies and even relatively sophisticated and precise eye-trackers are already embedded in commercially available HMDs (e.g., HTC VIVE pro eye with Tobii ET). Eye-tracking technology can help capture important measures, which are barely measurable by standard TPE [50]. The human sensorimotor activity including pupillometry, blink rates or fixations on specific objects in the environment (areas of interest) were within the ranges found in previous studies, that reliably indicate different states of a driver’s arousal, fatigue, deception, attention change or loss, hazard perception or complete loss of control [68,69,70,71]. The eye-tracking technology was also proved to have the capacity to predict neurocognitive illnesses [72], which might speed up the decline of driving skills and hence it may promote prevention in road safety. Based on sensorimotor activity, several cognitive and emotional phenomena can be induced. Eye fixation on relevant objects in the visual field was found to be related to driver’s situation awareness [69,70]. The way drivers look differs according to their experience—experienced drivers fixate more on higher positions and for a shorter time than the more inexperienced drivers [73]. Additionally, older drivers focus their attention on potential hazards around them, while younger drivers are focused on other vehicles [68]. Usage of eye trackers in virtual reality has an advantage to its usage compared to real traffic, because in a real environment there might be problems with uncontrollable visual noise [74,75], which affects perception and subsequently other cognitive processes.

Situational awareness is a complex construct where the majority of measured cognitive functions are involved, applied and demonstrated. Researchers should acknowledge that SA is influenced by the driver’s working memory capacity [76], divided attention capacity [77], spatial perceptual ability, fatigue [52], emotional state [78], level of understanding of the car’s driving system [79], monotonous task [80], conversation [81,82], hearing cell phone notifications [83], parallel driving of more vehicles in the virtual environment [84], complexity of the driving interface [85], integration of ear cons to the virtual environment [86] and availability of vestibular or somatosensory feedback [87]. Additionally, situation awareness might increase after seeing a hazardous event [88]. SA in virtual environments is also better when subjects have a lot of experience with 3D computer games [89,90] and might be dependent on cultural differences in cognition like Western analytical and Eastern holistic thinking [57]. With the use of the combination of VR and ET, SA might be measured as the number of eye collisions with other objects in the visual field [91] or differences between real positions of objects and places where subjects think these objects are on the map [92]. However, measurement of situation awareness with eye tracking is criticized for being unable to judge how a subject evaluates information [93]. This objection can be partially resolved by using various logging systems to obtain data triangulation [47]. Additionally, empirical evidence showed that experienced drivers usually keep a higher level of situation awareness than novices [94], which effect is expected to be due to different visual search and perception skills that are gained from practice [52]. Concurrently, experienced drivers also possess a greater ability to quickly identify potential hazards in the visual field, since novice drivers are usually only able to pay attention to one specific aspect of driving at a time.

Hazard perception and fatigue are very important factors related to SA. Hazard perception represents the ability to obtain information about potential danger quickly and accurately, which employs specific visual search strategies. Here, since the hazard perception is considered to complement driver situation awareness, specific visual patterns can be identified using ET technologies. The drivers’ ability to remember moving and stationary objects in the environment (i.e., SA) is related to their cognitive load. Cognitive stress was shown to determine memory performance where cognitive load decreases the accuracy of recollection of moving objects [95]. Regarding this, under a high perceptual load drivers are expected to fail noticing, e.g., an unexpected pedestrian [96]. Fatigue represents a significant cause of traffic accidents. Intensive cognitive overload is suggested to bring active fatigue, while passive fatigue is evoked by long-term monotonous tasks [1]. Passive fatigue leads to a lesser amount of engagement with the activity than the active fatigue, where drivers’ fear is involved [97] and thus passive fatigue was indicated to decrease the driver’s ability to avoid collisions. Regarding this, authors Xu, Min and Hu proposed a non-intrusive, real-time eye-tracking evaluation of fatigue in drivers while monotonous driving [69]. Based on real-time eye-movement data there were identified differences in domain value distribution of the pupil area under the condition with normal and fatigue driving state, which can be applied for fatigue assessment in VR settings, where pupillometry is one of the tracked features.

The most frequent cause of accidents involving motor vehicle drivers in 2020 in Czech Republic was non-driving—i.e., failure to pay attention to driving (20.4% of the total number of accidents caused by motor vehicle drivers [98]. The research on drivers’ attention remains a critical issue, where eye-tracking technology and virtual reality can be directly used. Diverting the driver’s attention from primary driving tasks to an unrelated activity is considered a distraction [99], which includes talking (personally or on the phone) or looking at unrelated content within (messages) and outside the vehicle (roadside billboards). In the case of a distraction, the limited cognitive resources are split resulting in lower levels of SA and potentially negatively affecting the decision-making abilities as well as vehicle operation [89,99]. The distractions of visual attention were frequently studied [100], especially their effect on the drivers’ ability to keep in their lane, or by using a dual task paradigm [101,102], specifically for keeping in the lane or depressing the brake pedal or keeping follow-up distance behind other cars. Le, Suzuki and Aoki [103] introduced an original method for simulating involuntary eye movement by combining the vestibulo-ocular reflex model and the optokinetic response. The difference between the predicted and observed eye movements was assumed to be a measure of the level of cognitive distraction, which can be applied within virtual simulations.

Traffic offences have traditionally been considered the crucial predictor of crash accidents. Voluntary violation of traffic regulations represents a serious issue since the most tragic cause of accidents was speeding, where 42.9% of people killed in accidents caused by motor vehicle drivers was by this cause or by the incorrect overtaking of vehicles (5.5% of people killed in accidents). From a philosophical perspective, deception in driving can be associated with the philosophical concept of a free rider [104,105], where this term literally applies to roads. Since lying is cognitively demanding, deceivers with limited cognitive resources are prone to cognitive overload, which results in increased speed and longer reaction times [106]. Deception, i.e., conscious violations of rules, can be however detected by pupillometry, hence ET technology can be used to identify potentially hazardous drivers. The combination of driving behavior and ET data demonstrated within the simulation can be further assessed in order to identify the potential psychopathic traits [107,108].

Previous studies also highlighted the importance of ET technologies for cognitive diagnostics and for potential cognitive disease tracking in progressive neurodegenerative conditions [72]. This technology can easily help to detect neurodegenerative dysfunctions in early stages and prevent the clinical population from driving.

2.2. Virtual Reality HMD User Interface

The specific interface formats of affordable and available VR platforms vary. Generally, the space representing the physical HW properties of the VR interface is labeled as a motor space, the visual space represents the virtual content behind the screen [109]. During the VR interaction, the user operates in a motor space, which is rooted in the physical world. The motor space is constrained by the technology used and its features such as visualization quality, field of view, control devices or available sensors. The visual space demonstrates itself via the visual representation of the virtual content, it is the virtual world as presented by the visualization technology. Regarding HW properties, the crucial structures in the VR systems are input and output devices [110]. Generally, the input device translates the physical signal from the user into a digital format and transfers it to the VR engine. The output device creates a specific (usually) visual, auditory, or haptic modality, which is presented to the user, and which ideally corresponds to the input data. A suitable combination of input and output HW devices considering the above discussed fidelity are promises of a realistic and immersive VR experience, which gains importance especially with respect to the fact that the cognitive as well as emotional response in humans was found to be determined by the specific interface properties [111]. In the case of driving, to secure realistic conditions, wearable VR HMD combined with the real-like cockpit interface represent a desirable immersive interface promoting a valid cognitive and emotional response.

Various devices for control as well as outputs can be further classified. The world-fixed devices are installed in the users’ surroundings, wearable devices are carried by the user and specific user activity directly affects the virtual scene (e.g., scene movement). Non-hand and hand input devices (if hands required) represent main categories, where non-hand input devices may include tracking systems for movement inputs: body tracking [112], head tracking (HMD systems), eye tracking, gait motion tracking (e.g., treadmill, omnidirectional treadmill) or pedal-type inputs. Auditory inputs are secured by microphone. With HMD systems embedded with head tracking, the VR environment movements correspond to the head movement of the user and so it has the potential to increase the users’ feeling of presence and at the same time positively affect the potential perceived discrepancy between user movement and visual inputs. The gaze point, which can be tracked by the eye tracking system as one possible variable, may offer valuable information about user sensorimotor activity—as is discussed above. Hand input devices are based on the hand-held controller, hand-worn or bare-hand tracking. Hand-held controllers usually include operation buttons which can be used for various actions (e.g., HTC Vive controller). Hand-worn types are represented e.g., by data gloves, which are worn directly on users’ hands. The third hand-based option to control virtual interfaces entails directly tracking the hand movement in which specific hand gestures are recognized. Both hand-held as well as hand-worn controllers may provide basic sensory haptic feedback–e.g., by vibrating. Hand input devices without any specific spatial tracking options are classified as world-grounded types, where the classical keyboard and a mouse are most acknowledged examples. The non-tracked hand-held controllers including gamepads and also specific controllers such as steering wheels (some using servo motors for haptic feedback) can be labeled as world-grounded controllers. Useful taxonomy of input devices was suggested for example by [110], basic summary is depicted in the

Table 1. Summary of VR input devices; adopted from [104].

Category	Type	Device
Manual operation	General	Keyboard, mouse, joysticks, etc.
	Customized	Customized instruments, operational platforms, etc.
Automatic tracking	Head	Accelerometer, gyroscope, etc.
	Hands	Data gloves, gyroscope, etc.
	Eyes	Camera, IR sensor, etc.
	Body	IR sensor, depth camera, etc.
	Voice	Microphone, etc.
	Position	Magnetic/optical/mechanics sensors, etc.

.

To provide a specific VR experience, multi-level sensory feedback needs to be saturated by the available outputs. The devices securing these outputs are classified with respect to the sensory cues they are providing. Generally, for humans a higher visual cue remains essential and predominantly influences the user’s presence in VR [113]. According to Argelaguet and Andujar, visual outputs are presented in the visual space which is experienced by the user [109]. The world-fixed types of visual displays are installed in the real world. They are represented by various kinds of displays, projector-based displays or monitors where the presented visual content does not react to the observer position or movement, the display does not change based on the users’ activity. Here, e.g., CAVE secured by multiple displays is designed to promote user’s immersion by providing a wide field of view [114]. More or less in opposition to the world-fixed types stand the head-mounted display types (HMDs), which currently represent the most immersive technology (as is discussed above). These can be further classified into non-see-through HMDs (smartphone-based HMD or assembled HMD—e.g., HTC Vive) and video see-through HMDs.

Haptic feedback is generally divided into passive and active types [115]. Active haptic is represented by tactile feedback or proprioceptive force feedback and can be installed in the real world as well as be worn by users. Tactile feedback (usually vibration) is transmitted to the skin of the user, while the proprioceptive force gives force feedback. A motion platform, e.g., used to mimic a vehicle, provides feedback such as motion which corresponds to an activity presented in a VE (moving in the vehicle). Passive feedback is obtained directly from structures built in the real world.

For auditory feedback speakers, earphones or headphones can be used. Since the quality of visual output needs to be secured concurrently with tracking options, adequate HMD technology needs to be considered for specific purposes. The current market offers many HMD devices. Probably most well-known are assembled devices such as the Oculus Rift and HTC Vive, but several more brands are dynamically evolving (VRgeneers, Varjo) and making VR more affordable in the near future on the worldwide market. Useful taxonomy of displays was suggested for example by [110], basic summary is depicted in the

Table 2. Summary of VR display devices; adopted from [104].

Category	Type	Number of User
Screen	Normal screen	Single
	3D screen	Single
Projector	Flat screen fabric	Single/multiple
	Curved/multi-screen fabric	Single/multiple
HMD	Small high-res screen	Single
	Small optical projector	Single
Holograms	Holographic emitter	Single/multiple

.

3. TPE HMD-Based Tool Concept and Design

Based on the above-mentioned background, we propose a specific design of the cost-effective HMD-based tool for valid measurement of driving skills. Methodologically speaking, the tool employs concurrent triangulation [47], which allows for gathering different types of data to create a complex picture of the studied phenomena. The key feature of the present TPE tool remains the ability to measure dynamic complex phenomena including several components of situation awareness, so we complement the TPE tool with several tracking options and extensive data logging. The presented tool is still in the stage of proposal and has not been tested yet. In the following sections, we discuss the key features of the proposed design, linking them to contemporary literature.

3.1. Measures in HMD-Based Tool

With respect to above discussed evidence [52,58,59,65], in the present article we identified situational awareness as the crucial construct for the VR assessment of driving skills. Driver’s SA represented a key element that affected driving decision-making as well as driving behavior. SA is directly predetermined by several driving related phenomena, which can be measured in a VR-based simulated driving scenario. Preceding influential factors such as distraction or driver states have been found to have significant effects on SA [52]. For the measurement of SA in driving assessment, we proposed the path diagram of the conceptual model including key factors relevant for TPE (see Figure 3) along with corresponding measures (

Table 3. Crucial latent variables according to proposed path diagram.

Driver factor	Characteristics	Age	Q
		Gender	Q
		Driving experience	Q
	Emotion	Fatigue	ET1
		Anxiousness/negative	ET1
Deception factor	Traffic offences	Violations (De1)	logging	ET1, ET2
Distraction factor		In-vehicle devices (GPS, Smartphone) (Di1)	ET2	head tracking
		Condition outside the vehicle (Di2)	ET2	head tracking
		Absent-mindedness (Di3)	ET2
		Conversation (Di4)	logging
SA factor	Perception	Vehicles or pedestrians (SA1)	ET2
		Traffic signs (SA2)	ET2
		Speeds (SA3)	ET2
		Perceived Hazards (SA4)	ET2
	Understanding	Location and speed of vehicles around (SA5)	logging	ET2
		Sign Content (SA6)	logging	ET2
		Sign line meaning (SA7)	logging	ET2
		Speed limit value (SA8)	logging	ET2
	Prediction	Safe overtaking (SA9)	logging	ET2
		Lane change ((SA10)	logging	ET2
		Acceleration (SA11)	logging

). The diagram is based on the previous studies on SA in combination with ET measurement options for driving assessment [52,68,69,70,71]. With respect to measurement limits and options of the used VR technology, the model was adjusted by including and excluding specific variables. The driver’s characteristics and states as well as distraction remain central variables affecting SA. Since the driving simulation represents driving activity per se, we did not include any standard cognitive measures because the driving performance is directly manifested. Traffic violations were identified as a crucial factor for traffic accidents occurrence, so we embedded a tendency to consciously deceive while driving into the diagram. Based on previous research discussed above [52,98,99,106], the proposed diagram is dedicated to describing the relationships between latent variables, where SA is affected by several mentioned factors as shown in Figure 3.

Regarding the discussed instrument options, we propose specific measures applicable in VR driving assessment. The basic driver characteristics will be captured by a brief questionnaire (age, gender, driving experience). The emotional state (i.e., anxiousness/negative emotional state) as well as fatigue will be measured with the help of ET technology based on the eye movement activity (below labeled as ET1) according to previously identified pattern analyses [116]. The distractions will be measured by ET based on the capturing specific fixations on the objects in the virtual environment (labeled as ET2) [68,69,70,71]. The Deception factor is represented by the conscious violation of traffic rules and could be identified based on a combination of ET1 and ET2 measures and activity log (e.g., driver saw speed sign but accelerated, considering also pupillometric measures to confirm consciousness rule violation) [108]. The situation awareness will be measured by wide logging and ET2 logging activity. The specific measures are depicted in Table 3.

Based on the above discussed studies [52,58,59,98,99,106] the proposed diagram (Figure 3) hypothesizes that (1) driver characteristics and states would affect distracting elements as well as deception and SA. Further, (2) distracting elements would affect SA and (3) deception would impact SA. Finally, (4) deception would affect distraction. Reflecting the presented structure, the SEM diagram of influential factors of drivers’ SA including suggested measures was obtained (see Figure 4).

3.2. User Interface

In order to test the proposed tool, we would use the HMD technology with original embedded eye-tracking technology HTC VIVE pro eye, which provides a satisfactory resolution of 1440 × 1600 pixels per eye (2880 × 1600 pixels combined), refresh rate 90 Hz and field of view 110 degrees. The device is equipped with several sensors: SteamVR Tracking, G-sensor, gyroscope, proximity, IPD sensor and eye tracking. Eye fixation on relevant objects was found to be related to driver’s situation awareness [70,71], as discussed above. ET provides binocular 120 Hz gaze data output with accuracy 0.5°–1.1° and 5-point calibration. Trackable field of view covers the whole 110°. The ET data output provides gaze origin, gaze direction and pupil position, pupil size and blinks (eyes openness). Headphone systems allow for auditory instruction (Figure 4, left). Since a movement metaphor (i.e., UI control of the interface) was identified as problematic regarding REAL-VR equivalence for the majority of previous studies, the current VR TPE tool was complemented with the corresponding haptic interface—i.e., Thrustmaster T300 RS—which represents basic, but fully functional cockpit interface, including all crucial control aspects (Figure 5).

The testing environment will be created using UNITY software and will employ a simple driving scenario, where several driving actions are measured concurrently with the driver’s behavioral and sensori-motor reactions. The instruction will be given by auditory input announced via embedded headphones. The simulation will be completed with the basic physics of the car movement and noises corresponding to the driving activity. Meaningful driving environments with an adequate number of visual stimuli can be adopted using graphical assets from the Unity Asset Store [117]. The virtual environment employed embedded interaction and logging components including eye-tracking, which is more deeply introduced in Snopková and colleagues [18]. The environment offers a controllable and persistent environment where the specific road characteristics are not manipulated. Regarding used technologies, the interface includes head tracking, eye tracking, haptic control and pedal-type inputs. The sensory outputs are visual, auditory and servo motors provide haptic feedback in the form of steering wheel resistance. Concurrent triangulation, automatic logging and UI sensors allow us to record robust data about human activity for further analysis. The measured variables were identified with respect to significant predictors of real driving abilities and they are presented in Table 3.

4. Discussion

In the present paper we discuss the current body of knowledge about the use of wearable VR technologies for traffic psychological examination (TPE). Since the standard TPE was found to possess several limitations, especially with respect to specific populations or limited ecology [6,10,11], new approaches for TPE are suggested. We emphasize the potential of combining cost-effective wearable HMD VR devices with interactive haptic driving interfaces as a promising platform for driving skills assessment, and we perceive situational awareness as a central construct for the TPE. The VR technologies, regarding their recent massive development as well as imminent features of cognitive and physical fidelity, immersiveness and availability, were found to be a viable trend for valid and effective driving assessment [44,46], which have the potential to surpass expensive high-fidelity driving simulators by mobility, dissemination, minimal maintenance and price. Regarding VR technologies, current as well as potentially desirable measures for driving assessment based on recent advances of low-cost virtual technologies are outlined. Considering the crucial position of situation awareness as a psychological construct significantly predicting driving decision-making and driving behavior, the specific role of eye-tracking technologies embedded into VR is emphasized [50,68,69,70,71]. Based on the empirical and theoretical summary, the specific HMD-based driving assessment tool is proposed. The potential of currently available technologies for variable measurement is discussed and selected to design a prototype of a mobile cost-effective driving simulator, which can be suitable for a safe driving simulation and follow-up driver assessment out of the research institutions. Further, the basic path diagram of crucial variables potentially affecting situational awareness is proposed as a suggestion for further research on VR driving assessment.

The abovementioned combination of VR technologies and eye movements tracking, in comparison to standard TPE, emphasizes the ecology of the driving assessment since it better represents the state of the real world where driving naturally occurs [44]. With the use of HMD interface, we can precisely and concurrently measure several aspects relating to the driving process in real time, so crucial variables such as SA can be measured. The examination scenario can be dynamically modified based on the real time (inter)actions of the driver or her/his recorded physiological state. The data can be subjected to the automated analyses. The eye-tracking technology can be used to study driving interfaces such as the strategies when reading GPS navigation, checking fellow passengers, conspicuity—i.e., the probability of perceiving an object in the visual field or the distractibility by visual smog. Next to ET technology, since the virtual assessment can be easily kept in the lab, the measuring interface can be complemented with measures such EEG, ECG, GSR or blood pressure measures or, e.g., saliva analysis, etc. [48,118]. The true strength of complementarity is enhanced by VR technology, where the real-time body feedback from the ecologically valid simulation can be effectively recorded and used for complex assessment of driving skills. The combination of the various cognitive, motoric physiological and emotional aspects measured at the same time may provide a comprehensive perspective on the actual level of the driver skills.

The next step of the research will be constructing the proposed HMD-based driving platform with the functional driving scenario. Using this platform, the empirical measures are about to be done to test the used HW setting and validate the proposed SEM path diagram. The further direction is creating norms in the driver performance on the HMD-based tool, further optimization and potential dissemination into TPE practice. Its potential is not solely diagnostic. Since virtual reality using HMD explicitly works with graphical game-like content, the transfer of knowledge from well-established and commercially successful platforms is possible. Commercial video games represent an attractive and available source of inspiration, which can be adjusted and used in many industrial areas. The gamification principle successfully promotes such a transfer. Virtual realities can be generally classified into three categories: simulations, games, and virtual worlds [119]. Games represent the most promising category for learning, especially regarding serious games [120]. These applied games represent content primarily created for purposes other than entertainment, i.e., for example defense, automotive industry, health care, education, scientific exploration, emergency and crisis management, urban planning or engineering, e.g., in [121,122]. The serious games scenarios provide interactivity and autonomy, which promotes proactive and self-induced engagement into the presented content, where principles of general simulations are persisted, but concurrently educational values of fun and competition are superstructured. The concept of serious games gains importance in the above discussed VR driving assessment since there are many existing platforms providing ready-to-use complex driving virtual environments. The great potential of VR-based driving simulators in traffic psychology lies within its subsequent use for therapy. VR simulations have a great potential for drivers who are afraid of driving, e.g., a fear related to traumatic experience. Virtual reality using HMDs can be also used as therapeutic tools for traffic accident victims. Literature suggests that VR is a viable option for such people. [123,124].

Relatedly, the next step is the training of drivers using suggested HMD-based tools, which, at least in the early phases, may supplement the real driving classes. Since driving combines motoric abilities (steering wheel guide) and sensory abilities, but also the knowledge of rules, in the early phases VR technology can save the time of a real tutor/instructor (also costs and environment), since the elementary procedures can be easily transferred, automated and simulated within the VR platform. In such a platform, collaborative peer learning can be included where young drivers create a peer community for common training and supervision [125,126]. Training can be adaptive, i.e., the diagnostic data can be used for the creation of specific training programs focused on problematic areas [127]. Last, but not least, the research of various driving aspects is emphasized. The suitability of traffic infrastructure in urban areas can be assessed before it is built [128], the specific influence of drugs on driving can be safely tested [129]. In the area of autonomous cars, complex simulations can be created involving AI agents as well as real participants, who can remotely interact, and researchers can study the “human-in-the-loop” [130].

As any other technology VR possesses several limitations, which should be always considered in its application. Even though the VR limits are progressively solved, they represent a permanent struggle for current development of applied VR tools. Dynamic visual exposition via HMD glasses, the absence of a muscular response when virtual movement and visual/cochleovestibular contradictions result in nausea, which is usually referred to as motion sickness, cybersickness or kinetosis [131]. This feeling of sickness plays an important role since it may bias the measured data, but primarily in extreme cases it can completely prevent users from using the VR technology. Some other researchers question the very nature of immersive virtual reality for driving research claiming that today’s VR technology used for driving simulations does not bring general advantage compared to flat screens [132], where its negative features especially protrude. Dynamic and continuous technological progress, however, sets VR technologies into the center of the research activities anyway, whether with the goal to explore in more depth their aspects, or to directly resolve mentioned limits.

The ecological validity of the proposed tool can be questioned from several perspectives. The issue of the immersiveness and fidelity of VR technologies have been discussed above, the standard test-taking motivation or test-taking effort remain, though. Regarding this, previous evidence suggests that driver’s speeds when driving their own private cars correspond with their speeds during the in-car observations. Ref. [133] showed that the drivers drove in the same way when being observed as they did normally, so the in-car observation was found to be a reliable and valid method to observe driver behavior. We can deduce that this notion applies also for VR in-car observation.

5. Conclusions

To summarize, in this paper HMD technologies were found to provide huge potential for TPE, especially with respect to their dynamic development, variety of additional sensors including crucial eye-tracking, and limitless options of virtual content creation. Driving skills can be observed in the well-controlled and ecologically valid environment with the extensive data logging options. Specific design of the driving assessment platform as well as key cognitive constructs relevant for TPE were proposed. Further, the cost-effective MHD-based solutions were suggested to be easily distributed worldwide and promote, e.g., training and peer-based training of the driving community all around the globe.