A Driving Simulator Study to Examine the Impact of Visual Distraction Duration from In-Vehicle Displays: Driving Performance, Detection Response, and Mental Workload

Abstract

This research explores the impact of visual distraction duration from multifunctional in-car displays on driver safety. Utilizing a driving simulator and eye-tracking technology, this study involved 35 participants in visual search and car-following tasks, assessing their performance and mental workload across different durations of distraction. The results show that distractions lead to a decrease in driving control and a rise in mental workload, characterized by deteriorated vehicle handling and longer reaction times. With continued exposure to distractions, drivers begin to adapt, indicating a non-linear relationship between the duration of distraction and its consequences. This adaptation points to a threshold beyond which the negative effects of distractions no longer intensify. This work aids in developing safer automotive interfaces by highlighting the effects of larger screen trends on driving behavior and proposing strategies to mitigate distractions. It enriches the discourse on human–machine interaction by offering fresh perspectives on how visual distraction duration from in-car displays influences driving dynamics and cognitive load, thereby enhancing road safety.

1. Introduction

With the advancement of the intelligent cockpit, it has transitioned from a novel concept to an increasingly ubiquitous trend in standard automotive design. The large-scale full-touch display (referred to as the full-touch human–machine interaction (HMI) mode) has emerged as the dominant mode of interaction between the driver and the system (see Figure 1). As the dimensions of the full-touch screen increase, it expands the range of features accessible to drivers via in-vehicle-connected technologies. Nevertheless, this increase concurrently intensifies the risk of driver distraction. Driver distraction is characterized as the deviation of attention from essential driving tasks due to competing activities [1]. According to the National Highway Traffic Safety Administration (NHTSA), in 2018, distractions behind the wheel were responsible for 8% of deadly incidents, leading to 2841 deaths, and constituted 15% of accidents causing injuries, with an estimated 400,000 individuals injured in the United States [2]. Recognition errors, primarily attributed to driver distraction, account for approximately 39% of all vehicular collisions [3].

While driving, individuals often experience external noise and visual distractions, especially from their touch screens. Visual distractions significantly compromise the safety of drivers by substantially impairing their driving capabilities and heightening the probability of accidents. Existing research has explored the phenomenon of visual distraction during driving, noting its adverse effects on recognition, perception, and various cognitive processes, whether the distraction is deliberate or accidental [4]. For instance, intelligent transport systems, by diverting the driver’s gaze from the necessary visual field, introduce visual distractions that can undermine safety [4]. Additional research indicates that visual distractions negatively influence reading behaviors and understanding, and sensory disruptions can detrimentally affect higher-order cognitive abilities required for learning from textual materials [5]. Driving performance is compromised as drivers’ visual focus shifts to secondary tasks, such as using the touch screen, and manual interactions that require disengaging from the steering wheel, thereby increasing the risk of accidents [6]. The presence of in-vehicle screen devices correlates with increased driver fixation duration on screens over the roadway, significantly contributing to vehicular collisions, near misses, and critical safety incidents [7]. Distraction intensity and driving risk increase with the secondary task’s visual and manual demands, which compete for cognitive resources crucial for safe driving. The World Health Organization has identified driver distraction as a significant factor in road traffic accidents [8].

An expanding corpus of studies has explored the visual workload of drivers [9], cognitive workload [6], and driving performance [10] associated with the utilization of the full-touch screen during vehicle operation, and most existing research focuses on the impact of single secondary task scenarios on driving distraction, such as making phone calls, sending text messages, adjusting FM, etc. With the development trend of the multi-screen and large screen in the intelligent cockpit, the continuous visual occupation of secondary tasks has become very common, and some special functions, such as navigation and setup functions, require many steps to complete. However, the cumulative effects of prolonged visual distraction tasks may differ and could impact various aspects of driving behavior, such as lateral and longitudinal control. To explore this, the present study aims to investigate the cumulative effects of prolonged visual distraction on driving performance and mental workload through a simulated experiment. Changes in driving performance and mental workload across different time windows will be examined. This study provides a new perspective for a more comprehensive understanding of the complex relationship between distraction and driving safety.

2. Related Works

This section synthesizes research on the in-vehicle HMI and visual distraction, reviewing the effects of various interaction modalities and visual distractions on driving performance, in addition to the effects of distraction duration on driving stability.

2.1. In-Vehicle HMI Interaction and Safety

In the vehicle Human–Machine Interface (HMI), safety evaluation holds a pivotal position in analyzing the implications of driver distraction and in guaranteeing secure operation. Undoubtedly, engaging in tasks using a full-touch interface alters a driver’s distraction behavior. Researchers investigating the HMI have examined the influence of various factors like touch gestures, aerial gestures, voice interaction, and touchscreen placement on driving performance and resource needs [11]. Studies into the effects of traditional buttons, voice commands, and touchscreens on driving safety have revealed that touchscreens significantly reduce safety, with their impact being more pronounced compared to other modes [12]. Additionally, studies indicate that touchscreen methods lacking nonvisual feedback are inferior in task performance compared to conventional physical buttons [13].

Touchscreens, as a crucial component of the HMI, can distract drivers and diminish their environmental awareness, thereby posing significant safety risks in the context of driving. While modern vehicles are increasingly equipped with intelligent safety features, such as path tracking control and assisted braking [14,15], which autonomously maintain vehicle control and prevent collisions, these technologies do not fully eliminate the risks of visual distraction. Research shows that drivers’ focus varies with the display’s position, and the lack of tactile feedback on screens requires more visual attention [16]. Moreover, some studies explore the impact of touchscreen size, user interface design, and subtasks on the visual demands of in-car tasks and the boundary of driver distraction [17]. To minimize distraction, studies suggest using directional touch gestures to lower visual demands and maintain focus on the road [18]. Others investigate how touchscreen size, interface design, and specific tasks affect visual demands and distraction levels in drivers [17]. O. Tsimhoni and colleagues investigated three address input methods during the driving process: word-based speech recognition, character-based speech recognition, and input on a touchscreen keyboard. The findings indicate that using touchscreen input for address entry results in a decrease in vehicle control capability. The experiments reveal that, in the context of navigating address input, the use of a touchscreen introduces certain safety risks and is unfavorable for drivers during driving [19].

The distraction caused by in-vehicle large screens is becoming increasingly severe. Numerous studies are exploring the impact of these screens on driving performance and mental workload. However, our focus is on the cumulative distraction effects of visual-distracted driving behaviors caused by in-vehicle screens, such as the frequent use of car navigation systems, on driving safety.

2.2. Attention and Visual Distraction

Attention diversion in drivers during vehicular travel involves a shift from focusing primarily on safe driving to engaging in secondary activities or objects unrelated to driving safety [20]. This shift in focus substantially impairs the driver’s ability to respond to unexpected events and slows reaction times, thereby increasing the risk of traffic accidents. The National Highway Traffic Safety Administration (NHTSA) categorizes driver distraction into four types based on the source of the distraction: visual distraction, manual distraction, auditory distraction, and cognitive distraction [11]. Visual distractions arise from tasks requiring visual engagement, whereas auditory distractions originate from tasks demanding auditory attention. However, this dichotomy is not rigid. For example, engaging in telephonic conversations while driving [21], though primarily auditory, can invoke visual recollections or imaginings in the driver. Such cognitive processes can adversely affect the driver’s visual search capabilities, diminishing both the efficiency and frequency of information retrieval, thereby creating potential visual blind spots.

Visual information serves as a pivotal source for drivers to acquire necessary cues. Drivers rely on visual inputs to maintain speed and stability following distances. Incomplete or biased visual information acquisition can lead to traffic accidents. Devices for tracking eyes are often employed to scrutinize the eye movements of drivers and explore how distractions influence their driving performance [22]. Furthermore, analyzing driver’s eye movement and attention data provides a deeper understanding of their mental activities and anticipated actions, thus offering a more comprehensive view of their behavioral patterns. For instance, ref. [23] carried out an extensive review of driver distraction across various simulated environments, including standard driving, visual–manual tasks, and cognitive burdens. This study identified a significant decrease in the proportion of gaze towards the road when engaging in visual–manual tasks, contrasting sharply with regular driving conditions. Conversely, the incidence of the driver’s gaze on the road conspicuously augmented during cognitively demanding tasks, surpassing that observed in normal driving conditions. Ref. [24] investigated the influence of hands-free telephone conversations on drivers’ visual attention through a practical field experiment. This study concluded that hands-free phone usage while driving marginally affects gaze behavior concerning the driving task, with a tendency for drivers to focus less on traffic-specific details. Complementary findings were reported by [25] during a simulation study that evaluated drivers’ visual behavior using eye-tracking technology, alongside assessments of driving errors and subjective workload. Ref. [26] also corroborated that visual secondary tasks diminish the scope of drivers’ visual search. Under such circumstances, drivers tend to compensate by increasing the amplitude and frequency of head movements to acquire a broader field of view.

Visual search, a crucial aspect of driving, involves the capacity to survey the surroundings and identify potential hazards. Research indicates that driver experience significantly influences visual search strategies [27]. The cognitive load from secondary tasks, such as conversations or operating devices, further narrows the visual search field, thereby reducing drivers’ ability to detect hazards.

Extensive research in the domain of driver distraction has employed eye-tracking technology as a pivotal tool to analyze drivers’ visual behavior and attention allocation. Building on this foundation, our study also employs eye-tracking technology to analyze visual distraction. We aim to further explore the intricate relationship between distraction duration and visual attention during driving.

2.3. Distraction Duration and Driving Behavior

Driving instability is profoundly influenced by the distraction duration. It is noteworthy that driving instability encapsulates deviations from standard driving behaviors, including variations in speed, deceleration, acceleration, and jerk [28]. Recent research by [28] has established a direct correlation between increased distraction duration and heightened driving instability. Studies have illuminated the process by which an increase in distraction duration during driving worsens vehicular control issues, subsequently elevating the risk of accidents or near-miss incidents [29]. Crash and near-crash risks increase significantly for certain activities, especially those that require both visual attention and manual interaction or those requiring longer gaze duration. These activities include dialing on a cell phone, texting, reaching for items, and responding to external disturbances [30]. Visual distractions from secondary tasks significantly impact speed stability and contribute to crash and near-crash risks, with the increasing distraction intensifying the probability of such incidents [31].

Mental workload affects driving behavior by altering visual search patterns, reducing attention, and increasing reaction times, thereby heightening accident risks. Defined as the required operator resources to handle specific task demands [32], mental workload has profound implications on driving safety. Research has established a link between the risk of accidents, the driver’s visual search patterns, and the level of mental load [33]. Both an excessive mental workload, often associated with stress, and an inadequate one, linked with vigilance, can impair a driver’s perception and attention, potentially leading to traffic incidents [34]. A driver’s mental workload develops through a gradual accumulation process, resulting in a deterioration of driving skills. As this workload reaches a certain threshold, it may cause a rapid decrease in the ability to drive, leading to errors in judgment, confusion in operation, and similar issues, all of which significantly raise the risk of traffic incidents [35]. As the demands of task processing increase, drivers require more time to process information effectively. Furthermore, driving distractions, predominantly visual and cognitive, exert a similar influence on drivers by affecting their performance and significantly contributing to traffic accidents [36]. Such distractions lead to an increased mental workload for drivers [37]. Visual distractions considerably affect drivers, leading to poorer performance compared to cognitive distractions. This suggests that visual distractions impose a more substantial mental workload on drivers than cognitive distractions [38]. Numerous eye-tracking metrics are recognized as reliable indicators of mental workload. These include changes in pupil diameter, the length of blinks, the spread of horizontal gaze, frequency of blinking, the duration of gaze, the number of saccades, the standard deviation in the rotation of the eyeball horizontally, and the variations in the points of gaze [39]. However, there is a paucity of literature examining the patterns of change in mental workload over time in response to varying levels of distraction.

Detection response is essential for understanding driving behavior, revealing that distractions and increased cognitive load significantly impair reaction times and overall driving performance, directly linking to safety on the roads. The detection response task (DRT), endorsed by the International Organization for Standardization (ISO 17488:2016) [40], is predominantly utilized to gauge distraction, particularly in driving studies, where it assesses drivers’ resource availability and the attentional demand of secondary tasks [41]. This technique has proven effective in assessing variations in cognitive load, indicated by changes in reaction times and miss rates. The DRT’s sensitivity to cognitive load changes is equal across auditory, tactile, and visual stimuli and is unaffected by the visual location of the stimuli, emphasizing that cognitive rather than visual demands reduce detection capability [41]. This approach is particularly reliable for assessing reduced visual attention and identifying safety-critical information in the periphery [42]. Evaluations of DRT performance through reaction times and hit rates offer a nuanced understanding of the competing secondary task’s difficulty [43].

Building on extensive research on the in-vehicle HMI and visual distractions, prior studies highlight the correlation between distraction duration and driving instability, emphasizing how visual distractions increase a driver’s mental workload and impair performance. Despite these insights, there remains a notable gap in understanding the specific long-term impacts of cumulative visual distractions on driving performance. Previous studies have often focused on immediate or short-term effects (1–12 s), leaving the prolonged impact less explored. Therefore, this study aims to utilize driving performance, the DRT, and eye-tracking data to deeply investigate how sustained attention to distractions (300 s) affects mental workload and driving behavior over time.

3. Experiments

3.1. Participants

This study, adhering to the NHTSA Guidelines, initially recruited forty subjects across four age categories: 18–24, 25–39, 40–54, and 55–64 years, with a gender distribution of twelve females and twenty-eight males.

Table 1. Demographic descriptive statistics (N = 35).

Character		Frequency	Percent
Gender	Male	25	0.71
	Female	10	0.29
Age	18–24	9	0.26
	25–39	9	0.26
	40–54	9	0.26
	55–64	8	0.22

shows the participants’ demographic information. Due to recording issues, the analysis included only 35 valid participants (24 males and 11 females). All participants were right handed with normal contrast sensitivity, visual acuity, and color vision, and were free from cardiac and mental health issues. Every participant possessed a legitimate driver’s license and a minimum of two years of experience behind the wheel, averaging 5.42 h of driving weekly. No participants had incurred any traffic fines or accidents over the previous years. This study received approval from the IRB ethical committee of the Beijing Institute of Technology.

3.2. Apparatus and Materials

Driving simulator. This study employed a driving bench to develop a driving simulator that comprised a steering wheel and pedals, a computer, and a television in front of the bench simulator. The driving simulator was equipped with SCANeR™ studio driving simulation software (SCANeR 2021.2). The computer was capable of storing the driving performance data of the participants and synchronizing it with the information acquired by the eye-tracking devices. As shown in Figure 2, an in-vehicle 12″ touchscreen display was used to display the detection response tasks (DRTs), and the touchscreen utilized an Honor Tablet 8 (12 inches, which is manufactured and supplied by Honor, a company based in Shenzhen, China) as the display device, featuring a resolution of 2000×1200 dpi, a pixel density of 195 PPI, and a peak brightness of 350 nits. The touchscreen was located on the central stack of the dashboard in a fixed position within the driver’s visual range, and the positional parameters were calculated based on the average values of the measured vehicles’ touchscreen parameters from the XPeng P7, Li Xiang One, BYD Song, and NIO ES8 (The XPeng P7 is manufactured by XPeng Motors, based in Guangzhou, China. The Li Xiang One is produced by Li Auto, headquartered in Beijing, China. The BYD Song is made by BYD Auto, located in Shenzhen, China. Lastly, the NIO ES8 is manufactured by NIO Inc., which is based in Shanghai, China). The traffic scenarios and virtual environment were generated with SCANeR™ studio software. The driving simulator SCANeR™ studio delivers a highly authentic virtual realm: it encompasses road settings, vehicle dynamics, traffic, sensors, both real and simulated drivers, headlights, weather scenarios, and scenario scripting (see Figure 2).

Eye tracking. Eye tracking stands as a highly sophisticated technique employed in usability assessment. Accumulating data from eye tracking provides significantly more insights into user behavior. The experiment utilized the Tobii Pro Glasses 2 (The Tobii Pro Glasses 2 are manufactured and supplied by Tobii Pro, a company based in Danderyd, Sweden.) as the device to monitor eye movements, operating at a frequency of 100 Hz for sampling and providing a robust dataset for analyzing the timing and duration of gaze behaviors. This feature is critical for understanding the nuanced ways in which individuals interact with various stimuli. The device’s ergonomic design ensures minimal intrusion, allowing for a more natural observation of participants’ visual behavior. Its compatibility with numerous software enhances its versatility, accommodating both qualitative and quantitative research in eye-tracking studies. In our experiments, we ensured consistent lighting to maintain data accuracy, confirming that our results reflect genuine participant interactions with the stimuli, free from external influences [44].

3.3. Experimental Design and Procedure

In this experiment, a visual-based dual-task experimental paradigm was adopted to explore the effects of distraction duration on driving performance and visual search task accuracy. The independent variable in this experiment is the duration of the visual secondary task. The primary task, the car following [45] task, which is one of the most common road conditions during driving on straight roads and an important requirement for safe driving, requires participants to follow another vehicle at a constant speed of 60 km/h on a straight urban road, while the visual secondary task involved performing a visual search.

Visual-based DRT task: To investigate the influence of visual load on drivers, a paradigm based on visual search tasks was utilized to establish links between gaze behavior and levels of attention [46]. A visual search and response behavior involve using the eyes to seek out relevant information within a given display or scene, subsequently guiding subsequent actions [47] while experiencing minimal or negligible cognitive load. In this visual search paradigm experiment, as shown in Figure 3, participants are instructed to respond to changes in shapes (circle/triangle/square) on the in-vehicle large screen by pressing a button on the side of the steering wheel within 1500 ms of the appearance of the triangle shape. There is a 1 s interval between the display of the triangle and circle shapes, each lasting 1000 ms, with the entire experiment lasting 5 min. Throughout the duration of the experimental protocol, stringent controls were established to ensure a constant illumination level within the experimental milieu, thereby aiming to attenuate the effects of extraneous variables on pupillary dilation variations. In consideration of the experimental goal of visual distraction, measures were taken to minimize the likelihood of drivers recognizing stimulus images in their peripheral vision. Specifically, the size and color of the stimulus images presented during the experiment were carefully chosen to blend in with the background color of the main experimental interface. This was performed to further reduce the possibility that drivers would recognize these images peripherally. This task was designed to simulate visual distraction tasks that drivers might engage in during driving, such as prolonged interaction with content on the in-vehicle large screen.

The experimental procedure began with online preparations. Participants digitally signed an informed consent form and completed several questionnaires. Upon arrival at the lab, participants were thoroughly briefed about the study procedures, including the objectives and their role in the experiment. This briefing was followed by an informed consent process and a demographic survey. Next, participants were introduced to the driving simulator. This introduction included detailed explanations about the simulator’s functionalities, the primary and secondary tasks, and the overall procedure of the experiment. Prior to each experimental drive, participants underwent a 4 min training session tailored to the specific driving conditions. This session was designed to familiarize participants with the simulated driving environment, the sensitivity of the steering wheel, and the car pedals. This practice focused on maintaining a consistent speed on a straight road, acclimating them to the driving environment. They also practiced the secondary visual task, which involved using an in-vehicle large screen to identify specific shapes with a high degree of accuracy (over 95%). In this instructional session, the participants practiced driving on straight paths, around bends, at crossroads, and transitioning from dual lanes to quadruple lanes. After the session, the participant put on eye-tracking glasses, and the experimenter adjusted the calibration of their gaze. Once the practice session was completed, participants were directed to focus on accuracy in the secondary visual tasks while ensuring steady and safe driving during the experimental runs. During this phase, the participant wore the eye-tracking equipment and the experimenter calibrated the gaze position of each participant. This equipment was crucial for collecting precise data on where and how often participants looked away from the road, providing valuable insights into the effects of distraction. Then, the first experimental drive, which lasted five minutes, commenced. After this session, the experimenter expressed gratitude to the participants and provided compensation for their involvement. Figure 4 illustrates a diagram outlining the experimental process.

3.4. Data Analysis

This study designates the duration of the secondary task as the independent variable. The dependent variables are driving measures related to car-following behavior.

(1)

Lateral control measures

The standard deviation of lane position (SDLP) (m): The SDLP was calculated as the standard deviation of the vehicle’s lateral displacement from the centerline for each driver [48]. An increased SDLP indicates reduced lateral control of the vehicle.

Standard deviation of steering wheel angle (SDSWA) (degrees): This measure offers insight into the vehicle’s lateral dynamics. A higher value in this measure suggests more significant lateral adjustments made by the vehicle [45].

(2)

Longitudinal control measures

The standard deviation of speed (SDS) (m/s): A higher standard deviation in driving speed suggests less efficient driving.

Standard deviation of acceleration (SDA) (m/s²): A larger standard deviation of acceleration indicates poorer longitudinal control over the vehicle.

(3)

DRT measures

Missing rate (MR): A higher missing rate signifies a decrease in the driver’s ability to detect and respond to secondary tasks.

Reaction time (RT): This indicator reflects variation in the driver’s reaction time due to interference from secondary tasks while driving.

(4)

Mental workload

Average pupil size [49] (APS): Pupil size is commonly believed to increase in response to the mental workloads imposed on a driver while driving.

Single fixation duration (SFD): A prolonged off-road single fixation duration indicates heightened distraction levels during tasks, suggesting that the driver is experiencing an increased mental workload.

Statistical evaluations of the driving and cognitive load data were performed using polynomial regression in GraphPad 9.5. For all statistical assessments, a p-value of less than 0.05 was used as the criterion for statistical significance.

4. Results

The correlation matrix is utilized to understand the relationship among driving performance, detection response, mental workload, and distraction duration. This matrix displays the Pearson correlation among its elements, where the value represents Pearson’s coefficient denoted as ‘r’. A larger r-value means a stronger correlation between these two components.

Regarding the longitudinal control of the vehicle, Figure 5 indicates that the standard deviation of acceleration and the standard deviation of velocity are positively correlated with distraction duration, and distraction duration was significantly associated with the standard deviation of acceleration (r = 0.7177, p < 0.001) and the standard deviation of velocity (r = 0.8107, p < 0.001).

In terms of the lateral control of the vehicle, Figure 5 indicates that the distraction duration was significantly associated with the standard deviation of lane position and the standard deviation of steering wheel angle. Correlation analysis also showed that distraction duration had a strongly positive correlation with the standard deviation of steering wheel angle (r = 0.2298, p < 0.001) and the standard deviation of lane position (r = 0.6741, p < 0.001).

In the context of detection response, Figure 5 shows a significant association between distraction duration and both missing rate and reaction time. Concurrently, correlation analysis reveals a negative association between distraction duration and missing rate (r = −0.6896, p < 0.001), as well as reaction time (r = −0.5554, p < 0.001).

Regarding mental workload, Figure 5 shows a significant association between distraction duration and average pupil size, with no significant correlation found for single fixation duration. Further correlation analysis indicates a negative correlation between distraction duration and average pupil size (r = −0.3647, p < 0.001).

4.1. Driving Performance

By analyzing the correlation between driving performance and distraction duration, we infer a connection between the standard deviation of steering wheel angle, lane position, velocity, acceleration, missing rate, reaction time, average pupil size, and distraction duration. To clarify this relationship, the cubic equation was used to model the trends of these variables.

The standard deviation of acceleration and velocity assesses the driver’s longitudinal control abilities, with higher values indicating poorer longitudinal control over the vehicle. The relationship between the standard deviation of velocity, the standard deviation of acceleration, and the distraction duration is displayed in Figure 6a,b. These relationships indicate that with the increasing duration of distraction, both the standard deviation of acceleration and the standard deviation of velocity exhibit a similar pattern of change, characterized by an initial ascent followed by a gradual tendency toward stability, ultimately re-emerging with an upward trend around the 300 s mark. As depicted in Figure 7, each plot corresponds to the rolling standard deviation of these metrics measured in 5 s intervals across a 300 s duration. The graphs show an initial variability that gradually stabilizes as time progresses, indicating that despite initial fluctuations, the control abilities tend to stabilize, suggesting an adaptation or habituation effect in the driver’s response to sustained driving conditions. The correlation test results in Table 1 indicate that the fitting coefficients obtained from the cubic regression consistently surpass 0.9. This finding highlights the favorable fitting performance of the curve. Furthermore, it suggests that with an increased distraction interval, the driver’s longitudinal control of the vehicle does not consistently decline but exhibits a certain degree of stability. The cubic regression equations are also displayed in

Table 2. The correlation between driving performance, detection response, mental workload, and distraction duration.

Correlated Variables	r-Value	Cubic Equation
The SD of acceleration—duration of distraction	0.9351	$\mathrm{y}={5.280\mathrm{e}-8\mathrm{x}}^3-3.300\mathrm{e}-5{\mathrm{x}}^2+0.006493\mathrm{x}+0.1091$
The SD of velocity—duration of distraction	0.9685	$\mathrm{y}={1.521\mathrm{e}-6\mathrm{x}}^3--0.00093{\mathrm{x}}^2+0.1836\mathrm{x}+1.933$
The SD of steering wheel angle—duration of distraction	0.6371	$\mathrm{y}=1.607\mathrm{e}-7{\mathrm{x}}^3-9.736\mathrm{e}-5{\mathrm{x}}^2+0.01694\mathrm{x}+1.905$
The SD of lane position—duration of distraction	0.6891	$\mathrm{y}=4.633\mathrm{e}-8{\mathrm{x}}^3-2.489\mathrm{e}-5{\mathrm{x}}^2+0.004217\mathrm{x}+0.3247$
Missing rate—duration of distraction	0.7053	$\mathrm{y}=-4.369\mathrm{e}-8{\mathrm{x}}^3-2.444\mathrm{e}-5{\mathrm{x}}^2+-0.004474\mathrm{x}+0.3802$
Reaction time—duration of distraction	0.3539	$\mathrm{y}=5.000\mathrm{e}-7{\mathrm{x}}^3--0.0001452{\mathrm{x}}^2+0.003762\mathrm{x}+0.8125$
Average pupil size—duration of distraction	0.4650	$\mathrm{y}=-1.523\mathrm{e}-8{\mathrm{x}}^3-1.192\mathrm{e}-005{\mathrm{x}}^2+-0.002813\mathrm{x}+3.741$
Single fixation duration—duration of distraction	0.0631	$\mathrm{y}=-8.581\mathrm{e}-6{\mathrm{x}}^3-0.003565+-0.3539\mathrm{x}+123.2$

.

Measures of lateral control evaluate the effectiveness of drivers in keeping their vehicles positioned correctly within a lane. These include the standard deviation of lateral position and steering wheel angle. Greater values of the SDLP and SDSWA indicate a diminished lateral control proficiency of the driver over the vehicle. The relationship between the standard deviation of the steering wheel angle, the standard deviation of lane position, and the distraction duration is displayed in Figure 6c,d. This relationship suggests that with the increasing duration of distraction, both the standard deviation of the steering wheel angle and the standard deviation of lane position exhibit a trend characterized by an initial ascent followed by a subsequent decline, ultimately concluding with a marginal resurgence. The correlation test outcomes presented in Table 1 reveal that the fitting coefficients derived from the cubic regression consistently surpass 0.6. This observation underscores the satisfactory fitting performance of the curve. Furthermore, it implies that an augmentation in distraction duration does not lead to a continual decline in lateral control of the vehicle but rather suggests the presence of a distinct stable phase. The cubic regression equations can be found in Table 1 as well.

4.2. Detection Response

Figure 8a illustrates the relationship between distraction duration and missing rate when participants were instructed to drive and respond to the detection response task (DRT). It can be observed that at the beginning of the distraction period, the missing rate initially increases. As distraction duration continues, the missing rate slightly decreases and subsequently stabilizes. Ultimately, the missing rate rises to 0.4 before decreasing to approximately 0.12. Figure 8b depicts the correlation between reaction time and response frequency. The data indicate that at the beginning of the distraction period, reaction time initially increases as the number of responses rises. Afterward, reaction time exhibits a marginal trend toward stabilization. Subsequently, after the 25th response task, reaction time tends to stabilize and slightly decrease until the end of the experiment, rising to 0.82 s before decreasing to 0.67 s. Despite the cumulative effects of prolonged distractions, drivers’ reaction times did not continuously increase but instead showed a brief period of stability before a slight reduction.

4.3. Mental Workload

It is generally believed that pupil dilation increases when a driver experiences mental workloads while driving [50]. An extended off-road single fixation duration signifies increased driver distraction during tasks, reflecting a higher mental workload for the driver. Figure 9a indicates that the average pupil size does not continuously increase with an extended duration of distraction. On the contrary, the pupil size gradually decreases and stabilizes at 3.54 mm, followed by an upward trend. Figure 9b illustrates a noticeable decline in the single fixation duration within the first 60 s of sustained distraction, reaching a minimum of 92 ms. Subsequently, the single fixation duration gradually increases with the prolonged distraction time until it decreases again at 223 s of distraction. This suggests that the driver’s mental load does not exhibit a linear escalation with prolonged distraction; rather, there appears to be a discernible plateau, and in certain instances, a reduction in mental workload may be observed.

5. Discussion

This study investigates the impact of prolonged screen fixation-induced visual secondary tasks on driving performance, detection reactions, and mental workload. Additionally, it explores how driving performance, detection response time (DRT), and mental workload vary with the duration of distraction. Driving distraction is a significant risk factor for accidents, as highlighted by [51]. Hence, quantifying the relationship between the duration of visually induced driving distractions and driving performance is crucial. Numerous studies have shown that a prolonged distraction duration can result in a deterioration of driving performance and an increase in mental workload. However, limited research has delved into how the cumulative effects of extended visual distraction, particularly from prolonged screen viewing, impact driving performance, response detection, and mental workload. This study aims to investigate the correlation between the duration of visual secondary tasks and their implications for driving safety. The findings are expected to enhance our understanding of the incremental impact prolonged driving distractions from large screens have on driving safety.

Referring to the effects of distraction duration on the lateral control of the vehicle, the findings reveal that as the distraction duration extends from 0 to 100 s for the driver, there is a non-linear rise in the standard deviation of lane position and steering wheel angle. Previous research also indicates a direct correlation between the extent of distraction and driving instability [29]. When the distraction duration increases from 100 s to 300 s, the standard deviation of lane position and steering wheel angle gradually tends to stabilize. This stabilization might be attributed to a decrease in the mental workload of the driver as they become accustomed to the distractions. This reduction in mental workload could result from the driver’s improved efficiency in processing the distractions over time, allowing for better allocation of attention to the lateral control of the vehicle. This finding suggests that after reaching a distraction time of 100 s, both the standard deviation of the steering wheel angle and standard deviation of lane position tend to stabilize rather than continue to rise, indicating that the driver’s lateral control of the vehicle does not further worsen with an increased distraction duration. When the distraction duration exceeds a specific adaptation threshold, there appears to be a counterintuitive positive correlation between increased distraction duration and improved driving performance. This suggests that beyond a certain level of exposure to distraction, drivers might adapt their behavior to manage the demands of driving and distraction more effectively, which is in line with the literature [52]. Therefore, these findings underscore the nuanced relationship between distraction duration and driving performance, suggesting that while prolonged distraction initially exacerbates driving instability, this study reveals a potential for stabilization or even modest improvement in vehicle control with increased duration of visual distractions, and this should not be interpreted as an endorsement of introducing routine distractions as a method to enhance driving skills. Instead, these findings highlight the adaptability and resilience of human attention and control mechanisms under prolonged exposure to distraction. Further research is needed to understand how these adaptations can be leveraged safely and effectively without compromising road safety.

Referring to the effects of distraction duration on the longitudinal control of the vehicle, as the duration of distraction increases, both the standard deviation of speed and the standard deviation of acceleration show a non-linear and significant rise, indicating that drivers experience a deterioration in longitudinal control ability with prolonged distraction, which is consistent with the findings by [21], who noted increased speed variation in tasks demanding higher mental workload. Moreover, as the distraction task diverts the driver’s focus, their ability to control speed diminishes, aligning with the conclusions presented by [38]. Nevertheless, with further increases in distraction duration, drivers exhibit stability or even improvement in longitudinal vehicle control, suggesting that drivers may adapt to the impact of distraction to some extent, leading to a reduction in its influence on longitudinal vehicle control. This suggests that as the cumulative duration of visual distraction tasks increases, drivers gradually regain and adapt their longitudinal vehicle control capabilities to the visual distraction tasks.

Referring to the effects of distraction duration on detection response, as the distraction duration increased, there was an observed elevation in the initial missing rate and reaction time. This indicates an increase in the psychological load on the drivers. Previous research also indicated that additional cognitive tasks can increase mental workload, subsequently leading to an increase in reaction time and missing rates [53]. However, with a further increase in distraction duration, the missing rate gradually decreased and stabilized at 0.12, while the reaction time initially remained stable, starting to decrease around 100 s, eventually reaching 0.67. This suggests that with the increase in distraction duration, drivers may experience a reduced mental workload in the detection response task, primarily due to increased proficiency and automation in handling these tasks. As task processing becomes more automated and attention allocation is optimized, drivers are able to maintain or even enhance task performance at a lower mental workload. This phenomenon aligns with the literature on long-term adaptation and cognitive resource optimization [54]. These trends affirm that as the cumulative duration of visual distraction tasks increases, drivers may demonstrate improved responsiveness and reduced missing rates. Despite the negative association observed between distraction duration and missing rate (r = −0.6896, p < 0.001), as well as reaction time (r = −0.5554, p < 0.001), our study results indicate that missing rate and reaction time initially increase, suggesting a rise in cognitive load. This aligns with previous findings that additional cognitive tasks elevate mental workload, leading to increased reaction times and missing rates. As distraction duration extended further, we observed stabilization and even a reduction in missing rate and reaction time. This does not imply that drivers’ overall performance improved but rather shifted from a very poor state to a less poor state. This phenomenon can be attributed to the drivers’ adaptation and increased automaticity in handling distraction tasks. Over time, drivers become more proficient at managing distractions, allowing for better attention allocation and reduced cognitive load.

Referring to the effects of distraction duration on mental workload, with increasing duration of distraction, we observed a swift escalation in both pupil size and fixation duration, indicating an accelerated increase in the mental workload of drivers. This could be attributed to the fact that the distraction task heightens the driver’s cognitive burden and intensifies psychological effort, consequently elevating their mental workload [55]. However, as the duration of distraction further increases, mental workload stabilizes and even experiences a slight decrease. This contradicts findings from a study that reported that driver distraction was found to increase mental workload [26,38]. This suggests that drivers may gradually adapt to distractions, and it may also imply that the impact of distractions on mental workload is no longer significant to a certain extent. Beyond a certain point, it does not lead to a corresponding increase in mental workload. This indicates that the cumulative effect of visual distraction does not continuously increase the mental workload of drivers; instead, it reaches a plateau or even decreases slightly. This suggests that drivers might adapt or become habituated to the distraction over time. As they learn to manage cognitive demands more efficiently, the increase in mental workload stabilizes and may even diminish. This adaptive response could have important implications for understanding how long-term exposure to distractions affects driver performance and safety.

Several limitations were present within this study. Although the participant sample included individuals from all age groups, there was a notably low representation of elderly participants. A portion of the older participants could not successfully complete the experiment, leading to a further reduction in their representation in the data. As such, the generalizability of the findings is constrained and should be considered within the context of this limited demographic coverage. Studies using driving simulators often induce atypical driving behaviors, attributed to the absence of real-life risks in simulated environments and the learning impacts from event repetitions [56]. The distractor task was auto-paced, differing from most real-world, driver-paced secondary tasks, thus affecting the applicability of our findings to everyday driving scenarios. The experiment’s duration, limited to 300 s, does not capture the complexities of longer driving sessions, which may involve prolonged and varied distractions.

6. Conclusions

Advancements in 5G and vehicle intelligence have notably increased full-touch screens’ prevalence in production vehicles’ intelligent cockpits. As vehicles become more connected and interactive, understanding the implications of full-touch screens on driver attention and safety has become crucial. This study, therefore, seeks to bridge the gap in understanding by focusing on the specific effects of touchscreen interfaces within these intelligent cockpits. Specifically, it investigates the impact of driver distraction caused by operating the central console touchscreen on driving performance and mental workload.

This study aims to explore how the duration of driving distraction influences changes in both driving performance and mental workload among drivers. The effects of distraction duration on vehicle control and mental workload reveal that initially, within the first fifty seconds, distraction leads to a deterioration in lateral and longitudinal control, increased missing rates, and reaction times, indicating higher mental workload. However, as the duration of distraction continues to increase, particularly after 150 s, drivers gradually adapt to the impact of distraction, rendering its effects on driving performance and mental workload less pronounced. This further indicates that there may be an upper limit to the influence of distraction time on drivers. Once this limit is surpassed, there is potential for stabilization or improvement in vehicle control to some extent, along with a reduction in the impact on mental workload. The SDLP increased from an initial value of 0.45 m to a peak of 0.78 m within the first 100 s of distraction before stabilizing around 0.65 m beyond 150 s. The missing rate for detection response tasks initially increased but then decreased from 0.40 to 0.12 over the duration of the experiment. Similarly, reaction times initially increased but eventually improved from 0.82 s to 0.67 s as drivers adapted to the prolonged distraction. Average pupil size, indicative of mental workload, initially increased but stabilized at 3.54 mm after 223 s of distraction, demonstrating a non-linear response to prolonged distraction.

This finding indicates a non-linear positive correlation between distraction duration and driving performance, emphasizing the dynamic nature of driver adaptation to distractions over time. By shedding light on the cumulative effects of visual secondary tasks from full-touch screens, this research enhances our understanding of the impact on driving safety by focusing on driver performance and mental workload. Additionally, it explores the temporal aspects of driver behavior, eye movements, and mental workload, providing critical insights into how drivers interact with in-car technology over time.