Studying Driver’s Perception Arousal and Takeover Performance in Autonomous Driving

Abstract

The driver’s perception level and takeover performance are two major factors that result in accidents in autonomous vehicles. This study’s goal is to analyze the change in drivers’ perception level and its influence on takeover performance during autonomous driving. A takeover behavior test platform is implemented based on a high-fidelity driving simulator. The fog zone is selected as the takeover scenario. Thus, a 2 (takeover request time: 5 s, 10 s) by 2 (non-driving-related task: work task, entertainment task) takeover experiment was conducted. A generalized linear mixed model is developed to explore the influence of the perception level on takeover performance. The study finds out that, after the takeover request is triggered, the driver’s gaze duration is shortened and the pupil area is enlarged, which is helpful for the driver to extract and understand the road information faster. Male drivers have greater perception levels than female drivers, and they prioritize leisure tasks more than professional ones. The drivers’ perception level decreases when age increases. The shorter the gaze duration is, and the larger the pupil area is, the shorter the takeover response time will be. In addition, drivers’ perception level has a positive effect on takeover performance. Finally, this study provides a reference for revealing the changing rules of drivers’ perception level in autonomous driving, and the study can provide support for the diagnosis of takeover risks of autonomous vehicles from the perspective of human factors.

1. Introduction

Autonomous vehicles have been proven to bring new possibilities to improve the safety [1], efficiency [2], and ecology [3] of transportation. The Society of Automotive Engineers (SAE) classifies autonomous vehicles into six levels (L0-L5) [4] where L3 autonomous vehicles share the normal driving tasks of the driver, resulting in the driver being transformed into a supervisor [5]. For the conditional autonomous vehicle, the driver must take over in many hazardous scenarios. In fact, on the freeway, fog zone, rain, storm, snow, and other weather conditions are more hazardous scenarios for autonomous vehicles whereas fog zone occurs more frequently on freeways. In addition, the probability of accidents in a such zone is ten times greater than on normal roads [6], which further increases the risk of autonomous vehicles driving in such conditions. Considering the experiment cost, the fog zone is analyzed as a case scenario in this study. Before the autonomous vehicle enters this zone, the driver is in a supervision state, and he must take over control of the autonomous vehicle at the instant of entering it. Moreover, many studies show that, when the driver is in a supervision state, the driver’s perception will be reduced, and the takeover quality will deteriorate in autonomous driving mode [7,8]. This implies that the drivers’ perception can affect the takeover performance. Therefore, the study of driver’s perception arousal and takeover performance, being the subject of this research, is of great importance for the safety of L3 autonomous vehicles in a fog zone.

In automated driving, the drivers’ perception ability is affected by driver’s individual characteristics, driving tasks and non-driving related tasks, and other environmental factors. From the human factors perspectives, Kaber et al. [9] pointed out that the increase of age will impair drivers’ cognitive ability and perception ability. Added to that, the complex and changeable driving environment will further undermine drivers’ perception ability. Young et al. [10] mentioned that distraction can lead the driver to losing his/her perception, which may result in safety drop. As for the perspective of non-driving-related tasks, De Winter et al. [11] found that when drivers’ cognitive resources released by autonomous vehicles are used to engage in non-driving related tasks, drivers’ attention level and perception will be significantly reduced. In fact, drivers’ attention span and perception will be greatly diminished when cognitive resources, made available by autonomous vehicles, are used for non-driving-related activities. Thus, when these released drivers’ cognitive resources are used to monitor the road, the perception ability will be significantly improved. However, excessive immersion in monotonous road monitoring tasks will lead to passive fatigue leading to a decrease in the perception ability of drivers due to low workload [12]. Ma et al. [13] found that non-driving-related tasks would deprive drivers’ cognitive resources and impair their perception ability. Moreover, it can be seen, based on the previously presented research, that drivers’ perception ability in autonomous driving is affected by many factors, and drivers’ perception ability is correlated with takeover performance.

When the driver’s perception is awakened, reliable takeover performance at the safety level is guaranteed for the autonomous vehicle. Moreover, the takeover request method is the key to awakening the driver’s perception. Bazilinskyy et al. [14] and Chen et al. [15] found that takeover request occupy the three channels of vision, hearing, and touch, and the multi-modal takeover request form can transmit more effective information and can significantly improve the takeover performance. Besides, the takeover request time will affect the driver’s takeover performance. In addition, Samuel et al. [16] found that when the takeover request time is 4 s, 6 s, 8 s, and 12 s, it takes at least 8 s for the drivers to awaken their perception. Kim and Yang [17] set four levels of takeover request times according to different scene complexities, and found that, with the increase of takeover request time, the maximum acceleration and the lateral deviation gradually decrease. As for other scenarios, the driver’s takeover behavior is different. In more detail, Zhao et al. [18] compared the drivers’ takeover behaviors when driving in fog zone and ordinary conditions, and the results showed that the takeover success rate in fog zone was higher although drivers needed longer takeover response time. Moreover, Wang et al. [19] analyzed the drivers’ takeover behavior in the accident zone, and the results showed that those with a high attention level would have faster avoidance ability. In addition, the individual characteristics of the driver can affect takeover performance. Based on a subjective survey and some experimental methods, Hardman et al. [20], Robertson et al. [21], and Nielsen and Haustein [22] found that male drivers were more interested in autonomous driving technology than female drivers; however, female drivers had a higher focused perception level and more cautious takeover behavior. In addition, Korber et al. [23], Scott-Parker et al. [24], and Clark and Feng [25] found that, with the increase of age, drivers’ cognitive ability and perception declines, leading to a decrease in takeover performance. For instance, Zhao et al. [18] compared the takeover performance of drivers with different ages through driving simulation experiments, and they found out that, in fog environment and accident zone scenarios, younger drivers had the shortest reaction time. Moreover, statistical results show that the number of male drivers successfully taking over autonomous vehicles is higher than female drivers. Salvia et al. [26] designed three No-Driving-Related Tasks (NDRTs) with different mental workloads, and they informed the drivers in advance. The findings revealed that reaction time and error rate increased along with mental workload. Furthermore, many scholars have studied methods to improve drivers’ takeover performance, such as Human-Machine Interaction (HMI) optimization [27] and driving training [28], which are closely related to drivers’ perception arousal.

To sum up, the main objective of this study is to analyze the change of the drivers’ perception level and its influence on takeover performance in autonomous driving. Added to that, an autonomous driving takeover test platform is developed in this study based on a driving simulator. This test platform realizes driver-vehicle-environment closed-loop testing and allowed the reproduction of various HMIs. The virtual scenario was developed based on the fog zone of freeway. A 2 (non-driving related tasks, work tasks and entertainment tasks) × 2 (takeover request time, 5s and 10s) driving simulation experiment is designed. Participants were invited to participate in the driving simulation experiment, and eye-tracking and driving data were stored. This study can provide a reference for revealing the changing behavior of the drivers’ perception level in autonomous driving and provide guidance for proposing preventive measures that can improve takeover performance and reduce takeover safety risks from the perspective of human factors.

To end up, this paper is divided as follows: in Section 2, the methodology of the work will be shown whereas the results and the discussion will be proposed, respectively, in Section 3 and Section 4. Finally, Section 5 consists of a conclusion that will sum up this work and will propose some future ideas.

2. Methodology

2.1. Development of Test Platform

The takeover behavior test platform of the L3 automated driving is developed based on a driving simulator. The platform included four parts: (i) L3 Level for the autonomous driving simulation system; (ii) data acquisition equipment; (iii) collaborative data processing renter; and (iv) vehicle terminal. As for the key connection technologies, they included: (1) the Application Program Interface (API), which adds the simulator script language to create different road, weather, and traffic conditions in the experimental scenario; (2) the User Data Protocol (UDP) interface, which establishes the data interaction between the control center and the collaborative data processing center; and (3) the wireless communication technology (Wi-Fi module), which enables data communication between the computer and the HMI (vehicle terminal) to realize the interconnection between the vehicle and driver.

The AutoSimAS driving simulation system is used and includes a driving simulator, six computers, a Huawei tablet, an eye tracking core+ eyeglasses, and a camera for equipment acquisition. The simulator software is the SCANeR1.9 system, which can dynamically collect its own data operation and manipulation as well as from the surrounding vehicles. The eye-tracker technology is an ETG 2w portable eyeglass eye-tracking system device, produced by SMI (company located in Germany). The device can obtain the driver’s eye movement data, including the position of the fixation points and their numbers, the fixation duration, the saccade number and duration, etc. Both driving simulation data and eye movement data are collected at a frequency of 20 Hz. The Hintsoft CAD and the 3D Max software were used to develop virtual scenes that were later on imported to the driving simulation system. To sum up, the structure and the equipment of the research platform are shown in Figure 1.

2.2. Experimental Design

2.2.1. Take-Over Request time (TOR)

TOR refers to the time duration from the instance the takeover request message is sent to the instance the automated driving is disabled. Previous studies have shown that when the TOR is 5 s, it is considered that the safety threshold to meet takeover requirements is established [17], and for 10 s, all takeover operations are met [29]. Therefore, the TOR is set as 5 s and 10 s in the study.

2.2.2. No-Driving-Related Task (NDRT)

NDRT refers to non-driving-related tasks performed by the driver during automated driving, it includes Work Tasks (WT) where drivers were asked to read and speak the same news article and Entertainment Tasks (ET) where drivers were asked to watch the same entertainment videos. which are conducted through the mobile phone.

2.2.3. Takeover Scenario

The fog zone in the freeway was selected as the takeover scenario. The freeway is a one-way path with three lanes (two fast lanes and one emergency lane), and autonomous vehicles can drive on the two fast lanes. According to the “Grade of fog forecast (GBT 27964-2011)” [30], the fog level is designed to be a heavy fog. The fog zone (refer to Figure 2a) involves a fog forming section, a fog section (visibility 725 m), and a fog clearing section. The traffic density is set at 6 pcu/km/lane (free flow). The speed limit of the ordinary roads is 120 km/h, and that of the fog zone is 60 km/h. The autonomous vehicle speed is 100 km/h. After switching from manual driving mode to automatic driving mode, the vehicle speed gradually goes back to 100 km/h.

To sum up, the four takeover scenarios (2 TOR × 2 NDRT) were formed based on TOR and NDRT. To mitigate mutual influence among takeover scenarios, these takeover scenarios (S1–S4) are distributed randomly on the two roads (Figure 2c), and all drivers were asked to go through Road 1 (experiment 1) and Road 2 (experiment 2). In addition, the starting point of both experiments was randomly chosen, and not all drivers started at the same location. The minimum distance from the starting point to point A (Figure 2a) is 1 km, so that the driver can achieve a stable driving state before entering the fog zone. A sufficiently long transition section is placed between the two scenarios in Roads 1 and 2.

2.2.4. Takeover Request Method

In this study, four different HMI were developed based on takeover requirements and existing studies [31,32], and the HMI is optimized according to ‘Human Factors Design Guidance for L2 and L3 Automated Driving’ [33]. The HMI is developed using HUAWEI tablet, which provides both visual and auditory takeover request information. The HMI is described as follows:

• Manual driving: Figure 3a shows that the automated driving system is unavailable due to the constraints of the surrounding traffic environment or the failure of the automated driving system. Thus, the driver manually controls the autonomous vehicle;
• The driver to the automated driving system: When the automated driving system is available (refer to Figure 3b), the driver can press the “Activate” button (shown in Figure 1b);
• Autonomous driving: Figure 3c shows that the automated driving system is activated, and the automated driving system operates normally;
• The automated driving system to the driver: When the vehicle passes the warning point (points B or C in Figure 2a), the HMI will send a takeover request information (as shown in Figure 3d), and the driver should press the “Takeover” button (displayed in Figure 1b).

2.3. Participants

42 participants were invited to take part in this study. Gender and age were considered to describe the driver’s individual characteristics and information, as shown in

Table 1. Driver’s individual characteristics.

Attribute	Level	Number	Mean	Standard Deviation
Gender	Male	32	-	-
	Female	10	-	-
Age	Young (18–35)	15	23.2	2.0
	Middle-aged (36–60)	14	46.5	6.6
	Older (>60)	13	63.7	2.9

. According to the ‘Medium- and long-term youth development plan (2016–2025)’ [34] and the ‘Law of the People’s Republic of China on the Protection of The Rights and Interests of the Elderly’ [35], the age can be divided into three levels: young (18–35), middle-aged (36–60) and older (greater than 60) whereas the measurement unit is the year.

2.4. Experimental Procedure

Each driver is required to participate in two experiments where the starting points are randomly selected. Experiment 1 consists of scenarios S1 and S2 whereas experiment 2 contains scenarios S3 and S4. The experiments procedures are described below:

(1)

The participant fills out the informed consent form and the basic information form;

(2)

Pre-experiment driver training where theoretical training, video training, and practical operation training (30 min to 40 min in total) are conducted;

(3)

Experiment 1: Experiment 1 begins when the driver wears the test equipment and the experimenter reads him the instructions. The driver passes through scenarios S1 and S2. Experiment 1 takes about 18 min for the vehicle to go from the start to the end of Road 1, including the automated and the manual-driven modes.

(4)

Experimental interval. After Experiment 1, the experimenter sorted out the equipment and prepared for Experiment 2 after having a rest for about 10 to 15 min;

(5)

Experiment 2: The process in Experiment 2 is the same as in Experiment 1. The driver passes through scenarios S3 and S4, and the duration of Experiment 2 is also about 18 min;

(6)

Experiment ends: Drivers fill out the subjective questionnaire and receive remuneration.

2.5. Data Preprocessing and Indicators Selection

42 drivers participated in 4 takeover scenarios, 168 takeover behavior data were collected through experiment, among which 160 valid data were collected after removing invalid data (the driver did not take over the autonomous vehicle within TOR). Visual indicators are used to quantify the drivers’ perception, and indicators of takeover performance are extracted from response time and stability perspectives. The two levels of indicators are described as follows:

Indicators of Drivers’ Perception

(1)

Gaze duration (unit: s)

The duration that the driver’s eyes stay focused at a certain fixation point. The longer the gaze duration, the longer the driver takes to extract and understand the road information, and the lower the perception level of the driver will be.

(2)

Pupil area (unit: mm²)

The average pupil area of a driver over a period. The larger the pupil area, the higher the driver’s visual workload, the more road information is acquired, and the higher the perception level of the driver will be.

Indicators of Takeover Performance

(3)

Takeover Response Time (TRT)

This refers to the duration required from HMI to send the takeover request information to the driver to press the “Takeover” button. It is an intuitive indicator to measure the driver’s takeover response.

2.6. Analytical Method

The test results show that the data does not meet the normal distribution, so the non-parametric statistical method is adopted. The Wilcoxon signed-rank test was used for statistical tests of two associated samples data whereas the Mann-Whitney U test was used for statistical tests of two independent samples data and finally, the Kruskal-Wallis H test was used for statistical tests of multiple independent sample data. The significant level was fixed at 5% (p < 0.05) whereas the marginal significance is considered at 10% (p < 0.1) [36].

As the driver’s takeover performance is affected by individual characteristics and external environmental conditions, this study explores the influence of awakened drivers’ perception level on takeover performance; thus, both drivers’ parameters (e.g., individual characteristics and external variables) should be considered together. Added to that, as the influence of the driver’s individual characteristics on the dependent variable is nonlinear, the generalized linear mixed model is selected. Furthermore, the generalized linear mixed model (GLMM) model has unique advantages in solving the problems of joint features among multiple data sets, complex correlation structure, and diverse sharing feature, and this method may be used in a variety of situations since it does not require that data fits into a normal distribution.

In more detail, the GLMM model [37] is based on the generalized linear model [38], which introduces random effects parameters. It can deal with a variety of research designs and data types, and it is suitable for data with non-normal distribution and complex correlation structure. The general expression of the GLMM model is:

$$Y=X\beta +Zu+\epsilon$$

(1)

where X and Z are, respectively, the construction matrix of the fixed effect variables and the random effect variables, β is the vector of unknown regression coefficients called fixed effects, u is the vector of random effect parameter, and ε is the vector of random error.

In this study, the GLMM model was used to explore the influence of the drivers’ individual characteristics (gender, age), perception level (gaze duration, pupil area), and external environmental factors (TOR, NDRT) on the TRT value. The main effect of each parameter on the TRT value, and the interaction effect between these variables and the TRT can be calculated. Therefore, the contribution of each variable over the TRT will be explored.

3. Results

3.1. Characteristic Analysis of Driver’s Perception Level

3.1.1. Driver’s Perception Restored

When the vehicle is in autonomous driving, the driver’s perception is focused on NDRTs. When the autonomous vehicle sends the takeover request, the perception of the driver shifts towards the driving task. To explore the changing rules of the drivers’ perception level after the takeover request is triggered, this study compares and analyzes the characteristics of drivers’ perception level within 10 s before and after the takeover request is triggered. The statistics and the test results are shown in

Table 2. Statistical results of drivers’ perception level restore indicators.

Indicator	10 s before TOR Is Triggered		10 s after TOR Is Triggered		p-Value
	Mean Value	Standard Deviation	Mean Value	Standard Deviation
Gaze duration (s)	0.281	0.095	0.255	0.115	0.037 **
Pupil area (mm2)	11.743	5.114	13.957	6.097	0.051 *

Note: ** p < 0.05 significant at 5%, * p < 0.1 marginal significance.

.

Furthermore, Table 2 shows that the takeover request warning has a statistical significance on the driver’s gaze duration (as the p-value is equal to 0.037). Thus, when the driver is awakened, the gaze duration decreases, indicating that the driver spends less time extracting and understanding road information in order to respond and to takeover in a faster way. The level of the drivers’ perception level has increased. Added to that, there is marginal significance between the pupil area of the driver before and after they are awakened (as the p-value is equal to 0.051). After the takeover request is triggered, the driver’s pupil area and the driver’s visual workload increase. This indicates that the driver receives more visual information, which is used to support takeover and subsequent control of the autonomous vehicle.

In conclusion, after the takeover request is triggered, the driver can improve the perception level by shortening the gaze duration and increasing the pupil area, which can reduce the takeover risk and improve the takeover performance.

3.1.2. Influencing Factors of Drivers’ Perception Restore

The differences of the drivers’ perception level occurred after the takeover request is triggered, and these differences are affected by the individual characteristics and the external environment parameters. This study analyzed the drivers’ perception level after being awakened under different individual characteristics (gender, age) and external environment (TOR, NDRT). The results are displayed in Figure 4 and Figure 5.

Furthermore, Figure 4 shows that the gaze duration for male drivers is shorter than for female drivers and the p-value shows significant differences (equal to 0.041), which means that male drivers can extract and understand information faster. Added to that, the driver’s gaze duration increases with age. There is a statistical difference in gaze duration between young and old drivers and the p-value is equal to 0.037, which may be related to the poor physical functional responses for old drivers as this category takes a longer time to understand and acquire information and has a lower perception level. The gaze duration of the work task is higher than that of the entertainment task without getting statistical differences (as the p-value is equal to 0.067). The work task requires more attention from the driver, and it takes a longer time to restore the perception level, which leads to a decrease in the speed of the driver’s information extraction and understanding. Finally, there is no significant difference between the takeover request time and the driver’s gaze duration.

Figure 5 shows that the pupil area of female drivers is higher than that of male drivers (as the p-value is equal to 0.023); however, the visual workload of female drivers is higher. In addition, the gaze duration of female drivers is longer, showing that female drivers are more careful and cautious in extracting and understanding road information. Pupil area decreases with age, and young drivers have the highest ability to obtain road information, as well as higher perception level. The pupil area of the work tasks is lower than that of the entertainment tasks (as the p-value is equal to 0.076). Finally, drivers require more attention to perform work tasks and have limited access to road information after being awakened.

3.2. Analysis of the Driver’s Perception Level on Takeover Performance

Before applying the GLMM, the perception level of the drivers needs to be divided into categorical variables. The K-Means clustering method was adopted to classify the three driver’s perception indicators already listed (i.e., gaze duration, pupil area) into two levels: low and high. The classification results are shown in

Table 3. K-Means cluster results of drivers’ perception level indicators.

Indicator	Group	Range	Numbers	Ratio	p-Value
Gaze duration (s)	➀ Low	[0.022, 0.275]	98	0.61	<0.001 **
	➁ High	(0.275, 0.530]	62	0.39
Pupil area (mm2)	➀ Low	[4.627, 15.618]	89	0.56	<0.001 **
	➁ High	(15.618, 31.743]	71	0.44

Note: ** p < 0.05 significant at 5%.

. To test the rationality of the classification results, the Wilcoxon W test was used to calculate the statistical significance of each level, and the results showed that there was prominent statistical significance between the levels. Therefore, the classification results of independent variables are reasonable.

Drivers’ takeover performance is affected by, not only the driver’s perception level, but also by the individual characteristics and the external environment. Therefore, the four different parameters (i.e., gender, age, TOR, and NDRT) are introduced into GLMM. In GLMM, drivers’ perception level indicators (gaze duration, pupil area), individual characteristics factors (gender, age), external environmental factors (TOR, NDRT) were used as fixed effect factors. Gender and age represent the heterogeneity of different drivers, so both variables are also considered as random effect factors. Takeover performance indicators include TRT. As for this study, it mainly focuses on the effect of drivers’ perception level on takeover performance. Thus, the parameter results of GLMM are shown in

Table 4. Parameter results of GLMM.

Indicator	Level	Baseline	Takeover Response Time (Unit: s)
			β	p
Intercept			1.226	<0.001 **
Gaze duration	High	Low	0.243	0.017 **
Pupil area	High	Low	−0.157	0.032 **
Gender	Male	Female	−0.032	0.087 *
Age	Middle	Young	0.043	0.073 *
	Older	Young	0.179	0.024 **
TOR	10 s	5 s	0.025	0.115
NDRT	WT	ET	0.037	0.085 *
Gaze duration × Gender	High × Female	Low × Female	0.132	0.031 **
	High × Male	Low × Male	0.293	0.012 **
Gaze duration × Age	High × Young	Low × Young	0.213	0.026 **
	High × Middle	Low × Middle	0.144	0.046 **
	High × Older	Low × Older	0.078	0.061 *
Gaze duration × TOR	High × 5 s	Low × 5 s	0.017	0.332
	High × 10 s	Low × 10 s	0.015	0.217
Gaze duration × NDRT	High × WT	Low × WT	0.113	0.041 **
	High × ET	Low × ET	0.021	0.196
Pupil area × Gender	High × Female	Low × Female	−0.027	0.109
	High × Male	Low × Male	−0.157	0.037 **
Pupil area × Age	High × Young	Low × Young	−0.189	0.031 **
	High × Middle	Low × Middle	−0.013	0.334
	High × Older	Low × Older	−0.014	0.317
Pupil area × TOR	High × 5 s	Low × 5 s	−0.009	0.513
	High × 10 s	Low × 10 s	0.012	0.411
Pupil area × NDRT	High × WT	Low × WT	−0.139	0.039 **
	High × ET	Low × ET	0.017	0.341
Omnibus test			χ2 = 18.881, p = 0.016 **

Note: ** p < 0.05 significant at 5%, * p < 0.1 marginal significance.

.

GLMM analyses the takeover response time, and the below conclusions are obtained:

-

With the increasing level of gaze duration, takeover response time was prolonged by 0.243 s (having a p = 0.017). The takeover response time decreased with the increase of pupil area. This indicates that the improvement of the driver’s perception level helps to enhance the driver’s reaction capacity.

-

Taking the female driver as the baseline, the takeover response time of the male driver was reduced. A statistical difference between age and takeover response time is obtained, that is the takeover response time decreased with the increase of age;

-

Non-driving related tasks have marginal significance on takeover response time (p = 0.085), while work tasks had longer takeover response time;

-

The driver’s gaze duration has an interaction with the gender, the age, and the non-driving related tasks on takeover response time, and the increase of the driver’s gaze duration yields in an increase in the driver’s takeover response time;

-

The pupil area has an interaction with the gender, the age, and the non-driving related tasks on takeover response time, and the increase of the pupil area decreased the driver’s takeover response time.

To sum up, the driver’s gaze duration is shortened, and the pupil area is increased during the takeover process. Both findings indicate that the driver’s perception level has increased. Combining Table 2 and Table 4, it can be concluded that the increase of driver’s perception level contributes to the improvement of reaction capacity. In addition, there was no statistical difference between the TOR and the takeover response time, and there was no interactive effect with the driver’s perception indicators concerning the TOR.

4. Discussion

In this study, high-fidelity driving simulation experiments are carried out. The driver’s eye movement data and takeover performance data are obtained, which can provide good support for the analysis of driver perception level and the takeover performance. The effectiveness of driving simulators has been fully discussed [39], so it will not discussed in this paper. However, the changes of drivers’ perception level, the influence factor of perception level, and the influence of perception level on takeover performance in autonomous driving are discussed in the remaining part of this section.

In autonomous driving, the driving operation and the monitoring tasks are handles by autonomous vehicles, and the driver breaks away from the loop to participate in non-driving related tasks, which will cause the change of the driver’s perception level. This study found that, after the takeover request was triggered, the driver could quickly extract and understand the road information by shortening the gaze duration and increasing the pupil area. Theus, the perception level of the driver was improved where it is conducive to increasing the response ability and making preparation for taking over the autonomous vehicle. Stephenson et al. [40] found that the skin conductance level of drivers increased after triggering the takeover request, indicating that the drivers’ perception was aroused. To sum up, after the takeover request is triggered, the perception level of drivers will increase. Drivers will improve their perception level by shortening the gaze duration and increasing the pupil area. The takeover request is an important factor in awakening the driver’s perception.

Driver’s perception level is influenced by individual characteristics and external environment. In this study, the driver’s perception level was represented by gaze duration and pupil area. The study found that the perception level of male drivers was higher than that of female drivers after the takeover request was triggered, and the perception level of drivers decreased with the increase of age. The perception level of entertainment tasks was higher than that of work tasks. Eisma et al. [41] found that more attractive non-driving related tasks would lower the perception level. Added to that, the driver’s trust level for the autonomous vehicle can affect his perception level during automated driving [42]. This provides a strong support for analyzing the perception changes of drivers.

Drivers’ takeover performance is affected by, not only the drivers’ perception level, but also the drivers’ individual characteristics [16] and the external factors [19]. In the study, the takeover response time decreased with the increase of the drivers’ perception level, which indicated that drivers’ perception was helpful to improve his takeover performance. The drivers’ perception level interacts with individual characteristics and external factors on takeover performance. Guo et al. [43] found that gaze duration was positively correlated with takeover response time and brake response time. In general, the improvement of drivers’ perception level can improve takeover performance, thus improving the safety of drivers and autonomous vehicles.

5. Conclusions

In this study, the driving simulation experiment of autonomous vehicle takeover in a fog zone was carried out. Based on the data statistics and the GLMM method, the change of the drivers’ perception level and its influencing factors, and the influence of the drivers’ perception level on takeover performance were analyzed. The research conclusions are as follows.

(1)

The driver’s perception level was quantified by gaze duration and pupil size. After the takeover request is triggered, the drivers’ perception level was significantly restored. The perception level of male drivers was higher than that of female drivers, and the entertainment task was higher than work task. The drivers’ perception level decreases with the increasing age.

(2)

The driver’s takeover performance is quantified by the takeover response time. In GLMM, drivers’ perception has a positive effect on takeover performance. The perception level of drivers interacted with gender, age, and non-driving related tasks on takeover performance. The shorter the gaze duration, the larger the pupil area, and the shorter the takeover response time.

This study analyzes the change of drivers’ perception level in the takeover process and the influence factors of the perception level are explored. Moreover, takeover response time was determined as a dependent variable whereas the perception level and other factors were considered as independent variables. Furthermore, the influence factors of the optimal design of the human-machine interface were obtained. The findings show that the driver attribute factors have an impact on the reaction time and the perception level, which indicates that driver differentiation training is necessary when dealing with autonomous vehicles. This study can provide a strong support for the design and the optimization of human-machine interaction system for autonomous vehicles from the perspective of improving the driver’s perception level. Moreover, gender and age parameters have an impact on takeover response time. Thus, differentiated training for different drivers can improve the drivers’ adaptability and perception level to control autonomous vehicles, yielding to improve the takeover response ability. Therefore, this study can also provide support for improving driver takeover response ability through driver differential training. In addition, the research may trigger follow-up studies that will be mounted through the theory of perception and reception and the other theories belonging to this paradigm. It will be possible to evaluate what is called “performances”, not only of drivers but also of other decision makers in a variety of fields, such as commanders on the battlefield, school teachers, and others. Finally, all these entities can join what distinguishes the post-heroic research paradigm based on the theory of perception [44].

However, this study has some limitations as the selection of non-driving-related tasks is reduced (results cannot be generalized over all non-driving-related tasks in the real world). As for the future studies, the influence of takeover scenarios, the individual characteristics, and the external environment on drivers’ perception level as well as the takeover performance will be further studied. Moreover, environmental conditions as rain, storm, snow and other scenarios will be also studied and analyzed.