The Decline of User Experience in Transition from Automated Driving to Manual Driving

Abstract

Automated driving technologies are rapidly being developed. However, until vehicles are fully automated, the control of the dynamic driving task will be shifted between the driver and automated driving system. This paper aims to explore how transitions from automated driving to manual driving affect user experience and how that experience correlates to take-over performance. In the study 20 participants experienced using an automated driving system during rush-hour traffic in the San Francisco Bay Area, CA, USA. The automated driving system was available in congested traffic situations and when active, the participants could engage in non-driving related activities. The participants were interviewed afterwards regarding their experience of the transitions. The findings show that most of the participants experienced the transition from automated driving to manual driving as negative. Their user experience seems to be shaped by several reasons that differ in temporality and are derived from different phases during the transition process. The results regarding correlation between participants’ experience and take-over performance are inconclusive, but some trends were identified. The study highlights the need for new design solutions that do not only improve drivers’ take-over performance, but also enhance user experience during take-over requests from automated to manual driving.

1. Introduction

Automated driving is an emerging technology that has received widespread attention recently, both in industry, academia and by the public. Even if the technology is rapidly evolving, fully automated vehicles which are able to operate during all conditions will not be widely available within the next few decades, according to predictions [1]. Until then, the control of the dynamic driving task will be shifted between the driver and automated driving system. In terms of the Society of Automotive Engineers (SAE)-taxonomy [2], conditional and high driving automation (L3–4) require drivers to resume manual control when the operational conditions of the system are no longer met. The transition from not driving to becoming an attentive driver within a restricted time period can be cumbersome. Thus, the performance of drivers who must resume control of the car has generated considerable research interest in recent years. This research shows that take-over times as well as aftereffects post take-over vary and are affected by factors such as the urgency of take-over [3], experience [4], traffic density [5] and engagement in non-driving related tasks [6,7,8]. For more extensive reviews, see McDonald and colleagues [9] and Zhang and colleagues [10].

To date, the majority of studies focusing on control resumption have mainly considered safety related aspects, such as take-over performance and quality. However, few studies have focused on the experience of resuming control of the car [11]. Therefore, this paper aims to explore how users experience the transition from automated driving to manual driving and how the experience relates to the users’ take-over time.

User Experience

User experience (UX) as a field of research has developed from various different disciplines such as cognitive science, design, psychology, and engineering [12]. There is a wide variety of descriptions of the constitution of UX; a commonly agreed definition is framed in ISO 9241-210: 2019, where it is stated that UX is the “user’s perceptions and responses that result from the use and/or anticipated use of a system, product or service”. Thus, UX includes the users’ perception and responses before, during and after use and is a consequence of, but not exclusively related to, system performance. UX goes beyond usability by also incorporating hedonic qualities, such as the user’s needs and emotional experience during product use [13]. Two important dimensions of UX are contextual and temporal dimensions, since experience is highly affected by the context that the product is used within and that it is present-oriented as well as changing over time [14,15].

UX research in automated driving has to date been scarce but is steadily growing [11]. Earlier UX research indicates that control, autonomy, security, and trust are important factors when the vehicle is in autonomous mode [16,17,18]. Furthermore, being able to manage time and socialize are considered to be critical for the adoption of the technology [19]. Some studies have investigated how UX can be increased by different design solutions. In a study by Frison and colleagues [20], their interface design increased the participants level of autonomy and stimulation when using a fully automated vehicle, and a study by Karjanto and colleagues [21] showed that a peripheral visual feedforward system increased UX. Regarding the contextual dimension, Bjørner [18] argues that UX must be considered and explored within the various contexts that it can be used, such as different road conditions, purposes, or social environments. The importance of temporality of UX in automated vehicles has also been emphasized, since UX evolves over time from first appeal, experience of direct use, to forming habits [22].

Several studies have investigated which aspects of different design solutions can reduce the negative impacts of the transition from automated to manual driving. Some of the studies have involved auditory feedback, including pulse audio pre-alerts preceding the more standard take-over prompts [23], various types of auditory (non-speech) feedback [24], multi-step speech outputs [25] and natural language reliability displays [26]. Other studies have investigated visual feedback in the steering wheel, including information about automation state and notifications regarding take-overs [27], and physical changes of the steering wheel [28]. However, few studies have focused on the UX of the transition per se, and how well it corresponds to the take-over performance.

2. Method

This paper is based on an on-road study where 20 participants experienced an advanced driver assistance system (ADAS) and an automated driving system using a Wizard of Oz setup. The study was conducted in the San Francisco Bay Area, CA, USA, with the purpose to explore how users experience driving a car with several levels of automation. The analysis presented in this paper focuses on how the participants experienced the transition from automated driving to manual driving. In addition, the participants’ transition experience is compared to the participants’ take-over performance in a cluster analysis.

2.1. Participants

The study included 11 female and 9 male participants ranging from 22 to 62 years old (M = 41.5, SD = 13.74). The inclusion criteria for the participants were: (i) holds a valid driving license, (ii) commutes by car daily, and (iii) drives a car equipped with automatic gearbox and driver assistance system in terms of cruise control. None of the participants worked at a vehicle manufacturer or other company related to vehicle development. The participants were recruited and reimbursed trough a local recruitment agency.

2.2. Setup

A Wizard of Oz setup was used, in order to explore the participants’ experience in a naturalistic setting. The setup consisted of a modified Volvo XC90 that was equipped with additional vehicle controls in the back seat, which were concealed for the participant in the front seat. The test vehicle had two different automated systems, an advanced driving assistance system and an automated driving system, where the later was simulated by a professional driver (the wizard), using the additional vehicle controls. Two different test drivers acted as ‘wizards’ and to ensure that they drove as consistently as possible, several practice runs were performed on the test route before the actual test. The test drivers also used an advanced driver assistance system to be able to keep a consistent acceleration/deceleration pattern and distance to lead vehicle. An additional person (the Human Machine Interaction (HMI) wizard), was also positioned in the back seat, managing the information provided to the user during automated driving. The test leader was positioned in the front passenger seat during the ride.

The advanced driver assistance system (ADAS) provided support regarding keeping a set speed, adjusting the speed with regard to vehicles in front, and provided steering assistance through lane-keeping support where the driver was still in control of the dynamic driving task. This system was available at any time and could be activated and deactivated by a short press on a button on the steering wheel whenever the driver preferred. An icon and graphics in the instrument cluster indicated when the system was active.

The automated driving system (ADS) was only available in congested traffic situations during the following conditions: (i) travel directions separated by center barrier, (ii) lead vehicle present, (iii) max speed of 60 kph, and (iv) similar speed of traffic in adjacent lanes as host vehicle. When these conditions were met, the driver was notified by an auditory cue and a visual cue in the instrument cluster, informing that the automated driving system is now available. To activate the system the driver had to make a sustained (600 ms) press on two buttons on the left and right side of the steering wheel. When the system was active, the vehicle performed the dynamic driving task (full lateral and longitudinal control) and the participants were able to engage in non-driving related activities, e.g., using the phone or eating. Whenever the availability conditions were no longer fulfilled, the participants were prompted to resume manual control by a visual and auditory prompt, as well as a pull of the seatbelt. If the participant did not take back the control within 10 s the prompts were intensified with a strengthened visual prompt, a more urgent auditory prompt, and an additional pull of the seatbelt. To deactivate the system the driver performed the same procedure as when activating it; a simultaneous sustained (600 ms) press on two buttons located on the left and right side of the steering wheel until the system deactivated.

2.3. Procedure

Before the driving sessions, the participants received general information about the test procedure from the test leader and their informed consent was obtained, in accordance with the General Data Protection Regulation (GDPR) [29] in accordance with EU law. They were also presented with oral and written instructions about the functionalities of the advanced driving assistance system and automated driving system, i.e., the capabilities and limitations of each system. The participants were informed that the automated driving system takes over the driving task completely under congested traffic conditions but did not receive information about the detailed availability conditions. They were also informed that the system will notify the driver when available (availability conditions fulfilled) and when conditions were no longer met, and the driver needs to take over. Information about how to activate and deactivate the function and where notifications would be available was provided, but not on the look of the notifications. When in the car, the driving wizard was introduced to the participants as a safety driver, the HMI wizard as a note taker, and the test leader described the functionalities of, and the interaction with, the systems once more.

The driving sessions were conducted during morning and afternoon rush-hour on highways and urban areas, and each session took approximately 1.5 h. The participants were encouraged to use the systems as much as possible during the drive, but the amount of availability differed depending on the traffic conditions. The number of occasions that the automated driving system was active during a driving session ranged from 2 to 5 (median = 3) and the total time the system was active during a driving session ranged from 12.67 to 46.58 min (M = 21.36, SD = 8.52). When the automated driving system was active, the participants were informed that they could engage in non-driving relate activities. Many participants (65%) engaged in non-driving related activities at least at one occasion but all of them still repeatedly looked at the road or instrument cluster. The activities consisted of: (i) using the center panel display (56%), (ii) using a mobile phone (33%), and (iii) eating and drinking (11%). Most (85%) of the transitions from automated driving back to manual driving were prompted by the automated driving system because conditions were no longer met, i.e., the traffic became lighter and the speed increased to above 60kph.

2.4. Data Collection

Both qualitative and quantitative data were collected in order to capture the participants’ subjective experience of the transition as well as the objective performance during the take-over.

2.4.1. Qualitative

Directly after the driving session, the participants were first interviewed about their experience of the take-overs and then interviewed, using a UX-curve procedure, to elicit their general experience of the driving session and the transitions. UX-curve procedures have previously been used in studies with manually driven cars [30] as well as in studies on automated vehicles [19,31]. The participants were asked to describe their general experience with the different systems and during transitions by illustrating their experience as a curve, on a paper template, and at the same time explaining why they drew the curve as they did. Most of the participants described a general assessment of the experience of using the systems and the transitions but some also described a change in experience over several transitions, for example describing how the experience improved after experiencing several transitions. The paper template was a modification of the original template [32] and consisted of two axes, a y-axis that ranged from very positive in the top to very negative in the bottom, and an x-axis with four different areas representing: (i) driving manually, (ii) using the advanced driving assistance system, (iii) using the automated driving system, and (iv) driving manually again. No specific references on the meaning of a very positive or a very negative experience were given to the participants; it was interpreted by the participants themselves. The UX-curve procedure was used to elicit the participants’ overall experience of the driving session as well as their experience of the transition from using the automated driving system to driving manually. All qualitative data were recorded and later transcribed with NVivo 12.

2.4.2. Quantitative

Before the driving session, data regarding participants’ attitude towards technology were collected using a questionnaire with three 6-point items, ranging from totally disagree to totally agree (6 indicating a positive attitude towards technology). The items concerned how skeptical the participants were toward new technology and if they were early adopters.

During the driving session, a wide range of signals from the controller area network of the car, as well as video data from three camera views (facing front traffic, facing driver from the front right and facing driver and HMI displays from the back), were recorded using a DEWESoft S-Box measurement system [33]. For each transition between the automated driving system and back to manual driving, the dependent variable take-over time (s) was collected. This was measured from the onset of the take-over request until the participants had fully completed the deactivation sequence. A total of 68 transitions were collected, but ten of those were discarded from the quantitative analysis since they were deactivated without the onset of a take-over request. Descriptive statistics of the take-over times are presented in

Table 1. Descriptive statistics of the take-over times (in seconds).

	All
N	58
Mean	8.73
Standard deviation	5.12
Minimum	1.18
Maximum	24.34
5th percentile	2.71
25th percentile	4.41
Median	7.69
75th percentile	12.40
95th percentile	18.83

.

2.5. Analysis

2.5.1. Qualitative Analysis

The interview data were analyzed using an inductive approach without predefined themes. The analysis focused on finding statements that described how the drivers experienced the transition and identifying reoccurring themes (clusters). The analysis identified three different themes: (i) relaxed, (ii) stressed, and (iii) confused. For example, one participant who described the transition as “Alarming. Sudden I’d guess I would say. I was like—I don’t know. It was fine, it probably was pressure” was categorized as stressed. Another participant who described the transition as “When it was asking (to take-over), I knew that it was asking but I just didn’t know why it was asking” was categorized as confused.

For the purpose of this paper, half of each UX-curve, the sections within the neighboring areas which goes from ‘automated driving system’ to ‘manual driving’, were used since the purpose of the paper was to investigate the UX during transitions and not UX of the whole ride. The data from the UX-curve procedure were analyzed in two steps. In the first step, the UX-curves were visually compared in order to identify groups of participants who experienced the transition differently. Visually similar curves were clustered together, resulting in four groups of curves with: (i) upward inclination, (ii) horizontal inclination, (iii) downward inclination, and (iv) very steep downward inclination. The corresponding transcripts were analyzed in a second step in order to further understand how the different groups experienced the transition and to also identify the underlying reasons for their experience of the transition. This was achieved through an inductive analysis where statements that included descriptions of why the participant experienced the transition in a certain way were clustered together. This resulted in 15 different reasons belonging to 7 groups of more overarching themes. For example, the explanation of a curve that stated “(…) but with all the bells and whistles, you are like ‘Oh, what’s going on?’. Being in a new car, but I think this over time will change” was categorized as under two different reasons being ‘A lot of feedback’ and ‘new experience’. All analyses were performed together by author 1 and 3, and discrepancies in coding were discussed to reach a final consensus.

2.5.2. Quantitative Analysis

To investigate the connection between the drivers’ subjective experience of the transition and their objective take-over performance, an exploratory data analysis was conducted. This was an attempt to compare the groups from the qualitative analysis to groups of take-over time data by applying clustering algorithms. Here, groups of take-over time data refer to bins or intervals that segment the data. This was done in order to investigate whether the take-over performance correlates with the participants’ experience of the transition. The mean take-over time for each participant on all their transitions was used since the main focus of the paper was to investigate their general experience of all the transitions and not how it differs over time. The data extraction and cluster analysis were performed in Python 3.8.

Two methods were used to find groupings on the take-over data: Quantile-based discretization function and Jenks natural break classification method [34,35]. The quantile-based approach divides the data into a user defined number of intervals by dividing the data’s probability distribution into areas with equal probabilities. A consequence of this is that the resulting groups contain approximately the same number of observations. Since the last-mentioned notion imposes a quite naïve assumption on the qualitative groups, this method is only chosen to provide descriptive figures to the analysis. Jenks method also divides the data into a user defined number of groups, but unlike the quantile-based method the groups are found by minimizing the within-group variance and maximizing the variance between the groups. This will result in natural take-over time intervals that can consist of unequal sizes. Since the number of clusters created by the Jenks method are user defined, a method to evaluate the cluster quality in relation to the number of clusters chosen is required. To avoid introducing unnecessary bias, two cluster evaluation methods based on visual inspection were chosen: (i) the goodness of variance fit using the heuristic known as the elbow method, and (ii) silhouette score [36].

The elbow method looks at how much of the variance in the clusters can be explained by the number of clusters chosen by plotting the percentage of variance explained against the number of clusters. By adding more clusters, the amount of variance explained will increase until a point where the gain will decrease which will create an “elbow” in the plot. At this point, the most recently added cluster doesn’t provide a better modelling of the data, but instead starts to overfit. The heuristic in this method is then to select the number of clusters at the point before the variance gain starts to decrease greatly.

The silhouette score is a measure of how similar cluster samples are within their own cluster and how similar they are to samples belonging to other clusters. This score ranges from −1 to +1, where a higher score indicates a good cluster belonging. The average silhouette score is calculated and the number of clusters with the highest average score is then selected as it appropriately clusters the samples.

3. Findings

The findings show that the participants in general often had a negative experience in the transition from using the automated driving system to driving manually. The data from the interviews show that almost half of the participants (n = 9) felt stressed in the situation and explained the request to take over as abrupt. About a third (n = 7) of the participants felt relaxed in the transition and some (n = 3) participants felt confused by the transition. Similar indications are apparent from the analysis of the UX-curves. A quarter (n = 5) of the curves that the participant drew to illustrate their experience of the transition had a downward inclination and almost half (n = 9) had a steep downward inclination. The analysis identified, in total, four groups of curves, being curves with: (i) upward inclination, (ii) horizontal inclination, (iii) downward inclination, and (iv) steep downward inclination (see Figure 1, Figure 2, Figure 3 and Figure 4). The curve groups are presented below, together with descriptions of the group and followed by the identified underlying reasons for the drivers’ experiences. The findings also show that the initial attitude toward technology was rather similar for all of the four curve groups. The group with downward inclination had a slightly more negative attitude towards technology (Mdn = 3, IQR = 2–5) than the three other groups (upward inclination (Mdn = 4, IQR = 1.75–5.25), horizontal inclination (Mdn = 4, IQR = 2–6), and steep downward inclination (Mdn = 4, IQR = 2–6)).

Upward inclination (n = 2)—The curves with an upward inclination started on a lower level when using the automated driving system and ended up higher when driving manually again, indicating a more positive user experience when driving themselves. The participants who drew the curves with the upward inclination both stated that they prefer to drive themselves and experienced some issues during the take-over procedure. One participant explained why his UX-curve went from negative to positive “I was fairly quickly disappointed that it was doing its own thing and I was not in control (…) Then you know, it is a nice car to drive”.

Horizontal inclination (n = 4)—The curves with a horizontal inclination indicated a steady experience at a very positive level, both when using the automated driving system as well as when driving manually. The participants that drew these curves often experienced the transition as ‘good’ or ‘smooth’. Many of them liked the automated driving system but also expressed that they felt comfortable with driving manually. One participant explained the curve by “With [the automated driving system] it was a point where I was like ‘Wow, this was really, really cool’, and then my ability to go back into manual was again seamless. I really enjoyed both halves”. However, a few also experienced some negative situations during take-over.

Downward inclination (n = 5)—The curves with a downward inclination were either high during the whole usage of the automated driving system and then went down when going into manual driving, or were initially low and then went up during the usage of the automated driving system and then down again when driving manually. Most of the participants explained that they preferred using the automated driving system over driving themselves, “after me getting over that initial anxiety, I felt a lot more comfortable with it and it was actually really nice to feel like a passenger instead. And then manual drive was just the same (indicating neutral on the UX-curve)”. Many of the participants also attributed the negative experience to the use of the automated driving system being a ‘new and unfamiliar experience’. A few of the participants also mentioned having negative experiences during the transition.

Steep downward inclination (n = 9)—The curves with a steep downward inclination often went from a relatively positive level when using the automated driving system to a comparatively negative level, often occurring during the transition from one state to the other. After the transition, some of the curves stabilized again or went up minorly while some of the curves continued to a very negative experience. Many of the participants felt that the transition was abrupt and some of them also experienced confusion during the take-over, such as not knowing who was in control and not knowing why transition occurred. One participant described the transition as “It felt abrupt and direct, it told me to re-engage. But then it felt like it needed to be done right away and for me to get thrown into the loop was kind of scary (…) There was some pressure for me to like get back on the steering wheel right away”. Some of the participants also explained that the negative experience of transitioning from automated driving to manual driving was due to it being a ‘new unfamiliar experience’ and that they liked the automated driving system.

As seen in the UX curve descriptions, the findings do not only show that the participants experienced the transition as clearly positive or negative, with a majority indicating it as a negative user experience, but with differing reasons as to why they experienced it in a certain way. The comments that related to the curves with steep downward inclination and horizontal inclination were often related to the take-over procedure and sometimes to a more general assessment of the transition (e.g., ‘good’ or ‘abrupt’). The comments that related to the curves with downward inclination and upward inclination were often related to a preference for the automated driving system or manual driving, or the lack of experience with the system but also sometimes related to the take-over procedure. Generally speaking, two aspects were identified that affected the user experience: (i) the interaction with the vehicle (the take-over procedure per se) and (ii) going from one driving mode into another.

3.1. Reasons Behind the Experience of the Transition

From the findings, 14 different underlying reasons affecting the participants’ experience of the transition from automated driving to manual driving were identified. As mentioned previously, the reasons differ in character and have therefore been organized into 7 categories: preference, exposure, general assessment, change of state, transition, feedback, and interaction.

Table 2. Reasons influencing the experience of the transition. n = number of participants mentioning the reason.

Category	Reasons	n	Impact
Preference	Familiar with driving	2	Positive
	Liked automated driving system	4	Negative
	Do not enjoy driving	1	Negative
Exposure	New experience	5	Negative
General assessment	Abrupt transition	6	Negative
	Good/smooth transition	3	Positive
Change of state	Relaxed state in AP	2	Negative
	High workload	2	Negative
Awareness	Control confusion	2	Negative
	Why transition occurred	2	Negative
Feedback	Clear haptic feedback	1	Positive
	Good combination of feedback	1	Positive
	A lot of feedback	1	Negative
	Haptic feedback (belt)	2	Negative
Interaction	Deactivation issues	2	Negative

presents an overview of the different reasons as well as the number of participants that mentioned each reason (reasons were only counted once per participant even if mentioned several times by one participant) and their impact. A participant sometimes mentioned more than one reason behind their experience of the transition and a single statement was sometimes categorized into several reasons.

Three of the reasons are not directly related to the transition procedure itself but have to do with participants’ preference for the automated driving system or driving manually. Some experienced transitioning to manual driving positively since they did not like the automated driving system because of the lack of control and felt more comfortable with driving themselves. Meanwhile some participants experienced the transition negatively since they liked using the automated driving system and did not enjoy driving themselves in condensed traffic. One participant explained that the fact of having to drive manually again created a negative experience “In the end I remember feeling “Ugh, I have to go back to manual”, so this is the end. This is hard”.

The exposure to transitions between modes was also considered by the participants to have affected the experience, where several mentioned that since it was a new and unfamiliar situation, it affected their experience of the transition negatively. One participant explained the confusion during the first transitions, “The first time was a little bit like I didn’t know what’s going on. But I mean once you know when it’s fine, it’s not surprising”.

However, many of the underlying reasons do relate more directly to the transition procedure itself and many participants provided a general assessment of the transition from automated driving to manual driving, with many experiencing the transition as quite abrupt, while some experienced it as good or smooth. One participant who felt it was good stated “The transition was very smooth. It was not like it would stop and continue”.

Furthermore, changing state from being relaxed when using the automated driving system to a more attentive state when driving manually and the increased workload during the transition procedure were both reasons that negatively affected the experience of the transition. A participant described the experience of suddenly going from one state to another “It wasn’t the most pleasant experience. Cause you go from like a really nice, relaxed feeling, to like I got to switch back to it”.

Going from one mode of driving to another also created confusion, where some participants were not aware if they were in control or not after the take-over and questioning why the transition occurred. One participant explained that they had a negative experience since they were not aware why the transition occurred, “I was irritated that it made me take back control … I shouldn’t have to be confused in a car”. Another participant explained how the confusion of not really knowing who has the control authority (the driver or the car) affected the experience; “the transition between [the automated driving system] and when you’re driving, gets me pretty nervous. Just because of that little delay, where I don’t know if I’m driving or the car is driving”.

Lastly, several underlying reasons relate to feedback from and interaction with the vehicle during the transition procedure. The reasons relating to the feedback from the vehicle concerns the multimodality of the feedback and the haptic feedback that participants received prior to the take-over. How the multimodality and haptic feedback affected the participants’ experience differed. While one participant found the combination of feedback to be good “You get the signal with the seatbelt, then you get on the dash, you’re getting a visual saying—giving you a message. The way it asks you to take over is very easy”, another thought it was too ambiguous feedback “It was still positive, but with all the bells and whistles, you are like ‘Oh, what’s going on?’”. Similarly, while one participant thought the haptic feedback from the seat belt was clear other participants felt that it affected their experience negatively. One participant explained how the haptic feedback affected the experience of the transition, “I just can’t get over the point of the seat belt. (...) It just put a pretty big hamper on the experience, but once I got over it, it wasn’t massive”. The reason relating to the interaction with the vehicle concerns issues of deactivation, such as not pushing the buttons long enough when deactivating the automated driving system, that the participants experienced when they were supposed to take over control of the dynamic driving task. One participant explained that the negative experience of the transition was due to a deactivation issue, “I didn’t wait until it deactivated. I turned the steering wheel to the right to merge to the 280 and it was really stiff, because I was in (automated driving). And that was like I felt the car, the steering wheel going this way and I wanted to go this way”. This indicates that the participants might also have expected to just take over without further interaction steps.

A further analysis of the underlying reasons showed that they do not only differ in character, as shown by the different categories (see Table 2), but also in duration and are derived from different events in the transition from automated driving to manual driving. By comparing the categories previously identified, it is recognized that some of the reasons can be tracked throughout the whole transition procedure, meaning also before and after the take-over itself, while others are for example only derived from events directly before the take-over. These differences between when the reasons start to have an effect on the experience and the duration of that effect, indicate that there are several different temporal dimensions that affect drivers’ overall experience of the transition from automated driving to manual driving. To illustrate these differences and their duration, a transition timeline with the relevant events illustrating the temporal differences is depicted in Figure 5.

As illustrated, preference and exposure have the greatest timespan and range outside of the transition procedure itself. Preference relates to the whole use of the automated driving system and the manual driving, both before and after the transition procedure. Exposure is derived from the repeated exposure to the transition procedure or lack of it. Change of state, awareness, and general assessment are all derived from the whole transition procedure. This ranges from the notification to take over control and continues until in full control of the dynamic driving task. Reasons relating to the feedback from the vehicle are derived from the notification until the take-over event. The shortest timespan belongs to interaction issues which arise when the user starts to engage in the procedure until deactivation of the automated driving system.

3.2. Comparison between Drivers’ Experience and Take-Over Time

The exploratory data analysis used two methods, a quantile-based approach and Jenks method, to compare the groups from the qualitative analysis to groups of take-over time data. This was undertaken in order to investigate the connection between drivers’ subjective experience of the transition and their objective take-over performance.

3.2.1. Quantile

For the quantile-based approach, two different numbers of quantiles were selected: 3 (terciles) and 4 (quartiles) (see Figure 6). The following tercile intervals (T) of take-over time were found: T1: [4.739 s–6.636 s], T2: [6.636 s–8.970 s] and T3: [8.970–19.322]. Both T1 and T3 contain 7 samples each, and T2 contains 6 samples. By looking at the short range in both interval T1 and T2, we notice that the majority of samples exist within 8.97 s. These intervals might however be too small and too close to each other to provide any meaningful segmentation of the data, especially considering that T3 spans over 10 s.

The following quartile intervals of take-over time were found: Q1: [4.739 s–6.337 s], Q2: [6.337 s–7.852 s], Q3: [7.852 s–10.035 s], Q4: [10.035 s–19.322 s]. All quartiles contain 5 samples each. Q1–3 are all quite short intervals, but could be interpreted as fast, medium and slow, respectively. Similar to T3, Q4 is too large (spans almost 10 s). The quantile-based approach shows that the participants were in general quite fast to resume control. As expected, the cluster ranges resulting from the quantile approach are too coarse to be interpreted in a meaningful way. To find more representative clusters the exploration continues by applying the Jenks method.

3.2.2. Jenks Method

Cluster Evaluation

Elbow method suggests that either 3 or 4 is the best number of clusters (see Figure 7). Note that this is still a heuristic and in certain cases, such as this, where we have two decreases in variance gain (between 3–4, and 4–5) it can be difficult to select the best one. Since the elbow method yielded inconclusive results (either 3 or 4), the silhouette method was used to test these two numbers of clusters.

The analysis shows that when using 3 clusters the average silhouette score is 0.515 and when using 4 clusters the average silhouette score is 0.449. All clusters have high averages, both when using three and four clusters. As a result, the analysis is inconclusive, and the evaluation indicates that either 3 or 4 clusters model the data best. Both 3 and 4 clusters are therefore presented.

Jenks Method Clusters

The Jenks method using 3 clusters (A) found the following intervals: A1: [4.739 s–7.472 s], A2: [7.472 s–12.171 s], and A3: [12.171 s–19.322 s].

The Jenks method using 4 clusters (B) found the following intervals: B1: [4.739 s–6.968 s], B2: [6.968 s–9.510 s], B3: [9.510 s–14.688], and B4: [14.688 s–19.322 s]. Figure 8 shows the Jenks cluster groups.

The data from the interview (top of Figure 8) shows a very similar distribution, for all three groups, between the different clusters both when having three and four clusters. This indicates that there is no clear correlation between the drivers’ experience identified in the interview and take-over time. The data from the UX-curves (bottom Figure 8) also show inconclusive results. However, there are some indications that participants with the curve type ‘horizontal inclination’ performed rather fast take-overs (except for one of the participants that performed very slow take-overs). Similar tendencies can be seen for the participants with the curve type ‘steep downwards inclination’, where a majority of the participants are organized into the fastest clusters (A1 and B1).

4. Discussion

The aim of the paper is to explore how users experience transitions from automated driving to manual driving and how the experience relates to the users’ take-over time. Earlier studies of transitions have often focused on safety-related aspects such as take-over performance, but few have investigated the drivers’ experience during the transitions. The findings show that the majority of participants experienced a decline of user experience during the transition from automated driving and manual driving, and inconclusive results regarding the correlation between UX in transition and take-over time.

4.1. The Decline of User Experience

The findings show that the participants in general had a quite negative experience of the transition from automated driving to manual driving. Even if both the data from the interview and UX-curve method indicate that most of the participants had a negative experience, the UX-curves show more of a negative tendency, with almost three quarters of the participants (n = 14) representing their experience of the transition with a downward or steep downward curve. The difference, even if small, between the results of the two methods may be because of the methods’ difference in character and focus. The interview focused on the take-over only, while the UX-curve procedure was performed to encapsulate a more holistic and reflective experience, as indicated by the participants’ explanations. It can also be difficult to compare participants’ subjective interpretation of their experience. In future studies, it may therefore be crucial to use multiple different types of data collection and analysis methods preferably during different parts of the study, as also argued by [37].

The findings also show that UX is affected by several types of reasons that differ in their temporal character, as also indicated by previous research [15,22]. Previously, temporality has often been considered as either present-oriented or over a prolonged period of product use, e.g., [14,32]. These two dimensions are also identified in the study by the momentaneous effects of feedback and interaction issues on the participants’ experience, as well as the apparent effect of the long-term exposure to transitions. However, the different underlying reasons for the participants’ experiences also have a timespan in between momentaneous and long-term use, indicating intermediate temporal dimensions that are important to consider when investigating UX in transitions to fully understand the drivers’ experience.

The different reasons are derived from different stages of the transition, which affects how one can design to reduce the negative impact of transitions on the UX. Reasons relating to the feedback from the vehicle and interaction issues, which are derived from the notification until the take-over event, may need one type of solution. For example, working with the modality of the feedback [25] or ways of interacting with the system [38]. However, reasons relating to the change of state and awareness are derived from the whole transition procedure. These range from when the participant gets the notification to take over until they are in full control of the dynamic driving task. For these reasons, other types of solutions may be needed. For example, preparing drivers for take-overs [27] or communicating control and responsibility [39].

4.2. UX of Transitions and Take-Over Time

The findings regarding the correlation between UX and take-over times were inconclusive, where data from the interview showed no correlation and data from UX-curves showed indications of trends. One possible explanation can be found in the identified reasons and in their temporal differences. They illustrate that the experience is not only affected by the take-over itself but also other reasons with a greater timespan. Thus, even if the participant performs well by resuming control quickly, other reasons (extending before and after) may affect the overall experience.

Furthermore, the analysis was performed on a small sample size of 20 and should therefore be considered as explorative. More studies should be conducted investigating the correlation between UX and take-over performance. In this study we chose to look at take-over time since it is a well-established indicator of take-over performance and is safety critical. Future studies should further investigate take-over time, together with other UX dimensions, but also consider other indicators of take-over performance. For example, a study by Said and Chauvin [40] identified that user experience was associated with the magnitude of the driver’s actions related to lateral and longitudinal control after a take-over.

However, even if results were inconclusive, some trends were apparent from the Jenks cluster method on the curve type data. According to the data that was analyzed, the UX-curve data seem to best explain the relationship between UX and take-over time. There are some indications that participants with the curve type ‘horizontal inclination’ and ‘steep downwards inclination’ performed rather fast take-overs. Interestingly, these two groups had quite opposite experiences of the transition procedure, compared to the participants with the curve type ‘steep downwards inclination’, who often had a rather negative experience of it. Meanwhile participants with the curve type ‘horizontal inclination’ had a rather positive experience of the transition procedure. A possible explanation is that the participants with the curve type ‘horizontal inclination’ liked the transition procedure, resulting in a positive experience of the transition. While the participants with the curve type ‘steep downwards inclination’ were stressed about the transition situation and therefore performed the take-over rather quickly or they became stressed because they perceived that they had to perform the take-over quickly. Thus, even if many performed well (fast take-over times) they also had a negative experience of the transition. This could possibly be explained by the feedback from the system, that it made them alert but also stressed, similar to what was seen by Kutchek and Jeon [24]. This illustrates the thin line of developing interactions and feedback that result in a good take-over performance but also a positive UX.

4.3. Methodological Influence on Take-Over Time and UX

An on-road Wizard of Oz approach was chosen to create a natural experience for the participants in order to have high ecological validity. As a consequence, the study does not have a controlled environment, resulting in different participants having different amounts of exposure with the system and experiencing different numbers of take-overs. Furthermore, even if most take-overs occurred in similar situations where transitions were system initiated because of traffic getting lighter and speeds increasing, the transition situations differed slightly between the participants. The differences in external contextual factors will most likely have affected the participants’ take-over time, since it affects what they need to observe in the take-over situation. Similarly, if the participants engaged in non-driving related activities or not, which differed between participants in the study, will probably also have affected the participants’ take-over time. As a consequence, the take-over performance will most likely partly affect participants experience, even if no apparent effects of take-over time were found in this study. Furthermore, the experience is also affected by personal preferences and previous experiences, which may shape the expectations and anticipation, an important part of drivers’ experience in driving automation [41]. Even though the focus of this study was not to investigate the underlying factors of the take-over performance, future studies should also consider external contextual factors, non-driving related activities and personal preferences in order to better understand the underlying factors of take-over performance and in turn UX of transitions.

5. Conclusions

The findings show that a majority of the participants experienced the transition from automated driving to manual driving as negative. This highlights that work needs to be conducted to enhance the user experience during transitions. Additionally, it was found that the UX seems to be shaped by several underlying reasons that differ in temporality and are derived from different phases of the transition. Therefore, it is important to consider all the temporal dimensions identified in this paper, since different design solutions may be needed for different dimensions. Future studies should, therefore, further investigate the different underlying reasons and possible ways of mitigating the negative user experience during transitions using the identified reasons as guidance in the design process.