Rear-Seat Productivity in Virtual Reality: Investigating VR Interaction in the Confined Space of a Car

Abstract

Ubiquitous technology lets us work in flexible and decentralised ways. Passengers can already use travel time to be productive, and we envision even better performance and experience in vehicles with emerging technologies, such as virtual reality (VR) headsets. However, the confined physical space constrains interactions while the virtual space may be conceptually borderless. We therefore conducted a VR study (N = 33) to examine the influence of physical restraints and virtual working environments on performance, presence, and the feeling of safety. Our findings show that virtual borders make passengers touch the car interior less, while performance and presence are comparable across conditions. Although passengers prefer a secluded and unlimited virtual environment (nature), they are more productive in a shared and limited one (office). We further discuss choices for virtual borders and environments, social experience, and safety responsiveness. Our work highlights opportunities and challenges for future research and design of rear-seat VR interaction.

1. Introduction

Since travel is becoming cheaper and faster, people travel more often and for longer distances. Not only has this had an effect on the tourism industry but also the number of commuters has increased. In big cities, such as London, nearly half of the work force commutes to their place of work [1]. Similarly, commuting is often seen as working time—in some cases, even remunerated as such. In this context, travellers seek to be productive during travel. Prior work found that the wish to be productive influences the choice of transport [2]. Thus, a train in which there is enough space and a WiFi network might be preferred over a car that users currently still must drive themselves. Of course, if the car is a chauffeur-driven “taxi” that lets users work in the back seat, their preference might change because they now can save time [2]. This is the subgroup of commuters we focus on. Rear-seat productivity may be perceived as an intermediate state towards automated vehicles. However, in contrast to automated vehicles, being productive in the rear seat of a car is already feasible and desired.

Currently, productive activities can be carried out on mobile devices, such as mobile phones and laptops. However, recent progress in wireless head-mounted displays (HMDs) provides an opportunity for travellers to be productive in virtual reality (VR) beyond the limited screen size of a laptop or mobile phone [3]. HMDs enable users to completely surround themselves with VR. Thus, they can immerse themselves into a virtual workplace that is larger than the confined physical space of the car. This may be a virtual replication of their actual office or any other virtual environment that fosters productivity performance. However, in VR, one cannot see the physical reality and its physical borders. A virtual mesh or grid—which we refer to as virtual borders—are built-in solutions that current headsets employ to warn HMD users about physical borders. This solution of showing virtual borders and alternative ones from prior work [4,5,6,7] are conceptualised and tested for labs and larger office/living rooms. Although the virtual mesh or grid solutions have recently been repurposed for seating solutions in confined spaces as part of the Oculus Quest release [8], this development may be unsuitable for in-car HMD usage as physical borders are very close and thus the virtual border would constantly be displayed. This diminishes the effect of an unlimited virtual environment, which also affects presence in VR—a common subjective measure for the success of a VR application [9]. Presence is measured by questioning how much of the physical reality users are aware of while they are in VR. It is unclear whether and how (constantly) displaying a virtual border in a car affects productivity performance and presence. Alternatively, it may also be considered unnecessary to visualise physical borders in VR for rear-seat HMD usage because users are aware that they are confined to a small space with (a) limited room for movement and (b) no disturbance from external factors, e.g., a colleague walking into the physical office space.

To summarise, the confined space of a rear-seat VR experience bears not only opportunities, such as the unlimited virtual environment, but also challenges, such as the proximity to physical borders. We want to explore both of these directions and the effect they have on performance, presence, and the feeling of safety. Our research question is as follows:

RQ: 

How can we utilise the design of virtual environments to make the confined space of a car suited for performing productivity tasks in VR, and is a virtual border display relevant in this context?

We present the results of a user study (N = 33), which suggests that performance and presence in rear-seat VR interactions are similar, whether virtual borders are displayed or not. Although the lack of borders resulted in an increase in confrontations with the physical borders of the car, this did not have any effect on performance, presence, and the feeling of safety. Finally, we found that a virtual office environment increases productivity performance significantly over a relaxing, natural environment. To our knowledge, this is the first paper that explores the need to adapt VR interactions for productivity in confined spaces, such as the rear seat of a car. Our findings are valuable for researchers and practitioners who want to create VR experiences for rear-seat passengers.

2. Related Work

To better understand the joint study context of automotives and VRs, we review the three main research lines, namely, mobile work in cars, presence and productivity in VR, and passenger VR experience.

2.1. Mobile Work in Cars

The trend of mobile work is shaped by ubiquitous technology and the range of portable devices, such as smart phones, tablets, and laptops. The continuously improving capabilities of these devices foster temporal and spatial flexibility and decentralisation in the workplace. These devices are small and portable but still provide powerful input modalities, such as foldable keypads and screens. In the context of this paper, we also consider HMDs as portable virtual offices/screens. Although there already exists limited work on productivity in HMDs [10], particularly productivity while commuting has not been extensively researched.

During a commute in various transportation settings (e.g., private or public, owned or shared, and short or long distances), users show distinct attitudes and concerns towards using technical devices for productivity: car commuters, for example, use slow-moving traffic and long waiting periods in the morning to perform work-related tasks with mobile devices [11]. Moreover, the choice of task also depends on the space constraints presented to the commuter [12].

While anticipating for future automated vehicles, Pfleging et al. [13] conducted a train-based survey regarding non-driving-related tasks (NDRTs) in driverless cars. Their results indicate that, in addition to the tasks performed during a public transport commute (e.g., phone calls, web browsing, and text messages), passengers will conduct a broader set of activities in the private space of their car. In specific, Stevens et al. [14] forecasted that owning a driverless “mobile home and office” increases the availability, predictability, and planning certainty for using travel time for NDRTs. Previous workshops in the AutomotiveUI community have shed light on supporting NDRTs in automated vehicles especially as mobile offices [15,16,17]. Recent studies further explored the design space for such automobile offices [18] and productivity tasks such as text comprehension and typing [19,20] in Society of Automotive Engineers (SAE) level 3 semiautomated vehicles [21]. These trends, together with the increasing need for productivity in the car, motivate our work that focuses on the rear-seat passenger.

2.2. Factors Influencing VR for Productivity

Prior work has investigated various aspects of productivity in VR, such as text entry [22,23], co-located bystanders or passengers [24], or collaborative communication between VR users and physical collaborators [25]. In addition to these areas, Grubert et al. [22] also pointed out the lack of privacy research for virtual offices. Similarly, George et al. investigated the observability of VR user interaction by a physical bystander during general productivity tasks, such as typing [26] and authentication [27]. In the context of VR, presence and immersion are factors that influence the subjective quality of the overall experience [9]: There are varying definitions, but summarised, presence depends on how a human reacts to immersion (here, defined as how many interaction modalities are coupled between VR and physical reality). Although prior work revealed varying results on the correlation between performance and presence [28], it is unclear how it relates to productivity. We consider productive tasks (e.g., writing) as a subset of performance tasks (e.g., running and playing games), as mentioned in a prior work [28].

To improve the presence perceived in a VR workplace, Grubert et al. [22] suggested customising virtual environments for users to control or interact beyond physical limitations, to diminish distractions, or to relieve users from a confined space in reality. Additionally, different virtual environments lead to different passenger VR experiences. When anticipating automated driving, users mostly demand relaxation, entertainment, and productivity when they do not need to drive [13]. Parades et al. [29] suggested that passenger-based calm VR applications are well suited for an automotive context. They demonstrate that a dynamic ocean scenario such as VR content leads to a calming experience. On the other hand, Holoride [30] was announced as a virtual gaming environment and aims for a more joyful passenger experience in modern cars. However, we did not find any recent work on virtual working environments for rear-seat productivity in VR. This gap motivated us to build and test a VR productivity system using sounds, visualisation, and input controllers. With this fundamental setup, we built an immersive productivity experience, in which we can test the impact of virtual working environments on presence and performance.

2.3. Productive VR Usage in the Car

One of the challenges in using VR in the rear seat of a car is the confined space. This was also voiced in prior work on passenger VR usage by McGill et al. [31,32]. While exploring users’ thoughts on rear-seat productivity in VR, Li et al. [33] found that commuters generally envision using VR in cars for working (e.g., making phone calls and reading text) and are most concerned with their physical integrity in virtual environments.

Although there are a variety of solutions for increasing situational awareness of the physical reality, the most prominent one is the built-in grid solution [4,5]. While this is essential for dynamic, room-scale experiences, it may be obsolete in the confined space of a passenger car. In this context, the user is sitting and has limited reach, and while the car is driving, the physical work environment will not change in contrast to a confined office desk, where a colleague may step into the physical space. In this paper, we focus on the difference that the confined space of a car affords, such as the need for virtual representations of physical borders and the effect they have on productivity performance, presence, and the feeling of safety.

As a part of automotive interaction, safety will be a critical premise in future rear-seat VR workplaces. However, in prior work on VR, safety has been more generally discussed as “situational awareness” (how aware is the VR user of the physical reality?) [34]. Similarly, in automotive research, “situational awareness” refers to how drivers perceive road situations, including how relevant are elements located in the environment [35]. We refer to the “feeling of safety” to discuss participants’ subjective understanding and perception of it. We measure this by determining how safe and comfortable they feel while conducting rear-seat productivity tasks in VR and how much they worry about hitting the car interior. Additionally, we observe how many times participants actually touch the car interior while performing the task in VR as an objective assessment. We first look at a stationary vehicle setup and implement visualisations of virtual working environments with and without virtual borders.

3. User Study

Our study explores how adapting a virtual workspace to the limited physical space of a vehicle affects performance, presence, and the feeling of safety. More specifically, we focus on virtual borders and virtual environments as two major influencing factors in this context. To answer our research question proposed before, the user study aims to verify the following hypotheses:

Hypothesis 1 (H1.1–H1.5).

Adding virtual borders makes the passenger feel safer (H1.1) and more comfortable (H1.2), worry less (H1.3), touch the car interior less (H1.4), and be more productive (H1.5) during a rear-seat VR workplace experience.

Hypothesis 2 (H2.1–H2.2).

The passenger feels more present (H2.1) and productive (H2.2) in an unlimited and secluded virtual work environment (nature) than in a limited and shared one (office).

3.1. Influencing Factors: Virtual Borders and Virtual Environments

This study uses a mixed design with 3 variables: the presence of virtual borders (Border vs. No Border) is a between-subject variable, while the type of virtual environment (Office vs. Nature) is varied within-subject. The resulting four visual conditions are shown in Figure 1. In Border, participants will see the virtual grids when they are closer to the physical car interior, while there is no indication in No Border. The two types of virtual environment differ visually and acoustically: in the Office, the user sits in a simulated open-plan office that is limited by walls. Animated avatars representing colleagues and ambient sounds of keyboard typing simulate a more realistic atmosphere. In Nature, users sit in a simulated open forest, which is not limited by any walls. They also hear ambient sounds of twittering birds to emphasise the secluded forest atmosphere. Both visual and auditory discrepancies aim for realistic experiences of the two hypothesised types of virtual working environments (unlimited and secluded vs. limited and shared).

Similarly, in addition to the type of environment, another within-subject variable is the presence of an incoming phone call: in both the office and the nature environment, users will either receive a Call or No Call. Half of the participants experienced the Office first, while the other half exeperienced Nature first. Similarly, half of the participants experienced a Call first, while the other half experienced No Call first. A Latin square was used to counterbalance these conditions in order to avoid learning or fatigue effects.

3.2. Dependent Variables: Performance, Presence, and Feeling of Safety

Previous work shows that reading or typing text are the most requested activities for productivity by rear-seat HMD users [33]. Therefore, we selected the reading span task [36] as the experimental task. Participants were asked to read sets of sentences aloud while trying to memorise the last word of each sentence and to recall all the words in the correct order by typing on a virtual keyboard. We adapted the original task to the study context by fixing the number of sets (12) and the set size (2–5 sentences per set) to enable a comparison of the performance between the participants. The sentences for the reading span task were derived from a custom-written software [37]. Each sentence was displayed for 6.5 s [38]. To measure performance, we calculated the partial credit unit (PCU) score [36], the mean proportion of sentences within an item that were correctly recalled. The perceived workload was measured with the NASA-TLX questionnaire [39]. We assessed presence using the igroup presence questionnaire (IPQ) [40].

In this exploratory study, we aimed to investigate the different variables that contribute to safety/situation awareness in a rear-seat VR experience and summarised them under the term “feeling of safety”. We measured this self-defined concept by asking the participant to provide a numeric rating on a Likert-scale between 1 and 7 with 1 indicating “strongly disagree” and with 7 indicating “strongly agree” according to the following criteria: (a) perceived safety: I felt safe while performing the task; (b) perceived comfort: It was comfortable to perform the task; and (c) worry of touch: I was worried about touching the car interior during the task. Furthermore, the experimenter counted how many times participants actually touched the car interior—which we refer to as “count of touch”.

3.3. Apparatus

Our VR experience was developed with C# using the Unity game engine on a MSI GT 03 Titan 8 GR. An Oculus Rift and the paired wireless controllers and tracking sensors were used. The productivity task was performed by the select, swipe, and type operations with the controller in the virtual workspace, which consisted of three virtual displays and a computer desk (see Figure 2).

The sentences are shown on the center display for reading text. The display on the right functions as a keyboard for typing. Notifications about an incoming phone call are shown on the left display. The position of the adjacent displays can be switched using a swipe on the controller. The user can only select items on the display that is in the middle; thus, what is in reach can be changed by a swipe with the controller. We designed our virtual grids by mimicking the existing Oculus guardian system [4]. In specific, while seated in the testing car and performing the productivity tasks, we set up the distance range showing virtual grids from transparent to opaque in the Unity software. We thus measured and designed the minimum interaction space required for the implemented tasks as 30 cm (from left to right with the participant in the middle) × 10 cm (from the participant to the front). This means that our participants can see the virtual grids with decreasing transparency when their controllers exceed this preset space till touching the front seat back, right side door, or the middle of the rear seat. The closer a participant gets to the physical car interior, the more opaque the virtual grid will appear in Border, while no visual cues are given in No Border.

The notification of an incoming phone call is set to pop up on the left screen during the productivity task. The interruption was designed to provide a realistic working experience and to increase the task difficulty. Our setup was inspired by a design fiction [41] about three information levels for rear-seat VR productivity, which are (a) environment level, (b) notification level, and (c) productivity level [33]. Following this information architecture, we concretised our application with nature and an office as variables at the environment level, an incoming phone call as an interruption at the notification level, and reading and typing text as tasks at the productivity level.

Setup and Hygiene Requirements

The study setup was in a static car parked in the garage at Audi AG (see Figure 3).

Two Oculus Rift sensors were mounted on the front seat to track the controllers’ movement. Due to the ongoing COVID-19 pandemic, some general measures to ensure the health and safety of participants and the experimenter were implemented. The hygiene requirements were made based on the guidelines of Audi AG. The participants and the experimenter put on face masks and gloves throughout the study. Additionally, both washed and disinfected their hands properly prior to beginning the study. Furthermore, participants were asked to wear a disposable VR facial cover during HMD usage. A 1.5 m social distance was maintained throughout the study.

3.4. Participants

A total of 33 right-handed participants (5 female) aged from 22 to 58 years (M = 37.24, SD = 8.82) were recruited for the study. Sixteen participants participated in the Border condition, and 17 participants participated in the No Border condition. All participants were Audi employees and worked in open-plan offices. Most of the participants travel by car as a passenger weekly (39%) or monthly (39%) and usually spend 15 to 30 min (27%) in the car. Only 18% of them had never conducted productivity tasks during these car rides before. Two participants reported using VR systems monthly, 13 tried it yearly or less, and more than half of the participants (55%) had no prior VR experience.

3.5. Study Procedure

Each participant was picked up at the building entrance and brought to the car. They were then informed about the hygiene requirements for the study and briefed about the study conditions. After signing a consent form and data protection declaration in concordance with local ethical review board requirements, each participant was asked to complete a questionnaire on demographics, prior rear-seat commute and VR experience, and the preference of productive environments. Participants were instructed on the usage of the wireless controller for the main productivity task. The functionality of the grids in the Border condition was explicitly explained. In the first trial round wearing the headset in the rear seat, the participant tested the swipe operation between the displays and type with the virtual keyboard.

For the main part of the study, each participant experienced two environments for rear-seat productivity in VR. The order of the Nature and Office as well as the Call and No Call conditions was counterbalanced by a Latin square. During the 15 min exposure in each environment, the participant performed the reading span task. In between, the participant did or did not receive a virtual phone call as an interruption (depending on wheether they were in the Call or No Call condition). The performance data of each participant and the screenshot of the whole session were recorded. After experiencing each environment, the participant filled out the NASA-TLX and IPQ questionnaires, and a questionnaire about the “feeling of safety”. After the experiment was over, the participant was asked to think aloud in a semi-structured interview (audio-recorded) about overall reactions, preferences, and suggestions. The entire user study took about one hour, in which each participant experienced two experiment conditions, namely Nature and Office.

4. Results

To verify the proposed research questions and hypotheses, we collected the perceived safety, comfort, and fear of hitting the car interior. We used objective assessments for monitoring performance and count of touch by the participant. Subjective assessment was used for presence, perceived workload, and preference for virtual environments. For each independent variable, we first present the statistical data, followed by the qualitative feedback and discussion based on our semi-structured interview.

Depending on the data distributions, we used various tests to determine the significance of our findings. In particular, we used dependent and independent t-tests for normally distributed data. For nonnormal distributions, we used a Wilcoxon test or a Mann–Whitney U test instead. Statistical significance is reported for $p\le 0.05$. A large effect size is reported if it is >0.5.

4.1. Influence of No Border or Border on Count of Touch

Overall, there is a limited fear of collision and participants actually touched the car interior less than once on average. Meanwhile, the perceived levels of comfort and safety are high in both the Border and No Border conditions. Participants in both conditions achieved medium to high performances with a fair workload.

The presence was rated moderate in both the Border (Mdn = 4.875, SD = 1.044) and No Border conditions (Mdn = 4.250, SD = 1.186) (Reported Mdn values are computed as the average of several actual medians and thus multiples of 0.125). As the data were not normally distributed, we used a Mann–Whitney U test and found no significant difference across conditions (U = 172.000, p = 0.192, d = 0.265, 95% CI = [−0.200, 1.500]).

4.1.1. Feeling of Safety

“Feeling of safety” was quantitatively measured by the perceived safety and comfort as well as the fear of touching the car interior (see Figure 4). We found no significant difference between the No Border and Border conditions for the three variables: (a) perceived safety: U = 100.000, p = 0.167, d = −0.265, 95% CI = [−0.500, 0.017]; (b) perceived comfort: U = 129.500, p = 0.824, d = −0.048, 95% CI = [−1.000, 1.000]; and (c) worry of touch: U = 169.500, p = 0.213, d = 0.246, 95% CI = [−0.035, 0.500].

The results from the Likert-scale (1–7) suggest that participants in both conditions felt comfortable during the task (Mdn_{No Border} = 6.0, SD_{No Border} = 1.171; Mdn_Border = 6.0, SD_Border = 1.040). Meanwhile, they barely felt concerned about touching the car interior in both conditions while performing the tasks in VR. However, they slightly (not significantly) felt safer (Mdn_{No Border} = 7.0, SD_{No Border} = 0.632; Mdn_Border = 6.5, SD_Border = 0.671) and worried less about touching the car interior (Mdn_{No Border} = 1.0, SD_{No Border} = 1.072; Mdn_Border = 1.5, SD_Border = 0.706) in the No Border condition compared to the Border condition.

These small differences without significance across conditions lead us to reject our hypotheses H1.1, H1.2, and H1.3. More specifically, in this exploratory study, we did not find strong evidence that showing virtual borders during rear-seat VR productivity tasks can enhance the “feeling of safety”.

4.1.2. Performance and Count of Touch

By assessing the PCU score, we found performance to be comparable between the Border (M = 0.743, SD = 0.127) and No Border conditions (M = 0.741, SD = 0.119). As the data were normally distributed, we used an independent t-test and found no significant difference (t(31) = 0.045, p = 0.964, d = 0.016, 95% CI = [−0.085, 0.089]). This suggests that showing virtual borders or not barely influences the participant’s performance in a rear-seat VR workplace. This contradicts our expectation of a better task performance in the Border condition (H1.5). As shown in Figure 5, participants in both conditions reported a fair workload in both conditions (M_Border = 66.159, SD_Border = 9.831; M_{No Border} = 64.257, SD_{No Border} = 9.396).

However, participants touched the interior significantly more when having no indication in the No Border condition (U = 70.000, p = 0.006, d = −0.485, 95% CI = [−0.500, −0.032]). This suggests that there is a medium effect and that we might want to gather more data to increase our confidence. Nevertheless, while performing productivity tasks, virtual borders help the participant touch the car interior significantly less in the Border (M = 0.063, SD = 0.171) condition than the No Border condition (M = 0.441, SD = 0.527). This confirms our hypothesis H1.4 that showing virtual borders helps participants touch the car interior less while performing a productivity task in VR.

4.1.3. Qualitative Analysis and Discussion of Virtual Borders

After performing the productivity tasks in VR, participants were asked different questions on how disturbing it was for them to see virtual borders in the Border condition or to not see them in the No Border condition. In general, participants found the lack of virtual borders acceptable during the productivity task in the No Border condition. In contrast, more of them reported virtual borders to be disturbing during the task in the Border condition (see Figure 6). To understand users’ thoughts on virtual borders in the rear-seat VR workplace, we developed a set of recurring themes through thematic analysis [42] on the interviews’ original notes and recordings. We organise our findings around the concepts of “safety perception”, “interaction demand”, and “spatial awareness”.

• Safety Perception (33/33)
  After touching the car interior once in the No Border condition, our participants cautiously reduced the scope of their arm movements. While in the Border condition, some could imagine learning in a tutorial clip demonstrating virtual borders only once at the beginning, others prefer having no indication as “…it would be more enjoyable” [P12], even with the risk that they could hit something inside the car. This shows that some participants simply found virtual borders unnecessary for the implemented rear-seat VR workplace because the required interaction could not seriously harm their concepts of safety (e.g., “…well then I just hit something” [P20]). Nevertheless, some users found virtual borders helpful and useful as “…they took away the fear of hitting something” [P28] and “…I feel that I don’t need to take care” [P7].
• Interaction Demand (10/33)
  In both the Border and No Border conditions, our participants perceived the required speed of arm movement in the experiment as overall gentle and within a small scale. Therefore, some were fine without virtual borders in this specific interaction “with such a small required space of movement…” [P29] and “…slow motion sequence” [P33]. Some participants even compared the interaction experience to other VR games, “where I have to throw a ball…” [P26] or “if hasty movements are included…” [P33]. In that case, they claimed a higher need for virtual borders. Furthermore, in the No Border condition, some participants found that the absence of virtual borders enhanced their concentration on the productivity task as they “…become unaware of the physical reality” [P3].
• Spatial Awareness (15/33)
  Although we found no statistically significant differences for presence between the Border and No Border conditions, qualitative data indicate that virtual borders affected participants’ sense of presence during the productivity task in VR. Without seeing virtual borders in the No Border condition, our participants felt moderately present in the virtual world. Such presence in VR may be accompanied by a subconscious awareness of the physical space. Some provided statements such as “I felt that I was not sitting in a vehicle” [P21], while others voiced that they “…know that the car door is on the right side” [P5]. In the Border condition, more of them found virtual borders disturbing by “…bringing one back to the physical environment in a car” [P19]. Finally, the spacious rear seat in the test vehicle seemed to ease the demand of some participants for virtual borders. However, they also expressed the need for an indication of physical limitations “…if the car is smaller” [P13].

4.2. Influence of Office or Nature on Productivity

Below, we examine the influence of the type of virtual environment on presence and performance in the rear-seat VR workplace. We present the self-reported presence (IPQ score), perceived workload (raw TLX score), and performance based on the assessed PCU score. Participants were also asked which type of environment they prefer. The majority (N = 22) selected Nature, while eight participants voted for the Office. The rest had no preferences for one or the other. On average, participants liked Nature (Mdn = 6.0, SD = 1.25) more than the Office (Mdn = 3.0, SD = 1.65). The overall ratings for both environments are shown in Figure 7.

4.2.1. Performance and Presence

Despite the preference for Nature, our participants performed slightly better in an Office. We found a significant difference in performance between the Office and Nature settings (t(32) = 2.602, p = 0.014, d = 0.453, 95% CI = [0.091, 0.808]). Specifically, participants recalled more words in the Office (M = 0.765, SD = 0.122) than Nature (M = 0.723, SD = 0.138). Meanwhile, as shown in Figure 8, the perceived workload was comparable across conditions (M_Office = 65.314, SD_Office = 10.846; M_Nature = 66.273, SD_Nature = 11.292).

Moreover, they perceived medium presence in the task across the Office (Mdn = 5.0, SD = 1.275) and Nature (Mdn = 4.5, SD = 1.198) settings. As the data were abnormally distributed, we used a Wilcoxon test and found no significant difference in presence across conditions (W = 131.000, p = 0.597, d = 0.134, 95% CI = [−0.339, 0.553]).

The lower performance and fair presence in Nature further contradict H2. We thus look at the following qualitative analysis and discussion for further clarification.

4.2.2. Qualitative Analysis and Discussion of Virtual Environments

Based on another thematic analysis, we present these results according to “virtual workplace” (unlimited vs. limited), “social experience” (secluded vs. shared), and “ambient sound” (natural vs. artificial).

• Virtual Workplace (26/33)
  Participants found Nature to be spacious, “…which contributes to relaxation” [P12]; beautiful, “…which makes everything pleasant” [P23]; or green, “….which is more comfortable than an office” [P22]. Nobody attributed productivity directly to an unlimited virtual environment. Moreover, Nature was not considered a usual workplace for our participants but rather a place for “…relaxation or leisure after work” [p18]. In contrast, they associated the Office with a daily workplace, where they can “…focus and get less distracted” [P5]. Still, some participants demanded Nature for “…variety in the virtual world” [P26], especially after sitting in the office for the whole day. In general, Nature was seen as a positive change to the everyday workplace and favoured by the participants, albeit out of non-productivity-related motivation.
• Social Experience (14/33)
  In Office, our participants perceived the virtual avatars that represented their colleagues as “…motivating for productivity compared to sitting alone in nature” [P18]. Moreover, they desired on-demand interaction with these avatars. Such as “…if I call a colleague then his avatar will appear” [P15]. In contrast, if there is no need, existing avatars became more of a distraction in the productivity experience to our participants. Combining the pleasant and comfort factors of Nature with on-demand virtual avatars, some participants envisioned a productivity scenario of “…collaborative work in the nature environment” [P28].
• Ambient Sound (19/33)
  The audio also seems to contribute to the comfort factor. Our participants found the audio in Nature “more calming” [P28] and “less disturbing” [P14]. Specifically, some disliked the noises in Office such as typing on a keyboard and others babbling around. Although these sounds are in fact a usual part of their familiar open-plan offices, users wanted to “…have my peace and focus on myself” [P25] while performing the productivity task in VR.

4.3. Other Findings

We also collected supplementary data to explore participants’ responses to notifications in rear-seat VR. Our results revealed that most of our participants (N = 15) picked up the call due to curiosity even with a greeting “Hello?”. A few rejected (N = 3) or ignored (N = 7) the notification as they wanted to focus on the task. The others (N = 8) had no time to react before the call was over. We asked how much (not at all = 1 to totally = 7) they would answer a phone call in VR or on a phone during the productivity task in VR. Our participants generally would pick up a call in VR (Mdn = 7.0, SD = 0.39) and would not remove the headset to answer on a phone (Mdn = 1.0, SD = 0.94).

Our original hypotheses did not question the interaction between virtual borders and virtual environments. However, for statistical completeness, we conducted the mixed factor align-and-rank ANOVA [43] and found no significant interaction effect for the dependent variables as follows: perceived comfort: F(1,31) = 1.172, p = 0.287; perceived safety: F(1,31) = 1.651, p = 0.208; worry: F(1,31) = 0.002, p = 0.969); count of touch: F(1,31) = 2.372, p = 0.134; performance: F(1,31) = 0.066, p = 0.799; presence: F(1,31) = 0.497, p = 0.486; and perceived workload: F(1,31) = 0.274, p = 0.623. This, together with the significant results and qualitative analysis above, contributes to the following findings: (1) irrespective of the virtual environment design, the virtual border significantly decreases the number of occasions in which participants touch the car interior, and (2) irrespective of the virtual border, the design of virtual environments has a significant impact on productivity performance. Passengers are more productive in a familiar working environment, such as an open-plan office.

4.4. Validity and Limitations

Due to the COVID-19 pandemic and the resulting university and local restrictions, we were forced to adapt our original study design from a moving to stationary car. Additionally, we were only allowed to recruit participants from within our respective institutions. This kept social interactions to a minimum and eased their traceability. Although motion sickness and external traffic noise are not the focus of our investigation, their resulting absence may have an effect on the generalisability of our results. For example, a moving car induces motion on the passenger themselves that they may not be able to control. This affects the number of times they are exposed to the physical and virtual borders and thus may affect our defined feeling of safety, e.g., it might intensify the passenger’s fear of collision with the car interior and break their presence in virtual working environments more frequently. Research on motion sickness is an ongoing parallel effort [44,45], which we later plan to integrate back into our study. In contrast, the noise problem has been actively addressed by today’s commuters via headphones, which we see as a minor limitation in our study.

5. Insights and Outlook for Rear-Seat VR Workplace

In this paper, we explored how to adapt VR for productivity in the confined space of a rear-seat car experience. We investigated the necessity of virtual borders and the opportunity to increase productivity through the design of virtual environments. The results of our user study (N = 33) are contradictory with regard to the existence of virtual borders: Although users (i) touched the car interior significantly less, (ii) performance and the feeling of safety were comparable in the Border and No Border conditions—which suggests that hitting the interior does not affect productivity and our participants’ safety concepts. Finally, although the secluded Nature environment was subjectively preferred, participants performed significantly better in the Office environment.

Our exploratory study aimed to investigate how existing tasks transfer to VR in the rear seat of a car and what challenges need to be addressed in future work. In specific, we transferred the reading-span task, which has been studied in the automotive domain, for example, using heads-up vs. auditory speech displays in a driving simulator study [19], to rear-seat VR in our stationary vehicle setup. Below, we discuss insights based on our findings and provide an outlook for future research.

5.1. Representation Alternatives for Physical Limitations in a Confined Space

In the experiment, the implemented grid-based representation of physical limitations in the car rear-seat did not enhance the feeling of safety in the VR productivity tasks. By reporting a low level of disturbance in the No Border condition, our participants agreed that the small-scope VR interaction without virtual borders was gentle. Touching the car interior seemed acceptable “…I did not find it disturbing to hit only two or three times” [P24]. Similarly, qualitative feedback indicates that virtual borders were found comparably disturbing but helpful and useful. In specific, the participant touched the car interior much less (with a significant difference) when the virtual border is visible.

This provides a two-fold approach when deciding whether to include virtual borders: Firstly, in situations where concentration is favourable, virtual borders may be turned off. This may be useful in social interactions, such as communicating with a colleague/collaborating or in a learning context [46,47], where a high presence leads to a better experience—even if this affords collisions with the car interior. Secondly, virtual borders may be displayed in situations where broad movements are essential, for example, editing/creating presentations while using multiple screens/sources—in anticipation of the car interior being touched due to the broad and continuous movements. Moreover, the wider field of view of newer VR headsets in future studies might enhance the participant’s awareness of virtual borders in the periphery and might change impacts on presence. In addition to the above considerations, virtual borders can be represented in alternative ways based on previous HCI studies and qualitative feedback voiced by our participants, which we describe below.

5.1.1. Abstract or Skeuomorphic Visualisation

Instead of the grid-based visualisation, our participants suggested representing physical borders in the form of contextual or environmental objects, “…for example a cupboard” [P10]. This is consistent with prior work that suggests that representing physical borders as virtual artefacts in VR more subtly increases the awareness of real physical limitations [48]. Specifically for a virtual working environment, today’s web pages can be created as a skueomorphic representation—a virtual window. This design principle might decrease users’ confrontations with the physical borders of a car and avoid diminishing presence. For future work, we propose a focus on skeuomorphic representations of physical limitations for the rear-seat VR workplace.

5.1.2. Easy-to-Learn Auditory Indication

We also propose representing the confined space in a different way, not limited to visual representations. As highlighted by our participants, the representation of virtual borders “…does not need to be a grid” [P7]. Prior work [49] found that acoustic transparency enhances the VR user’s safety perception. Moreover, an auditory indication for real-world boundaries is comparable to visual indicators with respect to performance in VR [34]. Auditory icons or other natural mappings between objects and sounds might be easy to learn and promise a lower cognitive load due to the user’s life-long learning [50,51]. Using auditory indications (e.g., the sound effects of bumping into surfaces ranging from soft to hard) might indicate different levels of proximity towards physical borders. With limited work on this topic for the rear-seat VR workplace, for future research, we propose a systematic study of auditory user interfaces for indicating the physical limitations of a confined space in VR.

5.1.3. Supportive Haptic Feedback from the Car Interior

A car interior design that matches the virtual working environments (or vice versa) promises better performance and lower cognitive workload through enhanced immersion [52,53]. Although our prototype is based on visual and auditory senses only, our participants are inclined to renovate the car interior for rear-seat VR (e.g., “…the physical space of the car should be adjusted to HMD usage” [P1] or “…using a physical keyboard integrated in the car for some haptics” [P17]). Based on these findings, we envision haptic feedback for productivity in VR to be built upon in the established haptic design of car interior [54] to support VR interaction for rear-seat productivity.

5.2. Virtual Environment Design for Productivity in VR

When designing a virtual space for productivity, which types of virtual environment facilitate productivity? Our results seem to suggest that the primary determinant is the user’s familiarity with their daily workplace, be it borderless or limited, shared or alone, or natural or artificial (e.g., “…open-plan office reminds me of my daily workplace, where I feel more familiar to work” [P5]). In our study, this affected how productive participants were in the virtual world.

In addition and consistent with the previous work [22], users favour active control over a productive environment pre-tailored to their personal requirements. The results from our qualitative analysis show that users would like to customise virtual environments for a customised in-car workspace, such as “…a virtual world where I can fully immersed in as it is easier for me to concentrate ” [P3]. In general, users prefer less obtrusive interaction when using private time to be productive. Future work may investigate novel forms of unobtrusive VR interaction for enhancing productivity in the car rear-seat.

Finally, although the study took place in a stationary car, we predict consistently with prior work that motion-sickness will be an influencing factor for in-car VR experiences. Motion sickness can be escalated while watching videos or reading in a moving car [55]. Various solutions were found, such as repeated exposure [48,56], calming experiences, controlled breathing or listening to music [57,58], and visual cues synchronised with vehicle movements [33,59]. Here, we emphasise the importance of motion sickness (as a part of virtual environment design) for a follow-up study in a moving car.

5.3. Enhance Social Experience for Rear-Seat VR Workplaces

Our work tested the notification design of an incoming phone call during the task. Productivity and the potential for video calls were the reasons why participants said they favoured having the phone call option in VR. Participants even imagined holding the call “…while making the other person visualised as an avatar in the virtual world” [P28] or “…working on mails simultaneously” [P11]. Meanwhile, they were also concerned about privacy “…when there are strangers in the car” [P13]. It bothers them “…when the others just hear my voice” [P15] and they would rather “…text the person instead” [P16]. A VR workplace has been shown to prevent bystanders from invading one’s privacy [22]. However, privacy beyond screen observation in the confined and social environments of rear-seat VR interactions requires research on mitigation strategies. Concerns from VR users about safety and a changing environment were also observed in prior work on user needs for using VR in public places [60]. These may be mitigated by implementing visual, auditory, or haptic notifications in the VR system [61,62]. In line with prior work and inspirations from our participants, we advocate future work on multi-modal interaction and notifications (avatar-based, voice, and text) to enhance the social experience, with a focus on mitigation strategies for privacy concerns in the rear-seat VR workplace.

5.4. Ambiguous “Feeling of Safety” during Rear-Seat Productivity in VR

We found limited related work on safety during VR interaction in the confined space of a car [32]. However, when designing VR interactions for productivity in the rear seat, we still noticed safety as the prior precondition for any automotive interaction on real roads by referring to its relevant concepts in both VR and automotive research [34,35]. In our study context of rear-seat VR productivity usage, we explored the self-defined “feeling of safety” by measuring the perceived safety, comfort, and worry of touching the car interior. However, independently of whether virtual borders were shown, we found no significant influences on our safety concept. It is unclear whether safety/situation awareness is inconspicuous in this rear seat VR productivity context. To further testify influencing safety factors and to promote considering safety in a more well-defined way in the HCI community, we call for future studies on valid measurements of “feeling of safety”.

6. Summary

Today, rear-seat VR is still in its infancy. Both industrial and academic research, however, predicts increasing opportunities while also raising solid questions for thinking about solutions. In this prospective work towards automated vehicles, we described the development and evaluation of a rear-seat VR workplace, with a focus on exploring the impact of a virtual border display and virtual working environments on performance, presence, and the feeling of safety. We conducted an exploratory user study (N = 33) with a prototype implemented in the vehicle. Our choice of task, reading text and typing via a keyboard, was the result of a pre-study [33], where participants described their current commuting tasks. We found that users touch the car interior less when seeing virtual borders and perform better in a familiar working environment, such as an open-plan office. We further discussed representation alternatives of physical limitations (based on visual, auditory, and haptic senses), virtual environment design (concerning productivity and motion-sickness), and considerations for social experience and proposed to investigate the feeling of safety in future research and to consider it in the design of rear-seat VR interaction.

We present the first exploration of rear-seat productivity in VR by discussing which variables are transferable (e.g., presence) which ones need further evaluation (e.g., safety) and provide opportunities for future work (virtual environment to improve productivity). We expect insightful research in this young crossover field. With a joint force between automotive and VR industries and expertise, we are certain that this research will contribute to the larger picture of future passenger experience.