A Multi-Sensory In-Store Virtual Reality Customer Journey for Retailing: A Field Study in a Furniture Flagship Store

Abstract

The choice of furniture in a retail store is usually based on a product catalog and simplistic product renderings with different configurations. We present a preliminary field study that tests a Multi-Sensory In-Store Virtual Reality Customer Journey (MSISVRCJ) through a virtual catalog and a product configurator to support furnishings sales. The system allows customers to stay immersed in the virtual environment (VE) while the sales expert changes the colors, textures, and finishes of the furniture, also exploring different VEs. In addition, customers can experience realistic tactile feedback with in-store samples of furniture they can test. The journey is implemented for a furniture manufacturer and tested in a flagship store. Fifty real customers show positive feedback in terms of general satisfaction, perceived realism, and acceptance. This method can increase purchase confidence, reduce entrepreneurial costs, and leverage in-store versus online shopping.

1. Introduction

The retail industry is experiencing radical changes concerning products, customers, and sales channels. The first aspect is product evolution [1]. The development in engineering and processes allows for a higher level of features and mass customization of configurations, materials, and finishes. Therefore, the customer’s choice is much more difficult today than in the past, providing less confidence in the purchase. On the retail side, the product variety leads to the impossibility of having the full catalog in the local warehouse.

Moreover, customers are changing. Digital connection makes them more informed about the product and allows them to seek products tailored to their needs by playing an active role in purchasing. However, most of them have difficulty imagining the final product (i.e., ergonomics, aesthetics, etc.) [2] and whether the product fits in the space (i.e., spatial visualization). In addition, the decision-making process is complex and constrained: budget, available space, aesthetics, ergonomics, time investment, environmental impact, and lifestyle [3]. Customers demand a novel [2,4], multisensory [5], entertaining [6], and social [7] experience. Thus, the shopping experience becomes fundamental as a sensory and emotional process of discovering products and providing confidence in the purchase [2].

Customer journey (CJ) is defined as the study of customers’ interactions with a service provider to achieve a specific objective, consisting of several touchpoints [8]. CJ has become an increasingly important concept for predicting and triggering complex customer behaviors and getting insights into their experiences. The term has been used in several disciplines since the 1990s, and its literature has grown more than sevenfold over the last eight years [9].

The last aspect is related to the recent growth of digital sales channels, such as e-commerce and others offered by digital opportunities [10,11]. Although physical stores can provide a more engaging experience than online stores, they are facing a deep crisis in delivering added value compared to their costs: real estate, management, personnel, and warehouses [12].

We present a novel approach to in-store retailing by leveraging virtual reality (VR) technology. VR is well-integrated into the customer journey and allows multisensory feedback by combining virtual stimuli such as audio and video with the sense of touch provided by samples of real objects. Our approach, called Multi-Sensory In-Store Virtual Reality Customer Journey (MSISVRCJ) (see Figure 1), aims to make the in-store experience competitive compared to other channels in engaging customers, supporting them in the right decisions, and strengthening customer–brand relationships [13,14].

2. Background

The first VR experience within the retail field occurred in 1993, with the simulation of the shopping environment [15]. The main purpose was to provide a shopping environment and experience as close as possible to the real circumstance. Later, immersive aspects of VR and interactivity were investigated [16]. Other researchers explore how the virtual store affects customer behavior compared to the traditional store [17]. The latest research is related to the impact of VR on customer satisfaction, enjoyment, engagement, and acceptance in relation.

Fox et al. (2009) [18] argued that VR has a utilitarian component in fulfilling the needs and expectations of some consumers. Later on, Farah et al. (2019) [19] consider that VR adopted by retailers also has an impact on improving the overall consumer experience by focusing more on the human senses. Indeed, as discussed in the literature, VR is a medium “to meet the needs of users’ sensorimotor channels” [20]. By adopting this technology, users will feel immersed in a virtual world, exploiting the various senses such as touch, sight, and smell [21].

Van Kerrebroeck et al. (2017) examine the potential of a relaxing in-store VR experience in a shopping mall to address the issue of crowding during shopping. The research allows retailers that at times suffer from high perceived crowding to provide a VR experience to shoppers as a remediating option to alleviate the negative effects of crowding on consumers and consequently diminish the negative consequences for the retailers [22].

Zenner et al. (2020) present a VR-based application to support consultations in furniture stores [23]. The system allows customers to work out different configurations of a sofa with a sales expert and allows customers to experience them through immersive VR in a variety of VEs. While the sales expert can change the sofa layout and fabric, the customer can stay immersed and experience a real tactile feel of the configured sofa through passive haptic feedback provided by a sample piece that the customer can sit on.

The metrics measured are subjective, such as the time of the experience (estimated by the users themselves), users’ behavior and attitude toward new technologies, and the immersiveness and usefulness of the system. A preliminary field study in a furniture store shows that the system is immersive, conveying realistic impressions of the sofa configurations. In addition, customers perceive the VR configurator as useful as it would make their purchasing decisions easier.

Recently, Virtual Reality Customer Journey (VRCJ) has been conceptualized as the use by businesses of interactive computer-mediated environments. VRCJ can offer sensory feedback to engage consumers, strengthen consumer–brand relationships, and drive consumers’ desired behaviors at any stage journey [13]. The VRCJ framework suggests that sensory feedback affects all VR engagement dimensions. However, a broad, systematic understanding of VR’s drivers and effects through the CJ remains poor [13].

These aspects motivate our research to gain further insights into the user behavior related to a novel journey (i.e., MSISVRCJ) as an in-store implementation of a multisensory experience.

3. Methods

The journey is designed to guide the shopping experience in collaboration with our industrial partners. The MSISVRCJ involves the following actors:

• Customer—MSISVRCJ accompanies customers through each touchpoint in a natural way to meet their needs and expectations.
• Sales Assistants—They are trained about the MSISVRCJ usage and objectives, and they introduce the VR experience according to their preferences and the specific situation.
• Technical Helpers—They support the MSISVRCJ hardware and software (e. g. help with wearing and set up of the head-mounted display, safety issues, etc.). They are as least intrusive as possible; in the long-term, their role can also be taken by the sales assistants.

MSISVRCJ requires a dedicated area in the shop (see Figure 1), 3 × 3 m, not a passageway, and quiet, as users can feel uncomfortable to be blinded and observed while in VR [24,25]. The experimental setup, including room size and location, was adapted to the flagship store where the experiment was conducted and agreed upon with the industrial partner. In addition, the space should offer comfortable seats (e.g., chairs or sofas) on the perimeter, as the seated position is safer and more accepted by first-time VR users (see Figure 2).

The central area must be free of obstacles (e.g., tables, hard chairs, cables, etc.) to avoid collisions and tripping hazards during standing sessions. A graphics PC runs dedicated VR software and drives the head-mounted display (HMD). The graphics PC is connected to the sales assistant’s tablet via a wireless network. The tablet supports the customer during the setup process before and after the VR session. The objective is to hide the technology and keep the experience similar to traditional shopping.

Another key feature of MSISVRCJ, compared to the other methods in the literature, is the multisensory feedback. This feature aims to compensate for the lack of touch in low-cost VR experiences, and it is achieved by simply presenting material samples as non-virtual haptic support. In the composite continuum [26], visual virtuality–haptic reality (vV-hR) refers to conventional visual virtual environments (VEs) with real touch. In this journey, vV-hR is achieved by using a tangible material sample to interact with virtual objects configured according to customer material choices.

The introduction of these real-world sensory elements can vary depending on the specific product. For example, a car configurator can use a real seat to show ergonomics and a piece of leather from a sample book under the hand to experience the material. At the same time, the user can see the multiple color combinations in the VE. The timing of these tangible experiences can also be adjusted to be included or not included in the VR experience. This approach requires optimal in-store sample design to provide an extensive catalog.

The Touchpoints

MSISVRCJ is a step-by-step process consisting of four touchpoints (see Figure 3)—with holistic involvement of store management and staff in terms of tools, motivation, and training. Customers enter the store (Touchpoint 1). Söderlund (2017) reported that when customers find a shop assistant present (vs. absent) in the store, they report significantly higher levels of customer satisfaction, with their increased levels of pleasurable feelings mediating this effect [27]. The sales assistant proposes that customers explore the current catalog in VR. Giving their full agreement, customers wear the HMD and start familiarizing themselves with the MSISVRCJ’s premade scenes called “moods” (Touchpoint 2). In order to be more comfortable with the interface, customers start in a sitting position, and later, if they feel confident, they can stand up and walk in the dedicated area to change their point of view (PoV).

The virtual scenes include products that are shown emotionally and effectively using traditional multimedia (e.g., color schemes, product arrangements, and combinations) and virtual multimedia (e.g., 3D animations and sounds). Customers start familiarizing themselves with the MSISVRCJ’s ability to change their PoV by asking the sales assistant. Typically, at this point, most customers ask to see the product they have in mind or like best. This process triggers Touchpoint 3, where the customer can be active and configure the virtual space. The sales assistant software configurator includes the following features (see Figure 4a) used to fulfill customer’s suggestions on the tablet:

• Projects—This feature loads and saves scenes from the virtual catalog, including the preset ones (i.e., the “moods”) and those created by customers.
• Scene\Product Navigation—This feature allows users to rotate, move, and scale the 3D models, including the entire scene.
• Product\Material Selection—This feature loads products, materials, and finishes.
• Teleport—This feature allows the quick rapid movement of the virtual camera/customer in the scene to avoid walking.
• Walls\Ceiling Turn-On/Off—This feature disables occlusion elements.
• Scale the Scene—This feature enlarges\reduces the virtual scene to have an overview or focus on details (e.g., stitch of a virtual sofa).
• Reset the Scene—This feature erases customer changes and loads the default settings.
• Camera Switch—This feature moves the customer PoV to different preset positions in the scene (e.g., sitting on the sofa) for quick navigation.
• Scene Transfer—This feature transfers the scene on the HMD.

The software was designed to avoid any technical interaction from the customer side to overcome digital shame [28]. The sales assistant customizes furniture through the customer’s verbal instructions, and the customer only sees the result.

4. Field Study: Flagship Furniture Store

We chose a furniture flagship store as furniture is a high-value industry ($247,836 million in 2021) growing at 2.17% annually [29]. Furniture is also an expensive and long-term purchase [3], needing high configuration and attention to materials, space constraints, textures, and surface touch [2]. Therefore, we conducted the field study in collaboration with a leading furniture design manufacturer, which operates worldwide and provides an extensive catalog of furniture products with the emblem “Made in Italy”. The manufacturer allowed each product the possibility to configure its size, a palette of materials (>500), and options such as recliners, automatic headrests, and even embedded speakers.

The manufacturer was very involved in the development and validation of the MSISVRCJ within the field study, but it imposed some conditions:

• Customer behavior investigation could not interfere with the sales process. In-store researchers could discreetly observe user behavior and interview customers with only a few concise questions.
• Customers should have been aware of and agreed to the experiment conditions (e.g., data collection, photos, feedback) for scientific purposes only.
• Sales assistants could decide whether to propose the MSISVRCJ to the customer based on the situation (e. g., lack of time).

These constraints excluded some well-accepted questionnaires used in the literature. Thus, we chose a qualitative analysis through user observation and a survey [30].

The field study objective aimed to evaluate in a real store scenario with real customers the following questions: “Do customers appreciate MSISVRCJ?”; “Are virtual scenes realistic?”, “Is MSISVRCJ perceived as useful and helpful?”.

4.1. Procedure

After giving their agreement, customers wore the HMD and started the MSISVRCJ by exploring the virtual catalog with preset “moods” (i.e., scenarios). Customers could customize the mood and ask for specific models, materials, colors, and finishes. Compared to the traditional journey, we introduced the figure of the shadowing researcher as a casual observer who discreetly took notes on user behavior and formally surveyed customers at the end of the experience [31].

The implementation of the VR system in the furniture store took place in a dedicated area (i.e., an area surrounded by sofas to avoid collisions with customers). The tangible aspect of MSISVRCJ consisted of a smart selection of sofas in the VR area and a book of material samples. The sofas provided different types of seat toughness, so the user could find the best feeling on the real sofa, seeing their preferred materials in VR, and a material samples book for textures and finishes (see Figure 5).

Once customers had chosen the model, materials, colors, and finishes, they could remove the HMD and eventually complete the purchase. Whatever the sales outcome, the researcher administered the survey to the customer.

4.2. Materials

The field study was carried out in a flagship store located in London. The flagship store is increasingly popular in major cities to build brand identity, present the latest trends, and showcase product collections [4]. In order to ensure an immersive and safe virtual experience, the in-store dedicated area was carefully chosen in a corner of the ground floor.

The HMD used for the experiment was the Samsung Odyssey. We decided not to provide users with the two wireless controllers since they were not supposed to interact directly with the virtual elements, as the shop assistant was in charge of configuring the elements of the VE according to the user’s requests. In addition, users had to keep their hands free to touch the real material samples while immersed in the VE, in order to experience realistic tactile feedback.

The PoV change was the only activity allowed to the user by naturally moving the head. The graphics PC was a mobile workstation with an Intel Core i7-7700HQ processor, 16 GB RAM, and an NVIDIA GeForce GTX 1070.

The company implemented several “moods” as VEs. Qualified 3D artists provided realistic scenarios, including countryside models, by capturing the feelings through sound effects and animations (e.g., the wind blowing the vegetation and sky). The experience also included some aroma emitters [32] of the local fragrances to convey the emotion of “being there”. The sofa collection was reproduced in every detail, such as the stitching, the cushion wrinkling, and even the recliner animations.

4.3. Participants

Customers entered the store during two experiment sessions: from 10 April 2019 to 11 April 2019 and from 9 May 2019 to 10 May 2019. They had to accept privacy conditions before starting.

4.4. Metrics

The researchers observed the customers, and they reported the data collected from the following indicators on the rendering PC:

• Time Duration—VR time duration from when customers wore the HMD until they ended the session or removed the HMD.
• Level of familiarity with VR—Users were informally asked to rate their level of familiarity with VR on a scale of 1 to 7.
• User Behavior—The main actions (e.g., try to touch, change the PoV, passively look).
• User Targets—The objects that captured user attention or interaction (e.g., furniture details, an overview of the virtual catalog).

The survey questions (see Appendix A)—only four allowed by the furniture manufacturer—were designed to evaluate customer experience [33], measured on a 7-point Likert Scale. Only the third question was followed by an open-ended question to better understand the user motivations.

The questions were the following:

• How do customers rate the MSISVRCJ?—to evaluate general satisfaction.
• How do customers perceive the realism of the virtual scenes?—for the cognitive and sensory evaluation.
• Does MSISVRCJ help customers with shopping? Why?—for the affective evaluation (see Appendix B).

5. Results

Sixty-five customers entered the shop during the session days, but only n = 50 (32 males and 18 females, with an age distribution n= 18, 18–30; n = 22, 30–45; n = 10, 45–70) accepted to take part in the experiment. The time duration in the VE was generally below 10 min (average 7.03 min ± 1.63 min).

Participants had little or any experience with VR. A total of 30 participants had never experienced VR experiences while 20 participants had previous experience with VR. To assess the level of familiarity with VR, users answered the following question on a 7-point Likert scale from 1 (=not very familiar) to 7 (=very familiar), “What is your level of familiarity with VR?”. Results indicated that the familiarity level was not very high (av. Sc. 3.45; σ² = 0.94).

We observed that all participants changed their position from sitting to standing while immersed in the VE. Moreover, changing PoV (see Figure 6) was the most prevalent user behavior (82%, n = 41). This result may have implications related to the immersiveness of the scene. Indeed, using the HMD allows users to be completely immersed in the VE and discover the furniture collection in a completely different way than browsing it through traditional printed furniture catalogs.

A minor but interesting aspect is how eight participants tried to touch the virtual objects as proof of the realism of the experience. The observation of the user targets (see Figure 7) shows that almost all participants could have an overview of the virtual catalog (88%, n = 44) and focus on some details of the furniture (94%, n = 47), such as hand rests, legs, and stitching. Some customers tried to look at the bottom of the sofa to evaluate the “quality of construction”.

We observed a common surprise and excitement reaction from most of the users, as reported by their expressions: “I feel well in here!”, “It’s crazy I can look around the sofa!”, “Amazing, I want to touch it!”, “It’s fun!”, “Very helpful, I want to see also my home with this technology”, “My wife is very happy, so I’m the happiest man in the world”, “When I will buy my sofa and my furniture I want to use this technology”, “It’s a good overview of the product and it’s better for me to take a decision”, “Technology is amazing, I never saw so clearly!”.

The survey shows a good level of satisfaction from the customers (av. sc. 6.12, σ² = 0.71) [33] engaged with the MSISVRCJ, as also seen during the observation.

The perceived realism results are very positive (av. sc. 5.98, σ² = 0.59). Moreover, the results suggest that users were immersed in the VE. The third question led to positive results (av. sc. 6.54, σ² = 0.73), also supported by the following reasons:

• They saw products and configurations not physically available in the shop.
• They observed the models arranged in a pleasant environment or in a place that reminded them of\simulated their home.
• They increased confidence by better visualization of spatial configurations and material combinations.

6. Discussion

The main result of the field study about the MSISVRCJ is the overall positive feedback. However, it is interesting to notice how the average duration of the VR experience is below our expectations, considering the variety of moods in the 3D catalog. This result can be explained by the common request of customers to skip the moods, as they already had in mind the exact model of their interest from other channels (advertisement, friends, etc.). This request was supported by the software that allows jumping to the configuration and, consequently, to the shopping activity. However, we can also argue that the relatively short time in VR can depend on subjective aspects such as the discomfort of using an unknown technology or being immersed in an unfamiliar world, or the isolation from physical reality in a public place. In fact, covering the eyes with HMDs means the loss of visual knowledge of the environment. As a consequence, the user might be uncomfortable if touched or might be worried about personal belongings while using the HMD [25].

Another aspect is the natural and spontaneous PoV activity observed in our study and its correlation with the environment immersion of “being there” in a real shop [34].

Interestingly, many customers were curious about the details (e.g., under the sofa) and the joints (e.g., the stitching between the cushions) as the products are so well-modeled that such details are present in the virtual scene. Furthermore, customers revealed how they felt uncomfortable in the real shop to look after such details, while in VR they felt more confident and private.

Another behavior denoting the quality of the experience is how some participants tried to reach or touch the virtual objects as an instinct for tactile engagement [35]. This provides positive support to the first two research questions, but it is interesting for further investigations.

The open-ended question helped to validate the customers’ acceptance, as an important parameter in VR experiences, in public settings [24]. Furthermore, it supports the third research question about the perceived value of MSISVRCJ (e.g., wider catalog, pleasant VEs, product visualization before purchase) and the capacity to build or strengthen consumer–brand relationships, as demonstrated in the literature [13,36,37].

7. Conclusions

We propose a customer journey that integrates immersive VR, an optimal warehouse, material samples, and a dedicated in-store area to provide a comprehensive product catalog with sensory and ergonomic feedback. The technology is hidden from the customer by the sales assistant’s dedicated interface to control the scenes, the point of view, and the customization process. The journey is implemented for a furniture design manufacturer and tested in a flagship store with real customers. Fifty real customers experience the shopping experience and show positive feedback in terms of general satisfaction, perceived realism, and acceptance. MSISVRCJ can provide the following advantages to the user:

• Improve the product customization and configuration process with realistic graphics and immersive VR technology.
• Help to evaluate the spatiality of customer-related scenarios with the support of the shop assistant.
• Keep the in-store feeling and enhance it with a broader catalog and experiences.
• Provide multisensory feedback on ergonomics, the touch of real material samples, and scents in the real store.
• Induce better confidence in customers and reduce the risks of an afterthought.
• Deliver an engaging and new experience, strengthening brand awareness and driving innovation and technology.

Furthermore, the journey has several strengths from the entrepreneurial side: reducing inventories requires smaller stores without compromising the sensory perception as expected by shop visitors.

An important aspect that we want to report is that we encountered initial resistance from sales assistants to adopt the novel approach alongside their well-known sales techniques. This meant that, especially in the early stage, many customers entering the store were not offered the MSISVRCJ.

Therefore, we want to pinpoint the importance of the involvement, motivation, and training of the store management and staff to properly and successfully implement the MSISVRCJ.

From a scientific perspective, we have encountered major limitations in the design of the case study because of the constraint imposed by the furniture manufacturer. Nevertheless, we are planning to use MSISVRCJ as a framework for further research in this growing area [13], introducing well-supported evaluation standards (i.e., system usability scale, NASA task-load index, user experience questionnaire), and comparing the results in other fields and store locations and dimensions (e.g., mall and shopping centers). Moreover, the self-selected sample for the field study should be considered the main limitation, since the consumers who volunteered for the VR experience could already be inclined to review the technology positively.