Research and Evaluation of Multi-Sensory Design of Product Packaging Based on VR Technology in Online Shopping Environment

Abstract

The development and application of virtual reality (VR) technology significantly enhances consumer immersion. Exploring a multi-sensory evaluation model for virtual packaging is valuable for integrating VR technology with packaging. This study developed a multi-sensory evaluation model for virtual packaging using the analytic hierarchy process (AHP). Eye-tracker experimentation was conducted to identify consumer attention indicators when interacting with virtual packaging. These indicators were quantified using Saaty’s nine-level importance scale and expert input, resulting in a comprehensive multi-sensory evaluation model. Subsequently, a VR shopping system focused on potato chips and cola as packaging design objects. This system was evaluated using the established model, and the results were analyzed. Based on the findings, improvements were made, and the system was re-evaluated using the modified model. The post-improvement evaluation demonstrated significantly enhanced sensory experiences. These results validate that the developed evaluation model effectively guides multi-sensory design approaches for packaging within a VR environment.

1. Introduction

With the rapid development of the Internet, online shopping has become one of the most commonly used shopping methods. The main advantage of Internet shopping is its convenience and the simplification of the sales process [1]. Consumers can skip the steps involving the seller, allowing products to be delivered directly to them by the supplier through courier services [2,3]. However, when shopping online, consumers can only understand the products through pictures, videos, and product descriptions, leading to a lack of experiential engagement and interaction [4,5]. The multi-sensory product packaging design allows designers to overcome the limitations of the traditional visual promotion model, beginning with the human senses of sight, hearing, taste, smell, and touch [3,5]. This approach stimulates consumers’ sensory functions in multiple ways, enabling them to understand the product more realistically and effectively guiding their consumption [6,7,8]. In the online shopping environment, the future development of packaging design will transcend traditional approaches to promotion, enhancing product identification through multi-sensory design [9]. This will integrate packaging into consumers’ lives in a more humane, interactive, and engaging manner.

Virtual reality technology (VR) combines computer graphics, artificial intelligence, sensor display technology, network technology, and other cross-disciplinary fields [10]. Customers benefit from reduced shopping time, 24/7 availability, and increased product information in the virtual reality shopping environment [11]. The proposed Shop-WISE method allows consumers to select, inspect, or search for products in a virtual environment [12]. Lee [13] compared the consumer interface of VR shopping malls with traditional shopping centers and concluded that VR shopping significantly enhances consumer satisfaction. Liu [14] studied the shopping experiences of the elderly population and proposed solutions based on the findings. Kim [15] expands the sensory shopping experience through VR, enriching the positive emotions of consumers to stimulate their imagination and creative thinking. Branca [16] analyzes consumer evaluations of packaged products in immersive VR, manipulating package structure and haptic cues and elucidating potential differences with consumer responses in real life. Zhao [17] provides a practical method for assessing the shelf prominence of product packaging, revealing how consumer attention is affected by design elements in product packaging within a shop. Virtual reality technology provides a sense of reality by enabling consumers to immerse themselves in a virtual environment from a first-person perspective and interact with objects through virtual reality devices [18,19].

Although virtual reality technology has been widely used in online shopping, its application in packaging is still being explored. Investigating a virtual packaging multi-sensory evaluation model that combines virtual reality technology with multi-sensory design is of great value. This study aims to examine the relationship between packaging materials and functions and consumer sensory experience in virtual environments and to establish an evaluation model for the multi-sensory performance design of packaging in virtual environments. This model is intended to guide the decision-making process of the consumer multi-sensory experience at the initial stage of virtual packaging design. The structure of the study is shown in Figure 1.

2. Related Work

Multi-sensory design in product packaging, including visual density and material composition, is crucial in shaping consumer perceptions [20]. Research on food packaging has demonstrated that various materials affect sensory properties, such as taste and texture, which, in turn, influence consumer expectations [21]. Increased visual density in packaging design has been shown to elevate sensory expectations and influence purchasing decisions, highlighting the importance of visual elements in multi-sensory marketing strategies [22].

The application of virtual reality (VR) technology to enhance consumer experiences has received significant attention, particularly in product design and evaluation [23,24]. VR has been successfully employed in the cultural and creative industries to create immersive product evaluation environments, allowing users to interact with products virtually [25]. Additionally, VR’s ability to simulate physical environments has benefited various fields, including healthcare, where VR-driven designs enhance user experiences by replicating real-world interactions [26].

Integrating immersive technologies, such as VR, into online shopping environments offers new opportunities for consumer engagement [27,28]. However, understanding how consumer behavior transitions from physical to virtual environments remains challenging. Despite advancements in multi-sensory design and VR technology, their integration within online shopping contexts remains insufficiently explored. Existing research primarily focuses on sensory design or immersive technologies, with few studies examining their combined impact on consumer behavior in virtual retail environments. This research evaluates the sensory experiences of virtual reality technology in packaging applications and explores the following aspects: In the third section, an evaluation model is constructed, and a virtual reality shopping mall is designed. The shopping environment, packaging sensory experiences, consumer immersion, and the expression forms of packaging materials are analyzed, designed, and implemented. The system is developed using two common packaging forms: bagged potato chips and bottled soda. In the fourth section, the developed system is evaluated and improved using the constructed virtual packaging multi-sensory evaluation model, and the results verify the feasibility of the evaluation model in the field of virtual reality technology. Finally, the fifth section provides some concluding observations.

3. Method

3.1. AHP-Based Model for Multi-Sensory Evaluation in Virtual Packaging

The analytic hierarchy process (AHP) is a subjective evaluation method developed by the American operations research scientist Saaty in the early 1970s [29,30]. The AHP involves decomposing the decision problem into a hierarchical structure consisting of the target, criteria, and sub-criteria layers. Using mathematical methods, a quantitative representation of each factor’s relative importance is then assigned based on subjective judgment, followed by determining the weight of each indicator’s importance. Finally, by comprehensively calculating the weights of the indicator’s relative importance at each layer, the combined weights of the criteria layer and sub-criteria layers are obtained [31]. These weights serve as the basis for evaluation and scheme selection [29]. The hierarchical analysis process is shown in Figure 2. The core of the analytic hierarchy process (AHP) lies in stratifying and quantifying influencing factors. It decomposes an abstract phenomenon or problem from complexity to simplicity, facilitating intuitive judgments and decision-making on complex issues. The sensory design of packaging in the Internet environment involves various indicators, which is fundamentally a decision-making process [32,33,34].

3.2. Hierarchical Modeling

3.2.1. Eye-Tracker Experimentation

The acquisition of evaluation indicators is a crucial component of the evaluation model. By conducting eye-tracker experimentation on constructed virtual reality packaging, factors affecting consumer experience and those of most significant concern can be identified as the basis for evaluation criteria [35]. The eye tracker mainly relies on eye-tracking technology to track eye-movement trajectory and fixation residence time in order to obtain real-time data on subjects’ visual cognitive activities during the experiment [36,37]. The experimental instrument is the Dikablis Glass 3 eye tracker manufactured by Ergoneers in Germany, which is equipped with several high-resolution cameras and sensors, including an eye camera and a scene camera, as well as built-in coordinate positioning technology. In this experiment, a ring box was selected as the 3D entity for recognition, and the specific model was created using 3DMAX modeling software (version 2021, Autodesk). The appearance and surface details of the box were refined using Substance Painter software (version 2021.1, Adobe), with textures, colors, and light sensitivity designed to generate the model’s maps. Finally, the designed maps were integrated with the model. The final model is shown in Figure 3.

A total of 25 men and women aged 18–30 were invited to participate in this experiment. Participants were exposed to the experimental samples for the first time. The experimental samples included static, dynamic, third-person perspective, first-person perspective, color matching, and other types of virtual reality packages. Before the experiment, participants were instructed to wear the Dikablis Glass-3 eye tracker and undergo a nine- or five-point calibration process, during which they needed to focus on specific points on the screen. This process allowed the system to calibrate pupil position and eye rotation vectors, ensuring accurate eye tracking.

The experiment was conducted as follows: participants scanned the virtual scene of the ring box presented on a mobile phone and operated it according to their personal preferences. A virtual playground scene was established to generate more interactive behaviors and facilitate the determination of evaluation indicators. The scene utilized a first-person perspective to enhance immersion, with visual field movement controlled by the button on the screen. The perspective could be adjusted by sliding the screen. The scene featured bright lighting, nearly complete amusement facilities, and environmental sound effects that mimicked real playground sounds, processed to enhance entertainment and create a romantic atmosphere. Interaction design was implemented for the pirate ship and the big pendulum by adding colliders and scripts to their models. When participants moved to the locations of the big pendulum or the pirate ship and clicked on the models, these facilities’ running status and sound effects were displayed in real time. Two balloons were positioned in the square in front of the castle. When users clicked on the balloons, a balloon explosion effect was triggered. After both balloons were clicked and exploded, an animation of the ring box model appeared, as shown in Figure 4.

Simultaneously, the eye-movement camera on the eye tracker captured the participants’ eye images in real time using a near-infrared light source and light reflection points. The computer system connected to the eye tracker calculated the eye vector based on the relative positions of the pupil and reflection points. This eye vector was then mapped to the scene camera’s image to determine the specific point and area of the participant’s gaze. Concurrent scene cameras recorded images of the environment from the participant’s perspective. These cameras were positioned outside the lens to offer a panoramic view of the participant’s field of view. Real-time eye-movement data were transmitted to the computer, where fixation points, fixation times, fixation paths, and changes in pupil diameter were analyzed. The resulting images displayed the participant’s visual attention distribution. As shown in Figure 5, the eye-movement hotspot map illustrated participants’ attention distribution in the playground scene, with red indicating the most concentrated areas of gaze and fixation and yellow and green representing areas with less fixation.

3.2.2. Evaluation Modeling

Firstly, this study aims to establish a virtual packaging multi-sensory evaluation model. Thus, the target layer is the virtual packaging multi-sensory evaluation model, designated as (A). Analysis of Figure 5, combined with expert evaluations and the functional distribution of virtual scene regions, reveals that critical red and yellow areas of interest are concentrated in interactive regions requiring user actions such as clicking, sliding, high color differentiation, or dynamic model changes. These areas can be summarized into five criteria within the virtual scene: interactive behavior, color transmission, sensory experience, scene experience, and packaging function. These five criteria are designated as the evaluation model’s criteria layer and numbered B1, B2, B3, B4, and B5, respectively. Based on participant feedback and expert analysis, the five main criteria are subdivided into 17 indicators: consistency of operational behavior, the accuracy of information transmission, rationalization of the control module, rationality of color application, coordination of color and environment, replaceability of color groups, hearing, vision, touch, smell, taste, adaptation of scene and packaging, scene immersion, scene entertainment, accuracy of packaging function realization, matching degree of packaging function, and the suitability of packing model and function. These 17 indicators are numbered C1 through C17. The evaluation model is shown in

Table 1. Virtual packaging multi-sensory evaluation model.

Target Layer	Criteria Layer	Sub-Criteria Layer
Virtual packaging multi-sensory evaluation model (A)	Interactive behavior (B1)	Consistency of operational behavior (C1)
		Accuracy of information transmission (C2)
		Rationalization of the control module (C3)
	Color transmission (B2)	Rationality of color application (C4)
		Coordination of color and environment (C5)
		Replaceability of color groups (C6)
Virtual packaging multi-sensory evaluation model (A)	Sensory experience (B3)	Hearing (C7)
		Vision (C8)
		Touch (C9)
		Smell (C10)
		Taste (C11)
	Scene experience (B4)	Adaptation of scene and packaging (C12)
		Scene immersion (C13)
		Scene entertainment (C14)
	Packaging function (B5)	Accuracy of packaging function realization (C15)
		Matching degree of packaging function (C16)
		The suitability of packing model and function (C17)

.

3.3. Construction of Judgment Matrices and Consistency Tests

3.3.1. Construction of Judgment Matrices

Establishing the judgment matrix involves quantifying consumers’ subjective assessments of each indicator. Fifty experts in packaging and virtual reality technology were invited to design a questionnaire using AHP. Regarding the experts, they are professionals with over 10 years of experience in packaging design and virtual reality applications. These experts provided consultation during the system development phase but did not participate in the experiment described in Section 3.2.1, which involved different participants. The questionnaire is designed to compare the criteria in the criteria layer and the indicators in the sub-criteria layer in a pairwise manner [38]. Experts must use Saaty’s nine-level importance scale to select the appropriate values for comparing pairs of indicators, as shown in

Table 2. Saaty’s nine-point scale and its meaning.

Scale	Meaning
1	Indicates that the two elements are equally important.
3	Indicates that one element is slightly more important.
5	Indicates that one element is more important but not overwhelmingly so.
7	Indicates that one element is more important.
9	Indicates that one element is extremely more important.
Reciprocal Values (1/3, 1/5, 1/7, 1/9)	If one element has a higher value of x than another, then the reciprocal of x (1/x) indicates the relative importance of the second element compared to the first.

. The options should follow logical consistency; for example, if A > B and B > C, then C cannot be greater than A.

In this study, 50 experts participated, and all 50 questionnaires were completed with valid responses. Survey results are summarized in Figure 6, indicating that respondents have varying opinions on different packaging function indicators. However, the accuracy of packaging function realization (C15) is identified as a particularly significant indicator. Additionally, the suitability of the packaging model (C17) is also recognized as necessary in certain aspects. Therefore, to better meet user needs and expectations, it is essential to consider the importance of these indicators comprehensively and make necessary optimizations and adjustments in practical applications.

Step 1: The judgment matrix is established using the questionnaire survey data, and the weights are subsequently computed. Calculating the weights is a crucial step in assessing the importance of indicators. For example, in Formula (1), $w_i$ represents the normalized weight, $a_{ij}$ denotes the element in the judgment matrix that indicates the relative importance between the $i$-th and $j$-th elements, and $n$ is the order of the matrix. To calculate the weight of each indicator, the judgment matrix must first be normalized. Then, the rows of the normalized matrix are summed to obtain a column vector. The weight vector is obtained by dividing each element in the column vector by $n$:

$$w_i={\displaystyle \frac{\sqrt[n]{{\displaystyle {\prod }_{j=1}^n{a}_{ij}}}}{{\displaystyle \sum _{i=1}^n\sqrt{{\displaystyle {\prod }_{j=1}^n{a}_{ij}}}}}}$$

(1)

Step 2: Calculating the maximum eigenvalue ${\lambda }_{\max }$ is a crucial step in evaluating the consistency of the judgment matrix. The calculation of the maximum eigenvalue of the judgment matrix is detailed in Formula (2), where $n$ is the order of the matrix, $A$ represents the pairwise comparison matrix, and w is the corresponding eigenvector. The term ${(Aw)}_i$ denotes the $i$-th element of the product of matrix $A$ and vector $w$, while $w_i$ refers to the $i$-th component of the eigenvector $w$:

$${\lambda }_{max}={\displaystyle \frac{1}{\mathrm{n}}}{\displaystyle \sum _{i=1}^n\frac{{(Aw)}_i}{w_i}}$$

(2)

Step 3: The consistency test verifies the conclusions’ rationality and calculates the indices’ importance weights relative to the higher criterion based on the judgment matrix. Formula (2). represents the consistency index, defined as the ratio of the maximum eigenvalue minus the matrix order to the matrix order. A consistency index of 0 indicates that the judgment matrix is perfectly consistent. A higher $CI$ value indicates a greater degree of inconsistency in the judgment matrix:

$$CI={\displaystyle \frac{{\lambda }_{max}-n}{n-1}}$$

(3)

Formula (4) calculates the consistency ratio (CR), which is the ratio of the consistency index (CI) to the random consistency index (RI). The value of RI is obtained from the random consistency index (RI) table, which is derived from simulations conducted by Saaty involving 1000 iterations, as shown in

Table 3. Random consistency index (RI) values for different matrix orders.

n	1	2	3	4	5	6	7	8	9	10	11	12
RI	0	0	0.58	0.90	1.12	1.24	1.32	1.41	1.45	1.49	1.51	1.54

[39]. The judgment matrix passes the consistency test if the CR value is less than 0.10; otherwise, the matrix must be readjusted. This involves recalculating and revising the pairwise comparisons until consistency is achieved. The judgment matrix and consistency test are constructed according to the abovementioned steps. The results of these tests are presented in

Table 4. Judgment matrix for pairwise comparison of criteria B1 to B5.

Matrix 1	Interactive Behavior (B1)	Color Transmission (B2)	Sensory Experience (B3)	Scene Experience (B4)	Packaging Function (B5)	Weight
Interactive behavior (B1)	1	3	1/3	1/3	2	0.164
Color transmission (B2)	1/3	1	1/3	1/2	1/3	0.080
Sensory experience (B3)	3	3	1	1/2	2	0.275
Scene experience (B4)	3	2	2	1	2	0.335
Packaging function (B5)	1/2	3	1/2	1/2	1	0.146

,

Table 5. Judgment matrix for pairwise comparison of criteria C1 to C3.

Matrix 2	Consistency of Operational Behavior (C1)	Accuracy of Information Transmission (C2)	Rationalization of the Control Module (C3)	Weight
Consistency of operational behavior (C1)	1	1/5	1/3	0.105
Accuracy of information transmission (C2)	5	1	3	0.637
Rationalization of the control module (C3)	3	1/3	1	0.258

,

Table 6. Judgment matrix for pairwise comparison of criteria C4 to C6.

Matrix 3	Rationality of Color Application (C4)	Coordination of Color and Environment (C5)	Replaceability of Color Groups (C6)	Weight
Rationality of color application (C4)	1	1/3	1/2	0.163
Coordination of color and environment (C5)	3	1	2	0.540
Replaceability of color groups (C6)	2	1/2	1	0.297

,

Table 7. Judgment matrix for pairwise comparison of criteria C7 to C11.

Matrix 4	Hearing (C7)	Vision (C8)	Touch (C9)	Smell (C10)	Taste (C11)	Weight
Hearing (C7)	1	1/6	1/3	3	2	0.131
Vision (C8)	6	1	3	3	3	0.450
Touch (C9)	3	1/3	1	3	2	0.233
Smell (C10)	1/3	1/3	1/3	1	1/2	0.073
Taste (C11)	1/2	1/3	1/2	2	1	0.114

,

Table 8. Judgment matrix for pairwise comparison of criteria C12 to C14.

Matrix 5	Adaptation of Scene and Packaging (C12)	Scene Immersion (C13)	Scene Entertainment (C14)	Weight
Adaptation of scene and packaging (C12)	1	1/3	1	0.210
Scene immersion (C13)	3	1	2	0.550
Scene entertainment (C14)	1	1/2	1	0.240

and

Table 9. Judgment matrix for pairwise comparison of criteria C15 to C17.

Matrix 6	Accuracy of Packaging Function Realization (C15)	Matching Degree of Packaging Function (C16)	Suitability of Packing Model and Function (C17)	Weight
Accuracy of packaging function realization (C15)	1	1/4	1/7	0.079
Matching degree of packaging function (C16)	4	1	1/3	0.263
The suitability of packing model and function (C17)	7	3	1	0.659

:

$$CR={\displaystyle \frac{CI}{RI}}$$

(4)

3.3.2. Relative Weight and Consistency Test

As shown in Table 4, the weights of the criterion layer are 0.164, 0.080, 0.275, 0.335, and 0.146, with the highest weights assigned to scene experience and sensory experience. These five weight values are multiplied by the corresponding indicators in the sub-criteria layer, followed by a consistency test to verify the rationality of the results. Formula (4) is employed to confirm that $CR=0.054<0.1$, demonstrating that the evaluation model passes the consistency test. Finally, the relative weight results, shown in

Table 10. Relative weights and ranking.

Indicators	Relative Weight	Ranking
Consistency of operational behavior (C1)	0.017	15
Accuracy of information transmission (C2)	0.104	3
Rationalization of the control module (C3)	0.042	9
Rationality of color application (C4)	0.013	16
Coordination of color and environment (C5)	0.043	8
Replaceability of color groups (C6)	0.024	13
Hearing (C7)	0.036	11
Vision (C8)	0.124	2
Touch (C9)	0.064	7
Smell (C10)	0.020	14
Taste (C11)	0.031	12
Adaptation of scene and packaging (C12)	0.070	6
Scene immersion (C13)	0.184	1
Scene entertainment (C14)	0.080	5
Accuracy of packaging function realization (C15)	0.012	17
Matching degree of packaging function (C16)	0.038	10
Suitability of packing model and function (C17)	0.096	4

, indicate that scene immersion (C13), vision (C8), and accuracy of information transmission (C2) are the most critical factors. The weights are 0.184, 0.124, and 0.104. The consistency of operational behavior (C1) and the rationality of color application (C4) hold relatively low importance. Among all indicators, the accuracy of packaging function realization (C15) carries the least weight and exerts the most minor influence on virtual packaging design. Additionally, the score (M) of the virtual packaging multi-sensory evaluation model is established, as shown in Equation (5). Each indicator’s score (G) is multiplied by the corresponding weight (W), and the total sum is calculated:

$$X={\displaystyle \sum _{i=1}^{17}{G}_i{W}_i}$$

(5)

3.4. VR Shopping Model Construction

3.4.1. Key Technologies

This study facilitates the interaction between the consumer and the model within the virtual reality system by scripting languages that control model animation playback, including pausing and adjusting playback speed. The hardware and software utilized in the virtual reality system are detailed in

Table 11. Key technology.

Technology	Function
3DMAX	The 3D models were created with 3ds Max software
Substance Painter	The 3D models were textured with Substance Painter
Maya	Model animation design
Unity3D	Engine and scene building
PICO Neo3	VR all-in-one
C#	Programming language

. The PICO Neo3 is an all-in-one VR headset featuring robust data processing, hardware driving capabilities, and high-definition screen resolution. It is a crucial device for users of VR systems, as illustrated in Figure 7. 3DS MAX, Substance Painter, and Maya were employed to model the virtual environment. The virtual scene was developed using Unity3D, and C# was employed for scripting to implement the VR shopping mall system. The technical roadmap is presented in Figure 8.

3.4.2. VR Shopping Model Development Practice

(1) Packaging modeling

Various methods exist for displaying goods in a VR shopping system, such as using product images to enhance display or constructing models to replicate real-world shopping mall environments, where products are often three-dimensional [40,41]. Compared to the first method, which allows for quick retrieval of goods but offers poor display quality of product packaging, the second method, which simulates natural shopping scenes, immerses consumers more effectively and provides a more tangible display of products and packaging, with more significant opportunities for interactive design. This study’s VR shopping system simulates a natural shopping environment. Bagged potato chips and bottled Coca-Cola were selected as examples of modeling and animation design. The models and textures were integrated to create the virtual representations of these two packaging products. Animation design was also required to achieve interactive dynamic effects within the VR environment. Following the tactile design concept, Maya was used to design two interactive packaging effects: the tearing of the potato chip bag and the opening of the Coke bottle cap, along with the deformation of the bottle body. Finally, static and force deformation effects were produced, as illustrated in Figure 9.

(2) Scene construction

To provide a shopping experience in the VR environment that closely mirrors real-life conditions and enhances consumer immersion, the system’s functional architecture is designed to replicate a typical shopping scene [42,43,44]. This includes entering the store, walking, maneuvering a shopping cart, browsing shelves, selecting items, and checking out. Additionally, functions are refined and augmented to meet consumer needs better, ensuring a realistic shopping experience. To address the challenge of calculating the total price when consumers select numerous items, a sub-function for electronic billing has been integrated into the VR shopping system, aiding consumers in managing their expenses. The VR shopping system is modeled after a small store. The prepared model is imported into Unity3D for integration and placement to create the scene depicted in Figure 10. The scene’s layout is based on actual store operations and consumer behavior within the VR shopping system. The store’s space is kept compact to avoid overwhelming users and prevent potential discomfort, such as dizziness. The layout includes three shelves, refrigerators, and a checkout counter.

(3) Packaging multi-sensory design solutions

In a virtual reality (VR) environment, packaging is represented as a model within the simulated space. The design of the multi-sensory experience for packaging in this virtual context aims to convey visual, auditory, and tactile sensations through alterations in the model’s shape, color, sound effects, and light intensity [45,46]. Effective simulation of olfactory and gustatory experiences in virtual reality remains unresolved. Therefore, the focus is on the visual, auditory, and tactile aspects of packaging and its materials. The tactile experience in VR primarily relies on visual and auditory cues to create a psychological sense of touch. The movement of the hand model in the virtual space is controlled by the VR headset’s controller, with distinct actions mapped to different buttons. This setup allows the virtual hand model to replicate real-world hand movements. Design considerations include aspects such as the opening process of the packaging, the distance and duration of tearing, the associated sound effects, collisions, and stress-induced deformation of the packaging, all of which contribute to a simulated tactile experience. Visual experiences are enhanced by meticulously replicating the packaging model and store environment, ensuring accurate color matching to mimic real-world appearances. This provides consumers with a realistic visual representation of the packaging and the store setting. Auditory experiences are achieved through environmental sound effects within the VR shopping system and interactive sound effects associated with the packaging model.

(4) Interaction design

Interaction design is pivotal in a VR shopping system [47,48]. As virtual reality technology immerses users directly into a virtual environment, it is crucial to screen and classify all models within this space, replicating real-world operational conditions and habits. Interaction design underpins the various actions consumers can perform in the VR store using VR devices. By employing different interaction designs, consumers can experience a range of sensory inputs within the virtual environment. Based on the initial conceptual framework, the system’s functionalities and sensory experiences are integrated into the interaction design module. The system utilizes a two-hand model for object selection and manipulation to enhance the realism of interactions within the virtual scene. This model controls two states—grasping and releasing—through the trigger button on the controller, enabling consumers to pick up and put down items. The system adds impact and gravity effects to models, such as the Kopic potato chips and Coca-Cola bottles. When a product is picked up, gravity is simulated, causing the product to fall when released. If the product collides with other items, the collision is visually represented.

Additionally, to assist consumers in product selection, the system features a product information display panel that appears when the trigger button is engaged. The package can be opened by holding down the button, accompanied by the sound of the package opening. This design provides a comprehensive sensory experience, integrating visual, auditory, and tactile feedback. The effects are illustrated in Figure 11.

(5) Purchase stage

The purchase phase function encompasses the shopping cart: after selecting items, consumers can add them to the shopping cart. During the checkout stage, consumers must proceed to the cashier. Clicking on the cashier will display a list of items, including their names and prices. After reviewing the list, consumers click “confirm payment” to complete the transaction. The effect diagram is presented in Figure 12.

4. Results and Discussion

4.1. VR Shopping Model Evaluation

The functionalities of the VR shopping system have been fully implemented, as detailed in the previous section. However, the sensory experience and consumer satisfaction require validation through practical application. The virtual packaging multi-sensory evaluation model is intended to assess and enhance the VR shopping system. The experiment involved 25 participants, both men and women aged 18–30, who had previously participated in the eye-tracking experiment described in Section 3.2.1. The VR shopping system was packaged into an installation file and deployed on the PICO Neo3 all-in-one device for testing. Test participants were instructed to wear the device and were trained in the operational procedures.

After completing the training, participants entered the VR shopping system. They engaged in activities typical of their shopping habits, such as walking, browsing products, viewing product information, picking up items, opening packaging, adding items to the shopping cart, and completing the checkout process. The test PICO Neo3 VR all-in-one duration was unrestricted. Upon task completion, participants evaluated the system using the virtual packaging multi-sensory evaluation model. The evaluation scale ranged from 1 to 10 points, with a score of 7 or higher considered passing or satisfactory. The evaluation scores for each indicator from all 25 participants were summed and then averaged to produce the results shown in

Table 12. Virtual packaging multi-sensory evaluation model score.

Criteria Layer	Sub-Criteria Layer	Score
Interactive behavior (B1)	Consistency of operational behavior (C1)	6
	Accuracy of information transmission (C2)	5
	Rationalization of the control module (C3)	6
Color transmission (B2)	Rationality of color application (C4)	7
	Coordination of color and environment (C5)	8
	Replaceability of color groups (C6)	7
Sensory experience (B3)	Hearing (C7)	7
	Vision (C8)	7
	Touch (C9)	5
	Smell (C10)	3
	Taste (C11)	3
Scene experience (B4)	Adaptation of scene and packaging (C12)	9
	Scene immersion (C13)	8
	Scene entertainment (C14)	7
Packaging function (B5)	Accuracy of packaging function realization (C15)	5
	Matching degree of packaging function (C16)	7
	Packaging model and workmanship (C17)	7
Total score		6.68

. The total score was subsequently calculated using Equation (5):

$$X_{\mathrm{VR} \mathrm{Shopping} \mathrm{System}}={\displaystyle \sum _{i=1}^{17}{G}_i{W}_i=6.68}$$

Table 12 presents the evaluation results. The VR shopping system received a total score of 6.68 points, indicating a generally low evaluation of the consumers’ sensory experience. Among the components receiving scores below 6 were the consistency of operation behavior, accuracy of information transmission, and rationality of the control module in interactive behavior, each scoring a maximum of six points. The overall score remains low. The effectiveness of the sensory experiences related to touch, smell, and taste was suboptimal. Given that the system did not incorporate the sensory experiences of smell and taste, the tactile experience was the primary focus. Additionally, the accuracy of packaging function realization requires improvement.

4.2. VR Shopping Model Improvement

4.2.1. Improvement Program

To enhance consumers’ sensory experience with the improved system, it is crucial to analyze the factors affecting their experience and contributing to low ratings before making improvements. The evaluation results indicate that interaction design, haptics, and the accuracy of packaging function realization require improvement. In the VR shopping system, interaction design encompasses three main aspects: basic walking, packaging interaction, and checkout. Haptic feedback primarily involves the opening and deformation of packaging, while the accuracy of packaging function realization pertains to the animation design of the packaging model. Analysis of the three areas for improvement reveals that their common realization path involves the interaction between consumers and product packaging. In the virtual environment, packaging exists in only two states: open and closed. When the packaging is opened by pressing a button to trigger the function, the animation plays directly without simulating the force application process observed in reality. This discrepancy from real-world interactions results in a suboptimal sensory experience for consumers.

In the virtual environment of the VR shopping system, the states of package opening and closing are controlled by animation. To allow consumers to experience the process of applying force when opening the package, analysis indicates that the virtual hand gradually increases force as the participant pulls the package open over time. The distance the hands pull outward approximately correlates with the magnitude of the applied force. As the numerical relationship between the hand-pulling distance and the applied force is not predefined, self-defined hand distances for opening the package (MaxDis) were established for potato chips and cola, as shown in Figure 13, to simulate the force application process in the virtual space. After iterative adjustments to MaxDis, the optimal hand movement distances were determined as follows: 20 cm for opening the cola bottle cap and 30 cm for opening the potato chip package.

4.2.2. Improved Validation

The experiment involved 25 participants, aged 18–30, who had previously participated in the eye-tracking experiment detailed in Section 3.2.1. The design and implementation environment were aligned with the previous experimental setup to ensure consistency. To verify the feasibility of integrating virtual reality technology with packaging using the virtual packaging multi-sensory evaluation model, participants evaluated the enhanced VR shopping system to determine its effectiveness in improving consumers’ sensory experiences. All 25 participants submitted valid responses. According to

Table 13. Virtual packaging multi-sensory evaluation model score.

Criteria Layer	Sub-Criteria Layer	Score
Interactive behavior (B1)	Consistency of operational behavior (C1)	7
	Accuracy of information transmission (C2)	6
	Rationalization of the control module (C3)	7
Color transmission (B2)	Rationality of color application (C4)	7
	Coordination of color and environment (C5)	8
	Replaceability of color groups (C6)	7
Sensory experience (B3)	Hearing (C7)	8
	Vision (C8)	9
	Touch (C9)	7
	Smell (C10)	3
	Taste (C11)	3
Scene experience (B4)	Adaptation of scene and packaging (C12)	9
	Scene immersion (C13)	8
	Scene entertainment (C14)	7
Packaging function (B5)	Accuracy of packaging function realization (C15)	8
	Matching degree of packaging function (C16)	7
	Packaging model and workmanship (C17)	7
Total score		7.36

, the improved VR shopping system received a score of 7.36, indicating a notable improvement in consumers’ sensory experiences. Improvements were noted in operational behavior consistency, information transmission accuracy, control module rationality, tactile feedback, and packaging function accuracy, all of which had previously contributed to poor sensory experiences. Visual and auditory sensory experiences also improved. Participants’ feedback confirms that the virtual packaging multi-sensory evaluation model effectively guides the application of virtual reality technology in packaging and enhances consumers’ sensory experiences:

$$X_{\mathrm{improved}}={\displaystyle \sum _{i=1}^{17}{G}_i{W}_i=7.36}$$

4.2.3. Statistical Analysis

Figure 14 illustrates that the scores for most participants increased following the improvements, indicating a positive impact of the enhancements made to the VR model. Notably, the most significant improvements were observed in vision (C8) and accuracy of packaging function realization (C15). Additionally, enhancements were recorded in the consistency of operational behavior (C1), the accuracy of information transmission (C2), rationalization of the control module (C3), hearing (C7), and touch (C9). However, the scores for smell (C10) and taste (C11) remained the lowest, with both receiving a score of only 3.

Smell and taste represented significant limitations in our research. While focusing on visual, auditory, and tactile experiences addressed the core aspects of sensory perception, it did not encompass the full spectrum of multi-sensory interactions. Incorporating smell and taste could offer a more nuanced and comprehensive understanding of user experience. The primary challenge in integrating these senses lies in the technical and logistical difficulties associated with accurately simulating and measuring these sensory inputs within a controlled experimental setting. Future research should aim to develop methodologies for effectively incorporating all five senses, thereby enhancing the comprehensiveness and efficacy of sensory research.

The results indicate that VR shopping systems hold substantial promise for enhancing marketing and shopping experiences but also present significant ethical challenges. On the one hand, virtual reality systems often gather extensive data regarding user behavior, preferences, and interactions within the virtual environment. These data may encompass sensitive information that, if inadequately managed, could result in privacy breaches or unauthorized use. On the other hand, the integration of VR technology into the shopping experience can markedly alter consumer behavior. While such changes may enrich the shopping experience, they also raise concerns about immersive technology’s long-term psychological and social effects. Prolonged exposure to virtual environments may impact users’ perception of reality, consumption patterns, and social interactions. To address these issues responsibly, it is crucial to tackle privacy concerns, assess the impact on consumer behavior, ensure accessibility, and secure informed consent.

5. Conclusions

The presentation of multi-sensory packaging and packaging materials within a virtual environment must address not only the functional implementation of system software but also emphasize the sensory experience of users. In this context, this study addresses the challenge of evaluating multi-sensory design in virtual environments by developing a comprehensive multi-sensory evaluation system and model for virtual packaging. The goal is to provide engineers with effective multi-sensory design strategies for packaging in virtual environments. A VR shopping system was constructed using potato chips and cola as case studies to validate the evaluation model. The findings and verification of the model are summarized below.

(1) This study presents a multi-sensory evaluation model for virtual packaging, developed using the analytic hierarchy process (AHP) to support the multi-sensory packaging design within virtual environments. The model is structured into five criteria layers: interactive behavior, color transmission, scene experience, sensory experience, and packaging function, with 17 indicators across these layers. Each indicator is quantified, and weights are computed using the nine-point scale method and expert judgment. This model offers engineers an effective decision-making tool for determining how packaging should be represented in virtual environments. However, certain limitations of AHP must be acknowledged. A significant issue is the rank reversal phenomenon, where the addition or removal of alternatives can alter the ranking of remaining options, potentially leading to inconsistent decisions. Furthermore, AHP assumes that the evaluation criteria are independent, yet in practice these criteria may be interdependent, influencing the overall decision-making process. Neglecting this interdependence can lead to biased or inaccurate results.

(2) In the Internet age, the influence of packaging on consumers’ sensory experiences within online shopping environments is of considerable importance. This study employs virtual reality (VR) technology to develop a VR shopping system, focusing on the display and interaction of packaging within this system. The evaluation results of the model utilized have led to significant improvements in the VR shopping system. These results demonstrate that the evaluation model proposed in this study effectively guides the multi-sensory packaging design in virtual environments. Furthermore, future research should focus on VR shopping systems’ long-term impact and consumer engagement to ensure sustained success. Investigating how consumer satisfaction and ease of use evolve over time will be crucial for identifying issues related to system fatigue or diminished novelty, thereby enhancing the VR system.

In the online shopping environment, the multi-sensory evaluation of product packaging based on VR technology is a multi-criterion decision problem, and the sensory experience is the subjective feeling of consumers. Moreover, there is no one-to-one corresponding relationship with the functional realization of packaging in the virtual environment, but through the combination of two or more packaging colors, interaction, environmental sound effects, etc., it provides consumers with different sensory experience. Based on this situation, this study proposed a virtual packaging multi-sensory evaluation model. Although we have studied the packaging sensory expression of the shopping environment in the VR shopping system, only visual, auditory, and tactile sensory experiences are realized in the system, and there is a single type of commodity packaging. How can the smell and taste experience that packaging brings to consumers in the virtual environment be realized? It is of great research value to explore ways of smelling and tasting in more types of packaging in VR shopping systems in future studies. In the Internet era, the technology iteration speed is fast, and this study only focuses on virtual reality technology for packaging multi-sensory research. In the future, more cutting-edge and emerging technologies can be applied to packaging multi-sensory research, and the evaluation model can be improved to make the model more suitable for packaging in the multi-sensory field.