Next Article in Journal
Extraction of Bioactive Compounds from Oxheart Tomato Pomace (Lycopersicon esculentum L.) Using Different Solvents: Characterization of Extracts
Previous Article in Journal
Field Evaluation and Application of Intelligent Quality Control Systems
Previous Article in Special Issue
Construction and Ranking of Usability Indicators for Medical Websites Based on Website User Experience
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Applied Sciences
Volume 14
Issue 16
10.3390/app14167141
applsci-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Comparison of Tutorial Methods in Virtual Reality Games for a Better User Experience

by
Yuryeon Lee
1,
Guyeop Kim
2,
Kang Hoon Lee
3,
Jaehyun Park
4 and
Hyun K. Kim
1,2,*
1
Department of Artificial Intelligence Convergence, Kwangwoon University, Seoul 01897, Republic of Korea
2
School of Information Convergence, Kwangwoon University, Seoul 01897, Republic of Korea
3
School of Software, Kwangwoon University, Seoul 01897, Republic of Korea
4
Department of Industrial and Management Engineering, Incheon National University (INU), Incheon 22012, Republic of Korea
*
Author to whom correspondence should be addressed.
Appl. Sci. 2024, 14(16), 7141; https://doi.org/10.3390/app14167141
Submission received: 12 July 2024 / Revised: 29 July 2024 / Accepted: 13 August 2024 / Published: 14 August 2024
(This article belongs to the Special Issue Human-Computer Interaction: Challenges, Opportunities and Emerging Developments)
Browse Figures
Versions Notes

Abstract

:
The commercialisation of virtual reality (VR) headsets has made them more affordable and popular in gaming and entertainment. The natural interaction between the VR environment and users can maximise immersion and is crucial to VR gaming. Despite their growing popularity, educational VR games prioritise learning over immersion and require users to learn to interact with and play games using tutorials. Herein, we developed a game named Numverse with an accompanying tutorial. After selecting the tutorial content, we programmed the user interface and proposed a delivery method for the tutorial. We evaluated the user experience based on the effects of the presence or absence of the tutorial and its mode of delivery. The tutorials were of three types: no tutorial, instruction-screen tutorial, and context-sensitive tutorial, with the latter being the most preferred. The evaluation results show that presence, ability to learn controls, intrinsic motivation, and learning effectiveness are higher for the instruction-screen and context-sensitive tutorials than for no tutorial. On average, users experienced more motion sickness in the no-tutorial case, with a significant difference in nausea items. This study asserts the importance of tutorials in VR games, and its findings could improve user experience in future VR games.

1. Introduction

Virtual reality (VR) is being rapidly adopted in fields such as tourism, medical care, and education [1,2,3,4]. Head-mounted display (HMD)-based VR games have become popular [5,6] owing to the affordable headsets [7,8] and the popularity of VR. The natural interaction between VR and its user is an essential element of immersive VR [9,10]. Considering that this maximises user immersion, the prospects for the development of VR for gaming are high. VR games can be used for various purposes, including education, training, treatment, and collaboration [11,12]. In education, VR games are gaining popularity as they are proving to be useful tools for learning and storytelling [10,13,14]. VR games are being used in various educational fields, such as science, technology, engineering, and mathematics [8,15].
An educational game provides users with both entertainment and learning experiences [16]. However, these games are not immersive, as they prioritise learning and provide ‘fun’ elements in a formal manner [17]. In VR games, where players use a dedicated controller to explore the game and interact with objects [18,19], the gameplay may be unfamiliar to players new to VR games [6]. Therefore, tutorials should provide help and information on interaction and game progression [5]. The provision of tutorials by game designers is a crucial element of these games [6]. Tutorials should be effective because users go through them at the start of the game [20] and thus learn how to play the game and interact with in-game elements [6,21]. The help provided to the user should be cognitively comfortable and a stress reliever, which can improve learning outcomes for educational purposes and the overall VR game experience [15]. A study revealed a positive effect on users when tutorials are offered in VR games [5,22]. However, the literature lacks published research on how to effectively train users to increase their participation in virtual environments [2,20] and on the effects of education tutorials.
A study divided tutorials into ‘instruction-screen’ and ‘context-sensitive tutorials’ [5]. An instruction-screen tutorial is a traditional form of tutorial with fixed positions and directions for the tutorial [5], whereas a context-sensitive tutorial provides information with a speech bubble when the player requests new information, which is fixed onto the controller [5]. Compared with instruction-screen tutorials, context-sensitive tutorials provide a more positive player experience, which may be attributed to the lack of or limited experience of users with playing VR games [5]. Additionally, the study emphasised that the effects of various levels of context sensitivity should be reviewed in the future. Therefore, we recruited participants primarily from a group of users with considerable VR experience and further expanded the scope of research related to tutorial comparison using directly developed content.
This study aims to determine user experience and VR motion sickness concerning the presence or absence of tutorials and delivery methods in a VR-based learning environment. Studying the effects of various tutorial methods on users can aid in designing effective VR game tutorials, leading to enhanced learning outcomes [20]. This highlights the importance of analysing the tutorial method for VR games and researching the user experience concerning the tutorial method. Herein, we develop a VR educational game named ‘Numverse’. For this game, we design and develop tutorials based on the in-game tasks. Adopting [5]’s approach, we divide the tutorials into instruction-screen and context-sensitive tutorials by their display time. An instruction-screen tutorial is displayed before the game is played, assuming that the player will progress and learn the game more exploratively than with the context-sensitive tutorial. Conversely, we provide a context-sensitive tutorial according to the task of the user during the game, enabling the player to play and learn the game in a specific manner. To understand the user experience, we prepared a VR engagement questionnaire (VREQ) for evaluation. Conversely, we use a simulator sickness questionnaire (SSQ) to verify the effect of tutorials on VR motion sickness. We formulate the following hypotheses:
H1. 
The user experience in VR-based games differs depending on the presence or absence of a tutorial.
H2. 
The user experience in VR-based games differs with the manner of displaying the tutorial.
H3. 
The degree of motion sickness experienced during VR-based games differs with the manner of displaying the tutorial.

2. Literature Review

2.1. Virtual Reality (VR) Interaction

Interaction is a series of communications between a VR environment or object and the user via an input/output device. It includes manipulation of virtual objects or browsing of the virtual environment [23]. A study utilised the NASA-TLX and system usability scale to assess workload and usability for emergency medical VR training applications and examine the users’ interactions with the application [24]. Ref. [25] designed and evaluated interaction techniques in VR vocational rehabilitation systems for people with autism spectrum disorder. Interactions for object selection and manipulation in the system were implemented using tangible object manipulation, haptic device interaction, touch and snap technique, and touchscreen interaction. Movement-related interactions were implemented using real walking and in-place walking [25]. To promote a positive player experience with VR games, another study analysed motion-controller interaction methods and proposed effective player interaction methods in first-person shooter games [26].

2.2. VR Tutorial

Kao et al. [20] compared three VR game tutorial design types: text, text + diagrams, and text + spatial. For text + diagrams, the controller diagram and text were displayed in front of the field of view (FOV) of the user, whereas in the text + spatial method, a tooltip was provided on the virtual controller of the player and text was displayed in front of the FOV. In third-person shooter games, the text+spatial method demonstrated the highest control learnability and led to higher performance, player experience, and intrinsic motivation compared with those resulting from the text method. In puzzle games, the text + spatial method exhibited higher control learnability and performance compared with the text method, whereas the text + diagram method had higher control learnability compared with the text method [20]. Similarly, in another study, VR wave shooter games were implemented with instruction-screen and context-sensitive tutorials, and the player’s experience concerning the tutorial method was evaluated [5]. The results from the usability evaluation demonstrated that the context-sensitive tutorial method led to higher positive emotions and motivation and lower negative emotions than those resulting from the instruction-screen tutorial method [5]. Ros et al. [27] developed immersive tutorials in a 3D video format and conducted usability evaluations for medical school students. The groups that used immersive tutorials achieved more positive learning outcomes than those that only read technical notes. Another study evaluated the learnability, effectiveness, and satisfaction of VR tutorials for users classified into beginner and expert groups. The beginner group demonstrated a relatively high workload compared with that of the expert group; however, learnability, effectiveness, and satisfaction were all high for both user groups [28].

2.3. VR Educational Game Content

VR educational games have been developed and researched with various types of content. Akman and Çakır [29] found that educational VR games fractionally increased academic achievement among elementary school students and positively affected their immersion and participation in mathematics. Abuhammad et al. [9] developed a VR educational game prototype (MedChemVR) that could help students learn medical chemistry (MC) subjects. Subsequently, a usability evaluation was conducted with pharmacy graduate school students, and subjective feedback regarding the advantages and disadvantages of the application was received. Participants mentioned the need for VR technology, and a common consensus was that it could increase fun or participation. Another study discovered that laboratory safety training through educational VR games was efficient and effective. Immersive VR simulation, desktop VR simulation, and existing manuals were compared, which revealed that immersive VR simulation was preferred to existing manuals with a statistically significant difference [30]. Another study developed the design review simulator (DRS), a VR education simulation game that evaluates and reviews residential buildings using the educational design framework A.D.D.I.E (Analysis, Design, Development, Implementation, and Evaluation). Thus, a new direction for future VR game development and design was presented [31]. Jansen and Fischbach [32] developed The Social Engineer, an immersive VR education game, to improve social engineering awareness among company employees. The Social Engineer is an immersive training tool that teaches social engineering awareness to enable and train employees to identify security vulnerabilities within the company.

2.4. Simulator Sickness Questionnaire

A typical side effect of using VR devices is motion sickness, which negatively affects the overall user experience of VR devices [33]. Kennedy et al. [34] noted that the motion sickness questionnaire (MSQ) is not a valid measure of simulator motion sickness because symptoms such as drowsiness have been seldom observed in studies related to simulator sickness. Therefore, some of the MSQ items were proposed as SSQ, which includes three categories: nausea (N), oculomotor (O), and disorientation (D) [34]. SSQ is used to evaluate simulator motion sickness. Recently, it has been employed in HMD-based virtual environments [33]. The formula for calculating the SSQ score is as follows:
[Nausea(N) = [A]*9.54, Oculomotor(O) = [B]*7.58, Disorientation(D) = [C]*13.92, Total = ([A] + [B] + [C])*3.74]

3. Tutorial Development

This study designed a tutorial for the VR educational game Numverse, which was newly developed in this study. The theme of the game is to learn the concept of quadratic arithmetic using numerical beads (Figure 1).
The tutorial was developed in the following order: selecting the content to be included in the tutorial (Section 3.1), designing a user interface (UI) that effectively displays the content (Section 3.2), and proposing the manner of displaying the tutorial (Figure 2).

3.1. Content Selection for Tutorials

The tutorial content in VR-based educational games informs the user of the interaction method using VR devices and the rules of the game. Therefore, deciding the main user tasks during gameplay and displaying to the user the components of each task is crucial. The tutorial in this study consisted of seven parts: (1) how to use the controller, which showed the types and functions of the buttons on the controller of the VR device (Figure 3a); (2) the player motion method, which introduced the method of manoeuvring the playable character using the controller (Figure 3b); and (3)–(7) user tasks required to play the game and to learn the concept of quadratic operations using numerical beads (Figure 3c–g). These tasks included grabbing objects, combining objects, lowering conversion devices, throwing numerical beads, and throwing fruits.

3.2. Tutorial Design Progress

The tutorial was designed considering (1) the design components that display the content, (2) the size and location of the tutorial interface, and (3) the colour and font size of the content. First, three games (AltSpace VR, First Steps, and First Contact), selected based on their popularity and provision of tutorials, were examined to analyse the design components that showed the tutorial content. Moreover, the documents mentioned in Section 2.2 were reviewed. The analysis revealed that the design components of tutorials were primarily of three types: text, controller image, and motion image (Figure 4). Therefore, the tutorial design included text to explain the gameplay, a controller image for interaction with the game, and a motion image informing the user of the gameplay by explaining the relevant user behaviours (Figure 5).
Subsequently, related papers were reviewed to determine the size and location of the tutorial interface. The FOV is an element that must be considered in the design of a VR interface because it defines the range in which the user can turn their head comfortably [35]. A typical user can comfortably turn their head from approximately −30° to +30° left and right and from approximately −12° to +20° up and down [35,36]. Alger [37] redefined the range proposed by Chu by applying it to VR. The main content range, which is the comfortable range of content from the centre, was derived to be a viewing angle of approximately −77° to +77° left and right and approximately −12° to +20° up and down, respectively [36,37]. Purwar [38] proposed that, on a 3600 × 1800 pixel-sized canvas, a 1200 × 600 pixel-sized user interface, which is equivalent to one-ninth of the canvas, should be placed in the centre. Therefore, in this study, the tutorial interface was designed to have a size of 1200 × 600 pixels.
Finally, related studies were reviewed to determine the colour and font size of the content. Maguire [39] applied the design principles to VR. Specifically, VR interfaces should use large interface elements to facilitate interaction with the system and avoid using large text blocks. Additionally, numerous studies have established effective interface design guidelines for learning [40]: for text, consistent font, colours, and short paragraphs should be used, and uppercase text should be avoided. Text and diagrams should also be properly integrated, applied, and contrasted with the background [40]. Additionally, colours should be consistently used throughout the page, and for long sentences, light colours should be avoided [40]. Therefore, in this study, the tutorial was designed with black, white, and blue colours and employed a font size of 15 points or more. The colour contrast between the background and the text was designed to be 11:1 or higher, and the text was typed primarily in lowercase. The complete tutorial design is illustrated herein (Figure 3).

3.3. Tutorial Method Proposal

Two main prototypes were developed, depending on how the tutorials were displayed on the screen. The instruction-screen tutorial method provides a tutorial only at startup (Figure 6). The location and direction of the tutorial were fixed, and the user could flip the seven tutorial pages shown in Figure 3 using the controller. By contrast, the context-sensitive tutorial method provided tutorials according to the user’s task in the game, and the X button allowed the user to check the tutorial at any time (Figure 7). The design of this method was similar to that depicted in Figure 3, with the difference that the tutorial was seen around the object without covering the object on the screen.

4. Usability Evaluation

4.1. Participants

The experiment was conducted with 42 men and women (23 men and 19 women) (mean age = 22.7, SD = 1.9) in their 20s, with a focus on individuals who typically enjoy playing games. Approximately 60% of the participants had VR game experience, whereas the remaining participants had no VR game experience. Approximately 80% of the participants had experience with controller-based games, whereas the remaining had no experience with controller-based games. The participants were recruited online and paid approximately USD 15 toward experiment expenses. This study was conducted with the approval of the IRB (7001546-20211125-HR(SB)-011-01) of Kwangwoon University.

4.2. Experimental Environment and Equipment

The experiment was conducted using an HMD (Oculus Quest 2) in a university laboratory environment (Figure 8). One of the two experimental hosts observed the progress of each subject with a monitor, and the other assisted the subjects in participating properly in the experiment. The resolution of Oculus Quest 2 was 1832 × 1920 pixels.

4.3. Experimental Procedure

The participants were required to fill out personal information, including their name, sex, age, and consent forms for the experiment, and were guided regarding the purpose of the experiment and the experimental procedure. Additionally, before the experiment, basic controller usage was taught for the benefit of participants who were not familiar with the controls.
The experiment was conducted based on a within-subject design, and the 42 participants were divided into two groups according to the order of tutorial provision. Figure 9 indicates the procedure for usability testing. One group first engaged in the experiment with the no-tutorial method, i.e., no in-game tutorials were provided to them, followed by the instruction-screen tutorial method (Figure 6) and context-sensitive tutorial method (Figure 7). The other group also first engaged in the experiment according to the no-tutorial method; however, in contrast to the sequence for the first group, it was followed by the context-sensitive tutorial method and then by the instruction-screen tutorial method. When performing according to each condition, the participants were asked to complete all seven user tasks. A survey was conducted after each experiment per condition, and interviews were conducted after all experiments for all conditions had been completed.

4.4. Evaluation Factors

The VREQ was developed based on a literature review focusing on keywords such as VR games, presence, learning, and enjoyment (Table 1). It comprised four major categories: presence, control learnability, intrinsic motivation, and perceived learning effectiveness. The items judged to be suitable for this study were selected based on previous studies. For presence, the items outlined by [41], consisting of spatial presence, experienced realism, and immersion, were used. For control learnability, the items outlined by [20], consisting of mental load, mental effort, and learnability, were employed. Intrinsic motivation was reconstructed based on items from [42,43,44] and consisted of enjoyment, continual intention to use, and player experience.
For perceived learning effectiveness, the items provided by [45] were used, and sub-items did not exist. Table 1 lists the items and sample questions from the VREQ. They rated the VREQ on a scale from 0 (totally disagree) to 10 (totally agree).
For the SSQ, the items from the [34] study were employed, and the score was calculated using the SSQ calculation method. They rated the SSQ on a scale from 0 (none) to 3 (severe).
Table 1. Virtual reality engagement questionnaire (VREQ) factors and items.
Table 1. Virtual reality engagement questionnaire (VREQ) factors and items.
FactorItemReference
PresenceSpatial PresenceI felt present in the virtual space.[41]
Experienced RealismHow much of your experience in the virtual environment seemed consistent with your real-world experience?[41]
InvolvementI was completely captivated by the virtual world.[41]
MappingThe method of interacting (playing) in a game environment was the same as the method used in real life. [41]
Controls’ LearnabilityMental LoadThe game’s controls were difficult to learn for me.[46,47]
Mental EffortI needed to put lots of mental effort into learning the game’s controls.[46,47]
LearnabilityWhen I wanted to do something in the game, it was easy to remember the corresponding control. [48]
Intrinsic MotivationEnjoymentI was very interested in VR games.[42]
Continual Intention to UseI wanted to play VR games longer without stopping.[43]
Player ExperienceThe realism of VR helps enhance my understanding. [7]
Perceived Learning EffectivenessI learned a lot of factual information from the VR game.[45]

4.5. Analysis Techniques

A one-way analysis of variance (ANOVA) was performed to analyse the differences in user experience and motion sickness between the no-tutorial, instruction-screen tutorial, and context-sensitive tutorial methods. Post hoc analysis was conducted using the Student–Newman–Keuls (SNK) technique on the items, and subsequently, significant differences were identified. The application results were analysed using the R-based jamovi software ver 2.2.3.

5. Results

5.1. VREQ

The results of the VREQ are shown in Table 2 and Figure 10. In terms of presence, the average scores for the no-tutorial, instruction-screen tutorial, and context-sensitive tutorial cases were 5.2 points (SD = 2.5), 6.4 points (SD = 2.2), and 6.7 points (SD = 2.3), respectively. For control learnability, the average scores for the three cases (in the same order) were 4.6 points (SD = 2.6), 6.7 points (SD = 2.2), and 7.5 points (SD = 2.1), respectively. For intrinsic motivation, the average scores were 5.2 points (SD = 2.4), 7.3 points (SD = 2.1), and 7.5 points (SD = 2.1), respectively. For perceived learning effectiveness, the average scores were 5 points (SD = 2.6), 6.9 points (SD = 2.3), and 7 points (SD = 2.3), respectively. A one-way ANOVA revealed statistically significant differences in the presence, control learnability, intrinsic motivation, and perceived learning effectiveness (p < 0.001, α = 0.05). The homogeneity of variance assumption was accepted; thus, the ANOVA results were reliable. Moreover, SNK post-analysis revealed differences between the no-tutorial and instruction-screen methods and between the no-tutorial and context-sensitive tutorial methods. The experimental conditions did not have a statistically significant effect on involvement (p = 0.176, α = 0.05), whereas statistically significant differences were identified in the remaining subitems (p < 0.05, α = 0.05).

5.2. SSQ

The results of the SSQ are illustrated in Figure 11. For nausea, the average scores for the no-tutorial, instruction-screen tutorial, and context-sensitive tutorial cases were 20.4 points (SD = 0.64), 8.4 points (SD = 0.37), and 8.2 points (SD = 0.38), respectively. For oculomotor, the average scores (in the same order) were 33.6 points (SD = 0.89), 19.9 points (SD = 0.65), and 21.5 points (SD = 0.71), respectively. For disorientation, the average scores were 46.7 points (SD = 0.67), 27.8 points (SD = 0.49), and 28.5 points (SD = 0.42), respectively. Overall, the average score for the no-tutorial case was higher than those for the instruction-screen and context-sensitive tutorials. The homogeneity of variance assumption was accepted. One-way ANOVA revealed no statistically significant differences for oculomotor or disorientation (oculomotor: p = 0.133, disorientation: p = 0.177, α = 0.05); however, it revealed a statistically significant difference for nausea (p = 0.006, α = 0.05). Moreover, SNK post hoc analysis revealed differences between the no-tutorial and context-sensitive tutorial cases and between the no-tutorial and instruction-screen tutorial cases for nausea.

6. Discussion

6.1. User Experience and Motion Sickness

Presence, control learnability, intrinsic motivation, and perceived learning effectiveness for instruction-screen and context-sensitive tutorials were higher compared with those for the no-tutorial case; they exhibited statistically significant differences. Therefore, Hypothesis 1 is supported, with the conclusion that users can more easily learn the game and controls and enjoy the game more with the instruction-screen tutorial or context-sensitive tutorial method than with the no-tutorial method. However, no statistically significant differences were observed between the instruction-screen and context-sensitive tutorials in terms of presence, control learnability, intrinsic motivation, or perceived learning effectiveness. Therefore, Hypothesis 2 is not supported. As the preferred tutorial method differs according to the personal inclination of the user, the statistical difference between the instruction-screen tutorial and context-sensitive tutorial methods was insignificant. These results are similar to those of previous studies. For example, Andersen et al. [2] compared the difference in experience between the two tutorial methods for a complex game, and Frommel et al. [5] compared this difference for a simple game. By contrast, this study compared differences in experience with educational games. Therefore, we confirmed that, for any type of game, a well-structured tutorial leads to a positive user experience regardless of the tutorial method. Regarding motion sickness, only the nausea item had a higher score for the instruction-screen tutorial and context-sensitive tutorial methods than for the no-tutorial method, which was a statistically significant difference. Therefore, Hypothesis 3 is partially supported. This suggests that, if a tutorial is not provided, the users may be burdened or tense while playing the game, and that such feelings might cause motion sickness. Moreover, no statistically significant differences were observed between tutorial methods for motion sickness.

6.2. Analysis of Qualitative Interview Results According to the Tutorial Method

The interview revealed that the no-tutorial method had the lowest user preference, whereas the instruction-screen tutorial method had the highest. Nonetheless, irrespective of the experimental conditions, most participants responded that they could play the game sufficiently without experiencing motion sickness. Regarding the no-tutorial method, numerous opinions of ‘it was difficult to understand the controller intuitively’ and ‘it was difficult to immerse myself in the game because I felt like I was alone in the game’ were noted. Conversely, regarding the instruction-screen and context-sensitive tutorials, numerous opinions stating that they were ‘helpful in understanding the game method’ were noted. Regarding the instruction-screen tutorial, an opinion stating ‘it was relatively better than no tutorial because it explained the game method’ was noted, alongside ‘it was difficult to remember the explanation provided at the start of the game while playing the game’ and ‘because it is provided at the beginning of the game, the game can be explored more, making it more immersive and interesting’. Regarding the context-sensitive tutorial, opinions that ‘it was the best way to learn how to use a game controller, but it was uncomfortable to see because it was provided around designated objects in the game’ and ‘because tutorials are provided for each user task, the process was well understood and was able to proceed sequentially’ were noted. The consensus was that the context-sensitive tutorial was more convenient in terms of explaining the controls, whereas the instruction-screen tutorial was more efficient in explaining the game process. Some users suggested using videos or simulations when providing tutorials and responded that they needed feedback on the buttons they pressed.
In summary, providing a tutorial is crucial to preventing motion sickness or improving the user experience, even in a simple game. Regarding context-sensitive tutorials, they help users to understand the detailed game progress method at each step; therefore, for an exploration game, an instruction-screen tutorial, which informs only on the progress method, is recommended. Furthermore, for context-sensitive tutorials, the backgrounds or objects in the game must be displayed in positions where they are not covered, whereas for instruction-screen tutorials, they must be placed in the centre of the screen based on the viewing angle. As both tutorial methods provide a positive user experience, they are both recommended.

6.3. Limitations

This study has several limitations. First, because all participants played the game without a tutorial first, they may have already learned part of the game by the time they were shown the instruction-screen and context-sensitive tutorials. However, if a user is not shown the tutorial after an introduction-screen tutorial or context-sensitive tutorial, evaluating the user experience for the no-tutorial method is challenging because the evaluation is conducted after the VR game mechanics are already learned. Therefore, in this study, the experiment was conducted in the order shown in Figure 9. Second, more diverse age groups could not be considered, and the age of the participants was limited to the 20s because the content of the game was quadratic arithmetic training, which may be more appropriate for teenage users. However, concerns were raised regarding the use of various VR devices by certain age groups. Specifically, several past studies have recommended that children under the age of 13 refrain from using VR [49]. Therefore, considering safety, only participants in their 20s were recruited. Reportedly, Oculus devices and the AR HoloLens 2 are unsuitable for use by children under the age of 14. Third, various tutorial interface designs were not considered. More diverse design types and manners of presentation exist in addition to those presented in this study.
In future studies, it will be necessary to recruit subjects of various ages other than those in their 20s to verify whether the results of this study can be obtained regardless of a specific age. In addition, this study designed the tutorial interface based on the contents of (1) basic operation, (2) moving method, and (3) user tasks required to play the game. Additional content may be required depending on the type of game, and the appropriate location of each content may be different. Therefore, the effectiveness of tutorials will be further verified based on more types of tutorial interfaces.

7. Conclusions

In this study, we developed a VR game and its tutorials and evaluated the user experience with and without the tutorials and the manner of their display. During these processes, we determined user tasks to be provided via these tutorials. Subsequently, we analysed and organised design considerations. Additionally, we proposed methods by which the tutorials could be displayed and used as the basis for user-experience evaluation. We considered three cases: no tutorial, instruction-screen tutorial, and context-sensitive tutorial. Presence, control learnability, intrinsic motivation, and perceived learning effectiveness were higher for the instruction-screen and context-sensitive tutorial cases than for the no-tutorial case. Users preferred the context-sensitive tutorial the most. On average, users experienced more motion sickness in the no-tutorial case, and we observed a major difference in nausea between the no-tutorial and tutorial cases. Thus, users were more satisfied with the VR game environment when the tutorial was provided than when it was not. This suggests that tutorials are an important tool for improving user experience, such as presence, control learnability, intrinsic motivation, and perceived learning effectiveness, in educational VR games. Additionally, tutorials can be regarded as an effective way to learn not only how to play games, but also how to perform interactions, such as operating a controller. The findings of this study can be used to improve the user experience in VR games in the future.

Author Contributions

Conceptualization, Y.L., G.K. and H.K.K.; methodology, Y.L., G.K. and H.K.K.; software, K.H.L., data analysis, Y.L., G.K., J.P. and H.K.K.; writing—original draft, Y.L.; writing—review and editing, J.P. and H.K.K. All authors have read and agreed to the published version of the manuscript.

Funding

This research was supported by the Ministry of Science and ICT (MSIT), Korea, under the ICT Challenge and Advanced Network (ICAN) program of HRD (IITP-2024-RS-2022–00156215) supervised by the Institute of Information & Communications Technology Planning & Evaluation (IITP). The present research was funded by a Research Grant from Kwangwoon University in 2022.

Institutional Review Board Statement

The study was conducted in accordance with the Declaration of Helsinki, and approved by the Institutional Review Board of Kwangwoon University (7001546-20211125-HR(SB)-011-01).

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

The data presented in this study are available upon request from the corresponding author.

Conflicts of Interest

The authors declare no conflicts of interest.

References

  1. Ahir, K.; Govani, K.; Gajera, R.; Shah, M. Application on Virtual Reality for Enhanced Education Learning, Military Training and Sports. Augment. Hum. Res. 2020, 5, 7. [Google Scholar] [CrossRef]
  2. Andersen, E.; O’Rourke, E.; Liu, Y.E.; Snider, R.; Lowdermilk, J.; Truong, D.; Cooper, S.; Popovic, Z. The impact of tutorials on games of varying complexity. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’12). Association for Computing Machinery, New York, NY, USA, 5 May 2012; pp. 59–68. [Google Scholar]
  3. McGovern, E.; Moreira, G.; Luna-Nevarez, C. An application of virtual reality in education: Can this technology enhance the quality of students’ learning experience? J. Educ. Bus. 2020, 95, 490–496. [Google Scholar] [CrossRef]
  4. Al-Ansi, A.M.; Jaboob, M.; Garad, A.; Al-Ansi, A. Analyzing augmented reality (AR) and virtual reality (VR) recent development in education. Soc. Sci. Humanit. Open 2023, 8, 100532. [Google Scholar] [CrossRef]
  5. Frommel, J.; Fahlbusch, K.; Brich, J.; Weber, M. The effects of context-sensitive tutorials in virtual reality games. In Proceedings of the Annual Symposium on Computer-Human Interaction in Play (CHI PLAY’17), Association for Computing Machinery, New York, NY, USA, 15 October 2017; pp. 367–375. [Google Scholar]
  6. Ho, J.C. Practice in reality for virtual reality games: Making players familiar and confident with a game. In Proceedings of the IFIP Conference on Human-Computer Interaction (INTERACT 2017); Springer: Cham, Switzerland, 2017; pp. 147–162. [Google Scholar] [CrossRef]
  7. Lee, P.W.; Wang, H.Y.; Tung, Y.C.; Lin, J.W.; Valstar, A. TranSection: Hand-based interaction for playing a game within a virtual reality game. In Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, USA, 18 April 2015; pp. 73–76. [Google Scholar]
  8. Wang, A.; Thompson, M.; Roy, D.; Pan, K.; Perry, J.; Tan, P.; Eberhart, R.; Klopfer, E. Iterative user and expert feedback in the design of an educational virtual reality biology game. Interact. Learn. Environ. 2022, 30, 677–694. [Google Scholar] [CrossRef]
  9. Abuhammad, A.; Falah, J.; Alfalah, S.F.; Abu-Tarboush, M.; Tarawneh, R.T.; Drikakis, D.; Charissis, V. “MedChemVR”: A Virtual Reality Game to Enhance Medicinal Chemistry Education. Multimodal Technol. Interact. 2021, 5, 10. [Google Scholar] [CrossRef]
  10. Checa, D.; Bustillo, A. A review of immersive virtual reality serious games to enhance learning and training. Multimed. Tools Appl. 2020, 79, 5501–5527. [Google Scholar] [CrossRef]
  11. Borstad, A.L.; Crawfis, R.; Phillips, K.; Lowes, L.P.; Maung, D.; McPherson, R.; Siles, A.; Worthen-Chaudhari, L.; Gauthier, L.V. In-Home Delivery of Constraint-Induced Movement Therapy via Virtual Reality Gaming. J. Patient-Centered Res. Rev. 2018, 5, 6–17. [Google Scholar] [CrossRef] [PubMed]
  12. Velev, D.; Zlateva, P. Virtual Reality Challenges in Education and Training. Int. J. Learn. Teach. 2017, 3, 33–37. [Google Scholar] [CrossRef]
  13. Ferguson, C.; Broek, E.L.v.D.; van Oostendorp, H. On the role of interaction mode and story structure in virtual reality serious games. Comput. Educ. 2020, 143, 103671. [Google Scholar] [CrossRef]
  14. Shi, A.; Wang, Y.; Ding, N. The effect of game–based immersive virtual reality learning environment on learning outcomes: Designing an intrinsic integrated educational game for pre–class learning. Interact. Learn. Environ. 2022, 30, 721–734. [Google Scholar] [CrossRef]
  15. Drey, T.; Jansen, P.; Fischbach, F.; Frommel, J.; Rukzio, E. Towards progress assessment for adaptive hints in educational virtual reality games. In Proceedings of the Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems, Association for Computing Machinery, New York, NY, USA, 25 April 2020; pp. 1–9. [Google Scholar] [CrossRef]
  16. McLaughlin, A.C.; Rogers, W.A.; Fisk, A.D. Using direct and indirect input devices: Attention demands and age-related differences. ACM Trans. Comput. Interact. 2009, 16, 1–15. [Google Scholar] [CrossRef] [PubMed]
  17. Yang, S.; Lim, C. How to use VR technology in educational games, prototyping. AJMAHS 2018, 8, 161–168. [Google Scholar]
  18. Tan, C.T.; Leong, T.W.; Shen, S.; Dubravs, C.; Si, C. Exploring gameplay experiences on the Oculus Rift. In Proceedings of the 2015 Annual Symposium on Computer-Human Interaction in Play, Association for Computing Machinery, New York, NY, USA, 5 October 2015; pp. 253–263. [Google Scholar] [CrossRef]
  19. Marougkas, A.; Troussas, C.; Krouska, A.; Sgouropoulou, C. How personalized and effective is immersive virtual reality in education? A systematic literature review for the last decade. Multimedia Tools Appl. 2024, 83, 18185–18233. [Google Scholar] [CrossRef]
  20. Kao, D.; Magana, A.J.; Mousas, C. Evaluating tutorial-based instructions for controllers in virtual reality games. In Proceedings of the ACM on Human-Computer Interaction, Association for Computing Machinery, New York, NY, USA, 6 October 2021; pp. 1–28. [Google Scholar] [CrossRef]
  21. Cao, S.; Liu, F. Learning to play: Understanding in-game tutorials with a pilot study on implicit tutorials. Heliyon 2022, 8, e11482. [Google Scholar] [CrossRef] [PubMed]
  22. Miguel-Alonso, I.; Checa, D.; Guillen-Sanz, H.; Bustillo, A. Evaluation of the novelty effect in immersive Virtual Reality learning experiences. Virtual Real. 2024, 28, 27. [Google Scholar] [CrossRef]
  23. Park, K.-B.; Lee, J.Y. Comparative Study on the Interface and Interaction for Manipulating 3D Virtual Objects in a Virtual Reality Environment. Korean J. Comput. Des. Eng. 2016, 21, 20–30. [Google Scholar] [CrossRef]
  24. Matthews, T.; Tian, F.; Dolby, T. Interaction design for paediatric emergency VR training. Virtual Real. Intell. Hardw. 2020, 2, 330–344. [Google Scholar] [CrossRef]
  25. Bozgeyikli, E.; Bozgeyikli, L.; Raij, A.; Katkoori, S.; Alqasemi, R.; Dubey, R. Virtual reality interaction techniques for individuals with autism spectrum disorder, Design considerations and preliminary results. In Proceedings of the International Conference on Human-Computer Interaction (HCI International 2016); Springer: Cham, Switzerland, 2016; pp. 127–137. [Google Scholar] [CrossRef]
  26. Krompiec, P.; Park, K. Enhanced Player Interaction Using Motion Controllers for First-Person Shooting Games in Virtual Reality. IEEE Access 2019, 7, 124548–124557. [Google Scholar] [CrossRef]
  27. Ros, M.; Debien, B.; Cyteval, C.; Molinari, N.; Gatto, F.; Lonjon, N. Applying an immersive tutorial in virtual reality to learning a new technique. Neurochirurgie 2020, 66, 212–218. [Google Scholar] [CrossRef] [PubMed]
  28. Fussell, S.G.; Derby, J.L.; Smith, J.K.; Shelstad, W.J.; Benedict, J.D.; Chaparro, B.S.; Thomas, R.; Dattel, A.R. Usability testing of a virtual reality tutorial. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting; SAGE Publications: Los Angeles, CA, USA, 2019; Volume 63, pp. 2303–2307. [Google Scholar] [CrossRef]
  29. Akman, E.; Çakır, R. The effect of educational virtual reality game on primary school students’ achievement and engagement in mathematics. Interact. Learn. Environ. 2020, 31, 1467–1484. [Google Scholar] [CrossRef]
  30. Makransky, G.; Borre-Gude, S.; Mayer, R.E. Motivational and cognitive benefits of training in immersive virtual reality based on multiple assessments. J. Comput. Assist. Learn. 2019, 35, 691–707. [Google Scholar] [CrossRef]
  31. Castronovo, F.; Nikolic, D.; Ventura, S.M.; Shroff, V.; Nguyen, A.; Dinh, N.H.; Gaedicke, C. Design and development of a virtual reality educational game for architectural and construction reviews. In Proceedings of the 2019 ASEE Annual Conference & Exposition, Tampa, FL, USA, 15 June 2019. [Google Scholar] [CrossRef]
  32. Jansen, P.; Fischbach, F. The Social Engineer: An immersive virtual reality educational game to raise social engineering awareness. In Extended Abstracts of the 2020 Annual Symposium on Computer-Human Interaction in Play; Association for Computing Machinery: New York, NY, USA, 2020; pp. 59–63. [Google Scholar] [CrossRef]
  33. Kim, H.K.; Park, J.; Choi, Y.; Choe, M. Virtual reality sickness questionnaire (VRSQ): Motion sickness measurement index in a virtual reality environment. Appl. Ergon. 2018, 69, 66–73. [Google Scholar] [CrossRef] [PubMed]
  34. Kennedy, R.S.; Lane, N.E.; Berbaum, K.S.; Lilienthal, M.G. Simulator Sickness Questionnaire: An Enhanced Method for Quantifying Simulator Sickness. Int. J. Aviat. Psychol. 1993, 3, 203–220. [Google Scholar] [CrossRef]
  35. Chu, P. VR Design: Transitioning from a 2D to 3D Design Paradigm; Samsung Developer Connection: San Francisco, CA, USA, 2014. [Google Scholar]
  36. Fröjdman, S. User Experience Guidelines for Design of Virtual Reality Graphical User Interfaces Controlled by Head Orientation Input. Available online: https://www.diva-portal.org/smash/record.jsf?pid=diva2%3A939381&dswid=9471 (accessed on 15 March 2023).
  37. Alger, M. Visual Design Methods for Virtual Reality. Available online: https://www.semanticscholar.org/paper/Visual-Design-Methods-for-Virtual-Reality-Alger/94d9115ffc204c4a904312d6cecaf3032101009c (accessed on 15 March 2023).
  38. Purwar, S. Designing User Experience for Virtual Reality (VR) Applications. Available online: https://uxplanet.org/designing-user-experience-for-virtual-reality-vr-applications-fc8e4faadd96 (accessed on 24 February 2020).
  39. Maguire, M. An exploration of low-fidelity prototyping methods for augmented and virtual reality. In Proceedings of the International Conference on Human-Computer Interaction (HCII 2020), Copenhagen, Denmark, 19–24 July 2020; Springer: Cham, Switzerland, 2020; pp. 470–481. [Google Scholar] [CrossRef]
  40. Kamaruddin, N.; Sulaiman, S. Understanding interface design principles and elements guidelines: A content analysis of es-tablished scholars. In Proceedings of the Art and Design International Conference (AnDIC 2016); Springer: Singapore, 2018; pp. 89–100. [Google Scholar] [CrossRef]
  41. Schubert, T.; Friedmann, F.; Regenbrecht, H. The Experience of Presence: Factor Analytic Insights. Res. Q. Exerc. Sport 2001, 10, 266–281. [Google Scholar] [CrossRef]
  42. McAuley, E.; Duncan, T.; Tammen, V.V. Psychometric Properties of the Intrinsic Motivation Inventory in a Competitive Sport Setting: A Confirmatory Factor Analysis. Res. Q. Exerc. Sport 1989, 60, 48–58. [Google Scholar] [CrossRef] [PubMed]
  43. Zhao, L.; Lu, Y. Enhancing perceived interactivity through network externalities: An empirical study on micro-blogging service satisfaction and continuance intention. Decis. Support Syst. 2012, 53, 825–834. [Google Scholar] [CrossRef]
  44. Lee, E.A.-L.; Wong, K.W.; Fung, C.C. How does desktop virtual reality enhance learning outcomes? A structural equation modeling approach. Comput. Educ. 2010, 55, 1424–1442. [Google Scholar] [CrossRef]
  45. Sholihin, M.; Sari, R.C.; Yuniarti, N.; Ilyana, S. A new way of teaching business ethics: The evaluation of virtual reality-based learning media. Int. J. Manag. Educ. 2020, 18, 100428. [Google Scholar] [CrossRef]
  46. Paas, F.G. Training strategies for attaining transfer of problem-solving skill in statistics: A cognitive-load approach. J. Educ. Psychol. 1992, 84, 429–434. [Google Scholar] [CrossRef]
  47. Sweller, J.; van Merrienboer, J.J.; Paas, F.G. Cognitive Architecture and Instructional Design. Educ. Psychol. Rev. 1998, 10, 251–296. [Google Scholar] [CrossRef]
  48. Ryan, R.M.; Rigby, C.S.; Przybylski, A. The Motivational Pull of Video Games: A Self-Determination Theory Approach. Motiv. Emot. 2006, 30, 344–360. [Google Scholar] [CrossRef]
  49. National IT Industry Promotion Agency. VR/AR Device Production and Use Guidelines; National IT Industry Promotion Agency: Seoul, Republic of Korea, 2020. [Google Scholar]
Applsci 14 07141 g001
Figure 1. Virtual reality (VR) game ‘Numverse’ (top: complete view, bottom left: grabbing objects, bottom right: lowering conversion device).
Figure 1. Virtual reality (VR) game ‘Numverse’ (top: complete view, bottom left: grabbing objects, bottom right: lowering conversion device).
Applsci 14 07141 g001
Applsci 14 07141 g002
Figure 2. Tutorial development process: Section 3.1. selecting tutorial content, Section 3.2. designing the tutorial user interface (UI), and Section 3.3. suggesting tutorial method.
Figure 2. Tutorial development process: Section 3.1. selecting tutorial content, Section 3.2. designing the tutorial user interface (UI), and Section 3.3. suggesting tutorial method.
Applsci 14 07141 g002
Applsci 14 07141 g003
Figure 3. Tutorial interface design: (a) basic operation, (b) moving method, (c–g) user tasks required to play the game.
Figure 3. Tutorial interface design: (a) basic operation, (b) moving method, (c–g) user tasks required to play the game.
Applsci 14 07141 g003
Applsci 14 07141 g004
Figure 4. Tutorial component examples (left: AltSpace VR, right: half-life alyx): The design components of tutorials are text, controller image, and motion image.
Figure 4. Tutorial component examples (left: AltSpace VR, right: half-life alyx): The design components of tutorials are text, controller image, and motion image.
Applsci 14 07141 g004
Applsci 14 07141 g005
Figure 5. Tutorial components: motion image: inducing user behaviour, controller image: describing interactions with the game, text: describing game methods.
Figure 5. Tutorial components: motion image: inducing user behaviour, controller image: describing interactions with the game, text: describing game methods.
Applsci 14 07141 g005
Applsci 14 07141 g006
Figure 6. Example of instruction-screen tutorial method: The location and direction of the tutorial are fixed.
Figure 6. Example of instruction-screen tutorial method: The location and direction of the tutorial are fixed.
Applsci 14 07141 g006
Applsci 14 07141 g007
Figure 7. Example of context-sensitive tutorial method: tutorial is seen around the object without covering the object on the screen.
Figure 7. Example of context-sensitive tutorial method: tutorial is seen around the object without covering the object on the screen.
Applsci 14 07141 g007
Applsci 14 07141 g008
Figure 8. Experimental environment.
Figure 8. Experimental environment.
Applsci 14 07141 g008
Applsci 14 07141 g009
Figure 9. Usability testing procedure: Group 1 experimented in the order of no-tutorial, instruction-screen tutorial, and context-sensitive tutorial, while Group 2 conducted it in the order of no-tutorial, context-sensitive tutorial, and instruction-screen tutorial.
Figure 9. Usability testing procedure: Group 1 experimented in the order of no-tutorial, instruction-screen tutorial, and context-sensitive tutorial, while Group 2 conducted it in the order of no-tutorial, context-sensitive tutorial, and instruction-screen tutorial.
Applsci 14 07141 g009
Applsci 14 07141 g010
Figure 10. User experience concerning tutorial conditions (means followed by the same letter did not differ significantly).
Figure 10. User experience concerning tutorial conditions (means followed by the same letter did not differ significantly).
Applsci 14 07141 g010
Applsci 14 07141 g011
Figure 11. Motion sickness concerning tutorial condition (means followed by the same letter did not differ significantly).
Figure 11. Motion sickness concerning tutorial condition (means followed by the same letter did not differ significantly).
Applsci 14 07141 g011
Table 2. Virtual reality engagement questionnaire (VREQ) descriptive statistics.
Table 2. Virtual reality engagement questionnaire (VREQ) descriptive statistics.
FactorMeanSD
PresenceSpatial Presence7.121.43
Experienced Realism5.461.85
Involvement5.981.81
Mapping5.942.43
Controls’ LearnabilityMental Load6.372.30
Mental Effort6.232.54
Learnability6.252.20
Intrinsic MotivationEnjoyment7.241.85
Continual Intention to Use7.412.04
Player Experience6.752.10
Perceived Learning Effectiveness7.081.83
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Lee, Y.; Kim, G.; Lee, K.H.; Park, J.; Kim, H.K. Comparison of Tutorial Methods in Virtual Reality Games for a Better User Experience. Appl. Sci. 2024, 14, 7141. https://doi.org/10.3390/app14167141

AMA Style

Lee Y, Kim G, Lee KH, Park J, Kim HK. Comparison of Tutorial Methods in Virtual Reality Games for a Better User Experience. Applied Sciences. 2024; 14(16):7141. https://doi.org/10.3390/app14167141

Chicago/Turabian Style

Lee, Yuryeon, Guyeop Kim, Kang Hoon Lee, Jaehyun Park, and Hyun K. Kim. 2024. "Comparison of Tutorial Methods in Virtual Reality Games for a Better User Experience" Applied Sciences 14, no. 16: 7141. https://doi.org/10.3390/app14167141

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop