3D Interactive Learning Environment as a Tool for Knowledge Transfer and Retention

Abstract

The article deals with the design of virtual reality (VR) interactive training as a teaching method and its effect on knowledge transfer and retention of students. The first part presents the methodology of the VR interactive training design. The second part utilizes the created interactive training for a case study to evaluate its effect on the teaching process and to examine the potential of VR interactive training as a sustainable teaching method. The study took place at the Department of Industrial Engineering, Faculty of Mechanical Engineering, University of Žilina. Volunteers were divided into two groups. The first group used VR interactive training as a teaching method, while the second group used the conventional method. Both groups then underwent tests. The main goal was to evaluate the effect of the VR interactive training on the teaching process in comparison to the conventional method while trying to identify the key elements of the VR interactive training design and its influence on knowledge transfer and retention in a sustainable learning environment. At the start of the case study, four hypotheses were formed, questioning the effect of interactive training on knowledge transfer and retention in the long and short term, and its overall influence on the teaching process. Obtained data were then used to evaluate these hypotheses.

1. Introduction

Nowadays, a selection of knowledge transfer method constitutes a crucial part of education practice [1]. It is important to keep participants immersed and entertained while providing knowledge in a comprehensible form [2]. However, as digital evolution continues [3], it is imperative to ensure students can transfer learnt skills into practical use.

Virtual reality (VR) and gamification in education is not a new concept, with various researchers and institutions having emphasized the positive effects of the implementation of training into teaching processes. Augmented and VR technologies are often integrated into games. This method provides a less stressful environment for students to learn and thus reduces learning anxiety [4,5]. In his study, Abad-Segura et al. [6] highlighted the need to integrate digital technologies, including virtual and augmented reality, into education and research. The result of this study emphasizes the importance of using augmented reality as a tool for educational games focused on sustainability, motivation, and equal education. In his study, Sprenger et al. [7] evaluated four digital learning technologies, namely electronic lectures, chat, a classroom feedback system (allowing lecturers to ask students multiple-choice questions on their electronic devices during, before, or after the lecture), and mobile VR. The results of this study showed that mobile VR has surprisingly shown a gradual decrease in perceived usefulness for students, compared to other evaluated technologies. Education in a VR environment is based on proper visualization and information provision. However, when too much information displayed this can overload the user and affect their cognitive abilities. This issue was addressed by Albus et al. in their study [8], where they researched the influence of text annotations in educational VR environments. The results of this study showed that text signaling can improve learning outcomes in terms of recalling the outcome but not comprehending it. The learning environment should be student-centered and provide an effective goal-oriented setting [9]. The use of virtual and augmented reality seems to be a suitable for the training environment and as an alternative to traditional exercises. In his study, Morélot et al. [10] investigated the effects of immersion, sense of presence, and interaction between them. The results of this research showed that immersion supports procedural but not conceptual learning.

On the other hand, there is a presumption that such a form of teaching could lead to students’ attention being drawn to graphic elements such as animations, sounds, and effects [11]. Students do not pay attention to the educational content, but instead to the visual side of the training or educational game. The design of didactic games and training should be the result of teacher-student collaboration and should include all elements required by both groups [12]. Recently, the use of virtual laboratories has been widespread. These virtual environments should be designed to remain attractive to students and provide a modern and sophisticated environment [13].

Several previous studies have been created at the authors’ workplace, these were focused on the research of the application of virtual and augmented reality in the field of industrial engineering [14]. The presented study took place simultaneously and, in some parts, builds on and complements previous research, which monitored the attractiveness and acceptability of educational games in VR for students in the university environment compared to traditional teaching methods, such as laboratory exercises, e-learning, instruction, demonstration, and lecture [15]. The results of the study show that university students are inclined to use educational games in the teaching process, are interested in them, and would accept their use to a greater extent.

As stated above, the application of VR has great potential to further improve the quality of education. On the other hand, it must be said that fully immersive training and educational games are still searching for their place in the educational processes of the Slovak Republic and therefore we consider research in this area to be highly topical and necessary from the point of view of sustainability, development, quality, efficiency, and attractiveness of higher education of the future. The presented study deals with the identification of key elements for the creation of interactive training and the transfer and preservation of knowledge for the achievement of sustainable education.

2. Materials and Methods

In the first part of the research, a methodology was created and used to produce interactive training, which later served as a starting point for a case study in collaboration with the Department of Industrial Engineering and Department of Design and Mechanical Elements, Faculty of Mechanical Engineering, University of Žilina.

In the second part, inspired by research findings [16,17,18,19,20,21], interactive virtual training for a drilling operation was created. The training was created in the Unity 3D game engine and several sets of the HTC Vive Pro display device were used for full immersion into VR, using the experience and knowledge of the authors in the field of virtual and augmented reality [22,23,24].

In the third part, a sample of two groups of volunteers participated in a case study. One group was equipped with interactive training through VR (VR group), while the other group studied the same educational content using conventional teaching methods (NoVR group). Subsequently, the volunteers performed a series of tests to compare the effects of the two different teaching approaches.

2.1. The Methodology of Interactive Training Creation

The creation of interactive training using VR is a relatively long and complex process. The methodology provides suitable principles and methods for each part of game creation that helps the user to utilize available resources more efficiently. The process consists of multiple stages that require different approaches and software support. The result is serious training based on VR that helps the user to be more immersed in the training process. This can make the transfer of knowledge into practical use more efficient. A creator needs to use various hardware and software to gather necessary data and used them to create an interactable virtual environment. The main goal is to take a real task and turn it into a VR application, with a focus on teaching. To help visualize the whole process, a flowchart was created showing all tasks needed to create a VR interactive training. The flowchart itself is shown in Figure 1. The methodology covers the basic outline of serious training creation. A similar methodology was used to create a 5S (a lean management method) educational game within the authors’ department [15]. However, the creation process may significantly vary depending on the type of training and teaching subject. Nevertheless, the core elements should remain the same. Therefore, this methodology can be used as a foundation for interactive training creation.

The methodology can be divided into four main parts:

• Analysis and reference collection;
• Asset creation (3D and 2D assets);
• Virtual environment and game scenario creation;
• Testing and utilization.

2.1.1. Analysis and Reference Collection

After the subject of the VR training has been chosen, it is vital to analyze every aspect to ensure the smooth workflow of VR training creation. Depending on the chosen activity, VR interactive training will most likely consist of various operations and scenes. Therefore, it is necessary to thoroughly observe the real-life process in order to successfully create its virtual counterpart. In some cases, only the core points of the process may be enough for a satisfactory result. However, if the educational game takes place in a specific workplace doing specific tasks (such as virtual employee training), these require a high degree of detailed observation to truthfully simulate the process. In the end, it depends on the subjects of the educational game to determine the depth of the initial analysis. Along with the process, the scene itself is a vital part of VR training immersion. The player will most likely interact with many objects. Therefore, these objects must be transformed into a virtual world. The objective is to create a virtual scene that contains every necessary object to simulate the real environment and allow the player to execute assigned tasks. For this reason, 3D models of these objectives must be created. The key is to determine which objects of the real-life scene are vital for the correct functioning of the VR interactive training.

A virtual scene should contain:

• Objects that the player will interact with;
• Objects that are necessary to simulate the selected process;
• Objects that are necessary to simulate the selected workplace (scene);
• Objects that enhance the immersion.

Input analysis serves as a checkpoint. The main objective is to sort the obtained references (such as pictures or videos) and discover possible shortcomings. Some activities or objects may not be captured well enough for their faithful replication. Reference quality may be low, or the perspective may be confusing. In cases such as these, recapturing every discarded reference is recommended.

The next step is reference sorting into categories. Depending on usage, references are placed into groups dedicated to a selected objective. The categories may vary and can be modified to suit the user preferences and type of training. For example, the user may use the classification mentioned in Section 2.1.2 (relation of referenced object and user). References can also be divided into groups based on referenced object properties, such as complexity, frequency of use, or type of object (machines, tools, furniture, etc.). These simplify the workflow and allow easier access to needed references in any part of the VR training creation. In combination with process specification (information about the process for gamification), references serve as a basis for successful VR interactive training creation.

2.1.2. Asset Creation

Each object deemed necessary must be replicated in 3D virtual form. To properly guide the player through the virtual interactive training, a set of visual aids is required. Along with instructions, these 2D objects help the user to correctly navigate through the virtual scene. They also should lower the chance of the player doing the task incorrectly and overall increase the ease of use. However, while creating a 3D model, it is important to know the purpose of the object within the educational game. Not all models share the same complexity level. Therefore, before modelling starts, it is good practice to sort objects into categories based on their complexity. Each category represents the different relationships between the user and an object based on the level of interaction. These categories are:

• Objects that the player will interact with (the most complex ones);
• Objects that the player will not interact with but are crucial to correct real-scene replication (less complex ones);
• Objects that enhance the aesthetics of the scene (the simplest ones).

Making less important objects simpler reduces the hardware demands on the scene and saves time during the whole modelling process. Objects that players will interact with are the most complex because they need to contain all the details to necessary to simulate their real-life functions. For example, a machine tool must contain all the control buttons, displays, and levers that are necessary to operate it in the same way in which it would be operated in real life. However, the creator can leave out details that will not contribute to the object operability, therefore saving time and hardware requirements. Another important part is the separation of interactable parts while creating interactable objects. The rest of the objects that will not be interacted with are not required to have the same amount of detail. However, they should still contain key features that make them easily recognizable. Bigger objects, which will be seen from up close, can have more details than small ones that will mostly be seen from a distance. In the end, is up to the creator and their resources to determine the level of complexity and the time invested. The computing power of available hardware is also a deciding factor.

Another important part of this stage is the training logic creation. Created 3D models need to respond in the same fashion as their real-life counterparts. Pressing a button on a virtual machine tool needs to produce the same outcome as pressing the same button on a real object. Therefore, it is important to analyze and write down possible actions of the user and reactions of the objects. Selected actions then need to be done in the correct order to successfully finish the training. This is called the training scenario. Training logic contains the possible actions of the user during training. Executing these actions in a correct order creates the training scenario. The training scenario can be created before the 3D models. It depends on the complexity of the taught process, however making the scenario beforehand can help in choosing which 2D object and hints must be created to successfully navigate the player through the process. All the visual aids (2D object and hints) are created during the “Creations of required UIs” stage.

2.1.3. Virtual Environment and Game Scenario Creation

The main goal of this part of the process is to create the virtual environment and game scenario. It requires turning the observed process and objects (training scenario and 3D models) into the sequence of activities done in a correct order that simulates the subject of training.

It is important to keep in mind that players will do unfamiliar tasks in order to learn them through VR interactive training. Therefore, it is vital to split the taught process into its most elemental parts. These parts then will be explained step-by-step to enhance the comprehensibility of the training. Every task should represent a sub-goal contributing to the completion of the process. Therefore, all of them should be separated by the instructions and completion verification. If the player does the task incorrectly, the training itself should warn them and only allow them to proceed to the next stage after correction. The task sequence logic is shown in Figure 2.

The first step in creating the VR educational game itself is environment creation. Using gathered references, the main goal is to create the 3D virtual scene that simulates the real environment required to perform the tasks, which represents the subject of the training. This can be done in various software or game engines, such as Unity 3D. With a 3D virtual scene already created, the next step is to create the required interaction between player and objects. Interaction programming can be considered the most difficult task. Its complexity depends on the subject of VR educational games. The main goal is to create every necessary interaction and logic to successfully simulate the chosen process. To do so, programming language knowledge is required in most cases. For example, for the game engine Unity 3D uses the programming language C#, which is directly integrated into the software. This allows assigning codes (scripts) to specific objects or scenes, achieving the required intractability. This includes all interactions that are needed to execute all tasks of the VR educational game. The first step is the creation of player movement. Even though the player does not directly interact with the objects, it is a vital part of the activity simulation. There are two most commonly used options of player movement through the scene: teleport and continuous movement. Continuous movement offers the most immersive experience further enhancing the simulation of the real process. However, many people may experience motion sickness after a few minutes of smooth movement. In that case, the teleporting option servers as a sufficient replacement.

The next step is enabling control elements of the object. For example, making the buttons pressable, levers pullable, etc. However, every control element should trigger a reaction, thus every trigger should be assigned a reaction. The type of reaction depends on the object, trigger, and subject of the training.

The next step is the creation of an instruction and hint system. All created 2D objects and a hint will be used for this. The main goal is to provide instruction as clearly as possible and warn the user if the task has been done incorrectly. Instruction for the next tasks will only show after the previous task is successfully finished. When all tasks are completed, there is the possibility to include a post-process evaluation, which will inform the user about how successful they were in completing tasks.

To make the whole process easier, it is possible to use SDKs (Software Development Kits that help users to create platform-specific applications) and import them to the software. These include premade interaction codes and other useful tools to simplify the process for a less experienced user.

2.1.4. Testing and Utilization

The last task before the utilization of the game is the testing phase. The main goal is to ensure that the game runs smoothly without any visual or design flaws. For this reason, two types of testing are recommended: testing by the creator (functionality test) and a user test (visual and ease of use test).

The objective of the first test is to make sure that all parts of the game function correctly, that all interactions are working correctly, and that the game can deal with user mistakes without disrupting the game logic. The user is the one learning the process and is bound to make mistakes. It is the game’s purpose to point out any mistakes and set a player on the correct path. Therefore, this test needs to ensure that the game will correct the user no matter what discrepancy occurs. The second test lets the future user try the game themselves. The objective of this test is to collect feedback to potentially improve the experience. The perspective of someone who is not familiar with a game can help in identifying if some parts are hard to understand or are not visually clear enough. Even though the game can be functionally perfect, the perspective of the user should not be neglected. After finishing all necessary tests and fixing all discovered shortcomings, the VR educational game is ready to be exported and used.

2.2. Design and Creation of VR Interactive Training for a Case Study Based on the Methodology

The presented methodology was used to create a sample of VR interactive training, which was used as a VR teaching method representative in a case study. Therefore, it had to contain all the core principles of the VR educational game. The game focuses on machine tools operation training, specifically, drilling workplace operation training. This workplace is located at the University of Žilina and serves as a teaching aid for students at the Department of Design and Mechanical Elements. Its suitable location and purpose made it a perfect candidate for gamification and the case study. Many students of the Industrial Engineering department will be tasked with workplace design in their future careers. Therefore, virtually experiencing a machine tool operation and workplace workflow while not leaving a classroom may be very beneficial. This is the main reason why it was chosen as a subject for the creation of a VR educational game. However, before the case study itself, many other tasks must be completed to achieve a fully operational educational game using VR.

As the methodology suggests, the first step in the educational game (virtual training) creation is an initial analysis. Firstly, the composition of the workplace was analyzed. This includes deciding which objects are important for correct workflow simulation and which can be omitted. The selected objects then need to have their virtual model created. Another part of the analysis is an observation of the drilling process itself. Training participants must reproduce tasks learned through virtual training in a real process. Therefore, an entire drilling operation must be an exact copy of its real-life counterpart. The authors tried the drilling workplace operation themselves under supervision. This provided even more data for the creation of a training scenario in the next phase.

After analyzing the workplace objects and workflow, the next step is reference collection. All necessary data were gathered to create every required 3D model and training scenario. Reference materials include photos of objects (drills, tables, etc.) and a 360° video of the drilling operation along with the technological process. Figure 3 shows examples of the collected references.

With all references collected, the project moved to the next phase. This phase includes 3D modelling, UIs creation, and the creation of the educational game scenario. Firstly, all required 3D models are created. This includes every object in the drilling workplace needed for its correct simulation. The main objects of the scene are two drills (MAS Kovosvit VO50 and Heltos VS32) for the main operation—drilling. These two objects were created with the highest attention to detail because they are key points for a correct simulation and their control is the most complex of all objects in the workplace. All control elements were modelled with great precision to simulate the operation of a real machine tool. They were also separated from the main model to allow individual manipulation (pulling a lever, pushing a button, etc.). For this task, the modelling software Blender was used. Figure 4 shows the process of the VS32 drill modelling.

The next step is the creation of the UIs. This includes all 2D objects that help the player (student) navigate through the game providing all necessary information. This task is deeply connected with the creation of a scenario, since every operation simulating real workflow will be explained through these 2D elements (instruction, hints, arrows, etc.). In this case, the drilling virtual training required precision, correct tool selection, and correct workpiece placement. Therefore, the main 2D elements created were arrows to highlight the correct objects and floating windows with hints explaining the entire process step-by-step. These were created using the graphic software Visio. The creation of 2D elements ensures the proper order of performed tasks and the correction of any mistakes incurred. Figure 5 shows an example of hint creation.

The last stage before the creation of the virtual training itself is the creation of a scenario. In cases in which the subject of the educational game of virtual training is complex, it is possible to do this task before the creation of UIs, to ensure there is enough information to create every necessary 2D element. For the virtual training scenario, an analysis of every process reference is required. The whole process was analyzed and divided into small tasks that can act as checkpoints to track a student’s progress. Moreover, separating the process into small parts ensured that instructions are not overwhelming and easy to follow. Every step has a condition that needs to be met to proceed to the next part. However, every small task needs to be logical and meaningful while naturally blending with other tasks to preserve the immersion. A correction system was also created to correct the user’s actions. If the user’s actions do not match set conditions, the user is warned and guided to the correct workflow. Since the scenario was used to program every interaction in the virtual training, it is vital to correctly visualize it to ensure that the creator of the interaction of virtual training has a great understanding of the entire process. In this case, the training scenario has been visualized using a flowchart, as shown in Figure 6.

With all preparation tasks completed, the project continued into the stage of virtual training application creation. This includes the creation of a virtual drilling workplace and programming of interactions and logic. For these tasks, the Unity 3D game engine has been used along with the programming language C# for the interaction programming. Firstly, 3D models have been used to create an exact copy of the drilling workplace. Machine tools, tables, palates, and other objects were placed in the same position as their real-life counterparts. Reference materials, such as photos or a 2D layout, helped to correctly replicate every position. 3D models were simply imported into Unity 3D and then dragged and dropped into the scene. Moreover, created 2D objects and hints were also imported. However, they will be placed during the interaction programming stage to make the scene less cluttered. In addition, another scene was created. An entry HUB created from a 360° picture serves as a menu, where the players can choose a type of virtual training, even though they can only choose one training for now. Figure 7 shows the virtual workplace creation using Unity 3D.

With the virtual drilling workplace completed, the next step was securing the player’s interaction with a virtual environment. Firstly, the means of the player movement were added. Moving through the scene is the basic interaction required for any other tasks. Moreover, with movement added, the scene could be tested to see if the workplace has any hidden irregularities. The whole process was simplified using Steam VR SDK with its pre-made movement assets. What is also important is the choice of locomotion. For this project, a teleport type locomotion was selected to reduce the possibility of motion sickness.

The final step of virtual training creation was securing the game intractability. This task was divided into two main parts: player-object interactions and the hint/instruction system. The main goal of player-object interactions is to ensure that a player can replicate the real-life drilling operation and sub-operation in a virtual environment. This required knowledge of programming and the drilling process itself. Numerous C# scripts (components that tell game objects how to behave and interact with other objects) were created and assigned to various objects, which were then tuned to meet set parameters. This includes the ability to grab and place objects, the ability to control drills, object physics, and drilling simulation.

At this point, the entire drilling process could be simulated, but only by someone familiar with the required operations. Therefore, the final step is the programming of the hint/instruction system. All created 2D (arrow, instruction signs, etc.) objects will appear on the scene to guide the player through the entire virtual training This was again achieved using scripts to set the rules regarding when a particular hint is to appear for the player. In this case, the process is fairly straightforward, therefore there was no need for correction signs. Correct placement or movement is always highlighted, and the game will not proceed until the correct execution of the task. Figure 8 shows a simple introduction script controlled by a button press. The rest of the hint system is integrated into scripts for process simulation, since hints are shown continuously as the virtual training progresses.

After the successful integration of player interactions, the virtual training was ready for the testing phase. For the testing, an HTC Vive VR headset was used along with necessary accessories such as controllers and base stations.

Finally, the interactive virtual training for the drilling operation was completed and ready to be utilized for the case study. Figure 9 shows images from the educational game gameplay. The first picture shows the entry hub and the rest demonstrate the training itself.

3. A Case Study

With all the preparation and interactive training completed, the case study was conducted. The case study took place in parts between March 2020 and May 2021, at the Department of Industrial Engineering in cooperation with the Department of Design and Mechanical Elements, Faculty of Mechanical Engineering, University of Žilina. Due to the ongoing global pandemic, the additional options for experiments, the course of the study, and the size of the sample of volunteers were greatly limited. The entire case study was realized under strict measures of a pandemic valid for universities in Slovakia. The participants consisted of students of the aforementioned departments. Researchers and educators also volunteered, however, not as study participants. They helped with the study organization and prior testing of the virtual training before its usage in the study.

The group of volunteers were bachelor students and engineering students from both departments. The age range of students was between 20 and 25 years old. The ratio of women to men was 1:2. Only 6 out of 100 participants had previous experience with VR or with interactive training in VR, which represents 6% of the sample.

Volunteers were divided into two groups of 50 people—the VR group and the NoVR group. Students in small groups were assigned a precise schedule according to which training and tests were performed. After 30 days, every member of the VR and NoVR group underwent all five tests and subsequently filled out a questionnaire.

The main objective was to examine the effect on knowledge transfer and retention of VR interactive training compared to a conventional training method. At the same time, the aim of the study was also to identify the key elements of the VR interactive training or educational game that helped the students assigned to the VR training group enhance their learning capabilities.

3.1. Research Questions and Hypothesis

VR technology offers the possibility of accurate simulation of real-life processes, simulating not only the environment but also the movements required to execute tasks. Therefore, it possibly creates an ideal environment for learning safely and effectively. The case study aimed to confirm or refute the following hypotheses:

• The use of VR interactive training allows more effective knowledge transfer and retention in a short-term application;
• The use of VR interactive training allows longer knowledge retention and easier knowledge transfer in a long-term application;
• The use of VR interactive training gives students a better understanding of the taught subject compared to conventional methods, making it easier to integrate gained knowledge into a variable environment during practical application;
• The effectiveness of the VR interactive training is heavily influenced by the chosen game mechanics and method of instruction delivery.

On the other hand, there are some concerns regarding the use of VR interactive training. Some may consider it too distracting, taking students attention away from the main topic. The case study also examines these two possible issues:

• Students will be too distracted by the novelty of the VR interactive training to pay attention to the taught subject/process;
• The playful approach of the VR interactive training may redirect a student’s attention to unimportant aspects of the lessons.

3.2. Study Schedule and Tests

The tests were performed under laboratory conditions. Firstly, volunteers (students) were divided into two groups: the VR group (group educated by interactive training) and the NoVR group (group educated by conventional methods—video and instruction manual). There were 50 students each group given the schedule for the experiment. At the beginning of the case study, the NoVR group participated in the training for the chosen subject (machine tool operation) using the video presentation and written instruction manual. The VR group participated in a VR training session using the designed VR interactive training (multiple VR headsets were provided to the VR group training). Subsequently, both groups would be tested on their knowledge, replicating what they had learnt in a real workplace. The details and schedule are shown in

Table 1. Details of study.

	VR Group	NoVR Group
Number of participants	50	50
Age	20–25	20–25
Study degree	Bachelor	Bachelor
	Engineering	Engineering
Training content	Video preview of the tasks	Video preview of the tasks
	VR interactive training	Self-study with a given instruction manual
Training length	90 min	90 min

and Figure 10.

The main goal for each test was to complete every task as best as possible without using the available aids during the test. There were five possible grades for each test:

• Perfect score (A)—all tasks done correctly without any help;
• Passed with minor help (B)—all tasks done correctly while using the aids only once;
• Passed with major help (C)—all tasks done correctly while using the aids more than once;
• Insufficient (Xf and Xt)—student could not perform the task correctly, even with the help of the aids (Xf) or could not meet the time limit (Xt).

In addition, there was one more observed factor—the time of the completion. The time needed for each student to complete each test was measured to provide more data for the final evaluation. A time limit for the test competition was also set. Even if the student can correctly reproduce the tasks in a real workplace, it is too ineffective if they cannot do so in a set time. Therefore, even a perfect execution of the test which exceeded the time limit was marked as insufficient (X). The time limit was set to 11 min. For reference, on average, an experienced user can finish all tasks in four minutes.

Each test had small changes to make sure that students have truly understood the machine tool operation in training. The exception was Test 1 which had no test modifications—the test faithfully copied the content of the training. Therefore, it was made evident if students could not adapt to the changes and were only mechanically following the training instructions without proper understanding of the subject.

During one test (Test 4) there were slight modifications in the workplace layout. This was done to test the student’s reaction to the unexpected changes and to see how well they could deal with them.

On the day of the last test (Test 5), the students of the VR group filled out a short questionnaire. Its main goal was to gather the students’ opinions on the following:

• The VR interactive training was distracting;
• The VR interactive training did not provide enough information to successfully replicate lessons in a real workplace;
• What is a crucial part of a well-designed interactive training?;
• What is the best way to provide instruction and feedback in a virtual environment (while playing the VR training)?;
• What was the most distracting element of the VR interactive training that complicated following the instructions and their subsequent application in the real workplace?

4. Results

After the participants finished the tests in a 30-day cycle of the case study according to a schedule, the results were evaluated.

4.1. Tests Results

The test results were evaluated from two perspectives. The first aspect was the rate of success (the grades are described in Section 3.2). The second aspect was the average time of the test completion.

The number of the students who retained a perfect score was higher than expected. Additionally, the majority of them still managed to pass the test with minor or major help (B or C). As is shown in Figure 11, Figure 12 and Figure 13, both groups had a 100% success ratio for Test 1 and Test 2. Everyone in the VR group kept the trend for Test 3 while the NoVR group registered the first failed test. The VR group did not register failure up until the fourth test. Even in the last test, the majority managed to pass the test. Only 24% of students in the NoVR group could not pass the last test. In this case, the VR group also managed to achieve better results, with only 12% of failed attempts.

On average, the students that underwent the VR educational game training achieved better results throughout all five tests.

Table 2. Percentage of grades achieved for tests.

Grade	A		B		C		Xt/Xf
Test	VR	NoVR	VR	NoVR	VR	NoVR	VR	NoVR
Test 1	92%	82%	8%	18%	0%	0%	0%	0%
Test 2	90%	80%	10%	12%	0%	8%	0%	0%
Test 3	88%	80%	10%	8%	2%	10%	0%	2%
Test 4	82%	64%	12%	12%	4%	14%	2%	10%
Test 5	70%	50%	8%	14%	10%	12%	12%	24%

shows the percentage of achieved grades for each test in both groups and

Table 3. Average times of tests in summary.

Grade	Test 1		Test 2		Test 3		Test 4		Test 5
Test	VR	NoVR	VR	NoVR	VR	NoVR	VR	NoVR	VR	NoVR
Average time (minutes)	06:36	07:02	06:47	07:04	07:02	07:56	08:06	09:02	08:04	09:10
VR—NoVR (difference)	00:26		00:17		00:54		00:56		01:06
Best time (minutes)	04:56	04:55	04:57	05:15	05:45	05:40	06:22	06:42	06:15	07:02

shows the average times in summary.

The fastest completion time for each test varied. However, this was solely dependent on the individual therefore not suitable for overall evaluation. Therefore, average test times were used for the evaluation. The VR group completed the tests 9% faster than the NoVR group.

The main reason for this may lie within the type of training itself. Students in the VR group had an advantage because they had already experienced the test environment during the VR educational game, which reliably simulated the test area (workplace). On the other hand, the students of the NoVR group needed extra time to transfer the knowledge from conventional training into a real application.

One of the measured factors is consistency—the ability to yield satisfying results even after the passage of time or unexpected changes. The students in the VR group showed better results when it came to knowledge transfer and retention in the short and long term, with 88% of them managing to pass every test (Figure 14). For the NoVR group, this percentage was noticeably lower, at 72%, meaning that 28% of students in the NoVR group failed at least one test throughout the entire experiment. Overall, students that used the educational VR game performed better than the students using a conventional teaching method, increasing the potential of VR gamification and VR interactive training as a sustainable teaching method.

In conclusion, the members of the VR group were less likely to use the available help and on average achieved better test completion times. These data, in combination with the following questionnaire, were used to evaluate the proposed hypotheses in the conclusion.

4.2. Questionnaire Results

On the last day of the experiment (after Test 5), members of the VR group were asked to fill out a short questionnaire. Its results were incorporated into the final evaluation to support or question the acquired findings. The highlighted data from the questionnaire are shown in the figures below. Figure 15 shows the results of the part of the questionnaire where students could answer with yes or no. They were also asked to elaborate if their answer to a question was a no.

The majority of students gave positive answers. As mentioned above, the students also elaborated on their issues with the interactive training if the answer was a no. Their answers for the question of distracting elements matched in two main points. The first is regarding motion sickness caused by VR games, and the second is the distraction caused by the novelty of the technology. A handful of the students experienced motion sickness during the VR training, significantly affecting their learning capabilities. This was most likely to affect them in their performance in tests. Secondly, some of the students had never tried VR technology before, which affected their focus. They focused more on the actual technology than the subject of learning. However, this issue will not be present if students are able to have access to try the technology before the lesson itself. On the other, cybersickness caused by VR is a separate issue. In this case, the length of the VR training is not very long, so the effect on the quality of training was not that significant.

The second part of the questionnaire focused on closed answers with multiple options. Students could express their opinion on what they considered to be the most important aspects when designing VR training, or which part of the training they found to be the most distracting. The question did not necessarily apply only to study, but also to the students’ overall experience with VR games.

The first question asked students their opinion on the most distracting part of the VR interactive training. As mentioned before, students could consider their opinion on VR games and training as a whole, not just the training provided during this study. The results are shown in Figure 16.

The most frequent answers included incorrect instructions positions, with 36%, and the over usage of pointers, with 32%. During the VR training, some of the students complained that they often had to turn their heads too much to see the instructions while performing the tasks. Moreover, students also found some directional arrows obsolete, highlighting obvious elements of the game whilst blocking the view. Therefore, these results were not surprising. The third most frequent option was the oversaturation of the UI, which can significantly increase the chance of motion sickness.

The second question focused on the student’s opinions regarding the most important parts of VR educational game design (Figure 17). They were able to pick multiple answers out of seven prepared options. The two most frequent answers (90%) were the clarity and relevancy of instructions and the faithfulness of simulation. They were followed by a good combination of types of instructions and feedback system. Most of the students selected faithfulness of simulation and feedback system, which are also important factors for other types of training done outside of the virtual world. Therefore, these answers suggest that students may be considering VR training as a replacement for the other training methods, such as on-site training. Thus, VR training should faithfully replicate the real environment while providing clear and relevant guidance with constructive feedback. This assumption matches with a general view of VR interactive training and educational games.

The third question asked students which type of instruction delivery they found to be the most helpful (Figure 18). Again, they could choose multiple answers. The majority of students chose text, graphical, and directional instructions. During the training, many students claimed that combination a of these three provided the most suitable instruction system. Half of the students did not prefer video and speech instruction. In a post-survey discussion, students claimed that these two options were not optimal, because they are not static and continue to play even if a player cannot keep up. The overall results of this question are shown in Figure 18.

4.3. Evaluation of Hypotheses

At the start of the study, four hypotheses were formed. Obtained results were then used to verify their veracity. The first hypothesis stated that the use of VR interactive training allows more effective knowledge transfer and retention in a short-term application. During the early phases of the testing, the students that used the VR educational game achieved better results than the volunteers in the NoVR group. After the training, they utilized gained knowledge more efficiently, performing tasks quicker and with more confidence. This fact was supported in the questionnaire, where a majority of students agreed that VR interactive training helped them to perform better. From the results, it can be assumed that the first hypothesis is correct. Similar to the first hypothesis, the second hypothesis also deals with the effectiveness of knowledge transfer and retention for VR interactive training, but for long-term application. Even the with largest time gap, the students of the VR group performed better. Significantly more students passed Test 5 without any issues than the students from the NoVR group. They also did so in a shorter time. Educational VR training allowed them to apply their knowledge more effectively even after time passage. Therefore, the results of the study suggest that the second hypothesis is correct. The third hypothesis stated that the use of VR interactive training gives students a better understanding of the taught subject compared to conventional methods, making it easier to integrate gained knowledge into a variable environment during practical application. As mentioned before above, the majority of students stated that they feel that the VR interactive training helped them to grasp the principles of taught tasks better. This was also supported by the overall better results of the VR group. Is also worth noting that during test observation, it could be seen that the students of the VR group spent less time thinking about the next step and their workflow was much more fluent. Their moves seemed more confident and efficient. As a result, the third hypothesis can be considered valid. The last hypothesis stated that the effectiveness of the VR interactive training is heavily influenced by the chosen game mechanics and method of instruction delivery. However, the data obtained from the study were not enough to confidently evaluate this statement. Even though the tests, questionnaire, and dialogue with students strongly suggested that there is a correlation between used elements of VR training and its effectiveness, the obtained data are not enough to state that the correlation is as intense as the hypothesis suggests.

5. Conclusions and Discussion

The effects of VR interactive training on the teaching process, modelling, and simulation are beneficial [25,26,27,28,29,30]. However, like with many other applications, this positive effect is dependent on correct execution. The usage of cutting-edge technology does not guarantee an improvement of a teaching process if the teaching aid is not correctly designed. Moreover, with immersive technology such as VR, the negative effects of a bad design are multiplied, achieving the exact opposite results.

Therefore, before testing the impact of VR interactive training and educational games on the teaching process, it is vital to ensure their correct design. A properly designed virtual training will secure the objective testing environment, where the evaluation of such educational games and training will not be influenced by design flaws. For this reason, the article proposes a methodology for VR game creation. The goal of the methodology is to ensure that the core of the VR interactive training is properly designed. A proper training foundation can be easily expanded upon creating a powerful teaching aid. However, even a small shortcoming of the VR interactive training design can greatly affect the teaching process. In simulating real jobs, students develop habits, which can be then applied to the real world. However, the same applies to situations in which the training is poorly designed, when students can possibly transfer bad habits into real task execution. Therefore, it is vital to ensure that the interactive training design closely follows its real-life counterpart.

The proposed methodology describes the methods and principles for the creation of interactive training using tools of VR. The methodology was used to create virtual training for a real drilling workplace at the Faculty of Mechanical engineering, University of Žilina. Firstly, the process was analyzed and necessary references were collected. Secondly, all required assets were created, such as 3D models, instructions, and a virtual training scenario. Finally, the virtual training itself was created and tested. Subsequently, the training was used as the main part of the study to examine the potential of VR interactive training as a sustainable teaching method.

The study consisted of initial training followed by five tests taking place in different time windows. During the training, one group (VR group) used the VR interactive training as a teaching method while the second group (NoVR group) used the conventional teaching methods. After the last test, the VR group also filled out the questionnaire to gain additional data. For students in the VR group, their error rate and execution times were lower while maintaining higher efficiency.

At the start of the study, four hypotheses were formed. The obtained results are reported in Section 4.2. The first three hypotheses were confirmed: (1) the use of VR interactive training allows more effective knowledge transfer and retention in a short-term application; (2) the use of VR interactive training allows longer knowledge retention and easier knowledge transfer in a long-term application; (3) the use of VR interactive training gives students a better understanding of a taught subject compared to conventional methods, making it easier to integrate gained knowledge into a variable environment during practical application. The fourth hypothesis: (4) the effectiveness of the VR interactive training is heavily influenced by the chosen game mechanics and method of instruction delivery; could not be evaluated with certainty.

In terms of identifying key elements for creating interactive training, the following elements were evaluated as the most important for well-designed interactive training: (1) faithful virtual simulation of the real place and process; (2) clear and relevant instructions; (3) a good combination of text-based and textless instructions. Moreover, these types of instructions were identified as most helpful in interactive training: (1) directional arrows and object highlighting; (2) graphical instructions (images); (3) text instructions. Surprisingly, the spoken word instructions were identified as the least helpful.

Finally, the study also dealt with possible negative effects of VR interactive training. The concern is that the students will be distracted by the novelty and playful approach of the VR interactive training. However, these concerns proved to be unfounded. Even though some students considered novelty and playfulness as a possible source of distraction, the effect on the final results was minimal. Moreover, as students gradually get familiar with VR technology this issue will be minimized even further.

The results and findings of this study will be used to improve the structure of the following studies and to examine the effect of VR virtual training and educational games in more detail. The proposed methodology will be also improved for wider use and more effective workflow. The article aspires to push VR interactive training a little further as a stable teaching method in a sustainable learning environment.