Development of a Virtual Reality Tool for Train Crew Training

Abstract

(1) Background: The article presents the results of research carried out as part of one of the project’s tasks, aiming to develop training scenarios for selected workstations related to the operation of passenger rail transport. The article aims to present the results of a training experiment to verify the effectiveness of the developed training scenario for conductors regarding lift operations for people with disabilities. (2) Methods: The study used an analysis of the execution times of individual training tasks, accompanying observation, face-to-face interviews, and a comparative analysis of the effectiveness of various training strategies. The research was carried out following the developed five-stage research procedure. (3) Results: The obtained research results confirm the benefits of using virtual reality in training conductors and allow for determining the most effective training strategy. The measurements of the execution times of individual training operations made it possible to identify the tasks in the training program that were the most time-consuming or characterized by highly diversified execution times. (4) Conclusions: Based on the research, it was possible to develop recommendations regarding the changes that should be introduced in the training tool to better adapt to the real conditions of service operations. The effectiveness of using this form of training in relation to the improvement of a selected group of employees was confirmed.

1. Introduction

The current employee market in Poland results in a high turnover of employees in many sectors. The employee turnover rate in 2022 in Poland was 21% [1], but in the railway sector, this level is lower due to task specificity. However, it should be noted that we can also observe an increased turnover rate compared to previous years in this transport branch, which is a challenge in the current management of railway personnel. Increased rotation causes an increased risk of human errors in service processes. Every newly hired employee needs to be prepared each time in order to perform their duties. An employee starting work in a new position is usually characterized by a lack of skill and limited knowledge of the tasks performed. They also do not have a sufficiently large experience curve (number of repetitions of a given operation), which determines the speed and precision of performed actions. For this reason, such an employee is not only exposed to making mistakes but also the work he/she provides is characterized by extended execution times. It is also worth noting the research on risk assessments for rail freight transport operations, which is presented in [2]. According to them, the majority of adverse events recorded in the selected rail transport system result from errors and the negligence of crew members.

Many researchers and practitioners point to the need for changes in training systems for new employees and training for experienced employees [3]. Solutions are sought to allow employees to acquire and improve the required competencies related to their daily duties faster and more effectively. Traditional forms of training are currently rated low by both participants and employers, who point to their low effectiveness in shaping practical skills and the extended time to acquire knowledge [4]. The preferred employee training system is the active teaching form, which enables employees to achieve the required knowledge and improve their practice skills. The answer to these needs is virtual reality (VR) technology, which enables a faithful representation of the working environment of operational employees. The developed training scenarios allow trainees to practically participate in the supported processes without exposing them to damage and disruptions. Thanks to training tools developed in this way, it is possible to reduce the occurrence of adverse events in which employees are the source of risks. For this reason, VR technology is widely used in various sectors, including rail transport. It should be noted, however, that the number of publications on the use of VR technology in the training of railway personnel is very limited. Searching the database by keys “VR” AND “rail” OR “virtual reality” AND “rail” only returned 85 references. Among them, only eight items concerned the direct use of VR technology in training people to operate rail transport. These publications primarily discussed the use of VR technology to prepare personnel serving passengers in the evacuation process (among others: [5,6,7]), training in the use of railway signaling systems [8], and maintenance training [9]. Only one publication concerned the use of VR technology in rail transit personnel training [10], but this was directed to managers and maintenance personnel. It should be stated that there are currently no publications on the use of immersive virtual reality for training passenger-serving staff on trains, especially with respect to conductors. The application area of VR technology defined in this way currently comprises a research gap, which is filled by the research presented in this article.

The article presents the results of research carried out as part of one of the project’s tasks, aiming to develop training scenarios for selected workstations related to the operation of passenger rail transport. These scenarios were developed based on accompanying observations, direct interviews with railway personnel and instructors, an analysis of the managerial staff’s needs, and analyses of the risk of errors made by employees employed at the examined workstations. The article aims to present the results of a training experiment conducted on a group of 57 volunteers and a proposal for a strategy for training based on VR tools with the support of an instructor. The conducted training experiment aimed at verifying the effectiveness of the developed training scenario for conductors regarding lift operations for people with disabilities. It became the basis for the modifications introduced in the developed VR tool. The structure of the article is shown in Figure 1.

2. Literature Review

2.1. Immersive Virtual Reality (IVR)

Virtual reality is an artificially created digital environment, and its task is to reproduce real life. Two attributes characterize this environment:

• “immersion”—means an objective level of sensory fidelity that a VR system provides [11];
• “presence”—refers to the user’s subjective experience resulting from being in the immersive environment [12].

However, some authors (e.g., [13,14]) indicate that one more attribute should be added to the basic features of VR technology—“interactivity”, which Steuer [15] defined as the degree to which a user can modify the VR environment in real time.

The level of immersion determines the extent to which high-fidelity physical stimuli (e.g., light patterns, sound waves) are delivered to various senses (sight, hearing, and touch) to enhance the illusion of reality [16]. For this reason, in many publications, VR technology is divided into two major types based on the level of interaction and immersive environment [16,17,18]: (1) non-immersive virtual environment (desktop VR) and (2) immersive virtual environment (IVE). In a non-immersive virtual environment, the training simulation is presented on a conventional personal computer and is typically explored using a keyboard, mouse, wand, joystick, or touchscreen. This is the simplest type of virtual reality application. The training is presented on multiple room-size screens or a stereoscopic, head-mounted display unit in an immersive virtual environment. Audio, haptic, and sensory interfaces may enhance these systems. In his research, Mandal distinguished the third group of VR technologies, which is an intermediate between the abovementioned groups. He referred to it as semi-immersive (fish tank VR) systems [16]. These systems support head tracking and improve the feeling of “being there” thanks to the motion parallax effect. They still use a conventional monitor but generally do not support sensory output.

The “Immersive training system for railway personnel using virtual reality technology” project aims to develop training solutions using IVR technology. IVR tools provide a seamless, scalable environment capable of simulating a full range of sensory stimuli and allowing the user to experience 360° visual immersion and manipulate virtual objects in real time [18,19]. For this reason, some authors emphasize that the three main features provided by IVR are as follows [11]: the sense of immersion, real-time interaction, and imagination. IVR in the training system allows isolating the participant from external cues and uncontrolled stimuli from his/her physical word. This allows the participant to experience immersive and interactive scenery and interact with objects and tools in a desirable manner [20]. However, attention should be paid to the results of studies by Janssen et al. [21], according to which learners with certain individual traits and characteristics will benefit less from learning in VR (e.g., people with more anxious or reserved personalities). In their research, Jensen and Konradsen [12] proved that immersive HMD learners were more engaged; spent more time on learning tasks; and acquired better cognitive, psychomotor, and affective skills. However, they also noted that the graphical quality of VR and the awareness when using VR, for instance, can reduce the sense of presence.

Some authors classify the phenomenon of immersion used in VR solutions. Two classifications presented in Figure 2 deserve attention here.

Adams distinguished three dimensions of immersion in his research: tactical, strategic, and narrative immersion. A similar classification was proposed by Björk and Holopainen, who used the term sensory-motoric, cognitive, and emotional immersion for analogous dimensions. In addition, these authors supplemented the classification with three additional dimensions marked in yellow in Figure 2. Thanks to this, immersion provided in a virtual environment can be considered according to six proposed impact areas on the user.

2.2. Use of Virtual Reality in Training

The use of VR technology in training programs is a response to the limitations of traditional training forms. Among the main limitations of real-world training, the authors usually mention the following [4]:

• It could be time-consuming due to the efforts and time needed to set up the real-world training site and to travel to the site;
• It could be expensive due to the cost of preparing real-world training materials and hiring human coaches;
• It could be unappealing and unintuitive due to the lack of visual hints, such as 3D animations for illustrating skills and processes;
• It could not be possible to train some skills in the real world, such as emergency procedures that can only be safely trained in simulators.

Although VR is not new, recent developments in immersive technologies—in terms of visualization and interactions—have made VR increasingly attractive to scholars [20]. Virtual reality is now widely used in various types of training for multiple sectors of the economy. Rogers [24], in his article in Forbes, referred to it as “the learning aid of the 21st century”, and the results of research, among others, Krokos et al. [25], suggest that students retain more information and can better apply what they had learned after participating in VR exercises. For this reason, an important area of scientific and practical importance is currently the improvement of the so-called “industrial skills” via virtual reality training tools. According to Radhakrishnan [26], these skills include four groups of competencies, which are presented in Figure 3.

As noted by [27], training in industrial skills is critical to nations’ economic development and competitiveness. Their effectiveness reduces the number of errors made by industrial workers, which can have catastrophic consequences for the product and total production costs [28]. Meanwhile, performance metrics used in VR tools make it possible to identify mistakes made by the trainee, and on this basis, their weaknesses are indicated [18]. Thanks to this, it is possible to adjust the learning parameters to the needs of the training participants so that they can deliberately practice critical skills and increase exposure to a wide variety of both common and low-volume cases [29]. At the same time, VR tools provide a so-called safe training environment with minimal exposure to dangerous situations (e.g., fires [30]; explosions, and natural disasters [31]). For this reason, in the initial development phase, VR technology was primarily used to improve KSAOs, for which training processes were associated with the risk of loss of the health or life of training participants or very high implementation costs [32]. Therefore, the first applications of VR refer to the training of pilots, paratroopers, firefighters, and military personnel [32].

It is worth noting that training tools using VR technology meet the requirements for an effective learning process today, which include the following [33]: focus, meaningful representation of information, multiple mappings of information, and reflective learning. At the same time, currently created virtual reality environments (VREs) have unique contributions to learning scientific visualization, instructions, sensory-motor performance, and training [34]. Xie et al. point to two crucial aspects of the use of VR technology in the training process [4]:

• The possibility of using an increased number of training scenarios while reducing training costs: This is because VR training scenarios primarily involve computer-generated 3D graphics. Developers can, therefore, create various scenarios from existing 3D resources (both proprietary and available online) that can be reused to train different people.
• The possibility of training in the comfort of one’s own space: This is especially important in cases where the trainee may feel uncomfortable about their actions in the real-world training process due to the presence of other participants (observers). At the same time, the same tool can be used in training situations that need to happen in the presence of instructors as they need to provide early feedback on the trainee’s performance and alert the trainee to issues that they may be experiencing, including negative trends in their performance.

The increasing popularity of VR technology is positively influenced by the advantages of this solution, which are identified in numerous experiments described in the literature. The key benefits offered by VR technology in the field of employee training are usually indicated [35]:

• Increased involvement in training—immersive environments and 3D representations make users more interested and committed to their training;
• Speed—workers can obtain and retain information efficiently due to the creative and attractive method that this digital tool offers. Many studies emphasize that VR allows for training through emotional responses, the best method to not forget what has been learned;
• Measurability—training is carried out in computer-generated environments; thus, researchers can quickly obtain any required statistics, such as the time it takes an operator to carry out a particular activity, the most common accidents, or the procedure with less difficulty;
• Reduction in work accidents—VR offers a much safer scenario where users can practice obtaining the necessary skills without risking their integrity;
• Personalization—scenarios can adapt to the requirements of each worker;
• Reduction in costs—VR helps to reduce infrastructure, materials, time, and personnel expenses that are required by real simulations.

When assessing the effectiveness of VR training, it is worth considering the results of the literature review in this area, which was prepared by Strojny and Dużmańska-Misiarczyk [36]. In their research, the authors emphasize that the effectiveness of VR training can be assessed in many ways. The analysis of the publication proves the following [36]:

• The most common approach involved using some objective method (e.g., knowledge test): such methods were applied in 82% of the reviewed studies;
• The subjective evaluation of the training tool, learning outcomes, motivation, or other psychological constructs was also used in 72% of the studies;
• Observation by an expert as a form of assessment of learning effectiveness was used sparingly (6%). It was primarily used where objective assessment methods could not be applied or for the qualitative evaluation of learners’ reactions during learning;
• Physiological measurements were the least frequently applied (3%).

The review’s authors also emphasize that more than one method was used in more than half of the analyzed publications; in particular, combining methods from two different categories was the most popular choice. Only 15 studies combined three different methods (this accounted for about 5% of the researched publications). In the remaining analyses, only one selected effectiveness assessment method was used.

Due to VR technology’s opportunities, it is currently used in training programs for many industrial and service sectors. Xie et al. [4] reviewed a total of 48 VR training applications and works. Based on their research, the dominant areas using VR in the conducted training can be distinguished. They are as follows [4]:

• First responder training (e.g., police officers, firefighters, and emergency medical services);
• Medical training;
• Military training;
• Transportation sector;
• Workforce training;
• Interpersonal skills training (in particular, openness, empathy, verbal, and nonverbal communication).

However, it should be clearly emphasized that these are not the only sectors that currently use VR tools in training.

3. Methodology

The project task concerned verifying the developed training tool that uses IVR technology and offers a dedicated elevator operation scenario for people with disabilities by railway staff (conductors). The study involved 53 volunteers who had not previously operated the device. They were also informed that the obtained results would be used for scientific purposes (including publications) and to improve the developed training program.

The research carried out as part of the presented project task aimed to answer four research questions. Therefore, six research tasks were set, the implementation of which made it possible to answer the formulated questions. The research questions and developed tasks are presented in Figure 4.

The entire experiment was carried out following the developed research methodology, which consisted of five stages of the procedure. The research procedure with the assigned tasks is presented in Figure 5.

In the conducted research, the project team was supported by four professional instructors who completed training for conductors at a selected railway carrier. The experiment was carried out in a specially prepared training room. The first stage of the investigation was to introduce the IVR tool to the course participants and discuss all the activities that should be performed with the instructor during the entire training period according to the training scenario. The instructor confirmed the understanding of all instructions by the course participants after discussing the guidelines for the implementation of the training. The training was carried out in several training groups so that it was possible to verify various training strategies.

All performed tasks were measured during the implementation of individual operations by the training participants. The measurement was carried out traditionally outside the virtual environment. Members of the research team measured each trainee individually. The analysis of the duration of the operation by the trainees was based primarily on three statistical parameters: arithmetic mean, standard deviation, and coefficient of variation. The calculated arithmetic average for each activity formed the basis for estimating the time-consuming execution of each procedural step. On this basis, activities with long execution times and that suggested a high degree of difficulty in their implementation were identified. The calculated standard deviation formed the basis for identifying discrepancies in the execution times of individual participants. The coefficient of variation is calculated based on the ratio of the standard deviation to the arithmetic mean. This measure provided the basis for identifying operations for which their performance depends on the skills and psycho-physical characteristics of the trainee.

Interviews with trainees and instructors were conducted by the members of the project team immediately after the end of the training. The survey took the form of direct, unstructured interviews. The study aimed to collect participants’ opinions about their experiences (emotions) during the training, opinions about the training tool, and the identification of potential for improvement.

Training strategies were developed based on literature studies and direct observations. Two methods were used to assess the effectiveness of the developed training program—a practical test carried out on a real object and an accompanying observation carried out by two experts (a member of the project team and an experienced instructor). On this basis, a static analysis was carried out using time measurements collected during the conducted observation.

4. Results

4.1. Scope of Training Using Virtual Reality

One of the training scenarios developed within the framework of the project was the operation of an elevator for people with disabilities, which is used in Polish long-distance train vehicles made by Stadler. This training is significant for two reasons: (1) because of the level of service offered to passengers with special needs, particularly the time it takes to handle them during the exchange of passengers on the platform, and (2) traditional training requires temporarily taking the vehicle out of service, which is usually difficult to correlate with the timing of the training. Thus, this operation requires quick and error-free execution by the conductor. At the same time, it is not a standard operation and is often repeated during the course of their duties, so new personnel do not have the opportunity to develop a routine. In the training process, the problem is ensuring the availability of a vehicle equipped with such an elevator and out-of-current operation, allowing many repetitions to be performed without time constraints. Conducting this training using a virtual reality tool provides an opportunity for the trainee to perform multiple repetitions, allowing him to gain experience and proficiency in performing this operation.

The study of the fit and effectiveness of the developed training scenario was conducted using an experiment with 53 volunteers. The group of volunteers consisted of people aged 22–29 who want to pursue the conductor profession in the future, as well as people interested in rail transportation. Approx. 28% of those surveyed indicated previous experience with virtual reality, while 69% said they regularly play or have played computer games.

In the experiment, participants were tasked with the nine activities illustrated in

Table 1. Milestone activities performed by the participants during the experiments in the VR tool.

Symbol	Activities	Visualization
A1	Taking a board with the description of the procedure and carrying it under the train
A2	Opening a tray and retrieving the square key
A3	Opening the lid and lever, storing the key in the tray, and opening the entire elevator
A4	Unfolding the platform and wings of the elevator and securing the carriage from rolling away from the side of the carriage
A5	Lowering the elevator to the station platform level using the remote control
A6	Raising the elevator by remote control
A7	Lowering the elevator hydraulically using two valves
A8	Elevator raising with manual lever
A9	Closing the elevator and exiting the train

.

The first activity, i.e., moving the board, verifies the ability to observe the virtual reality tool in terms of grasping, moving, and orienting. These skills are required in further stages of training; therefore, their verification in the initial activity is essential. Subsequent activities already result from the elevator operation procedure. This procedure describes in detail the next steps of the procedure regarding the preparation of the device for operation (activities A2–A4), the ongoing operation of the device during different modes of operation (activities A5–A8), and the termination of the device (activity A9). It is also worth noting that the virtual reality training element includes procedures for securing the correct operation of a person with a disability (under activity A4).

4.2. Analysis of the Processing Times of Training Activities

In the first stage of the study, the execution time of contracted operations was measured using a virtual training tool. This measurement was to determine the following:

• The time intensity of the execution of individual operations;
• Identification of activities with high variation in execution times, which may indicate a strong dependence of the execution of the operation on the predisposition of the trainee;
• Identification of activities with low variation in execution times but a high parameter of average execution time—identification of activities that are difficult in virtual reality, regardless of the predisposition of trainees.

To carry out the research task defined in this way, the obtained time measurements were subjected to quantitative analysis based on three parameters: arithmetic mean, standard deviation, and coefficient of variation.

Table 2. Operations performed within a training iteration.

Symbol	Activity	Average (min)	St. Dev. (min)	Variation Coef. (-)
A4	Unfolding the platform and wings of the elevator and securing the carriage from rolling away from the side of the carriage	03:22	03:56	1.17
A9	Closing the elevator and exiting the train	03:35	03:25	0.95
A7	Lowering the elevator hydraulically using two valves	02:16	01:50	0.81
A2	Opening a tray and retrieving the square key	01:47	01:23	0.77
A5	Lowering the elevator to the station platform level using the remote control	01:19	01:00	0.76
A3	Opening the lid and lever, storing the key in the tray, and opening the entire elevator	03:15	02:27	0.75
A6	Raising the elevator by remote control	01:14	00:54	0.73
A8	Elevator raising with manual lever	02:24	01:29	0.62
A1	Taking a board with the description of the procedure and carrying it under the train	00:33	00:13	0.41

shows the obtained results arranged in descending order according to the coefficient of variation.

The analysis of the most time-consuming execution of the indicated handling operations by individual participants in the experiment was aimed at identifying which operations cause the most difficulty in execution and whether the identified difficulties are due to the lack of skill relative to trainees, the incorrect representation of reality in the virtual environment, or the complexity of the operation itself.

Therefore, an additional analysis was carried out on the impact of a trainee’s experience on the time he or she takes to perform specific tasks. For this purpose, data from a pre-training survey of participants were used, in which they specified, among other things, their experience of regularly playing computer games and virtual reality.

As can be observed in

Table 3. Comparison of times for completing a full training iteration by people with different experiences using computer games and VR tools.

Comparing	Statistics of VR Tool Use
	Average (min)	St. Dev. (min)	Variation Coef. (-)
VR and gaming experience	11:42	02:11	0.19
VR but no gaming experience	11:46	04:58	0.42
No VR but gaming experience	17:11	07:03	0.41
No VR and no gaming experience	18:47	05:38	0.30

, participants with experience using VR tools obtained significantly shorter execution times per iteration of the training cycle, even if they did not play regularly on computers. In contrast, the times are significantly longer for participants using VR tools without experience. For those who additionally do not use computer games, the average time to complete a full training iteration increased by 7 min.

The most time-consuming operations were found to be A2 (03:35), A1 (03:22), and A6 (03:15). Two of them (A1 and A2) also have the highest variance in their performance by trainees. In the case of activity A1, the value of the coefficient of variation was higher than 1, which means that in the group of volunteers, there were people who performed this activity in a relatively short period of time. At the same time, some people could not cope with the activity. This could mean that the activity was mapped incorrectly in the virtual environment and requires a certain correction.

Activities that required in-depth qualitative analyses were singled out based on the analysis of the variability index’s value. These activities included the following:

• Unfolding the platform and elevator wings;
• Closing the elevator and exiting the train;
• Lowering the elevator hydraulically using two valves;
• Opening the lid and lever, stowing the key in the tray, and opening the entire elevator.

Two research methods were used for the qualitative analysis: (1) accompanying observation and (2) unstructured face-to-face interviews with training participants. Based on the collected results from both research paths, inferences were prepared regarding the possibilities for improving the developed training tool. Noteworthy among them are the following recommendations:

• Introducing a uniform way of locating the square-type key for closed/open positions: e.g., a key in the vertical position means that the lock in question is open, and the horizontal position means it is closed;
• Verifying the correct direction of turning the square-type key to open/close locks;
• Entering and exiting the carriage smoothly and not jumping with the entry button;
• Opening/closing of doors should be implemented, considering the operation of buttons;
• The functionality of the door locking lock should be reproduced;
• The elevator’s location and immediate surroundings should correspond to the location of the actual vehicle;
• The buttons for opening/closing the doors and the handle for opening the elevator underneath them should be placed a little higher;
• The visible edge of the elevator cover should be added so that the edge hints at where to grab it;
• On the front of the elevator platform, there is a safety device that automatically lowers upon contact with the platform; its functionality should be recreated;
• The step under the door should extend and retract, adhering to reality;
• The ergonomics of unscrewing the valves for manually lowering the elevator needs improvement.

Points (1), (2), (6), and (11) are seemingly minor recommendations. However, observations made during training and an exam on a real device indicated that their current representation in virtual reality could perpetuate behaviors that hinder work on a real object. An example is recommendations (1) and (2), which refer to the habit of turning the key. It is also worth noting that recommendation (11) was created in response to the surprise among trainees caused by the functionality of the real valves compared to the virtual model.

4.3. Fitting a Training Strategy

In addition to a virtual tool to support the didactic process, developing an appropriate training strategy is necessary to allow participants to acquire the required competencies actively. For this reason, four possible training strategies were analyzed:

• Strategy 1 (ST1)—individual guidance of one trainee by one instructor—the instructor explains step by step what to do;
• Strategy 2 (ST2)—the trainee uses a storyboard describing the steps after a prior observation of the instructor’s demonstration—instructor support is limited;
• Strategy 3 (ST3)—instructor adaptively supports on request—people observed once by the instructor and once by another participant;
• Strategy 4 (ST4)—instructor adaptively supports on request—individuals observed the instructor once and another participant twice.

However, it should be noted that in each of the strategies highlighted above, the instructor discussed all the activities to be performed in the task realized in virtual reality (the group had a preview on a large screen) Once. For each strategy, an analysis was carried out on the execution times of all training activities in virtual reality. For this purpose, all volunteers were divided into four groups, and each group underwent training according to a different strategy.

It can be observed that there is a variation in the average times of completing activities using the virtual environment, as well as in the values of the standard deviation. Therefore, the following hypotheses can be put forward:

• The execution time of the first iteration of the tasks is longer if the trainee is to self-learn from the available materials and after only one observation of the activity;
• The time to complete the first iteration of the virtual exercise decreases with the number of observed iterations, and after two iterations of the observation of other trainees and one instructor, it reaches the level of individual guidance of one trainee by one instructor;
• The most time-disadvantageous case is to have trainees perform the exercise on their own after a single observation by the instructor (only with the support of training materials and limited assistance from the instructor);
• The variability in completion times as expressed by the standard deviation decreases most significantly after a single observation of other trainees; therefore, the prediction of exercise completion times becomes more precise at a more advanced stage of training.

The above hypotheses were confirmed statistically. Confirmation was obtained by constructing appropriate statistical parametric tests for the equality of the means of general populations and the equality of the standard deviations of general populations. Confirmatory results are shown in

Table 4. Comparison of training strategies.

Statistical Parameters	ST1	ST2	ST3	ST4
Group	1	2	3	4
Sample size	12	13	12	11
Mean (mm: ss)	14:05	19:26	15:44	13:50
Std. deviation (mm: ss)	08:57	08:16	01:55	01:39
Hypothesis H0	${\mu }_1={\mu }_2$	${\mu }_2={\mu }_3$	${\mu }_3={\mu }_4$	${\mu }_1={\mu }_4$
t-statistics	1.49	1.45	2.42	0.09
Reject of H0?	yes	yes	yes	no
Critical t-quantile value	1.32
Hypothesis H0	${\sigma }_1={\sigma }_2$	${\sigma }_2={\sigma }_3$	${\sigma }_3={\sigma }_4$	${\sigma }_1={\sigma }_4$
F-statistics	1.18	18.45	1.33	28.93
Critical F-quantile value	2.22
Reject of H0?	no	yes	no	yes

.

Using observations made during the experiments, it was found that with successive iterations of the exercise, which other participants observe, a particular awareness of the activities to be performed develops. Therefore, increasing the group supervised by one instructor is possible with successive iterations. The maximal number of trainees supervised by one instructor is limited to three and optimistically to four. This is because it is necessary to keep in mind the required availability of an instructor for a given trainee in terms of content and the safety of operating the virtual environment. The balanced scenario indicated in

Table 5. Proposal of a virtual reality training structure.

Stage	Instructor 1	Active Participants	Passive Participants	Estimated Net Time = Mean + Std. Deviation (min)
1	Demonstrates the operation of equipment and performs exercise activities in virtual reality	none	They observe, together on a large screen, the instructor’s virtual activity and the instructor’s physical activity	5
2	The instructor guides one trainee by explaining the tasks to him step by step	1—conducted individually by the instructor	They observe, together on a large screen, the virtual activity of the trainee, ask questions, and listen to concerns	23
3	Instructor adaptively supports participants in virtual reality, prompts upon request	2—conducted adaptively	They observe, divided into two subgroups, the virtual activity of one of the two trainees on dedicated screens, asking questions and listening to concerns	17
4	Instructor adaptively supports participants in virtual reality, prompts upon request	3–4—conducted adaptively	They observe, with divisions into three/four subgroups, each virtual activity of one of the trainees on dedicated screens, ask questions, and listen to concerns	14
5	Instructor adaptively supports participants in virtual reality, prompts upon request	4—conducted adaptively	They observe, with a division of four subgroups, each virtual activity of one of the trainees on dedicated screens, ask questions, and listen to concerns	14
			Total training time for ten participants (min)	73

shows the feasibility of an elevator exercise for a person with disabilities by one instructor for 10 trainees within a (net training) time period of about 73 min (average times were increased by the value of the standard deviation).

5. Discussion

The results presented in the article were to verify the effectiveness of using a training program for conductors using VR technology and to answer four research questions formulated by project team members and presented in Section 3, “Methodology”. Figure 6 shows the relationship between the results of the completed research tasks and the answers to the questions posed.

The measurements of the execution times of individual training operations made it possible to identify the tasks in the training program that were the most time-consuming or characterized by highly diversified execution times. A significant time commitment could result from three premises: (1) a high level of precision or complexity in performing a given task, (2) an incorrect representation of reality in a virtual environment, and (3) training participants’ predispositions. For this reason, it was necessary to conduct additional interviews with trainees and instructors. Thanks to these interviews, it was possible to identify the reasons for the recorded times of completing individual tasks. On the one hand, these interviews confirmed that the trainee’s personal characteristics and skills may affect the trainee’s level of immersion and the times of the operations performed. This result is consistent with the results reported by Janssen et al. [21]. On the other hand, it was possible to identify elements of the virtual world for which their mapping required improvement. The potential for further improvement of the developed training tool was also formulated.

An important element of the conducted research was the assessment of the effectiveness of the developed scenario and training tool. The experiment used two methods of evaluating training effectiveness, as in most studies described in the publications analyzed by Strojny and Dużmańska-Misiarczyk [36]. In our case, these were accompanying observations conducted by two experts, and the verification of skills was carried out based on a practical test conducted on a real object. In this way, it was possible to assess the level of participants’ skills and identify the current limitations of the prepared VR training tool. Thanks to this, it was possible to introduce improvements to increase trainees’ immersion level and the effectiveness of training scenarios.

Based on the interviews and accompanying observations, the research team members determined the benefits of using the VR tool in the developed training scenarios. Most of them were consistent with the benefits described by Naranjo [35], namely the following:

• Increased activity of participants during the training and increased involvement in the implementation of the tasks assigned to them;
• Greater emotions accompanying the implementation of tasks, which had a positive impact on remembering the scope of performed operations;
• Ability to adapt scenarios to the needs of training participants.

At the same time, developing scenarios in a virtual environment allowed the development of many training scenarios and offered a comfortable form of training. As part of the experiment, the trainees performed tasks under the supervision of an instructor in a training room. However, the developed tool can also be used individually in a safe and comfortable space created by the user. This aligns with the positive aspects of VR highlighted by Xie et al. [4]. What primarily speaks in favor of the use of virtual reality in the case of the analyzed scenarios of elevator operations for people with disabilities is the lack of the need to shut down authentic wagons for training purposes or the implementation of training during breaks taken when disposing of these wagons for. Training participants also praised the possibility of making mistakes, based upon which they learned the correct behavior. In their statements, they emphasized the comfort of not having health risks and the pressure related to the costs of damage caused by wrong decisions. Researchers also identified boundary conditions for the training of train crew members by the Polish railway carriers. The average group of trainees is about ten to sixteen people. Thus, strategies for training were also analyzed. As a result, researchers found that one teacher can effectively use four VR goggles to teach fourteen students about lift operations. The time effort is half the time spent on a real object.

The main limitation of the conducted research is the age of the training participants. All volunteers were no more than 30 years old. The experiment confirms the effectiveness of VR technology training among young people with no or minimal professional experience. However, most have experience in games based on virtual reality. Nevertheless, the developed training tool should be used for people of different ages. Meanwhile, using VR tools can be a problem for senior staff. Therefore, the research should be supplemented with an experiment in which people aged 35–50 and 50–60 would also participate. Thanks to this, it will be possible to assess the effectiveness of the developed tool for various groups of railway personnel.

6. Conclusions

The research described in the article presents the results of verifying a training tool using IVR technology, which was developed as part of the project “Immersive training system for railway personnel using virtual reality technology”. The research confirmed the higher efficiency of using VR tools in conductor training than traditional training forms. Training participants showed more significant involvement in the performance of training tasks and remembered the sequence of operations that had to be performed faster. Participants primarily emphasized the comfort of training in virtual reality, which was possible thanks to providing an educational function for mistakes made by participants. Critical results also concerned recommendations regarding changes to be made to both the scenario and the training tool. The introduced modifications and the developed training strategy will increase the effectiveness of the training conducted using the developed solution.

The results obtained in the research are of scientific importance and constitute the answer to the formulated research questions. The obtained results confirmed the benefits of using VR tools described by other researchers. It is worth noting that the innovation, in this case, was the development of IVR scenarios dedicated to the training of conductors. So far, such a solution has not been described in the literature. At the same time, various training strategies were verified, and on this basis, the most effective solution was selected. The practical contribution of the presented results is also important. Based on the conducted research, it was possible to develop recommendations regarding the changes that should be introduced in the developed training tool to better adapt to the real conditions of service operations. At the same time, the effectiveness of using this form of training concerning improving a selected group of employees was confirmed. Operators of passenger rail transport can, therefore, use the developed scenario and training tool.

As noted in Section 5, a limitation of the experiment conducted was the age of the volunteers who participated in the training. They were young people and were most familiar with the virtual reality environment. However, it is necessary to conduct a similar experiment on a group of people over 35. In this way, it will be possible to identify new potential for improving the developed tool and indicate recommended modifications. The results obtained in this age group will also allow for assessing the effectiveness of VR training in various groups of railway employees.