Application of Virtual Reality Method in Aircraft Maintenance Service—Taking Dornier 228 as an Example

Abstract

Flight safety and airlines operation have been at the center of research since aircraft were first invented, as even slight errors in aircraft maintenance may cause serious accidents. Thus, aircraft maintenance is critical to the aviation industry all the time. To prevent maintenance errors, it is important to train for aviation maintenance. Therefore, an aircraft maintenance virtual reality (AMVR) system was developed in this study. For a Dornier-228 aircraft, a walk-around visual inspection of its fuel system was designed and tested in a virtual environment. For the system, CATIA V5 and Unity 3D software were used for designing the 3D model of the aircraft and developing the visual environment, respectively. With the software, the visual environment of the aircraft hangar was created for the system. The developed system was tested by students to validate the effectiveness of using the AMVR system in training. The students acknowledged that the system was beneficial to their learning, which proved that the developed system is highly effective for training students to improve aircraft maintenance skills.

1. Introduction

The aviation industry is the most modernized transportation industry and has become essential for the economy. However, safe operations are always critical, as accidents during flights cause serious damage. Thus, the development of the aviation industry is closely related to that of inspection and maintenance technology. Therefore, it is necessary to train personnel for inspection and maintenance with appropriate tools, and the training methods need to be improved with the intensive development of science and technology. However, the cost of training with adequate facilities is significantly high and also requires a high level of safety. Along with this, a variety of engines for various aircraft require even higher costs for training. Therefore, it becomes critical to find a new approach to replace and improve the traditional training methods.

Virtual reality (VR) is one of the prominent technologies in the era of Industrial Revolution 4.0. As an advanced technology that allows experiences in the virtual and 3-dimensional environment, VR has three important aspects: immersion, interactivity, and imagination [1]. Recently, VR has been widely used in the fields of medical science, military, culture, tourism, and scientific research. VR applications are used for entertainment and education, as well as inspection and maintenance in the design and manufacturing industry.

In this study, we developed an aircraft maintenance virtual reality (AMVR) system for training students in the aviation industry and evaluated the effectiveness of the system. Focusing on the inspection of the aircraft fuel system of the Dornier-228 aircraft, we created three scenarios of inspection and maintenance, including walk-around inspection, disassembling, and re-assembling of the horizontal stabilizer of the aircraft. The developed system provides an effective way to train students with lower costs and higher efficiency.

2. Literature Review

Virtual reality (VR) is a technology that uses 3D modeling to create a virtual environment. In a virtual environment, the user becomes a part of the system. For example, in one of the applications, people move freely in the virtual space and interact with virtual objects. Conversely, the virtual environment reacts or responds to the user’s actions. These effects follow the laws of mathematics and physics, making users feel in the real world.

The interactivity and attractiveness of VR contribute primarily to immersion, providing on-screen actions. VR provides a “realistic” feeling as it involves different human senses. For example, users see floating 3D graphical objects, control objects in a virtual environment, and touch and feel them through controllers or sensor gloves. The most important feature of VR technology is real-time interaction. The user’s interactions in the virtual world allow a similar feeling of the real objects in real-time. VR technology has three critical characteristics, interactive, immersive, and imagination. A VR system consists of five components: software, hardware, network, users, and applications (Figure 1).

The hardware of the VR system includes personal computers (PC) or a workstation with strong graphics configuration and input and output devices. Input devices are capable of stimulating human senses to create a sense of existence in the virtual world. Output devices such as head-mounted displays (HMDs), microphones, and stereo headsets are showing what the user is looking or pointing, with head-trackers and wire gloves. Audio equipment (speakers) and sensory feedback (Haptic feedback such as gloves) create tactile sensations when touching and holding objects. Force feedback devices create impact force such as when cycling, riding on bumpy roads, etc.

The software is the essence of a VR system. In principle, any programming language or graphics software can be used to model and simulate VR objects. For example, languages such as OpenGL, C#, Java3D, VRML, and X3D, or commercial software including WorldToolKit and PeopleShop, are used for the VR system. VR objects are modeled with this software or converted from 3D models that are designed by other computer-aided design (CAD) software such as AutoCAD and 3D Studio. The VR software simulates the object’s kinematics, dynamics, and behavior. VR is applied in many fields of science and technology, including medical technology [2,3], military [4,5], architecture [6], and automotive industry [7], to meet the needs of research, education, and commercial purposes.

The aviation industry is also benefiting from the application of VR. From manufacturing and pilot training to flight experiences, VR enables safer and more efficient testing, training, and flight experiences. In addition, the immersive aspect of VR is used to reduce potential accidents through training. Therefore, the integration of VR into different processes and systems is of great significance to the aviation industry. The most popular application of VR in the aviation industry is pilot and crew training. Flight simulators are popular, even at home. Companies like Boeing have created similar aircraft cockpits to the real ones for training purposes. While effective as a training tool, these simulators are expensive and bulky. With VR, all possible scenarios during flight can be presented in virtual format. Students have opportunities to practice procedures without using real aircraft and airports to improve their skills at reduced costs. For example, Li [8] designed a virtual multimedia platform based on real cabin features. Virtual reality modeling language (VRML) and 3D Max were used to model the 3D model, and X3D and JavaScript were used as programming languages to complete the multimedia teaching system equipment. The virtual intelligence training program allowed students to apply their theoretical knowledge to the operation of virtual scenes with realistic reconstructions and intelligent guidance. In addition to training pilots and flight crews, VR is used to educate passengers about aviation safety.

In the development of virtual environment, Chittaro and Buttussi [9] presented an immersive HMD-based game that allowed passengers to experience multiple aircraft emergencies to show how to survive. At the same time, they compared this educational approach with the traditional method (safety cards) typically used by airlines. The HMD-based game turned out to have more detailed information. Chittaro et. al. [10] used VR techniques to design a mobile VR tool and evaluate the effectiveness of this tool between the safety education tools used by airlines. Sixty-eight participants, aged 20–24, were divided into two groups (app group and card group). The results shown that the app was more engaging than the card. Besides, instructions given by the app were perceived as more effective than those of the card. Moreover, after using two instructional media, participants in the app group were able to don the life preserver in less time than those in the card group. Benthem K. V. and Herdman C. M. [11] examined the efficacy of a virtual reality (VR) cognitive health screening tool (integrated into simulated flight scenarios) in identifying general aviation pilots who experienced a critical incident during flight in a full-scale Cessna 172 simulator. The fifty-one participants were between 17 and 71 years old, and they were divided into two groups: under 50+ years old group and 50+ years old group. During the VR cognitive screening, user-experience queries were incorporated into the screening process to investigate whether older, as compared to younger pilots would experience cybersickness. There was no significant difference in flight simulator preference, with both older and younger participants mostly preferring the VR system over the standard simulator. On the other side, the results also showed better piloting performance in the VR test predicted critical incidents from the quasi-real world flight simulation.

Maintenance is also important in the aviation industry. Using VR as a maintenance training tool provides a safe learning environment that minimizes the risks that may occur during training with significantly reduced costs. Quan et al. [12] proposed a virtual teaching and training system to meet the needs of modern aeronautical engineering education. The method was compared with traditional methods for several factors such as learning efficiency, cost, achievability, and convenience and showed positive results. Shao et al. [13] used HTC VIVE and Leap motion to develop an aircraft virtual equipment maintenance system based on gesture recognition to improve employees’ understanding and maintenance skills. Vu et al. [14] used VR for training smart aviation maintenance training service for aircraft’s preflight check.

In fact, although engineers are trained in a highly secure environment, there is still some training that can be potentially dangerous in hazardous environments, such as practicing in certain fire and explosion situations, inspecting and repairing fuel system components in the dark, and small spaces in the wings. Meanwhile, using VR, engineers only need to be in a secure small area to sit still or move around, but still be able to train in the same situations in the virtual environment. Aircraft models and aircraft parts required for aviation training are often very expensive. In addition, mistakes inevitably occur during training, which can lead to equipment damage. Instead, through VR, engineers will be trained on 3D models, minimizing errors that lead to failure and lowering training costs. Another problem that can be solved is that engineers will have more time and chance to practice, thus increasing the effectiveness of the training program.

3. Research Methodology

As mentioned in the previous research literature, VR can operate and practice in virtual space to provide the conditions and environments that cannot be achieved in general actual environments, such as preventions of explosion and aircraft fire, operation of replacing expensive instruments, and using VR to achieve the purpose of practice and training. An aircraft maintenance virtual reality (AMVR) system [15] was designed with three different scenarios: walk-around inspection, fuel system observation, and horizontal stabilizer maintenance for the Dornier 228. The aviation AMVR system is developed with VR technology, which has the training effect to improve the accuracy of maintenance and can also record the number of operation errors, which is key for guiding the operator to prevent mistakes. From these statistical data, the maintenance procedures can be improved, and the manual operation errors can be reduced. In addition, it also can improve flight safety and the designs of components and products. To design the AMVR system, the creation process of William and Alan [16] has been referred to. They have provided a simple and easy-to-understand workflow and detailed descriptions of the components needed to build VR applications, thus helping new VR developers to easily access and reference when developing VR applications.

The research follows the steps shown in Figure 2.

To generate scenarios and assign interaction functions, multiple software, programming languages, game engines, and 3D modeling tools are required (

Table 1. Commonly used software for VR systems.

Game Engine	Programming Language	3D Modeling Software
Unity [19], Unreal Engine, CryEngine, id Tech, Source, GameMaker Studio, …	C++, C#, Java/Java Script, Python, Swift, …	Blender, CATIA, SolidWorks, Autodesk Maya, Autodesk 3Ds Max, SketchUp, …

). Unity 3D version 2019.3.0f6, C#, and CATIA V5 are used to develop for VR maintenance training system in the current research. VR devices are indispensable for VR applications. To immerse in the virtualized space, users need a headgear with VR glasses. We used HTC VIVE Pro McLaren Limited Edition of HTC [17] and Oculus Quest 2 [18].

4. Results and Discussion

4.1. Virtual Reality System Design

CATIA V5 software was used for creating a 3D model of the fuel system of the Dornier 228 [20,21] as shown in Figure 3. The horizontal elevator and the fuel system were designed for a detailed inspection. The 3D model is imported to Unity 3D software after changing the file type to *.fbx. Then, the preselected software development kits were installed for designing the complete AMVR system. Finally, after completing the design, the AMVR system was evaluated to know whether the system design met the requirements or whether further modifications were required. The results of the evaluation process are shown in Figure 4, Figure 5, Figure 6 and Figure 7. The A questionnaire survey was conducted to evaluate the efficiency of the AMVR before and after students experienced the AMVR system for inspecting the aircraft.

The walk-around inspection included preflight check, subsequent flight check, and daily check. All the maintenance check requirements were performed by maintenance personnel according to the maintenance practices. Small screens were designed to assist students to inspect and understand the operation of various components extensively. The aircraft inspection was carried out for wings, fuselage, propellers, and power supply. During the walk-around inspection, students followed the inspection procedure and filled out a checklist based on instructions provided on the display.

For fuel system inspection, the schematic diagram, location, and the number of components were displayed as images and 3D models, helping students observe the system with HMDs. When maintaining the horizontal stabilizer, students disassembled and reassembled the horizontal stabilizer according to the instructions of the aircraft maintenance manual displayed on the display.

In all three scenes, a user interface object called “Hint” was used to guide how to use and be familiar with the controllers. The other UI object called “Switch Scene” was used to change the training scenes or exit the VR training system, as shown in Figure 7.

4.2. Questionnaire Survey

This study uses virtual reality technology to evaluate the influence on aviation maintenance. Its assumptions are that (A) there are no other external factors in this study, and students are not affected by emotional and physical stress. (B) The test subjects are not affected by gender. (C) Select the first year students of university with a preliminary concept of aircraft as the research objects. Twenty-six students were invited for a questionnaire survey and asked to fill out the survey before and after the training to perform the walk-around inspection of the Dornier 228 with the AMVR system first. The questionnaire survey consisted of 17 questions into five categories: problem organization, problem analysis ability, problem comment ability, problem-solving ability, and teamwork. The collected data were analyzed using SPSS. The mean and standard deviation were calculated to compute the efficiency of the training. The Cronbach’s α [22] was calculated as Equation (1). The results were used to evaluate the reliability of questionnaire and to make consistency. The results are presented in

Table 2. Cronbach’s α of the question in the questionnaire survey.

	Number of Questions	Cronbach’s α
Problem Organization	3	0.828
Problem Analysis Ability	3	0.902
Problem Comment Ability	4	0.902
Problem Solving Ability	4	0.897
Teamwork	3	0.916

.

$$\alpha =\frac{N·\overline{c}}{\overline{v}+\left(N-1\right)·\overline{c}}$$

(1)

where N is the number of the items, $\overline{v}$ is the covariance between the item pairs, and $\overline{c}$ is the average of the variance.

The result shows that the values of Cronbach’s α in all the five groups are higher than the minimum value of 0.8 specified by Nunnally and Bernstein [23] for applied research. In the experimental results and investigations of this paper, there should be some comparisons between the controlled group and the experimental group. The experimental group adopts VR technology to discuss aircraft maintenance technology, while the controlled group adopts the general mode (such as traditional teaching or computer digital images) to discuss the aircraft maintenance technology, and the two groups were compared before and after the test. According to the references [9,10,12], it is pointed out that the learning effect of using VR technology is more attractive, convenient, and will improve learning efficiency than ordinary computer with digital images. In addition, in terms of 3D stereoscopic experience, VR is able to experience 3D space feeling. It can be integrated into the environment and immersed in three-dimension. These VR operation techniques are just in line with the technology required by the aircraft maintenance system. The method of the controlled group cannot attract students’ attention and experience three-dimensional objects. Due to the above reasons, this study does not present the survey of the controlled group, but only conducted a questionnaire survey of the experimental group before and after the test to understand the impact of the use of VR on the aircraft maintenance technology.

The problem organization category consisted of 3 questions to assess the students’ understanding of the problem.

Table 3. Results for the problem organization.

Number	Question	Pre-Test	Post-Test
1	For the Do 228 walk-around inspection, students can find the defects by themselves.	3.52	4.15
2	When trained for the Do 228 walk-around inspection, students can find the required information in the manual.	3.76	4.00
3	When studying walk-around inspection, students can predict possible problems.	3.59	4.27
	Average	3.62	4.14

and Figure 8 show the statistics of the category. For question 1, the AMVR helped the students find the problems by themselves more easily than the traditional training method, showing an increase in the average score from 3.52 to 4.15. For question 3, the score also increased by 0.68, from 3.59 to 4.27, showing a significant change in the predictability of possible problems with the AMVR system. Besides, the standard deviation of 0.67 also showed high consistency among students. The overall problem organization average increased by 0.52, from 3.62 to 4.14, indicating that students’ ability to organize problems was enhanced using the AMVR system.

The problem analysis ability was asked to evaluate the ability to analyze data obtained after students were trained with the AMVR system.

Table 4. The results for the problem analysis ability.

Number	Question	Pre-Test	Post-Test
4	Faced with a complicated aircraft maintenance manual, students can quickly complete the sorting and classification.	3.38	4.15
5	Student can infer information and learn from Do 228 walk-around inspection data.	3.52	4.15
6	Student confidence in the logical thinking ability about the Do 228 walk-around inspection.	3.41	4.12
	Average	3.44	4.14

shows the evaluation questions and scores, and Figure 9 illustrates the results. For question 4, due to the enhanced ability to organize the problem, hence, the score of the ability to analyze and classify the information of the students also improved, from 3.38 to 4.15. The score of the student’s confidence level in logical thinking ability improved by 0.71 from 3.41 to 4.12. The scores for the questions showed a relatively low standard deviation of 0.73 and 0.77, respectively.

For the problem comment ability presented in

Table 5. Results for the problem comment ability.

Number	Question	Pre-Test	Post-Test
7	Students can judge the problems that others have not found.	3.59	4.00
8	Students can think backward to discuss problems.	3.72	4.08
9	Students can share their opinions on the current situation.	3.83	4.15
10	In the discussion of the Do 228 walk-around inspection, they talk about the problems with each other.	3.86	4.19
	Average	3.75	4.11

and Figure 10, a uniform change was observed in the results obtained before and after training with the AMVR system. Students become more active in sharing their opinions and discussing problems encountered with each other after using the AMVR system, as evidenced by an increase in scores by 0.32, from 3.83 to 4.15, in question 9 and by 0.33, from 3.86 to 4.19, in question 10. The score of the student’s ability to think backward also improved from 3.71 to 4.08. Overall, the result shows that the ability in comment of each student improved significantly.

For the problem-solving ability, 4 questions were asked to assess students’ problem-solving ability with the AMVR system to perform the walk-around inspection for the Dornier 228.

Table 6. The results for the problem-solving ability.

Number	Question	Pre-Test	Post-Test
11	When encountering a walk-around inspection problem with the Do 228, students prefer to find information to find solutions.	4.07	4.23
12	In the process of solving a walk-around inspection problem of the Do 228, students are able to formulate a solution strategy and review method.	3.66	4.19
13	In the process of solving a walk-around inspection problem of the Do 228, students monitor their execution progress.	3.72	4.12
14	Students can draw a flowchart of the Do 228 walk-around inspection.	3.34	3.96
	Average	3.70	4.13

and Figure 11 present the questions and the results. In question 14, the ability to draw diagrams for the inspection of the aircraft improved significantly with the average score, increasing by 0.62 from 3.34 to 3.96. This was followed by an improved solution strategy and review methodology (question 12), with the average score increasing from 3.66 to 4.19. The standard deviation of the score of question 11 was 0.71, which shows the consistent influence of the AMVR system on students’ problem-solving abilities. In general, students’ problem-solving ability has improved significantly after using the AMVR system, with the average score changing from 3.7 to 4.13.

The scores of three questions in the teamwork category are shown in

Table 7. Results for the teamwork.

Number	Question	Pre-Test	Post-Test
15	For the study of the Do 228 walk-around inspection, students can understand what others mean.	3.79	4.23
16	For the study of the Do 228 walk-around inspection, students can discuss with others to solve problems together.	4.03	4.15
17	During the study process, students coordinate work or cooperate with others in assignments.	4.03	4.23
	Average	3.95	4.20

and Figure 12. Through an improvement in problem comment ability, it can be seen that students’ ability to work in groups has also improved as well. Follow that, students can solve problems together or cooperate with others in assignments with the improvement score changing slightly from 4.03 to 4.15 (question 16) and from 4.03 to 4.23 (question 17). Moreover, the standard deviation was 0.71 for the score for question 15, which shows the influence of AMVR use on teamwork among students.

In

Table 8. Results of the questionnaire groups.

Questionnaire Groups	Pre-Test	Post-Test
Problem Organization	3.62	4.14
Problem Analysis Ability	3.44	4.14
Problem Comment Ability	3.75	4.11
Problem Solving Ability	3.70	4.13
Teamwork	3.95	4.20
Average	3.69	4.14

, the average score for each category is presented, based on which the effectiveness of the AMVR system is evaluated. The results show that all the students had a definite improvement in each category. The average score before and after training increased from 3.69 to 4.14. The category with the most significant improvement was the problem analysis ability with an increase of 0.70 (from 3.44 to 4.14). The problem organization, problem-solving ability, problem comment ability, and teamwork also improved in an order of the average score. Thereby, the AMVR system helped the students improve their knowledge, ability to learn, cooperation with colleagues, and application of the knowledge in maintenance. At the same time, the innovation of this training method also stimulated students’ interest in learning more than the traditional methods.

To find out the significance of the difference in the knowledge of Dornier 228 and the before and after using the AMVR system, we performed the paired samples t-test [24,25] as shown in

Table 9. Mean, the standard deviation of the scores, and t-test result of the questionnaire survey using AMVR system regarding the Dornier 228 (n = 26).

Number	Pretest		Posttest		t-Value	p-Value
	Mean	Stand-Devi	Mean	Stand-Devi
1	3.52	0.831	4.15	0.675	−2.961 **	0.007
2	3.76	0.827	4.00	0.748	−1.272	0.215
3	3.59	0.905	4.27	0.667	−2.979 **	0.006
4	3.38	0.862	4.15	0.732	−3.453 **	0.002
5	3.52	0.831	4.15	0.784	−3.048 **	0.005
6	3.41	0.912	4.12	0.766	−3.333 **	0.003
7	3.59	0.869	4.00	0.894	−1.726	0.097
8	3.72	0.797	4.08	0.845	−2.282 *	0.031
9	3.83	0.711	4.15	0.784	−1.786	0.086
10	3.86	0.883	4.19	0.749	−1.841	0.078
11	4.07	0.721	4.23	0.710	−1.044	0.306
12	3.66	0.862	4.19	0.749	−2.981 **	0.006
13	3.72	0.788	4.12	0.766	−2.026 *	0.054
14	3.34	1.014	3.96	0.774	−2.685 *	0.013
15	3.79	0.824	4.23	0.710	−2.388 *	0.025
16	4.03	0.728	4.15	0.784	−0.778	0.444
17	4.03	0.824	4.23	0.765	−1.000	0.327

* p < 0.05 ** p < 0.01.

. The scores of questions 1, 3, 4, 5, 6, 8, 12, 13, 14, and 15 show significant differences before and after using the AMVR training system, with p values less than 0.05. The scores of questions 2 and 7 do not show significant differences with p values greater than 0.05. However, it seems that the students have improved their knowledge with the help of the PBL method to a certain extent. The pretest scores of questions 9, 10, 11, 16, and 17 greater than 3.83. indicate that students have problem comment ability and understand teamwork.

5. Conclusions

An aircraft maintenance virtual reality (AMVR) system was developed for educational and training purposes in the aviation industry. CATIA V5 software was used for developing the 3D model of the Dornier 228 aircraft. Unity 3D software is also used for designing the virtual environment in which the aircraft’s 3D model was imported. Walk-around inspection, fuel system introduction, and horizontal elevator assembling/disassembling were included in the AMVR system.

The developed AMVR system was applied to training students and evaluated by them. Questionnaire surveys were conducted to analyze the effectiveness of the developed system for 26 students before and after training them with the AMVR system. The survey results show that the application of the AMVR helped help students improve their learning capability. After using the AMVR system, each student’s ability to organize, analyze, comment, solve, and work in groups improved significantly when compared to the pre-survey result. Thus, there is a great potential of the AMVR after using AMVR to perform the Do 228 walk-around inspection in the aviation industry.

However, there is still room for improving the user experience, such as the resolution of the application and reflecting the user’s movement in the VR environment to enhance the interaction between users and 3D objects. It is also required to upgrade the virtual environment to make it more realistic. In addition, the hand-tracking technology on VR devices is still incomplete, leading to a subpar user experience.