Towards a More Socially Sustainable Advanced Pilot Training by Integrating Wearable Augmented Reality Devices

Abstract

For flying all types of turbine-engine airplanes, a pilot must undergo an intense type rating (TR) course. This study investigated the learning conditions and TR course content, and specifies the most difficult course elements that could be tackled by augmented reality (AR) applications. Because women are underrepresented in the worldwide pilot population, it is important to address gender-specific preferences and needs in the development of AR-based wearable technologies for advanced pilot training. A gender-sensitive survey of the learning conditions and course contents was conducted with 31 pilots and 22 instructors. Despite many similarities, the results confirm that there are gender-specific needs and preferences for the development of future AR-based applications for TR training. In addition, the views of both pilots’ and instructors’ are required to obtain a comprehensive assessment of the learning contents and conditions related to TR. The results also show that time pressure increased the perceived difficulty of the course for some trainees. These results are important because they indicate the directions to be taken in developing future AR-based training applications for a more learner-centered and inclusive TR training. Future directions to foster a socially sustainable development of AR-based training means for TR with special focus on gender diversity are presented.

1. Introduction

The aviation industry has a high degree of standardization and control. Safety is the top priority, which has made flying the safest means of transportation. Pilots flying complex aircraft in commercial air traffic require specified levels of qualifications. Apart from the initial license, the pilot needs an advanced, aircraft type-specific training, which is endorsed in his/her license. Regular check flights are required to maintain the rights associated with a type rating (TR) license. During TR training, pilots learn the specifics of certain types of aircraft. The training consists of a theoretical and a practical part, which usually takes place on a certified full-flight simulator that is specific for the aircraft type. These devices are expensive and, depending on the aircraft type, rare, especially in business aviation, depending on the worldwide aircraft fleet size.

Since particular flight simulators are spread at different locations across the world, pilots must travel long distances for the TR training. Travel times of twelve hours or more are usual. The overall TR course duration can reach 30 or 45 days. Depending on the aircraft type, a TR training can cost up to EUR 80,000 per pilot, which is a high investment for an aircraft operator or for a pilot.

Anticipating the technological changes in aviation, research on improving the quality of TR training and the learning conditions is important for the industry.

Because aeronautical education is highly standardized, the training providers must comply with standards and receive approval for their courses from the competent aviation authority. As a consequence, the courses offered on the market are similar in both content and extent. Methods for differentiation from the competition are customer service, availability of courses, use of modern equipment, and pricing.

The practical part of pilot training is mainly conducted in modern full-flight simulators as these replicate the aircraft in detail and provide a safe and efficient environment to practice normal and emergency procedures. Theoretical knowledge is acquired using computer-based training (CBT), lectures, self-study and physical devices, representing cockpit instrumentation. The latter area is showing an increased use of new technologies, such as virtual reality trainers or other advanced digital solutions. New technologies will prevail, if they add extra value to the education or replace existing applications in terms of cost and learning success. In the course of this, training providers are facing the challenges arising from the interaction between production costs versus profitability, and the provision of high-quality training. Actual market trends show that virtual reality applications are finding their way into pilot training. The European Union Aviation Safety Agency (EASA) approved the first virtual-reality-based flight simulator in April 2021. This represents an evolution and is likely to make more cost-effective training devices available to complement full-flight simulators. In this regard, large pilot training providers have an advantage because they have the market power and resources to develop their own applications, which they can distribute to a large number of users.

As the TR course is comprehensive and complex, priorities need to be set for its future development. For a learner-centered and inclusive TR training, various social and economic factors need to be considered. Because women are underrepresented in the pilot population worldwide [1,2], flying an aircraft is stereotypically assigned to men. Thus, technological developments in support of advanced pilot training can be prone to gender bias [3]. As the aviation industry is interested in attracting talented women [4], it is important to address gender aspects in shaping the future of TR training.

Objective and Structure

The present study aimed to explore the preferences and difficulties experienced by pilots during the advanced TR training. In addition, preferences for the development of AR applications were investigated. By considering gender-specific preferences and difficulties in the development of future AR-based learning aids, which were aimed at in this study, the advanced pilot training should become more inclusive and learner-centered.

The rest of the article is structured as follows: In related work (Section 2), more information is given on the contents this study is based on. In methods (Section 3), the participants, the survey, and the data-processing methods are presented. Section 4 is dedicated to the results, followed by discussion (Section 5), conclusions (Section 6), and recommendations for future research (Section 7).

2. Related Work

2.1. Gender Aspects Related to Flight Training

Gender differences in flight training and barriers experienced by women who may have different starting conditions and experiences than men have been reported [5]. Research of cognitive processing has shown that men were better in visuospatial tasks that require mental rotation of objects, and women were better in verbal tasks [6]. However, research confirms that specific types of training can help both women and men to overcome initial gender differences and to reach comparable levels of performance. In an experiment, Neubauer et al. [7] found that although men performed better than women in two-dimensional visuospatial, mental rotation tasks, training had a positive effect on the performance of both genders, and as a result the gender difference decreased after training. Interestingly, gender differences were absent when tasks were performed in three-dimensional, as compared to two-dimensional, settings [7].

Research into flight training confirms these findings, showing that women and men can reach a similar level of flight performance after the same training periods, especially when the training concept used an inclusive design. Bauer et al. [8] found that despite initial gender differences in the pre-test phase, both women and men significantly improved their performance of upset recovery and reached similar levels of performance during post-test, after the same number of simulator exercises. In addition, Koglbauer and Braunstingl [9] investigated the effects of training traffic separation and airport procedures in a naturalistic flight simulation environment. They did not find gender differences in performance but in the subjective workload related to specific flight tasks that involved processing of both visuospatial and verbal information [9]. Ideally, the future developments in TR training should be made with full knowledge about gender diversity and should address the needs and preferences of diverse learners.

2.2. Theoretical Instruction

Theoretical flight instruction can be performed in the classroom or remotely by means, for example, of CBT, distance learning, or online instruction. Synchronous learning requires the trainees and the instructor to participate at the same time in a real or a virtual classroom. Asynchronous learning allows trainees more flexibility, as they can learn from pre-recorded material (e.g., texts, animations, and explanatory videos) at their own pace. Research from various domains shows that students can achieve similar levels of performance when attending a class synchronously or asynchronously [10]. In addition, research shows that blended online synchronous and asynchronous learning could improve student engagement with class activities and their experience of connectivity with their peers and instructors [11]. Trainee communication, interaction, and collaboration are considered to be more straightforward in synchronous learning (e.g., video, audio, or web conferencing; live chat; white boarding; and application sharing) [12].

2.3. Practical Instruction

Flight skills can be acquired and maintained by practice not only in the aircraft but also in flight simulators with various degrees of fidelity. Research has investigated the effectiveness of simulator training for pilots. For example, Hays et al. [13] found the combination of simulated and real flight more effective than training with one single training method only. Research into this shows a significant improvement in pilot procedural memory and generalization of skills acquired by pilots in a flight simulator and positive transfer to a new situation [14]. Furthermore, real flight performance can be significantly improved by training in a flight simulator [8,15,16,17,18,19].

2.4. Augmented Reality

The concepts of virtual reality (VR) and augmented reality (AR) are best described with the reality–virtuality continuum by Milgram et al. [20]. Here the space between reality and virtual reality is seen as a continuum referred to as mixed reality, which includes AR [20]. In the last decades, AR and VR were increasingly present in a multitude of use cases. Vergara et al. [21] mentions a number of examples for using virtual environments in education to attract and inspire students from different disciplines, e.g., electrical installations, hydraulics, or medicine. Among others, AR training means have been explored in virtual laboratories and 3D visualizations [21,22,23], educational gaming [24], interactive manipulation of holographic learning materials [25], and enrichment for the content of a book with pop-ups including 3D or animated AR content [26,27]. Wang et al. [28] analyzed AR in the area of education, showing the utilization for different pedagogical approaches. They concluded that AR technology represents an enormous potential for innovate learning and teaching. A similar conclusion was reached by Yuen et al. [27] where various educational applications were explored. In AR books the publication is extended with virtual information using AR, or in skill training AR facilitates the practice of certain tasks or procedures [27]. Virtual environments foster the transfer of theoretical knowledge into practice and allow the risk-free usage of expensive devices [22]. Additionally, AR can improve the learning experience and collaboration among student peers or between instructors and students [29]. AR was interesting from the start for the aviation industry, e.g., the term AR was coined by Caudell and Mizell [30] while working for Boeing. AR can be used in both synchronous and asynchronous learning. In aviation AR applications have been explored especially in relation to training materials [31,32,33,34,35]. In addition, the AR benefit to enable flight instructors the monitoring of pilot’s visual scan during simulated flight was highlighted by Vlasblom et al. [36].

2.5. Study in the Context of Sustainability

In addition to the primary goal of this study to explore direct benefits for the pilots from the use of AR in TR, we also discuss implications of this study in the wider context of sustainability. In the course of this we follow [37] and thereby base our considerations on the concept of sustainability as defined in the ”Report of the World Commission on Environment and Development: Our Common Future” [38] (p. 39) to, “[…] seek to meet the needs and aspirations of the present without compromising the ability to meet those of the future”. This general concept and goal of sustainability has been further developed and has influenced various research initiatives. In the context of business model innovation for example the research field “New Business Models (NBM)” [39,40,41] evolved, which investigates the development and consequences of (strongly) sustainable business models [42] and thereby aims to create business models that are “[…] simultaneously creating financial rewards, social benefits, and environmental regeneration” [42] (p. 17).

Sustainability has emerged as an important topic for the recovery of aviation after the COVID-19 pandemic. During the pandemic, some pilot training providers experienced increased competition and a decline in the number of customers (e.g., individuals and air-operators) due to the grounding of fleets, aircraft retirements, and reductions in crews. The forecasts for the post-pandemic recovery predict a return of the industry to operational levels close to those of pre-pandemic volumes over the next two years. How quickly and broadly the desired recovery will relieve the industry remains unclear. Concerns about pollution and lack of sustainability, however, are indicators of change: New inventions and developments, such as biofuel and alternative energy, will drive the industry in coming decades, while new pilot skills and specific training will also be required.

The focus of this study was especially on the social benefit and increased gender diversity in the advanced pilot education. The economic benefits of introducing AR in TR training, although expected, are not addressed here.

3. Methods

A survey was conducted with TR pilots and TR instructors to explore the difficulties of TR training and the potential of AR to improve the delivery of contents and learning conditions. The survey consisted of three parts. The first part addressed biographical aspects such as age, gender, pilot/instructor ratings, and activity.

The second part included items for assessment of TR training contents and conditions:

• Did the learning conditions make it difficult for some students to complete learning activities?
• Did the assessment conditions make it difficult for some students to complete assessment activities?
• Suggest support mechanisms for improving the learning outcomes of trainees.
• Name parts of the training perceived as most difficult.
• Select from the list of TR contents the items that were most difficult to learn.
• Name parts of the training pilots enjoy the most.
• Select from the list of TR contents the items that were easiest to learn.

The third part began with a video demonstration of typical AR features and applications. Three different AR use cases were introduced by presenting short videos to the participants to give them insights in the use of this AR technology. The shown use cases were not related to the aviation industry in order not to influence in specific directions. Subsequently, the participants were asked to select from a list of TR contents the items that could benefit from AR. The list included generic TR contents that were specified in the European aviation standards.

Table 1. Extract of the syllabus of theoretical knowledge for class or type ratings [43].

Detailed listing for airplane structure and equipment, normal operation of systems, and malfunctions

dimensions

engine including auxiliary power unit

fuel system

pressurisation and air conditioning

ice and rain protection, windshield wipers, and rain repellent

hydraulic system

landing gear

flight controls and high lift devices

electrical power supply

flight instruments, communication, radar and navigation equipment, autoflight, and flight data recorders

cockpit, cabin, and cargo compartment

emergency equipment operation and correct application of the following emergency equipment in the airplane

pneumatic system

presents a part of the syllabus “theoretical knowledge”.

The instruction program for TR has two main parts: the theoretical and the practical one. Theory aims to provide knowledge about complex aircraft, i.e., technical systems such as the fuel or electrical system, the operation of the systems, malfunctions, and limitations. Furthermore, knowledge about performance, flight planning, and monitoring is provided, which includes route planning and performance calculations such as take-off run and distance. Load and balancing is another important part of the theoretical instruction, as well as emergency procedures.

The practical part consists of eight training sessions and a final license skill test, all performed on a full flight simulator, representing the aircraft type for which the training is performed. The duration of each training session is usually four hours, in which each pilot performs two hours as the pilot flying and two hours as the pilot monitoring. The duty of the pilot flying is to control the path of the aircraft and order configuration changes (e.g., extend/retract landing gear). The pilot monitoring assists the pilot flying in respect of maintaining situational awareness and communication with air traffic control.

The training sessions start with a one-hour-long briefing, covering the learning objectives and expected program. The first sessions cover basic flying techniques and also focus on transferring the theoretical knowledge into the cockpit environment. Simple abnormals (e.g., failed back-up systems) are introduced before more severe situations (e.g., engine failures and fires and loss of main electrics) are thought and trained. Besides technical knowledge, interpersonal decision making and collaboration tools are also part of the course. The final goal is to get the student ready to successfully perform the license skill test. The program investigated here is defined by EASA and covers among others operation with a failed engine and different types of approaches.

3.1. Data Analysis

The results are presented descriptively in percentages and relative frequencies. Data obtained by selecting from a list of training elements were grouped in 15 topic categories, which are shown in

Table 2. Categories and assigned training elements.

Category	No. of Elements	Excerpt of Training Elements Assigned to This Category
Aircraft structure and equipment	13	Fuel system, hydraulic system, landing gear, electrical power supply, etc.
Limitations	4	Engine limitations, systems limitations, minimum equipment list, etc.
Performance	4	Flight monitoring, load and balance, etc.
Emergency procedures	2	Actions according to the approved abnormal and emergency checklist
Special requirements	2	For extension of a TR, for “glass cockpit”
Flight management system	1
Flight preparation	6	Performance calculation, cockpit inspection, etc.
Take-offs	8	Normal take-offs with different flap settings, crosswind take-off, etc.
Flight maneuvers	3	Turns with and without spoilers, etc.
Normal flight procedures	15	Engine, pressurization and air-conditioning, etc.
Abnormal and emergency flight procedures	12	Fire drills, engine failures, etc.
Instrument flight procedures	5	Adherence to departure and arrival routes and ATC instructions, etc.
Missed approach procedures	3	Go-around with all engines operating, etc.
Landings	4	Normal landings, landing with critical engine simulated inoperative, etc.
Instrument approaches	4	Rejected take-off at minimum authorized RVR, etc.

. The number of responses per topic category were weighted to counterbalance the varying number of items in different categories by dividing the number of answers per category by the number of items of the category. In addition, the responses were weighted to counterbalance differences in group sizes (e.g., women versus men or pilots versus instructors). The number of responses per group in the context of a specific category was divided by the total number of responses provided by the individuals included in the respective group (e.g., women, men, or instructors). In summary, the data are presented as percentages and weighted relative frequencies.

Responses to open-ended questions were separately analyzed for pilots and instructors and men and women. Nominations of specific contents were counted.

3.2. Participants

For obtaining pilots’ views, 31 pilots with a TR were surveyed. In total, 24 male pilots aged between 22 and 57 years (mean = 36.67, median = 35, SD = 9.04) and 7 female pilots aged between 29 and 48 years (mean = 36.29, mMedian = 37, SD = 7.57) participated in the study. The pilots held TR from different types of aircraft such as Boeing 767; MD-11; Cessna Citation types C525, C560XLS, and C750; Airbus A320 and A350; and Junkers Ju 52. For obtaining instructors’ views, 22 type rating instructor (TRI) were surveyed. All instructors were men aged between 29 and 65 years (mean = 45.90, median = 43, SD = 10.25).

4. Results

The results are structured in two parts: assessment of the course content and aspects related to learning and assessment conditions.

4.1. Type Rating Contents

The course content was analyzed with respect to the following four questions: course contents, which are the most difficult to learn followed by the contents, which are the easiest to learn. The third section examines contents, which participants enjoyed the most. The forth concludes with the contents, which are considered to benefit the most from AR.

4.1.1. The Most Difficult TR Contents to Learn

Participants could select from a list of course elements those that they experienced as being most difficult to learn. The contents were grouped in 15 categories for data analysis. Figure 1 shows that women assessed missed approach procedures as the most difficult content category, followed by landings, the flight management system, abnormal and emergency flight procedures, and instrument flight procedures. A number of female pilots rated contents from the categories of aircraft structure and equipment, flight maneuvers, instrument approaches, take-offs, normal flight procedures, and limitations as being the most difficult to learn. There were no contents in the categories of performance, emergency procedures, special requirements, and flight preparation; however, those were rated by women as being difficult. Similarly, male pilots rated missed approach procedures as the most difficult content, followed by the flight management system, landings, take-offs, and abnormal and emergency flight procedures. A number of men rated contents from the categories of instrument approaches, special requirements, flight maneuvers, instrument flight procedures, normal flight procedures, aircraft structure and equipment, limitations, and flight preparation as difficult. None of the contents in the categories performance and emergency procedures were rated by men as being difficult. The overall scores of the pilots including both genders indicate that the missed approach procedures were the most difficult content, followed by the flight management system, landings, abnormal and emergency flight procedures, and take-offs. By comparison, the TR instructors identified take-offs as the most difficult content, followed by the flight management system, missed approach procedures, aircraft structure and equipment, and emergency procedures. The contents missed approach procedures and the flight management system appeared in the top five for each group. Interestingly, the flight instructors did not rate course contents in the categories limitations, special requirements and instrument approaches as being the most difficult for their trainees.

Table 3. Categories: contents most difficult to learn and most promising for using AR from pilots’ and instructors’ perspective.

Category	Most difficult item	Content That Could Mostly Benefit from AR
Aircraft structure and equipment	Electrical power supply (TRP); fuel system, pressurization and air conditioning, ice and rain protection, windshield wipers and rain repellent, landing gear, pneumatic system (TRI)	Cockpit, cabin and cargo compartment (TRP); dimensions: minimum required runway width for 180° turn (TRI)
Limitations	General limitations (TRP)	General and engine limitations (TRP); general limitations (TRI)
Performance	Flight monitoring, on ground, servicing connections (TRI)	On ground, servicing connections (TRP); flight planning for normal and abnormal conditions, on ground, servicing connections (TRI)
Emergency procedures	Emergency procedures actions according to the approved abnormal and emergency checklist (TRI)	Emergency procedures (TRI, TRP), emergency procedures actions according to the approved abnormal and emergency checklist (TRI)
Special requirements	Special requirements for “glass cockpit” aircrafts with EFIS (TRP)	Special requirements for extension of a TR for instrument approaches down to decision heights of less than 200 ft (60 m) (TRP), special requirements for “glass cockpit” aircrafts with EFIS (TRP, TRI)
Flight management system	Flight management system (TRP, TRI)	Flight management system (TRP, TRI)
Flight preparation	Aircraft external visual inspection; location of each item and purpose of inspection (TRP, TRI)	Aircraft external visual inspection; location of each item and purpose of inspection (TRP, TRI), Cockpit inspection (TRP)
Take-offs	Take-offs with simulated engine failure between V1 and V2 (TRP); take-offs with simulated engine failure (TRI)	Crosswind take-off (TRP); take-offs with simulated engine failure, shortly after reaching V2, between V1 and V2 (TRI)
Flight maneuvers		Tuck under and Mach buffets after reaching the critical Mach number, and other specific flight characteristics of the aircraft (e.g., Dutch Roll) (TRP, TRI), normal operation of systems and controls engineer’s panel (TRP)
Normal flight procedures	Electrical and hydraulical system (TRP); pitot/static system, landing gear and brake (TRI)	Engine (TRP); pitot/static system (TRI)
Abnormal and emergency flight procedures	Fire drills, e.g.,  engine, APU, cabin, cargo compartment, flight deck, wing, and electrical fires including evacuation, smoke control and removal, incapacitation of flight crew member (TRP); steep turns with 45° bank, 180° to 360° left and right (TRI)	Wind shear at takeoff/landing (TRP); fire drills e.g., engine, APU, cabin, cargo compartment, flight deck, wing and electrical fires including evacuation, smoke control and removal (TRI)
Instrument flight procedures	Circling approach (TRP, TRI)	Holding procedures (TRP); circling approach (TRI)
Missed approach procedures	Manual go-around with the critical engine simulated inoperative after an instrument approach on reaching DH, MDH, or MAPt (TRP); go-around with all engines operating during a 3D operation on reaching decision height; other missed approach procedures (TRI)	Manual go-around with the critical engine simulated inoperative after an instrument approach on reaching DH, MDH, or MAPt (TRP, TRI); go-around with all engines operating during a 3D operation on reaching decision height; other missed approach procedures; rejected landing at 15 m (50 ft) above runway threshold and go-around (TRI)
Landings	Landing with simulated jammed horizontal stabiliser in any out-of-trim position (TRP); crosswind landings (a/c, if practicable); traffic pattern and landing without extended or with partly extended flaps and slats (TRI)	Landing with simulated jammed horizontal stabiliser in any out-of-trim position (TRP), crosswind landings (a/c, if practicable); traffic pattern and landing without extended or with partly extended flaps and slats (TRP, TRI)
Instrument approaches	Rejected take-off at minimum authorised RVR, go-around after approaches on reaching DH (TRP)	Rejected take-off at minimum authorised RVR, go-around after approaches on reaching DH (TRP), CAT II/III approaches: in simulated instrument flight conditions down to the applicable DH, using flight guidance system (TRP, TRI)

gives an overview of the most difficult items for each category rated by type rating pilots (TRPs) and TRIs.

In addition to the selection from the list, there was an open-ended question asking pilots “What part of your TR training was the most difficult? Why?” The question for instructors was “What part of your TR training was the most difficult for your trainees? Why?” The responses address both training contents and learning conditions such as stress. Six female pilots, 21 male pilots, and 20 instructors responded to this question. One woman mentioned language barriers because all the lessons were held in English. Two others identified the theoretical learning of aircraft systems as being most difficult. One woman replied that “the long theory days” were difficult. Three men pointed out the stress caused by the very short course duration and the sheer volume of information to assimilate. Seven male pilots brought up the learning of theoretical systems’ contents and six mentioned the practical parts as being most difficult. Twelve TRIs stated issues in the practical part, and three emphasized malfunctions during practical training. Furthermore, six TRIs referred to theoretical contents; two mentioned that candidates are nervous before tests; and two noted issues with multi-crew operations. One instructor mentioned the short time and the huge quantity of information contained in the advanced pilot training due to the complexity of aircraft systems.

4.1.2. Easiest to Learn TR Contents

Pilots were asked to indicate the easiest contents of the course by selecting items from the list. Women selected contents in the categories limitations, emergency procedures, take-offs, landings, and normal flight procedures as being the easiest to learn, as shown in Figure 2. Flight maneuvers were the easiest for men, followed by take-offs, instrument flight procedures, landings, and aircraft structure and equipment. For pilots of both genders, overall, the easiest contents were in the categories: flight maneuvers, limitations, take-offs, landings, and flight preparation. TRIs rated the easiest contents as follows: limitations, special requirements, performance, flight preparation, and emergency procedures. The easiest content mentioned in all groups was in the category of limitations. Interestingly, take-off and landing contents were selected in both the most difficult and easiest to learn contents, an indicator of the diversity of students’ experiences with the same material.

4.1.3. The TR Contents Participants Enjoyed the Most

Another open-ended question asked pilots “What part of your TR training did you enjoy the most? Why?” The question for instructors was “What part of your TR training do your trainees enjoy the most? Why?” These questions were answered by six female pilots, 23 male pilots, and 21 instructors. Five women said that they enjoyed simulator training. Two women explained that they like hands-on training, and two said that they like learning new things. Fourteen men mentioned either the simulator or practical training as the most enjoyable part of the training. Fourteen instructors noted that their trainees enjoyed simulator training the most.

4.1.4. The TR Contents That Were Considered to Benefit from AR

After a demonstration of typical AR use cases, all pilots and instructors were asked to select course contents that could benefit from AR to improve students’ development of mental models/knowledge during training. As illustrated in Figure 3, women rated special requirements as the category to benefit most from AR, followed by emergency procedures, the flight management system, aircraft structure and equipment, and instrument approaches. The flight management system was selected most often by men. Other contents among the top five selected by men were: emergency procedures, aircraft structure and equipment, flight preparation, and instrument flight procedures. This order also corresponds to the order of overall pilot scores. In addition, the instructors considered the category aircraft structure and equipment to benefit most from AR, followed by emergency procedures, flight preparation, performance, and abnormal and emergency flight procedures. The potential benefit of AR for learning emergency procedures and aircraft structure and equipment was rated highly by all groups. More information on the specific items of each category which can benefit from AR is given in Table 3.

4.2. Learning and Assessment Conditions

The learning conditions for pilots participating in TR courses are characterized by long travel times, because they often come from abroad. The course duration depends on the content and costs of the course.

Figure 4 (left) indicates that most of the male and female pilots as well as most of the instructors did not consider that the learning conditions created difficulties for some students to complete the course. None of the female pilots reported such difficulties. There were five responses from male pilots and four from instructors. They mentioned stress and time pressure, differences in the previous knowledge of the learners, and problems with self-study.

In addition, the participants were asked if the conditions made it difficult for some students to complete the assessment. As can be seen from Figure 4 (right), only between 0 % and 16 % of the participants thought that this was the case. Two male pilots and three instructors mentioned time pressure, the need to manage the pressure on students, and the need to better prepare for the assessment. Difficulties in learning were also reflected in difficulties with evaluation.

Moreover, participants were asked “Can you suggest any support mechanisms to improve the performance/ results of the students?” A female pilot, seven male pilots, and nine instructors responded to this question. The woman had no suggestion but said the course was very well done. Men suggested having more classical classroom instructions but also mentioned additional CBTs to improve performance and results. Furthermore, they suggested allocating more time to difficult topics. One pilot suggested deviating from the course plan in the event of the pilots not being able to achieve the learning performance expected from them as planned. TRIs proposed giving more time for learning, standardizing instructions, using other media (e.g., videos), and using a mock-up or procedure trainer.

5. Discussion

The aim of this study was to identify gender-specific preferences and difficulties in TR training that can be addressed by future development of AR-based training means. Thus, the advanced pilot training could become more inclusive and learner-centered, and this would bring both social and economic benefits [1,2,3,4]. Despite many similarities, the results of this study confirm that there are gender-specific needs and preferences in advanced flight training, as revealed in previous research [5,7,8,9]. Due to the underrepresentation of women in the pilot population, their learning needs and preferences can easily vanish in the pool of the collected data. The results also show that both trainees and instructor views are required to obtain a comprehensive assessment of the learning contents and conditions. For future development of TR training means in the context of theoretical and practical content, a variety of AR applications are listed in

Table 4. Potential AR applications for TR training.

Category	Content
	Theoretical	Practical
Aircraft structure and equipment	✔	✔
Limitations
Performance		✔
Emergency procedures		✔
Special requirements		✔
Flight management system		✔
Flight preparation	✔	✔
Take-offs
Flight maneuvers		✔
Normal flight procedures		✔
Abnormal and emergency flight procedures		✔
Instrument flight procedures		✔
Missed approach procedures		✔
Landings
Instrument approaches		✔

.

In the following two categories AR applications can be developed for both theoretical and practical training: aircraft structure and equipment and flight preparation. Possible methods for teaching aircraft structure and equipment, which can be used individually or combined are:

• Synchronous and asynchronous training materials [31,32,33,34,35,44],
• AR training means 3D visualizations [23],
• Educational gaming [24,34],
• Interactive manipulation of holographic learning materials [25],
• Book content with pop-ups including 3D or animated AR content [26,27].

For addressing the learning conditions, which play an important role for a successful completion of the course, AR could be integrated in future applications to support:

• Collaboration among peer students or between instructors and students [29],
• Skill training with AR-facilitated practice of certain (part) tasks or procedures [27],
• Flight instructors monitoring of the pilot’s visual scan during simulated flight as highlighted by Vlasblom et al. [36].

As research shows, the proposed AR solutions are seen as complementary to the classroom and full-flight simulator training. We propose the use of AR for the initial part of the training in blended online synchronous and asynchronous settings that have been shown to improve student commitment and the feeling of connectivity with peers and instructors [11]. This could mitigate the stress and time pressure factor related to the conditions of TR training that were highlighted by a number of pilots and instructors. Future AR applications for synchronous learning in TR training need to consider the richness of media for communication, interaction, and collaboration [12] such as application sharing and remote consultation that is enriched with virtual annotations and synchronous video sharing [45]. Asynchronous interactions [12] could be implemented in AR (for example, video recording of a practice session and guidance videos enriched with holographic on-site animations [46]).

Research [22] has shown that students liked the interactivity enabled by virtual laboratories, because the “learning process is more pleasant” and it “enhances their comprehension of the concepts and ideas developed in theoretical classes”. The tools used enhanced learning and led to better performance of the students. [22] This effect was also shown when using computer simulations [47]. According to [48], it is important to consider individual and social perspectives in the learning process and thus to shape it in line with group and individual activities.

The positive impact of AR on learning gains and the increase in motivation was also confirmed by a meta-analysis from [49,50]. With respect to the level of education, they found that “AR has a large effect on Bachelor’s or equivalent level and a medium to large effect on short-cycle tertiary education” and in the field of education context “engineering, manufacturing and construction” records the largest effect [50]. In a recent review, [51] showed that AR enables “learning by doing” (kinesthetic learning), and the integration of AR in pedagogical approaches supports the creation of “human-centered learning environments”. More efficient visualization, enhancing of collaborative learning, and boosting motivation and engagement are, among other points, some of the key benefits of using AR. However, technical issues, information/cognitive overload are some of the key challenges identified [51]. The positive impact of AR on student motivation was also reported [52], showing an increase in attention, satisfaction, and confidence. Research [53] has identified educational benefits of AR such as fun, dynamics, facilitation, interactivity, and entertainment. However, the main limitations of the device should be mentioned among a number of others. The integration of AR in learning concepts facilitates more interactivity. In e-learning environments AR offers more possibilities to discover technical concepts by using 3D-visualizations, which help to lower the barriers and at the same time to increase the motivation, both of which lead to better performance results.

There are also several examples for the use of VR in teaching in various disciplines to improve the learning process and understanding of it by the students. The usage of virtual laboratorys (VLs) has various benefits such as enrichment of the teaching–learning process with interactive elements, which in turn affects student motivation, brings in new possibilities for visualization (e.g., see through components), and gives more autonomy to students in the matter of how often and when they use the learning tool. In addition, VLs mitigates the risk of damage to a real machine/product or of students endangering themselves when they are inexperienced in handling them, and VLs are significantly less investment intensive than “real” devices. VLs also allows training with devices that would not be available in reality because of the (extremely) high costs involved [21,44]. The latter is of particular relevance to TR courses, because a real aircraft is often not available during the course.

Social sustainability aspects such as gender diversity and inclusivity can be addressed both in design [3,34,35] and evaluation of AR-based training means. Specific methods and techniques for gender research in aviation are detailed in the recent literature [3,8,9].

Sustainability aspects in terms of AR engineering issues and costs could be considered in future research. Ref. [54] mentions that if the application is not on the latest technological level, the user motivation and interactivity decreases until the application is updated again. This factor has a major influence on the operational life of the AR application, its maintenance, and the resulting costs [54,55,56].

6. Conclusions

Social sustainability aspects such as gender diversity and inclusivity were addressed in the evaluation of the TR course. This study with pilots and flight instructors identified gender-specific preferences and difficulties related to the content of TR training but also related to training and assessment conditions. Pilots of both genders and flight instructors noted that simulator training was the most enjoyable part of the course. The most difficult content categories for female pilots were missed approach procedures, followed by landings, the flight management system, abnormal and emergency flight procedures, and instrument flight procedures. The most difficult content categories for male pilots were missed approach procedures, followed by the flight management system, landings, take-offs, and abnormal and emergency flight procedures. Flight instructors and a number of male pilots mentioned difficult training conditions such as stress, time pressure, differences in the previous knowledge of the learners, problems with self-study, and the need to better prepare for the assessment.

The results show a variety of TR training elements that could potentially benefit from AR-based training means. The potential benefit of AR for learning emergency procedures, aircraft structure and equipment was rated highly by pilots of both genders and by flight instructors. In addition, female pilots rated special requirements, the flight management system, and instrument approaches as course contents to benefit most from AR-based training means. Besides the flight management system, flight preparation and instrument flight procedures were supplementary categories highly rated by male pilots. Further categories highly rated by flight instructors were flight preparation, performance, abnormal and emergency flight procedures. In addition, AR features were specified that could be used to address a number of learning conditions by facilitating the remote training of particular elements and collaboration among peer students or between instructors and students. The proposed AR solutions are seen as complementary to the classroom and full-flight simulator training. The results are used to inform the future development of AR-based training means for pilot training.

7. Future Research

Flexibility for TR trainees to learn and practice remotely and asynchronously using wearable AR devices could support their acquisition and maintenance of aircraft knowledge and flight skills. Controlled experiments [8,14,15,17,19] could be used to assess the effect of AR-based training means on pilot’s procedural memory, acquisition, and generalization of skills and transfer of training in a gender-sensitive manner.

In addition, various aspects of economic and environmental sustainability could be pursued in future research. Economic benefits for flight schools may be generated by novel AR-enhanced CBTs with an increased level of automation of the teaching and assessment processes.

CBT and possible new forms of virtual training fostered by AR may also have positive environmental impacts such as reduced travel demands for the students and instructors. These environmental sustainability effects of AR-based training could be investigated in dedicated case studies, field experiments, surveys, and long-term studies.