A Study of Applying Fuzzy Theory in Simulation-Based Education

Abstract

Among all aviation accidents in Taiwan’s general aviation industry from 1998 to 2016, human factors account for the most at 51.2%, including negligence of external obstacles, poor autopilot flight ability, poor resources management of the crew, inability to follow aviation regulations, lack of understanding of the landing area, not fully comprehending the operational functions and not alert to situations. Those factors have seriously affected flight safety. Resources management training for crew members may thus be the best measure to prevent human errors. Following the Evidence-Based Training (EBT) promoted by ICAO and International Air Transport Association (IATA), this study constructs the assessment indices of EBT for helicopter crews. After collecting the opinions of helicopter flight instructors of military and civil helicopter units, we apply the Fuzzy Delphi Method (FDM) for the preliminary assessment guidelines, use the Fuzzy Analytic Hierarchy Process (FAHP) to construct the hierarchy, and then calculate each criterion and each criterion weight.

1. Introduction

Casualties due to environmental disasters, such as wind disasters, have dramatically increased in recent years, corresponding with increased helicopter demand for disaster relief and rescue. Due to their particular characteristics, helicopters can fly at low altitudes and run through environmental settings such as cities, valleys, or mountains. Since these flights are usually under visual flight operation, flight crews must pay close attention to external inspection and monitoring. Avoiding interference from other aircraft or external environmental impact make cockpit operation very difficult. Although aircraft safety has continually improved from design and production perspectives, helicopter accident rates have not improved significantly. Taiwan’s civil aviation council classified 2007–2016 helicopter flight accidents as being largely operator caused (51.2%: 44.2% related to pilots and 7% to other personnel such as maintenance or air control) with the second most common cause being other aircraft, and only 18.6% due to environmental issues [1]. In comparison, USA average helicopter accident rate for 2001–2005 was 7.91 per 100,000 h, with 1.31 per 100,000 h fatality rate [2]. The main factor for helicopter accidents in Taiwan was 90% man-made errors due to deficient training, illegal flights, poor vigilance and mechanical failure. Increased employment pressure also affected pilot working attitudes, increasing aggressiveness and mission focus [3]. Other factors contributing to helicopter accidents were improper operation procedures, airspeed and distance miscalculation, delayed or improper decision, poor team coordination, carelessness, space obsession and inexperience [4].

The International Air Transport Association (IATA), various national civil aviation authorities, airlines, aircraft manufacturers, and training institutions have all highly recommended flight crew training programs to integrate personal techniques and capabilities with teamwork, and hence help prevent human negligence. However, training results show remarkable differences in effectiveness due to different operators and aircraft models during training sessions [5,6]. Therefore, it is critical to identify suitable helicopter cockpit resource management (CRM), combining communication, flight decision, teamwork, workload management, condition vigilance, automation and other key factors to effectively monitor the impact on flight performance. This study used the International Civil Aviation Organization (ICAO) evidence based training (EBT) methods to verify key CRM factors under emergency conditions. We tried to understand available methods for aviation crews to conduct well-organized flight management when facing dangerous situations. This study will provide civil and military aviation authorities and training organizations with references to integrate into junior and senior pilot education and training courses to improve management literacy and flight safety.

The remainder of this article is organized as follows. Section 2 reviews relevant literature, and Section 3 introduces the assessment methodology. Section 4 presents an example using practical Taiwan pilot training data to verify the proposed methodology. Section 5 summarizes the outcomes, concludes the paper, and discusses some useful future research directions.

2. Literature Review

The core issue in this study is “A study of spplying fuzzy theory in simulation-based education.” To reach the human error reduction goal, the most common theories and tools put into use are as follows: EBT and CRM. This “Helicopter flight crew EBT” study was based on ICAO “Annex 6 to the Convention on International Civil Aviation” Part 1 International commercial aviation transportation, section 9.3 flight crew training program and 9.4.4 flight crew competency check handbook which defined the core competencies, such as application of procedures, communication, leadership and teamwork, problem solving and decision-making, SA, workload management, aircraft flight path management, automation and aircraft flight path management and manual control.

2.1. Empirical Evidence Based Training

ICAO developed DOC 9995 in 2013 to provide aircraft users and training institutions with EBT reference guidance for regular retraining and technical testing for flight crew training. In particular, flight crews could repeat scenarios to improve their capabilities by implementing EBT models, producing remarkably improved outcomes compared traditional fitness tests. EBT models helped aircraft users and training institutions to establish flight crew training programs complying with regulatory needs and international norms, greatly improving operational safety [5,6]. Reality based training and appraisal programs using qualified flight simulation training equipment, i.e., flight simulators, provide training environments much closer to reality. The various training programs employed differentiated baseline EBT models for the various aircraft operational details.

Enhanced EBT could also record relevant flight data and evaluate key menace and risk factors that affect safe operation through simulation aircraft training processes. Therefore, without changing existing flight course outlines, adopting EBT principles allowed focus to be placed on analyzing root causes for improper operation by flight crews. Training was not limited by the number of fixed flight hours, and work task analysis ensured flight crews achieved observable operating standards during training by monitoring flight crew development.

2.2. Communication

Communication was defined as sharing, universal, and cognitive, i.e., information was circulated and shared between speaker and listener to reach a consensus. In particular, opinions, information, ideas, thoughts and feelings were exchanged through concise and clear expression, providing bilateral understanding. Communication included effective oral, non-verbal and written forms in normal and abnormal situations [6]. Speech move theory extended traditional theoretical models based on grammatical correctness alone, with focus on communication fluency and sentence function [7]. In addition to mastering some vocabulary and grammar knowledge to understand correct sentence structures, it was also important to understand how to use languages properly to interact with interlocutors and achieve effective communication.

Basic language interaction theory considers communication dialog includes three steps: speak, reply, and continuous question and answer, which have been incorporated into ICAO and Federal Aviation Administration (FAA) dialog steps as indication, reply, and confirmation, respectively.

• Speak. Speak can be directed, requested, ordered, communicated, questioned, etc. Speakers took a linguistic form to achieve a desired action.
• Reply. Listeners reply when they have received and understood the speaker’s message by responding to a speak request or expressing that they understand the message contents. Responses can be positive, negative, or compromised.
• Continuous question and answer. This is the original speaker’s role, and can be endorsement, concession, or simple confirmation.

The steps form a complete set of dialog streams, ensuring that when crew needed to send messages, they would take action only after being sure they understood the message meaning. Effective communication occurred when the resultant action was in accordance with the message sender’s original intention, otherwise the communication was considered invalid. Many factors or barriers could cause interference, obstruction, or otherwise reduce communication effectiveness, including personal factors, cognitive capabilities, social behaviors and culture. Lasswell [8] proposed a linear communication theory where communication was linear between a single speaker and receiver only. The other flight crew was particularly attentive to identify and send warning messages about possible confusion. The pilot had to listen carefully when receiving such messages, and reduce the frequency of numerics in the message transmission. For example, it was improper to send more numerics such as height, course and altimeter setting through several dialogs to improve communication quality.

Communication between crew members could be significantly improved if it was a specific CRM training focus as well as standard operating procedures. However, it is important to focus on “what was the correct message?” rather than “who was correct?” This can be achieved by

(1)

mutual respect and encouragement for participation; firm description, and active listening;

(2)

implementation of team member task prompts, clear expression of the determination for aircraft operation wherein the captain should question or ask for opinions;

(3)

aggressively ask for information and guidance from others if necessary;

(4)

insistence on goodwill or seeking assistance from others to ensure flight safety when facing an unknown situation, together with timely self-review and encouragement to others; and

(5)

using appropriate skills to deal with interpersonal and work conflicts.

2.3. Situation Awareness

Sater and Wood [9] proposed situation alert or situation awareness (SA): understanding the entire situation through ongoing situation assessment. Personnel should also have excellent understanding of the work they are engaged in. They should be capable of understanding relevant meaning, function, and possible impact on the things they observe [10]. SA for pilots includes understanding all relevant information in a flight environment and predicting what could occur to affect operations [5]. SA allowed crew to develop correct pictures regarding ongoing situations and what might happen in the future, and helps individuals to remain alert when facing the surrounding environment and matters [11]. Every team member should be capable of remaining alert to the surrounding work situations [12]. Therefore, SA capabilities include understanding how to perceive the work situation using already known modes and methods, and continual reassessment.

Situation awareness capability is often limited due to physical exhaustion, psychological stress, or emotional impacts [11,13]. Generally, SA errors were due to crew members not fully understanding aircraft systems, performance, and/or operational restrictions. When indications appeared, poor SA worsens situations without isolation and handling when they were still controllable. Although aircraft systems were operable normally, pilots could not remain alert to possible error causing factors due to distraction, off flight course, or blindness to the flight environment.

Thus, it is critical to develop methods to train crew members how to be aware of situations, enhance team communication skills, and continuously evaluate performance in work situations [9], ensuring each crew person could understand factors affecting team alertness. Helping individuals to improve themselves, reducing human negligence and improving overall crew SA capability and flight quality [14] will establish an organizational culture of sharing [15].

2.4. Teamwork

Shonk [16] defined a team to include two or two persons where the team members coordinate to accomplish common tasks. Jessup [17] contended a team should not only focus on achieving overall goals, but also emphasize mutual dependence and commitment between team members. A team has the authority to decide how to implement tasks, schedules, and assignments where rewards and performance feedback were determined in accordance with overall team performance [18]. The most striking feature of a team was that team members could decide the highest priority to achieve team goals. Individual team members had their own professional skills, but could mutually support, cooperate, and communicate clearly and openly due to sharing common commitments [19,20,21].

Improving teamwork effectiveness and increasing knowledge and specialties were helpful for teams to face various situations, helping to integrate ideas and skills, create new practices, and enable teams to work together effectively. Consequently, team effectiveness was always far beyond individual achievement levels. Leadership was defined as combining power and influence. Influence was the leader’s capability to modify team member’s psychology and behavior when communicating with others. Natural leaders exhibited pleasant psychological and behavioral performance voluntarily. Power was a mandatory form of leadership to make subordinates passive and obedient. Daft [22] argued that leadership was a relationship between leader people and the people being led, where leaders attempted to cause real changes to achieve shared goals.

2.5. Workload

Maslach and Goldberg [23] contended that work overload was basically heavy workload, i.e., overload being beyond team member’s maximum capabilities. Thus, individuals had to deal with too much work in a valid time. However, due to excess work or long working hours, employees commonly have to complete their work within a specified time. This situation will finally result in mental and/or physical incapability. This was effectively workload.

Workload can be divided into quality and quantity. Work overload in quantity means excessive workload such that workers could not complete their tasks as scheduled. Work overload in quality means workers did not have the capability to produce the required quality, or performance standards were too high.

Crew members were particularly prone to errors, even in daily missions, when different conditions occurred due to bad weather or aircraft failure. These situations resulted in particular errors occurring among flight crews. Low workload can also lead to reduced alertness, and flight crews can become complacent about the mission underway. Thus, workload can cause different errors, and the consequences of these mistakes could be key factors for crew insecurity.

Cavanaugh et al. [24] contended that workload could often generate challenging pressure, provided it did not exceed individual capability limits. Workload stimulated team member intrinsic internal arousal, helping them to work harder with a positive impact [25,26]. Hence individuals should be motivated to overcome workload, meet work requirements, and then achieve high performance. Conversely, if an individual viewed workload as an obstructive source of stress, i.e., a threat, they were incapable of completing the work satisfactorily, and it followed that they were unwilling to overcome the workload.

2.6. Aeronautical Decision Making

The USA FAA defines pilot judgment to mean processes to recognize and analyze available data regarding selfness, aircraft, aviation environment and flight purposes with subsequent reasonable assessment using other methods. It also meant timely decision making to ensure safety. Therefore, pilot judgment includes assessing attitude and appraisal capabilities for team members dealing with crises, as well as their decision making based on personal knowledge, skills and experience. Jensen and Hunter [27] proposed that flight decisions in the cockpit involved analyzing and comparing all possible available alternatives to respond correctly in a timely manner when flight safety was threatened.

Pilot decision making was usually based on uncertain messages or performed in dangerous situations, where often thoughtful and logical decisions could not achieve the desired results. Therefore, many flight safety accidents were due decision making errors. Making decisions in-flight is a complex cognitive process deeply affected by flight conditions and operational environment factors [28]. The particular situations forced pilots to make decisions while unable to confirm misjudged conditions or improper operational responses [29,30]. Pilots often used simple rules and procedures to implement flight tasks, but when facing complex emergencies, they must decide from multiple alternatives or solutions with considerable time pressure [31]. Most pilot reactions were related to flight decisions about establishing or changing flight postures, as a direct response to operating aircraft including understanding and judging instrument accuracy within the cockpit and outer environment. Pilot decision making required external signals, such as aviation control instructions, inspection card procedures, aircraft postures, or meter displays to ensure correct judgment or operation. Generally, this information needed to be free from conflict to provide a complete explanation quickly. If misunderstanding occurs in these situations, further understanding could help reduce human error likelihood [32].

Pilot judgment and decision making involve different judgment modes, which can be broadly divided into perceptive and cognitive judgment. Decision making and judgment often includes perceived elements (visual, auditory), such as distance or proximity to the ground, which can be very important for pilots to safely operate aircraft. Cognitive judgment occurs when perceptive elements are less clear, and usually requires more time for consideration. Generally, where there are more than two choices and the risk of either option was difficult to assess. Therefore, junior pilot judgement tends to be more cognitive judgment, whereas senior pilot judgement tends more to perceptive judgment. Changing from cognitive to rapid and effective perceptive judgment requires long learning/training times. Many decision making training systems help pilots to clearly understand the decision making process and potential mistakes for each step.

3. Methodology

This study used Fuzzy Delphi Method (FDM) and Fuzzy Analytic Hierarchy Process (FAHP) as the research tools. Among the EBT, definitions listed in Tables 2–8, Evaluation Competency and Scenario could be another tool. The FAHP can use the fuzzy analytic hierarchy process to check and weight the EBT Evaluation Competency and Scenario. FDM and FAHP were extended respectively from the Delphi Method, AHP, and fuzzy set. Thus, the concept of fuzzy set is introduced, then FDM and FAHP, as follows:

3.1. Fuzzy Set Theory

Fuzzy Set Theory was proposed by Zadeh in 1965 [33]; since then, fuzzy mathematics became popular and widespread in all kinds of fields. For example, during the decision making process, the decision maker will face various uncertainties. If fuzzy theory is introduced during the evaluating process, it would make the decision result match reality more closely. Thus, this study took these theories and applied them to military helicopter simulator training critical factors. The basic fuzzy set theory concepts are listed as follows:

Fuzzy numbers:

Dubois and Prade [34] defined any of the fuzzy numbers as a fuzzy subset of the real line. A fuzzy number is a quantity whose values are imprecise, rather than exact as is the case with single-valued numbers. Among the various shapes of fuzzy numbers, triangular fuzzy number and trapezoidal fuzzy number are the most commonly used membership function. If fuzzy number $\tilde{A}$ is a fuzzy set and belongs to membership function:

$${\mu }_{\tilde{A}}\left(X\right):U\to [0,1],x\in X$$

(1)

Triangular Fuzzy Numbers (TFN) has linear representations on its left and right side such that its membership function can be defined as:

$${\mu }_{\tilde{A}}\left(x\right)\{\begin{array}{cc}\hfill \frac{x-l}{m-l},& l\le x\le m\hfill \\ \hfill \frac{x-u}{m-u},& m\le x\le u\hfill \\ \hfill 0,& others\hfill \end{array}.$$

(2)

Fuzzy number algorithms:

Zadeh [33] modified AHP fuzzy theory and, based on his “Extension principle,” used triangular fuzzy number as every evaluation index. FAHP converts the opinions of experts from previous definite values to fuzzy numbers and membership functions presents triangular fuzzy numbers in paired comparison of matrices to develop FAHP. The triangular fuzzy number can be defined as follows:

$${\mathsf{\mu }}_{\widetilde{\mathrm{A}1}}\left(\mathrm{x}\right)=({\mathrm{l}}_1,{\mathrm{m}}_1,{\mathrm{u}}_1);{\mathsf{\mu }}_{\widetilde{\mathrm{A}2}}\left(\mathrm{x}\right)=({\mathrm{l}}_2,{\mathrm{m}}_2,{\mathrm{u}}_2);{\mathrm{l}}_{\mathrm{i}}\le {\mathrm{m}}_{\mathrm{i}}\le {\mathrm{u}}_{\mathrm{i}}.$$

• Fuzzy number addition

  $$({\mathrm{l}}_1,{\mathrm{m}}_1,{\mathrm{u}}_1)(+)({\mathrm{l}}_2,{\mathrm{m}}_2,{\mathrm{u}}_2)=({\mathrm{l}}_1+{\mathrm{l}}_2,{\mathrm{m}}_1+{\mathrm{m}}_2,{\mathrm{u}}_1+{\mathrm{u}}_2)$$

  (3)
• Fuzzy number subtraction

  $$({\mathrm{l}}_1,{\mathrm{m}}_1,{\mathrm{u}}_1)(-)({\mathrm{l}}_2,{\mathrm{m}}_2,{\mathrm{u}}_2)=({\mathrm{l}}_1-{\mathrm{u}}_2,{\mathrm{m}}_1-{\mathrm{m}}_2,{\mathrm{u}}_1-{\mathrm{l}}_2).$$

  (4)
• Fuzzy number multiplication

  $$({\mathrm{l}}_1,{\mathrm{m}}_1,{\mathrm{u}}_1)\left(\mathrm{x}\right)({\mathrm{l}}_2,{\mathrm{m}}_2,{\mathrm{u}}_2)=({\mathrm{l}}_1{\mathrm{l}}_2,{\mathrm{m}}_1{\mathrm{m}}_2,{\mathrm{u}}_1{\mathrm{u}}_2),{\mathrm{l}}_1\ge 0,{\mathrm{l}}_2\ge 0.$$

  (5)
• Fuzzy number division

  $$({\mathrm{l}}_1,{\mathrm{m}}_1,{\mathrm{u}}_1\left)\right(/\left)\right({\mathrm{l}}_2,{\mathrm{m}}_2,{\mathrm{u}}_2)=({\mathrm{l}}_1/{\mathrm{u}}_2,{\mathrm{m}}_1/{\mathrm{m}}_2,{\mathrm{u}}_1/{\mathrm{l}}_2),{\mathrm{l}}_1\ge 0,{\mathrm{l}}_2>0.$$

  (6)

Defuzzification:

“Competency and Scenario” evaluation values are fuzzy numbers. This research employs fuzzy AHP to find the fuzzy preference weights. This is also called “Defuzzification.”

Center of area, COA or Center of gravity, COG:

$$COG=\frac{{\displaystyle \sum \mu (y_i)·y_i}}{{\displaystyle \sum \mu (y_i)}}.$$

(7)

Center-of-sums, COS:

$$COS=\frac{{\displaystyle \sum y_{jc}{\displaystyle \int {\mu }_j}(y_j)d{y}_j}}{{\displaystyle \sum {\displaystyle \int {\mu }_j(y_j)d{y}_j}}}$$

(8)

Mean of maximum, MOM:

$$MOM={\displaystyle \sum _{j=1}^m\frac{z_j}{m}}$$

(9)

Teng and Tzeng [35] used the gravity method to defuzzify the fuzzy weight to obtain the best “Nonfuzzy” or so-called “Best crisp value.” Supposing $\widetilde{{\mathrm{W}}_{\mathrm{i}}}$ = (${\mathrm{l}}_{\mathrm{i}},{\mathrm{m}}_{\mathrm{i}},{\mathrm{u}}_{\mathrm{i}}$), the best “Nonfuzzy” value ${\mathrm{NF}}_{\mathrm{i}}$ will be:

$$N{F}_i=\left[\left(u_i-l_i\right)+\left(m_i-l_i\right)\right]/3+l_i.$$

(10)

After, the defuzzified fuzzy number could gain a certain value which was based on the weight to be prioritized. At the second stage of this study, the collected and analyzed questionnaire in this study obtained a fuzzy value of the “Competency and Scenario.” This study used Equation (10) to conduct defuzzication of the values and converted those values into definite values. Then, we may understand the “Competency and Scenario’s” weight of military helicopter EBT’s decision process.

3.2. Fuzzy Delphi Method (FDM)

The definitions of fuzzy sets were manipulated to develop Fuzzy multi-criteria decision making (FMCDM) methodology to resolve the lack of precision in assigning importance weights of criteria and the ratings of alternatives against evaluation criteria. The FDM steps are as follows:

Setting up triangular fuzzy numbers:

This study used FDM to obtain fuzzy evaluation values via experts, and converted via TFN. Supposing there were m experts and n competency factors, the number e expert’s initial importance evaluation values f, the equation will be:

$$\begin{gathered} {\tilde{\mathrm{W}}}_{ef}=(x_{ef},y_{ef},z_{ef}) \\ e=1,2,3,\dots m,f=1,2,3,....,n,1\le x_{ef},y_{ef},z_{ef}\le 9. \end{gathered}$$

(11)

As shown in

Table 1. Initial evaluation values of importance.

Serial No	Fill in □ with “V”	Very Unimportant	Not Important	Fair	Important	Very Important	Fuzzy Number
1	Communication			☑			(3,5,7)
2	Communication				☑		(5,7,9)
3	Communication			☑			(3,5,7)
4	Communication				☑		(5,7,9)
5	Communication			☑			(3,5,7)
6	Communication			☑			(3,5,7)
7	Communication				☑		(5,7,9)
8	Communication				☑		(5,7,9)
9	Communication				☑		(5,7,9)
10	Communication					☑	(7,9,9)
11	Communication					☑	(7,9,9)
12	Communication				☑		(5,7,9)
13	Communication					☑	(7,9,9)
14	Communication			☑			(3,5,7)
15	Communication				☑		(5,7,9)
16	Communication				☑		(5,7,9)
17	Communication			☑			(3,5,7)
18	Communication					☑	(7,9,9)
19	Communication				☑		(5,7,9)
20	Communication		☑				(1,3,5)
21	Communication					☑	(7,9,9)
22	Communication			☑			(3,5,7)
23	Communication			☑			(3,5,7)
24	Communication			☑			(3,5,7)
25	Communication				☑		(5,7,9)

, there were 25 experts that underwent the questionnaire survey in this study. Taking the f competency factor (communication) as an example, the initial evaluation values will be converted to fuzzy weight number.

The importance of f competency factor’s triangular fuzzy number ${\tilde{\mathrm{W}}}_{\mathrm{f}}$:

$$\begin{array}{c}{\tilde{\mathrm{W}}}_f=x_f,y_f,z_f,\mathrm{f}=1,2,3,\dots ,n\\ x_f=Min\left\{x_{ef}\right\},y_f=({{\displaystyle \prod }}_{e=1}^m{y}_{ef}{)}^{\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$m$}\right.},z_f=\mathrm{Max}\left\{z_{ef}\right\},\hfill \\ x_{ef}\le y_{ef}\le z_{ef}\\ \mathrm{If}{x}_f=1\hfill \\ y_f=\sqrt[25]{\begin{array}{c}5\times 7\times 5\times 7\times 5\times 5\times 7\times 7\times 7\times 9\times 9\times 7\times 9\times \\ 5\times 7\times 7\times 5\times 9\times 7\times 3\times 9\times 5\times 5\times 5\times 7\end{array}}=6.30\hfill \\ z_f=9\hfill \\ \mathrm{Fuzzy}\mathrm{value}=\frac{1+6.30+9}{3}=5.43.\end{array}$$

(12)

3.3. Fuzzy Analytic Hierarchy Process (FAHP)

In this study, the geometric mean method was extended the AHP to the situation of using linguistic variables. In the FAHP methods, Buckley [36] used Fuzzy pairwise comparison matrices constructed by using linguistic evaluations with respect to the decision makers’ judgments. In replacement of AHP definite value disadvantage, an advantage might be reached.

FAHP converts the opinions of experts from previous definite values to fuzzy numbers and membership functions present triangular fuzzy numbers in paired comparisons of matrices. In this study, the three-point marking method proposed by Zhong [37] was used to obtain the opinions of experts, using the approach human thinking model to achieve more reasonable evaluation criteria. For example, when comparing the importance of A and B, if m is “quite strong” its mental feeling might be between u “extreme strong” and l “little bit strong.” The parameter l denotes the smallest possible value, m the most promising value, and u the largest possible value that describes a fuzzy event. Those who answered the questionnaire filled m in m ∈ [l,u], as shown in

Table 2. Three-points marking method [37].

	A:B
	Absolute Strong		Extreme Strong		Quite Strong		Slightly Strong		Equal	Slightly Weak		Quite Weak		Extreme Weak		Absolute Weak
	9	8	7	6	5	4	3	2	1	1/2	1/3	1/4	1/5	1/6	1/7	1/8	1/9
A			$\mathrm{u}$		$\mathrm{m}$		$\mathrm{l}$											B

, m filled in at 5, the real situation, 3–7 can also be a possible answer.

The process of FAHP Buckley [36] proposed is listed as following:

(1)

Setting up hierarchy architecture

“Scenario Aspects and Criteria” went through the expert questionnaire and FDM, and based on the subject of this study, “Evidence-based Training on the Crews Researches Management of Helicopter Flight crew” was used to construct hierarchy model shown as Figure 1. The uppermost level was the core of this study which analyzed every “Scenario Aspects and Criteria,” evaluated the weight on the critical decision making process; the second level was “Scenario/Aspect”; and the third level was the “Competency/Criteria”.

(2)

Constructing pairwise comparison matrices

According to Buckley’s [36] proposed method, this study used a questionnaire survey, three-point marking method, and triangular fuzzy numbers to construct pairwise comparison matrices (supposing n elements):

$$\tilde{A}=\left[{\tilde{a}}_{kr}\right]=\left[\begin{array}{cccc}{\tilde{a}}_{11}& {\tilde{a}}_{12}& \cdots & {\tilde{a}}_{1n}\\ {\tilde{a}}_{21}& {\tilde{a}}_{22}& ⋮& {\tilde{a}}_{2n}\\ ⋮& \cdots & \ddots & ⋮\\ {\tilde{a}}_{n1}& {\tilde{a}}_{n2}& \cdots & {\tilde{a}}_{nn}\end{array}\right]$$

(13)

where, ${\tilde{a}}_{kr}$ k, and r are fuzzy comparison values.

(3)

Fuzzy Matrix consistency

Supposing A = [${\mathrm{a}}_{\mathrm{kr}}$] is a “Positive reciprocal matrix,” and $\tilde{\mathrm{A}}$ = $\left[{\tilde{\mathrm{a}}}_{\mathrm{kr}}\right]$ is a fuzzy positive reciprocal matrix, then each value is presented in ${\tilde{\mathrm{a}}}_{\mathrm{kr}}$ = (${\mathrm{l}}_{\mathrm{kr}}$, ${\mathrm{m}}_{\mathrm{kr}},{\mathrm{u}}_{\mathrm{kr}}$). According to Buckley [36], it was proposed that the consistency of fuzzy matrix will be as follows:

If A = [${\mathrm{a}}_{\mathrm{kr}}$] is consistent, then $\tilde{\mathrm{A}}$ = $\left[{\tilde{\mathrm{a}}}_{\mathrm{kr}}\right]$ will be consistent.

(4)

Calculating fuzzy weighting

If $\tilde{A}$ = $\left[{\tilde{a}}_{kr}\right]$ had T elements, the fuzzy weighting of number k element ${\tilde{w}}_k$:

$${\tilde{w}}_k={\tilde{r}}_k\otimes \tilde{\beta },k=1,2,.....T.$$

(14)

Additionally, ${\tilde{r}}_k=({{\displaystyle \prod }}_{k=1}^T{a}_{kr}{)}^{\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$T$}\right.}$, $\tilde{\beta }={(\widetilde{r_1}\oplus \widetilde{r_2}\oplus ....\oplus \widetilde{r_T})}^{-1}$.

(Definition of $\oplus$ is shown in Equation (3))

(5)

Calculating scenario factor fuzzy weight in hierarchy structure

According to Buckley [36], the calculating should take all fuzzy weight of scenario factors to multiply the fuzzy weight of competency factors. Then, we can obtain the scenario factor fuzzy weight in the hierarchy structure.

(6)

Defuzzification

Using simple center of gravity method (Equation (7)) to defuzzify the fuzzy weight, each scenario factor and competency factor can obtain a definite value. Then, we can conduct the prioritizing action to evaluate critical factors of the helicopter crew resource management process that may influence capabilities.

4. Empirical Analysis

This study conducted a two-stage questionnaire survey; the first stage surveyed various types of helicopter instructors with the contents. Then, it used FDM to analyze the data and delete the less important results. The second stage used a pairwise comparison questionnaire and selected its importance; then, FAHP was used to analyze its data to obtain the relative importance of all scenario factors and competency factors; then those factors were prioritized.

4.1. First Stage of Questionnaire Analysis

The objective of this stage was to obtain a contents evaluation via military and civil helicopter instructors’ professional competencies and experiences. It took the whole month of March 2017 to conduct the expert questionnaire survey. Thirty questionnaires were issued, with a 100% retrieval rate, and 25 were valid. These flight instructors were from Taoyuan D Airfield, Taichung W Airfield, and Tainan X Airfield; the average flight experience was above 18 years with 2099 flight hours. Detailed information is listed in

Table 3. First stage of research objects of the questionnaire survey.

Item	Airfield	Aircraft Type	Professional Position	Flight Experience (Years)	Gender	Flight Hours
1	D	A	Instructor	15	M	1580
2	D	A	Instructor	14	M	1027
3	D	A	Instructor	15	M	1778
4	D	A	Test pilot	17	M	1845
5	D	U	Instructor	19	M	1590
6	D	O	Instructor	15	M	1889
7	W	O	Instructor	23	M	3530.
8	W	A	Instructor	19	M	2756
9	W	A	Instructor	15	M	1640.
10	W	A	Instructor	22	M	3920
11	W	A	Instructor	13	M	1238
12	W	U	Instructor	18	M	1855
13	W	U	Test pilot	17	M	1590
14	W	U	Instructor	19	M	1946
15	W	U	Instructor	15	F	1220
16	W	U	Instructor	17	M	1670
17	W	O	Instructor	19	M	2830
18	X	O	Instructor	19	M	4200
19	X	U	Instructor	15	M	1930
20	X	C	Instructor	18	M	1688
21	X	C	Instructor	19	M	2375
22	X	T	Instructor	19	M	1740
23	X	T	Instructor	19	M	2030
24	X	U	Instructor	23	M	2240
25	X	U	Instructor	28	M	2388

. There are seven types of helicopters used in the military and civil aviation: AH-1W (military 4) and AH-64E (military 4); C for Cargo helicopters, such as CH-47SD (military 2); O for Light Observation helicopters, such as OH-58D (military 4); and U for Upgrades and variations on helicopters or full-featured helicopters, such as TH-67A (military 2), UH-1H (4), and UH-60M (military 2 and civil. 3). Regarding gender, M represents male and F represents female.

The calculation of questionnaires was explained in Section 3, and all aspects and criteria fuzzy numbers are listed in

Table 4. Triangular fuzzy numbers of aspects and criteria of initial evaluation.

Aspects	Criteria	Mini V	Mean V	Maxi V	Total	Defuzzification
Evidence-based Training on the Crew’s Helicopter Resources Management	Take-off with different tailwind but not informed	5	7.9	9	21.9	7.3
	High-altitude area or high payload made horsepower insufficient	3	7.83	9	19.83	6.61
	Multiple external NAV failure. NAV capability deteriorated	3	7.65	9	19.65	6.55
	Under “Visual Flight Rule” (VFR), visibility deteriorated (cloud or rain)	5	7.21	9	21.21	7.07
	Recognized “Air Traffic Control” (ATC) unsafe terrain clearance	3	7.56	9	19.56	6.52
	Increasing tailwind on final approach (not reported)	3	7.59	9	19.59	6.53
Take-off with different tailwind but not informed	Communication	3	7.68	9	19.68	6.56
	Problem Solving and Decision Making	3	8.13	9	20.13	6.71
	Workload Management	1	7.13	9	17.13	5.71
High-altitude area or high payload made horsepower insufficient	Problem Solving and Decision Making	3	8.28	9	20.28	6.76
	Communication	3	7.59	9	19.59	6.53
	Leadership and Teamwork	5	7.18	9	21.18	7.06
	Workload Management	3	5.76	9	17.76	5.92
Multiple external NAV failure. NAV capability deteriorated	Application of Procedures	3	7.68	9	19.68	6.56
	Communication	3	8.1	9	20.1	6.7
	Automation	1	6.29	9	16.29	5.43
	Problem Solving and Decision Making	5	7.36	9	21.36	7.12
	SA	3	8.16	9	20.16	6.72
	Workload Management	3	6.39	9	18.39	6.13
Under “Visual Flight Rule” (VFR), visibility deteriorated (cloud or rain)	Communication	3	7.8	9	19.8	6.6
	Leadership and Teamwork	3	7.56	9	19.56	6.52
	Problem Solving and Decision Making	3	8.37	9	20.37	6.79
	SA	3	7.38	9	19.38	6.46
Recognized “Air Traffic Control” (ATC) unsafe terrain clearance	Application of Procedures	3	6.19	9	20.19	6.73
	Communication	3	7.83	9	19.83	6.61
	Problem Solving and Decision Making	5	7.18	9	21.18	7.06
	SA	1	7.1	9	17.1	5.7
Increasing tailwind on final approach (not reported)	Communication	3	6.63	9	18.63	6.21
	Problem Solving and Decision Making	3	7.68	9	19.68	6.56
	SA	3	8.04	9	20.04	6.68

. For screening out numerous factors, this study set the fixed threshold value as the deleted reference.

At this stage of the expert questionnaire, the importance of the initial aspects and criteria were evaluated by experts with a “five-response scale.” If the initial important evaluation value was higher than 5, this meant the initial scenario had different importance levels. According to Klir and Folger [38], when decision making has too few measurement indicators available, the threshold value can be lowered to solve this problem. In this study, the average threshold value was 6.50; the detailed results are shown in

Table 5. Fuzzy delphi method screening evaluation indexes.

Aspects	Criteria	Mini V	Mean V	Maxi V	Total	Defuzzification
Evidence-based Training on the Crew’s Helicopter Resources Management	Take-off with different tailwind but not informed	5	7.9	9	21.9	7.3
	High-altitude area or high payload made horsepower insufficient	3	7.83	9	19.83	6.61
	Multiple external NAV failure. NAV capability deteriorated	3	7.65	9	19.65	6.55
	Under “Visual Flight Rule” (VFR), visibility deteriorated (cloud or rain)	5	7.21	9	21.21	7.07
	Recognized “Air Traffic Control” (ATC) unsafe terrain clearance	3	7.56	9	19.56	6.52
	Increasing tailwind on final approach (not reported)	3	7.59	9	19.59	6.53
Take-off with different tailwind but not informed	Communication	3	7.68	9	19.68	6.56
	Problem Solving and Decision Making	3	8.13	9	20.13	6.71
High-altitude area or high payload made horsepower insufficient	Problem Solving and Decision Making	3	8.28	9	20.28	6.76
	Communication	3	7.59	9	19.59	6.53
	Leadership and Teamwork	5	7.18	9	21.18	7.06
Multiple external NAV failure. NAV capability deteriorated	Application of Procedures	3	7.68	9	19.68	6.56
	Communication	3	8.1	9	20.1	6.7
	Problem Solving and Decision Making	5	7.36	9	21.36	7.12
	SA	3	8.16	9	20.16	6.72
Under “Visual Flight Rule” (VFR), visibility deteriorated (cloud or rain)	Communication	3	7.8	9	19.8	6.6
	Leadership and Teamwork	3	7.56	9	19.56	6.52
	Problem Solving and Decision Making	3	8.37	9	20.37	6.79
	SA	3	7.56	9	19.56	6.52
Recognized “Air Traffic Control” (ATC) unsafe terrain clearance	Application of Procedures	3	6.19	9	20.19	6.73
	Communication	3	7.83	9	19.83	6.61
	Problem Solving and Decision Making	5	7.18	9	21.18	7.06
Increasing tailwind on final approach (not reported)	Problem Solving and Decision Making	3	7.68	9	19.68	6.56
	SA	3	8.04	9	20.04	6.68

.

According to Table 5, we can understand the fuzzy values of the importance of aspects and criteria of helicopter crew resource management (fuzzy values were between 6.50 to 7.30). After prioritization from high to low, the meanings were as follows.

The competencies that did not reach a fixed threshold value were as follows:

• The “workload management” competency of the following three scenarios: “Increasing tailwind on final approach (not reported),” “High-altitude area or high payload made horsepower insufficient,” and “Multiple external NAV failure. NAV capability deteriorated” represents “Manages available resources efficiently to prioritize and perform tasks in a timely manner under all circumstances.” After a discussion with experts, most of them considered these three scenarios as not sudden occurrences and preventable because the technology of modern aircraft systems or equipment, especially backup and warning systems, has progressed dramatically, allowing the flight crew to make judgments and react in time; therefore, work pressure or workloads can be reduced. Thus, “workload management” competency did need to be listed in those three scenarios.
• In the scenario “Multiple external NAV failure. NAV capability deteriorated,” automation means “controls the aircraft flight path through automation, including appropriate use of flight management system(s) and guidance.” Most of the experts considered the aircraft operation, navigation, and/or the instrument systems of helicopters to be inferior to those of commercial fixed-wing aircraft and relied on manual operation for most of the operation. Thus, a partial system abnormal state might affect automation operation. If the flight crew complied with the emergency operation procedure and overrode automation to employ manual operation, this might allow them to seek the Ground Control Approach (GCA) for help. Therefore, automation can be ruled out.
• In the scenario “Recognized ‘Air Traffic Control’ (ATC) unsafe terrain clearance,” SA means “Perceives and comprehends all of the relevant information available and anticipates what could happen that may affect the operation.” Most experts thought that nonstandard landing zones or hindrances around mountain areas were unpredictable. Normally, the flight crew should conduct pre-landing checks in advance and remind and communicate with each other to confirm hindrances around mountain areas and execute an emergency departure procedure when necessary. Thus, SA was not the most important consideration and should be deleted.
• In the scenario “Increasing tailwind on final approach (not reported),” communication means “Demonstrates effective oral, non-verbal, and written communications in normal and non-normal situations.” Both sides should use standard terminology with correct, clear, and precise communication in mind to communicate to reduce communication errors. Therefore, most of the experts thought that harsh wind might have a greater influence on helicopters. In addition, unlike fixed-wing aircraft, helicopters possess low airspeed and low-altitude hovering characteristics. It is unnecessary for a long and straight runway for the final approach for landing; but when the altitude and speed decreased, ground hindrances were closer, and sudden crosswind or gust wind was encountered, which might have resulted in operations due to rotary-wing aircraft limitations and increased the risk of coming closer to ground hindrances. Thus, the experts thought that most control pilots would respond to handle the situation first instinctively but not engage in crew communication. Thus, communication was inapplicable in this situation and should be deleted.

After screening via FDM, the results were as follows: In the scenario “Increasing tailwind on final approach (not reported),” workload management was deleted; in the scenario “High-altitude area or high payload made horsepower insufficient,” workload management was deleted; in the scenario “Multiple external NAV failure. NAV capability deteriorated,” automation and workload management were deleted; in the scenario “Recognized Air Traffic Control (ATC) unsafe terrain clearance,” SA was deleted. Six competencies within six scenarios were deleted, so of the 24 competencies, 18 were left as the basic reference for “Evidence-based training on the crew’s helicopter resources management” second-stage questionnaire survey (see Table 5).

4.2. The Second-Stage Questionnaire Analysis

The objective of this stage was to obtain the experts’ evaluation via military and civil helicopter instructors’ professional competencies and experiences. As shown in

Table 6. Aspects and criteria of Evidence-Based Training (EBT) on the CRM of helicopters.

Aspects	Criteria
Evidence-based Training on the Crew’s Helicopter Resources Management	Take-off with different tailwind but not informed
	High-altitude area or high payload made horsepower insufficient
	Multiple external NAV failure. NAV capability deteriorated
	Under “Visual Flight Rule” (VFR), visibility deteriorated (cloud or rain)
	Recognized “Air Traffic Control” (ATC) unsafe terrain clearance
	Increasing tailwind on final approach (not reported)
Take-off with different tailwind but not informed	Communication
	Problem Solving and Decision Making
High-altitude area or high payload made horsepower insufficient	Workload Management
	Problem Solving and Decision Making
	Communication
Multiple external NAV failure. NAV capability deteriorated	Leadership and Teamwork
	Workload Management
	Communication
	Leadership and Teamwork
Under “Visual Flight Rule” (VFR), visibility deteriorated (cloud or rain)	Problem Solving and Decision Making
	SA
	Application of Procedures
	Communication
Recognized “Air Traffic Control” (ATC) unsafe terrain clearance	Automation
	Problem Solving and Decision Making
	SA
Increasing tailwind on final approach (not reported)	Workload Management
	Application of Procedures

, the scenario and competency evaluation criteria were weighted to understand the importance of “EBT on the CRM of helicopters” critical factors. The questionnaire survey was finished in June 2017. Thirty questionnaires were distributed and recollected, and 26 were valid. The objects of this questionnaire survey were the same as in stage one.

4.2.1. Consistency Testing

To ensure the validity of this questionnaire survey, after recollecting the questionnaire, the consistency testing followed. Saaty [39] suggested the “Consistency Index” (CI) and “Consistency ratio” (CR) for conducting consistency resting. In this study, the second-stage questionnaire content had seven items, and each relative scenario importance and the competency factors were evaluated. The scenarios were listed as follows: take-off with different tailwind but not informed; high-altitude area or high payload made horsepower insufficient; multiple external NAV failure, NAV capability deteriorated; under “Visual Flight Rule” (VFR), visibility deteriorated (cloud or rain); recognized “Air Traffic Control” (ATC) unsafe terrain clearance; and increasing tailwind on final approach (not reported). To avoid the inconsistency of pairwise comparison, these six scenarios and competency factors underwent consistency testing to obtain their CRs. When CR ≤ 0.1, the evaluation matrix was highly consistent; if CR > 0.1, the evaluation matrix was deleted and not used in fuzzy weighting. The CR values are listed in

Table 7. Consistency ratio (CR) values of the second-stage questionnaire.

No. of Questionnaire	Relative IMPORTANCE of Scenario Factor and Competency Factor	Take-off with Different Tailwind but not Informed	High-Altitude Area or High Payload Made Horsepower Insufficient	Multiple External NAV Failure. NAV Capability Deteriorated	Under “Visual Flight Rule” (VFR), Visibility Deteriorated (Cloud or Rain)	Recognized “Air Traffic Control” (ATC) Unsafe Terrain Clearance	Increasing Tailwind on Final Approach (not Reported)
1	0.0024	0	0.0001	0.0001	0.0001	0.0001	0
2	0.013	0	0.0001	0.0003	0.0025	0.0001	0
3	0.0018	0	0.0022	0.0012	0.0009	0.0023	0
4	0.0028	0	0.0013	0.0011	0.0009	0.0001	0
5	0.003	0	0.003	0.0024	0.0024	0.0030	0
6	0.0025	0	0.0001	0.0106	0.0002	0.0001	0
7	0.0049	0	0.0010	0.0036	0.0032	0.002	0
8	0.0032	0	0.0003	0.0007	0.0007	0.0001	0
9	0.0004	0	0.0001	0.0007	0.0001	0.0001	0
10	0.0034	0	0.0024	0.0001	0.0003	0.0003	0
11	0.001	0	0.0023	0.0012	0.004	0.0005	0
12	0.0106	0	0.0002	0.0008	0.0006	0.0001	0
13	0.0005	0	0.0013	0.0002	0.0005	0.0001	0
14	0.0005	0	0.0011	0.0016	0.0011	0.0003	0
15	0.0097	0	0.0045	0.005	0.0045	0.0002	0
16	0.0026	0	0.0017	0.0025	0.0041	0.0001	0
17	0.0027	0	0.0010	0.0002	0.0001	0.0003	0
18	0.0024	0	0.0024	0.0024	0.0027	0.0030	0
19	0.0012	0	0.0001	0.0002	0.0011	0.0043	0
20	0.0005	0	0.0002	0.0005	0.0001	0.0001	0
21	0.0054	0	0.0024	0.0012	0.0001	0.0003	0
22	0.001	0	0.0036	0.0027	0.0032	0.003	0
23	0.0026	0	0.0031	0.0018	0.0027	0.003	0
24	0.0047	0	0.0026	0.0016	0.0005	0.0040	0
25	0.0007	0	0.0011	0.0019	0.0004	0.0010	0
26	0.0019	0	0.0005	0.0001	0.0001	0.0001	0

. In this study, after all the experts completed the questionnaire survey and checked each factor CR ≤ 0.1, the evaluation matrix was acceptable except take-off with different tailwind but not informed and increasing tailwind on final approach (not reported). These two scenarios only had two competency factors to compare and did not have any inconsistencies; thus, they were accepted into the prioritizing process.

After the second-stage screening evaluation indexes, the fuzzy weighting values of the scenario and competency were obtained; then, we used the simple center of gravity method to defuzzify the fuzzy weight to obtain the definite value as the reference for prioritizing. The results are listed in

Table 8. Weighting values of criteria.

Aspects		Criteria Scenario and Competency Factors	Fuzzy Weighting Value	Nonfuzzy Weighting Value	CR Value
Scenario 1		Take-off with different tailwind but not informed	(0.228,0.230,0.227)	0.228	0.0689
Scenario 2		High-altitude area or high payload made horsepower insufficient	(0.249,0.249,0.244)	0.248
Scenario 3		Multiple external NAV failure. NAV capability deteriorated	(0.167,0.169,0.169)	0.168
Scenario 4		Under “Visual Flight Rule” (VFR), visibility deteriorated (cloud or rain)	(0.136,0.134,0.135)	0.135
Scenario 5		Recognized “Air Traffic Control” (ATC) unsafe terrain clearance	(0.116,0.114,0.118)	0.116
Scenario 6		Increasing tailwind on final approach (not reported)	(0.104,0.103,0.108)	0.105
Scenario 1	S11	Communication	(0.575,0.581,0.575)	0.577	0
	S12	Problem Solving and Decision Making	(0.425,0.419,0.425)	0.423
Scenario 2	S21	Communication	(0.266,0.263,0.264)	0.264	0.0021
	S22	Leadership and Teamwork	(0.293,0.289,0.290)	0.290
	S23	Problem Solving and Decision Making	(0.441,0.448,0.447)	0.445
Scenario 3	S31	Application of Procedures	(0.214,0.208,0.205)	0.209	0.0109
	S32	Communication	(0.180,0.174,0.174)	0.176
	S33	Problem Solving and Decision Making	(0.283,0.281,0.282)	0.282
	S34	SA	(0.324,0.337,0.333)	0.333
Scenario 4	S41	Communication	(0.186,0.182,0.184)	0.184	0.0016
	S42	Leadership and Teamwork	(0.189,0.185,0.187)	0.187
	S43	Problem Solving and Decision Making	(0.299,0.298,0.294)	0.297
	S44	SA	(0.325,0.335,0.335)	0.332
Scenario 5	S51	Application of Procedures	(0.449,0.449,0.445)	0.448	0.0006
	S52	Communication	(0.211,0.208,0.214)	0.211
	S53	Problem Solving and Decision Making	(0.340,0.343,0.341)	0.342
Scenario 6	S61	Problem Solving and Decision Making	(0.438,0.431,0.438)	0.436	0
	S62	SA	(0.562,0.569,0.562)	0.564

.

4.2.2. Aspects and Criteria of Questionnaire

As the weighting values listed in Table 8, take-off with different tailwind but not informed; high-altitude area or high payload made horsepower insufficient; multiple external NAV failure, NAV capability deteriorated; under “Visual Flight Rule” (VFR), visibility deteriorated (cloud or rain); recognized “Air Traffic Control” (ATC) unsafe terrain clearance; and increasing tailwind on final approach (not reported). These six scenarios had different criteria and independent training courses.

The meanings of the weighting values comparison were as follows:

(1)

The most dangerous scenario was No. 2, “high-altitude area or high payload made horsepower insufficient,” weighting value 0.248; all experts expected each flight to take-off and land safely. However, helicopters have performance limitations, particularly when operated in a highly complicated environment. During flight in a high-altitude mountain area, flight crew teamwork and decision-making capabilities were even more important.

(2)

The second most notable scenario was No. 1, “take-off with different tailwind but not informed,” weighting value 0.228; the third was the No. 3 scenario, “Multiple external NAV failure. NAV capability deteriorated,” weighting value 0.168. The experts considered helicopters mainly operated under VFR, airfield, and flight route as not fixed without a related standard, which made the flight route and weather changes hard to control. That is why the IFR SOP, a backup plan, decision making and alertness regarding environmental changes are important.

(3)

The fourth most notable scenario was No. 4, “Multiple external NAV failure. NAV capability deteriorated,” weighting value 0.135. The experts thought that flight at a low altitude might result in radio communication and radar control difficulties. Thus, due to compound weather condition changings and instrument navigation system malfunctions, the flight crew should pay more attention to the rapidly changing weather environment with vigilance, and flight crew communication and teamwork are needed to make instantaneous decisions.

(4)

The fifth most notable scenario was No. 5, “Recognized “Air Traffic Control” (ATC) unsafe terrain clearance,” weighting value 0.116. The experts thought that CAA did not establish and renew the hindrance information for the light-weight aircraft visual flight corridor, such as high-voltage electric wires, aerial lifts and cable transports since the Aviation Law did not regulate the temporary airfield application or related regulation. The responsibility rested on the flight crew mission application orientation, which had to check possible hindrance locations, conduct a flight crew crosscheck, and ensure good communication and decision making.

(5)

The last most notable scenario was No. 6, “increasing tailwind on final approach (not reported),” weighting value 0.105; the value seemed slightly low, but this did not mean it was unimportant. The experts thought that the helicopter was operated in the outer airfield without wind speed and wind direction information, and the rapid weather change and payload difference led to the tail rotor malfunctioning. Thus, the flight crew should pay more attention to flight environment changes and contingency decision making to cope with sudden occurrences.

4.2.3. Criteria Analysis of Questionnaire

From the weighting values listed in Table 8, the competencies’ weighting values and importance of different scenarios that the flight crew was faced with should be determined to cope with these emergency situations.

Aspect 1: Take-off with different tailwind but not informed: Helicopter operation relies on VFR mainly; it was short of a standard airfield and without meteorological personnel to offer a weather report. To prevent LOC-I occurrence, the flight crew should reinforce the “Communication” (0.577) and “Problem Solving and Decision Making” (0.423) CRM criteria.

• All systems should be checked in accordance with the procedure before take-off. If there is any system failure or damage (such as Test No Go), the flight crew should discuss it orally. If the ground crew witness a smoking situation (such as engine overheating), they should notify the flight crew with hand signals, which is “postural communication,” to shut down the engine and extinguish the fire. Alternatively, after finishing radio system failure troubleshooting, which needs a control tower to conduct a radio check. All the situations mentioned above belong to the scope of communication and a pattern of two-way communication. Dai Qingji [40] points out that the receiver can evaluate and judge the message from the sender; we may conclude that this is good two-way communication. Good communication depends on effective verbal or non-verbal language, which includes eye contact, body language and hand gestures. The flight crew should have been aware of basic communication problems first and fully understood that effective communication is one of their responsibilities [41].
• Aspect 2: High-altitude area or high payload made horsepower insufficient: The helicopter operates in a mountainous or marine area, which is often complicated. To prevent LOC-I occurrence, the flight crew should reinforce the “Problem Solving and Decision Making” (0.445), “Teamwork” (0.290), and “Communication” (0.264) CRM criteria.
• The flight crew should choose from multiple alternatives or propose different solutions under extreme time pressure [31]. The judgment of pilot depends on their attitude to handling the crisis and their risk evaluation capability, which includes their personal knowledge, skills and experience, to generate a suitable decision and solve the crisis they are facing. All the factors mentioned above are related to flight decision making; this requires clear and specific responsibility allocation among the flight crew and a full understanding of cockpit instrument monitoring and the outside environmental situation. The message sends out its demand, is understood, and then action will be taken. If the action matches the expectation of the sender, it is an effective communication and flight safety can be achieved effectively.
• Aspect 3: Multiple external NAV failure. NAV capability deteriorated: Currently, the light aircraft visual flight corridor has an area of responsibility, flight route altitude restriction, weather constraints etc. Since there is no accurate reference available, the flight route obstacles are altitude and location along the visual flight corridor, low-visibility weather, arrival and departure separations; the helicopter location relies on radio traction, which might be ineffective when operating within mountainous and marine areas outside of the radio-monitoring range. Furthermore, poor weather conditions are an issue. To prevent CFIT occurrences, the flight crew should pay attention to the “SA” (0.333), “Problem Solving and Decision Making” (0.282), “Application of Procedures” (0.209), and “Communication” (0.176) CRM criteria. The flight crew should evaluate the conditions of their flight environment at all times and ensure they are under the best SA conditions. This is not an easy task even for a veteran flight crew. It requires the flight crew to obtain all the flight information, fully understand the whole situation and follow suitable rules. In other words, the operation relies on all sensation information and cockpit instrument reading observations to accomplish its intended purpose. This may allow the flight crew to have more time to draw the flight plan and execute better problem solving and flight decision making. For example, flight decision making to deal with a weather sudden change might include turning back to the home base or continuing toward the destination airfield or an alternative airfield. If the flight crew decides to maintain its flight plan, they should follow the IFR to conduct their operation with a standard radio communication procedure to contact ATC personnel; then, accurate, clear and precise communication can be well established and reduce the likelihood of miscommunication situations [42].
• Aspect 4: Under “Visual Flight Rule” (VFR), visibility deteriorated (cloud or rain): The helicopter operates at a low altitude or within a mountainous area, which might obstruct radio contact, and it might be hard for the ATC radar to identify low-altitude aircraft. To prevent CFIT occurrences, the flight crew should pay attention to the “SA” (0.332), “Problem Solving and Decision Making” (0.297), “Leadership and Teamwork” (0.187), and “Communication” (0.184) CRM criteria.
• Aspect 5: Recognized “Air Traffic Control” (ATC) unsafe terrain clearance: Currently, the Civil Aviation Law does not regulate temperate airfields. The airfield of a helicopter is different from a civil aircraft airfield. In other words, the helicopter did not have a safety standard to follow. The flight crew should confirm and understand an obstacle’s location before flight operation. To prevent CFIT occurrences, the flight crew should pay attention to the “Application of Procedures” (0.448), “Problem Solving and Decision Making” (0.342), and “Communication” (0.221) CRM criteria.

The reasons for this are avoiding an indistinct landing zone and uncleared obstacles and to address the importance of the application of procedures. The CRM practical and effective methods were recommended to adapt the CRM criteria into SOP for the flight crew to follow, such as the Special Call-out, Checks, Guidance, Normal Checklist, Quick Reference Handbook, Abnormal/Emergency Procedure, Manuals, and Job Aid complemented by the LOFT and EBT simulation training, which may ensure the accomplishment of the operation and safety at the same time.

• Aspect 6: Increasing tailwind on final approach (not reported): The helicopter did not have standard airfield or meteorological personnel to offer weather information, which exposed the helicopter to adverse environmental and meteorological conditions, such as the altitude of the operation area and payload. To prevent CFIT occurrences, the flight crew should pay attention to the “SA” (0.564) and “Problem Solving and Decision Making” (0.436) CRM criteria.

5. Conclusions

The CRM is the most powerful tool for reinforcing the flight crew performance of flight safety prevention efforts. This study tried to implement the CRM in the EBT training program via understanding the critical capabilities the flight crew should have when dealing with emergency and unexpected scenarios.

This study involved a literature review, case collecting, and analysis of experts’ opinions. Scenario-Based Training (SBT) or Evidence-Based Training (EBT) took the ICAO 9995 Appendices 6 Training Program Development Guidance-Generation 2 (Turboprop) as evaluation and training matrix references. These aspects were selected in accordance with what a helicopter might experience frequently: adverse weather, aircraft system management, aircraft system failure, hindrance, demanding weather conditions, payload/fuel management/performance errors, and six scenarios to simulate and construct the foundation of the evaluation aspect description. The simulation scenario was the foundation of this study. Then, we used the Fuzzy Analytic Hierarchy Process (FAHP) to conduct the evaluation of applying CRM to the EBT training mode and concluded with six aspects and 18 criteria, to develop the evaluation and training matrix references and to achieve the best CRM training effectiveness. This is the innovation and contribution of this study.

Innovation and Contribution

(1)

The EBT training program is applied to the civil aviation industry for commercial aircraft (jet proportion and turbo propeller engine) mainly. This study confirmed that the aspect and scenario of EBT can be expanded and applied to the helicopter CRM training program as well.

(2)

This study combined fuzzy theory to explore CRM training criteria, which were difficult to quantify. In addition, this study used FAHP to calculate the importance of applying CRM to the EBT training program. This study found that this research tool can extend its application to different types of aircraft flight crews. In other words, this tool is practical and extendable.