Pilots’ Performance and Workload Assessment: Transition from Analogue to Glass-Cockpit

Featured Application

Trainee pilots were observed during a transition from analog instruments to glass-cockpit display. Their heart rate variability and maneuver accuracy were monitored in order to understand effects of the transition on performance and workload. Results show that the transition may be considered as a performance, but not workload influencing factor. Observed knowledge should be used for aviation training applications.

Abstract

During their professional career, pilots often experience a change in workplace conditions in the form of an aircraft cockpit ergonomics change. Change of working conditions may impact their perception of flight data or the pilot’s psychophysiological condition, especially in cases of inexperienced pilots. The presented study deals with the influence of cockpit ergonomics change on the performance and pilot workload during a training course. We divided 20 subjects with no previous practical flying experience into two training groups (Gr. A and Gr. B). The flight training was focused on acquisition of basic piloting skills where both groups experienced cockpit ergonomics change in different training phases. The performance (piloting precision) was assessed based on deviations from predetermined parameters of the monitored flight manoeuvres. Heart rate variability qualified the extent of workload. The study showed the influence of the cockpit arrangement on piloting precision, where the transition to other type of cockpit ergonomics did not influence pilots’ subjective workload with statistical significance.

1. Introduction

Ergonomic flight deck design has become a crucial element of continuous improvement in aviation safety. Flight deck arrangement plays an essential role in pilot’s ability to obtain flight parameters and control the aircraft. Analog dials had been dominating aircraft instrument panels for many decades. The location of instruments on early aircraft was dictated by manufacturing requirements. Indicators were often installed in the immediate vicinity of measuring probes or sensors. On 1920s aircraft, it was not uncommon to find engine instruments installed on engine cowlings outside of the flight deck. Pilots’ efforts to effectively obtain information was hindered by seemingly random location of instruments [1]. Additionally, the lack of standard instrument layout made transition between different types of aircraft very difficult [2,3]. Each aircraft manufacturer and even some aircraft operators appeared to have their own ideas of the best instrument layout [4,5,6]. Although, the need for a standardized ergonomic panel arrangement was well recognized during World War Two [3], it was not until 1957 that a common standard was adopted by the Civil Aeronautics Board. The new amendment to 14 CFR Part 4b required all transport category aircraft to feature a “Basic T” arrangement of four instruments which present basic information as to airspeed, attitude, altitude, and direction of flight. This requirement remains unchanged and the exact same wording appears in the current regulation (14 CFR part 25 §25.1321 and CS 25.1321). Although, the location of indicators follows the same layout on both classic panels and glass cockpit panels, the visualisation of flight information is different. Glass cockpit panels feature vertical moving tapes for airspeed and altitude instead of standalone round dials. Visualisation of direction of flight on electronic displays differs among equipment manufactures: a depiction of a classic horizontal situation indicator (original Garmin G1000, Garmin Ltd., Olathe, KS, USA; Avidyne Entegra, Avidyne Corporation, Melbourne, FL, USA ), a section of a compass rose (recent Boeing models, The Boeing Company, Chicago, IL, USA ), horizontal moving tape (present Airbus models, Airbus SE, Toulouse, France ) or even a moving map with course guidance (Garmin G1000 NXi, Boeing 787, Garmin Ltd., Olathe, KS, USA ). Pilots ability to obtain information from instruments may be affected by the type of visualisation.

Pilots are trained to utilize the “Basic T” layout and develop an effective instrument scan technique. Identical instrument scan patterns are trained for both classic panels and glass cockpits [7,8]. However, by comparing eye tracking studies on classic panels [9,10,11] and recent research on glass cockpit aircraft [12,13,14], different instrument scan techniques are revealed for each type of equipment. Eye tracking studies focus on the role of central vision and the role of peripheral vision is not thoroughly examined [15]. Peripheral vision can be particularly important on large modern displays with the artificial horizon extending through airspeed and altitude tapes to the edges of the screen (e.g., Garmin G1000, Honeywell Aspen, Honeywell International Inc., Charlotte, NC, USA, Boeing 787 and 737MAX ). Such a design may also affect pilots’ perception and require adjustments to instrument scan.

Much attention is given to development of training syllabi for experienced pilots transitioning to modern aircraft [16] and full utilization of glass cockpit aircraft in initial training [17,18]. However, the effects of transition from analog instruments to glass cockpit panels during initial pilot training remain largely unexplored. One notable exception is study on performance of novices in simulated training [19]. In the study, 62 psychology students with no previous flight training experience received short theoretical instruction and performed three training flights in either a traditional display cockpit or a glass display cockpit. Flight performance was then measured in a test flight using either the same or different cockpit display. The results revealed consistently poorer performance on the test flight for participants using the glass cockpit compared to the traditional cockpit. This study demonstrated glass cockpits may present some difficulties for pilots and shows that further research in an environment closely representing flight training may be more informative. Wright et al. noted that typically, pilots will not transition between traditional and glass cockpits until receiving their private pilot license. Although, this was true at the time of their study, today, pilots may experience such transition at a very early stage of their training [19].

Full glass cockpit panels have been used on airliners since the introduction of Airbus A320 in 1988. It took another 15 years until this technology became available on general aviation aircraft; Cirrus SR20 with Avidyne Entegra was certified in 2003. The majority of general aviation aircraft have been available with glass cockpit avionics since late 2000s. However, The Great Recession significantly slowed down aircraft sales [20]. Flight schools seldom acquired new aircraft and if so, it was used in advanced phases of training. Recent growth in flight training forces flights schools to add new aircraft to their fleets rather than replace old equipment, thus mixed fleet is used in training. To add on complexity, old aircraft are being retrofitted with new avionics. One aircraft model can feature three different versions of flight deck: a new glass cockpit panel, an original analog instrument panel or a combination electronic flight instruments and analog engine instruments. The three different versions may be simultaneously used in one phase of pilot training, training pilots on all types of panels. European Flight Crew Regulations are very vague in terms defining requirements for transition between different aircraft variants, i.e., learning objectives focus on acquisition of theoretical knowledge of computer system design rather than on adjustment on new method of visualisation.

Transition from classic panels to glass cockpits and vice versa can cause deficient distribution of the pilot’s attention, increase in workload, and emotional and mental stress [21,22]. A distressed pilot may overlook or inadequately process information which ultimately leads to making improper decisions. Therefore, understanding effects of such transition gains in importance.

This paper investigates the influence of cockpit ergonomics change on performance and workload during early stages of initial pilot training on general aviation aircraft. An experimental training schedule was proposed and followed by two groups of subjects. The training was designed to address the transition from analogue to glass cockpit. The experimental setup aimed to discover whether the cockpit ergonomics change can have a negative influence on piloting and whether such change can negatively influence pilot’s psychophysiological condition.

2. Materials and Methods

The overall setup of research activities was carried with the main focus on precise determination of pilot performance and workload during the transition from analogue to glass cockpit (see Figure 1). This change was carried during experimental training.

2.1. Participants

Subjects were selected among full-time undergraduate university students of aviation study fields with no previous practical experience with flying. Applicants for the subjects (n ≈ 100) had to demonstrate basic theoretical knowledge in the field of navigation and the basics of flight, which was verified by testing. Further, applicants underwent performance testing, the so-called OR-test [23], which is focused on reaction time, short and long-term memory assessment. Apart from that, there were medical requirements pursuant to a class 2 medical certificate as per Commission Regulation (EU) No 1178/2011, Annex IV (Part-MED), as amended. Subjects were selected to ensure maximum sample uniformity, i.e., based on testing results comparability and not success rate.

This way, 20 subjects were selected and accepted for the study. Next, the subjects were randomly distributed into two groups—Group A (Gr. A) and Group B (Gr. B), and anonymized (each subject was given a number). One part of the experimental training was the same for both groups, the other was focused on a transition to a different cockpit ergonomics (see Section 2.2).

Uniformity of the established groups from the perspective of initial experience with flying, psychophysiological condition, age-match and sex-match was crucial to reduce inter-group evaluation bias. Age distribution of the study subjects was also comparable, i.e., average age of Gr. A participants was 22 ± 5 years and average age of Gr. B was 23 ± 3 years. The ratio of sex representation, i.e., ratio of men/women was 8/2 for Gr. A and 9/1 for Gr. B. Note that the results presented in Section 3 do not indicate that there would be significant differences in the monitored parameters with respect to the subjects gender. The evaluated parameters create distributions with no apparent outliers corresponding to respective genders.

All subjects received basic information about the principles and demands of the experiment, non-invasive means of data collection and the way of personal and collected data anonymization in line with the ethical principles for medical research involving human subjects [24]. The subjects, however, were not informed about the details of the prepared experimental training flights or the cockpit ergnonomics change (from analogue to glass-cockpit), which was scheduled during the training.

2.2. Training Description

Because subjects had no previous experience with practical flying, the training schedule was designed for them to acquire basic skills of simple piloting. Before the training flight execution, it was necessary to brief the subjects about cockpit ergonomics and the functionality of basic flight instrumentation. Although it was not the goal to train the subjects to pilot the aircraft in a way to ensure safety of flight in all its phases, all basic elements of aircraft control were explained during joint theoretical preparation, including the procedures for take-off and landing.

An example of executed flight and practical training schedule is depicted in Figure 2. During individual flights, four predetermined manoeuvres were always executed: horizontal stable flight, horizontal 360${}^{\circ }$ turn with 30${}^{\circ }$ bank, 180${}^{\circ }$ climb and descend turn with prescribed 15${}^{\circ }$ bank and vertical speed of 500 ft/min (see Figure 2A). During the manoeuvres, subjects had to maintain predetermined flight parameters, where the instruction to execute respective manoeuvre was given by instructor. For complete uniformity, all manoeuvres were executed in the mentioned sequence, three times during a flight. Average duration of a single flight was 66 ± 8 min.

The training (Figure 2B) was divided into three parts, where flight simulator was utilized for executing first 11 flights. The second part was dedicated to real flying, specifically flights Nr. 12, 16 and 17 were executed in real conditions with Diamond DA-40 aircraft. Real flights were complemented with simulator flying. Until this part, the training was the same for both groups and all flights were carried exclusively with analogue display of flight, navigation and engine readings (see Figure 1A). Data were collected from flights Nr. 2, 11, 12 and 17 where subjects were flying alone, with no intervention by the instructor. Data collection pertained to piloting precision and, simultaneously, heart electrical activity. Detailed description of data collection and pre-processing is provided in Section 2.3. Subjects were informed about the measurement first upon their arrival to the location, where the flight was to be executed.

The last part of the experimental training was dedicated to the transition from analogue to glass-cockpit, so the aircraft was now equipped with Garmin G1000 (Garmin Ltd., Olathe, Kansas, U.S.) flight instruments (see Figure 1B). Aircraft type was retained and Gr. A subjects were not informed about the cockpit ergonomics change. Data collection was performed during flight Nr. 18 (simulated) and 19 (real) with no intervention of the instructor.

Before the measurement, Gr. B subjects carried four simulated flights where the instructor explained principles of scanning the glass-cockpit flight instruments and their utilization. Data collection about piloting precision and heart activity was then performed during flights Nr. 22 (simulated) and 23 (real).

Note, backup analog gauges were covered in the case of glass-cockpit flights in Gr. A as well as Gr. B.

Efforts were spent to schedule real flights so that all were executed under visual meteorological conditions. These flights were executed during spring and summer (May to August). Both simulated and real flights were executed in the terminal manoeuvring area of Košice International Airport. The instructor was holder of a certificate pursuant to Commission Regulation (EU) No 1178/2011, Annex I (Part FCL) as amended, and respective Commission Implementing Regulations.

2.3. Data Collection and Pre-Processing

Data collection was performed in two ways. The first was derived from pilot performance measurement by means of piloting precision evaluation. The second was oriented to psychophysiological condition assessment by means of heart rate variability (HRV).

Piloting precision was assessed by instructor who noted maximum deviations from predetermined flight parameters of individual manoevures (see Figure 2A). These data were subsequently used for assessment. This method was selected because of the impossibility to download aircraft flight recorder’s data, especially with analogue setup. Even though other data exist and it was possible to record them in both flight simulator and aircraft equipped with an electronic flight instrument system (EFIS); with respect to the aforementioned, data obtained from the instructor were used. Efforts were spend to limit potential bias due to subjective measurement by employing only one instructor for the assessment. Additionally, assessment was partly harmonized in this way among simulated and real flying, since during real flying, apparent piloting error may have occurred due to meteorological factors, such as wind gusts.

Overall, maximum deviation from constant altitude to be maintained during horizontal steady flight and horizontal 360${}^{\circ }$ turn was assessed. Next was assessed maximum deviation from banks during horizontal, climb, and descend turns. In addition, vertical speed error was evaluated for climb and descend turns. As the last parameter, magnetic heading deviation during horizontal steady flight was evaluated.

In case of psycho-physiological condition measurement, heart activity was measured and evaluated for the purpose of HRV analysis. The main reason for the choice of HRV was ease of measurement and no limitation for the pilot during his or her activity. At the same time, HRV and parameters obtained from the RR intervals analysis is perceived as significant indicator of psychophysiological condition [25,26]. From the perspective of psychophysiological condition quantification, it is also possible to use other measurements, such as skin resistance, brain activity (EEG) or respiratory rate [27]. On the other hand, there were many limitations related to these measurements, such as complicated measurement bothering the pilot during his or her activity related to flying (EEG cap), influence of the measurement by external forces (skin resistance), unclear methodology in terms of sensor location (temperature) or many artefacts following the nature of the measurement and used sensors (respiratory rate). Heart activity analysis, i.e., the HRV, is the most used and examined from all methods [26,28]. The use of such data stems from the idea that autonomous nervous system reacting to external stimuli regulates body functions, including the heart activity, i.e., the change in psychophysiological condition leads to change in heart activity, including the HRV [25]. Based on this knowledge and many studies proving the suitability of this signal for the given purposes and ease of its measurement, including the fact that the very measurement does not limit or bother the pilot during his or her activity related to flying an aircraft, the HRV has been selected for this study.

Heart activity measurements were performed by means of FlexiGuard (Czech Technical University in Prague, Prague, Czech Republic) mobile telemetry system [29,30] dedicated to physiological parameters recording. The main monitored parameter was heart activity, so the data collection was based on recording RR intervals (time intervals between R peaks of QRS complex when measuring electrocardiogram) [31].

During particular flight, the measurement was performed with presence of an operator who manually marked beginning and end of respective manoeuvre. These times were recorded by means of SW which was part of the measurement system, based on the instructions from the instructor. Consequently, apart form RR interval time-series, the data contained also timestamps delimiting individual manoeuvres.

Processing of such short RR interval time-series would significantly limit further analysis and options for data processing, e.g., by means of spectral analysis. Therefore, the processed time-series was extended to 210 s with respect to manoeuvre duration, i.e., if the manoevure initial time ($t_{MS}$) and end time ($t_{ME}$) are known, then the initial time ($t_{NS}$) and end time ($t_{NE}$) of the evaluated interval are given as:

$$t_{NS}=t_{MS}-\frac{210-(t_{ME}-t_{MS})}{2},$$

(1)

$$t_{NE}=t_{ME}+\frac{210-(t_{ME}-t_{MS})}{2},$$

(2)

where all times are in seconds. The selection of such modified time interval was due to HRV matrices and norms for ultra-short term measurements [32,33]. From these time-series, selected HRV indices were calculated.

The selection of parameters used in this study was done considering the signal length. In general, a signal of a minimum 300 s is required for HRV indicies calculation, which is considered a gold standard for short-term recordings [28,34,35]. In case of shorter signals, such as in this study, it was not appropriate to use some standard parameters. Studies dealing with suitability of standard HRV parameters for shorter signals (ultra-short term recordings) essentially agree in one parameter of time analysis—root mean square of successive RR interval differences (RMSSD) [36,37]. Usability of other parameters is at least arguable. In the case of standard deviation of NN intervals (SDNN, NN interval is synonymous for the RR interval) there are studies discouraging or not recommending its usage for ultra-short term recordings [37], but these studies work with 10 s records. Other studies indicate the possibility of its usage. Marks and Lightfoot show high reproducibility between 150 s and 300 s signals [38]. Similar results are also presented by Munoz et al. for signals longer than 120 s [35]. Other parameters were discussed as rather not suitable for ultra-short term recordings, i.e., for the purpose of this study [39].

Descriptive statistics appeared also suitable. Most often, mean RR interval (Mean RR) parameter is used. This parameter seems also applicable for short-term recordings [34,38,39].

From other approaches, frequency analysis of RR intervals time-series is typically used. From the perspective of this analysis, HF (power spectral density at high-frequency band, 0.18–0.4 Hz) parameter appears the most suitable for ultra-short record [34]. Its standalone utilization, however, is not effective. In the case of LF (power spectral density at low-frequency band, 0.06–0.10 Hz) parameter, and the most important parameter—the LF/HF ratio, some studies indicate effectiveness from 120–150 s of record [28,38]. By contrast, there are studies pronouncing frequency analysis unsuitable for ultra-short records and accept only the standard of 300 s as the minimal record length [40]. Due to this, frequency analysis was not considered in this study.

For the purpose of ultra-short term heart rate variability (HRV) in the time domain, two parameters—RMSSD and SDNN—were calculated based on the RR intervals. Further, mean RR interval value (Mean RR) was computed. RMSSD parameter served the description of autonomic nervous system (ANS), consequently used as index of cardiac vagal control [41]. RMSSD value reflects the deviation between two heart rhythms, being the time domain measure for short-term variability monitoring, and heart rhythm high-frequency fluctuations [41,42]. Calculation of RMSSD follows the knowledge of individual RR intervals (ms) with subsequent calculation of the squared difference of two consecutive RR intervals. The resulting RMSSD value equals the root of the averaged squared differences of the consecutive RR intervals, i.e.,

$$RMSSD=\sqrt{\frac{1}{N-1}\sum _{i=1}^{N-1}{(R{R}_{i+1}-R{R}_i)}^2},$$

(3)

where RR is i-th interval between two R peaks, with $i=1\dots N-1$.

RMSSD value is highly influenced by parasympathetic nervous system. During high workload, stress and in pathological states, its value drops compared to the baseline and, vice versa, increases in idle times [41,42,43]. This means that a decrease in RMSSD parameter may indicate stress [43].

SDNN was the second monitored parameter in the time domain, reflecting variability in the period of recording [28]. It is calculated as a standard deviation between each R-peak of the QRS complex. SDNN reflects spontaneous changes in sinus rhythm, following the influence of sympathetic and parasympathetic inputs [44]; decreased SDNN may be the consequence of increased sympathetic and/or decreased parasympathetic heart tonus, and eventually the consequence of HRV drop. SDNN decrease may thus indicate stress [45].

Heart activity was further assessed by means of Mean RR, which was the average R-R interval value (ms), i.e., the time interval between QRS complexes, or R-waves. Because RR intervals were used to compute heart rate (HR), i.e., the amount of heart rhythms in a minute, there is a correlation between the two values. The shorter the RR interval, the higher the heart frequency. Consequently, it was important to consider the correlation between Mean RR and Mean HR, although it was not linear. As in the previous example however, it holds that the lower the Mean RR, the higher the Mean HR. Drop in Mean RR indicates increased heart rhythm. This means that a decrease in Mean RR may indicate stress [46].

2.4. Statistical Analysis

Piloting precision data, i.e., records about error rates during individual executed manoeuvres, were used to create composite score. The score was created primarily to allow only one variable indicating piloting precision. To this end, it was necessary to standardize the data so that each dataset comprising respective monitored variable, corresponding to each subject during his or her entire training, was standardized. Data standardization for each subject was carried by the equation [47]:

$$Z_p=\frac{x_p-{\mu }_p}{{\sigma }_p},$$

(4)

where Z are the standardized data by means of Z-score, x is observed value, $\mu$ is average value of the monitored parameter p, where $p=1\dots 4$ and represents flight parameters, i.e., altitude, bank, vertical speed and magnetic heading; and $\sigma$ is standard deviation from the monitored parameter p. Next, summation of all individual flight scores was done for each subject:

$$CompositeError=\sum _{i=1}^n\frac{Zi,p}{2},$$

(5)

where $Z_{i,p}$ is a matrix with i rows and p columns, with columns corresponding to mentioned flight parameters. Summation is for the sake of practicality divided by 2, because precisely two flight parameters were monitored during each manoeuvre. In other words, each standardized distribution of a monitored parameter has average zero and standard deviation (SD) equal to $\pm 1$, i.e., division by 2 retains the SD ratio.

Repeated measures ANOVA [48] was then applied on the adjusted data. To determine whether repeated measures ANOVA can be applied, individual datasets were subjected to normality testing by means of Shapiro–Wilk test [49]. Normality was not indicated for all datasets. Nevertheless, the repeated measures ANOVA was applied taking into consideration that datasets not indicating normality may increase type I error of the performed analysis. On the other hand, when rejecting the hypothesis about normal distribution, the testing was near to accepting the hypothesis and it is also necessary to note that this type of analysis is robust against violation of normal distribution assumption [50,51].

Mauchly’s test [52] was used to test the hypothesis about sphericity and it did not confirm the assumption about meeting compound symmetry. Due to that, Greenhouse–Geisser correction [53] was used for p-value determination.

Using Wilkinson notation, it is possible to describe the model used for applied repeated measures ANOVA as:

$$Fl1-Fl6\sim Gr+Man+Gr\times Man,$$

(6)

where $Fl1-Fl6$ is notation for the measured flights (see Figure 2), $Gr$ is independent categorical variable indicating a group (i.e., Gr. A a Gr. B), and $Man$ is independent variable representing manoevure type. Within subject design is, therefore, based on individual measurements during the training. For the subsequent post hoc analysis, i.e., comparison of data pairs, Dunn–Sidak correction [54] was applied.

Statistical analysis was performed in MATLAB environement (MATLAB R2017a, MathWorks, Inc., Natick, MA, USA) with the use of statistical toolbox.

3. Results

The analysis did not show statistical significance in the interaction of Group × Measurement × Maneuver and Measurement × Maneuver for the monitored HRV measures. This indicates that the manoeuvre type had no influence on the monitored physiological parameters. This held for piloting precision results as well. Note that between group differences for particular measurements in Figure 3 are shown only informatively. Most likely, these differences were caused by inter-individual variability. For the purpose of groups comparison, Group × Measurement interaction was applied with testing whether the pattern of change was different for the groups [55].

3.1. Heart Rate Variability Measures

The results of the repeated measures ANOVA for the monitored Mean RR, SDNN and RMSSD parameters, considering the interaction of the executed flight and monitored group of subjects, show that there is statistically significant difference between Gr. A and Gr. B with respect to the distribution of respective HRV measures during the training. Specific results of the repeated measures ANOVA for Mean RR show $F(5,1160)=26.504$, $p\approx 2\times {10}^{-25}$; for SDNN $F(5,1160)=5.545$, $p\approx 1.6\times {10}^{-4}$ and for RMSSD $F(5,1160)=11.337$, $p\approx 7.7\times {10}^{-9}$.

Mean RR, SDNN a RMSSD distribution during the training of both groups is depicted in Figure 3 by means of error plots. The results of post-hoc multicomparison analysis are shown in Figure 4. Note that increase in Mean RR indicated decrease in heart rate (beats per minute) a vice versa.

The monitored HRV measures for Gr. A indicated uniformly that the initial contact with glass-cockpit arrangement in form of simulated flight (Fl. 18) was the least stressful for the group (see Figure 4A–C). Fl. 11 measurement also showed significantly higher values of the monitored parameters among all monitored HRV measures (except for RMSSD, where no statistically significant difference was identified between Fl. 11 and Fl. 19). During Fl. 11 and Fl. 18 measurements, the subjects exhibited statistically significant R-R interval separation (indicating lower heart rate) and higher short-term and long-term heart rate variability. For this group, however, it was not possible to determine the level of workload based on the HRV between Fl. 2 and the real flights. This could mean that the workload was at the same level, or there was not enough evidence for decision about the difference between these flights. The results for RMSSD parameter, however, showed that during real flying with glass-cockpit (Fl. 19) the subjects exhibited no statistically significant difference between Fl. 11 and the rest of the measured flights. Having a look on Figure 4C, this flight appeared to be intermediate between low and high workload for Gr. A. Because it was not possible to reliably distinguish respective flight from other flights, such as Fl. 2, Fl. 12 and Fl. 17, by means of SDNN and Mean RR, it is possible to conclude that Fl. 19 did not exhibit the assumed workload stemming from the experiment. With the experiment setting, it was assumed that glass-cockpit would play the role of influencing factor, affecting psychophysiological condition or workload. The presented results, however, did not support this. Even more, the results from the perspective of subjects’ psychophysiological condition exhibited rather an improvement with the cockpit ergonomics change.

All Gr. B HRV measures results are of more complex nature. Assessing the training by pairwise comparison evaluation (see Figure 4), it is possible to claim that the subjects exhibited the lowest workload during the flights Fl. 2, Fl. 11 and Fl. 22. These flights were executed on flight simulator and showed no statistically significant difference in Mean RR and SDNN. In case of short-term variability, i.e., RMSSD (see Figure 4C), there was statistically significant difference between Fl. 11 a Fl. 22. In this case, it was difficult to determine the sequence of the simulated flights only by workload change based on HRV measures. It is, however, possible to claim that there was no statistically significant difference between Fl. 2 and Fl. 22 in any HRV parameter, i.e., unlike for Gr. A., workload was at the same level during the first measurement and the measurement on simulator with glass-cockpit. In comparison with Gr. A, this went against the training of Gr. B, which completed additional training on glass-cockpit simulator but seemingly exhibited higher workload. Next, the results presented in Figure 4A–C indicated that the subjects experienced the highest psychophysiological stress during their first real flight (Fl. 12). Between the second measured flight with analog representation (Fl. 17) and real flight with glass-cockpit (Fl. 23), it was not possible to observe statistically significant difference in HRV parameters. Nevertheless, in all cases it was possible to see difference between Fl. 12 a Fl. 23, indicating that additional training may have had some influence on stress reduction.

In any case, for both Gr. A and Gr. B there was not enough evidence that a transition to other type of cockpit ergonomics has influence on workload or pilot stress. The results reflect rather the training progress and acquired piloting skills in individual groups. From Figure 3 and Figure 4A–C, it follows that significant change in psychophysiological condition, based on heart rate change and heart rhythm variability, manifests mainly during the transition to real flights. Possible changes can also be observed during skills acquisition e.g., between the first and the last measured flight on the simulator (Fl. 2 vs Fl. 11).

3.2. Piloting Precision

The results from repeated measures ANOVA for piloting precision showed that there was statistically significant difference between Gr. A and Gr. B, considering the training course for both groups, $F(5,1160)=100.7$, $p\approx 1.4\times {10}^{-59}$.

The results of piloting precision represented by the Composite Error variable show that for Gr. A, the trend depicted in Figure 3D already reflects significant change in performance during the transition to glass-cockpit arrangement. For further interpretation it is necessary to realize that increase or decrease in piloting error is related to average value of the overall training, which for composite error oscillated around zero.

Specific pairwise changes and behavior of individual groups during the training can be learned from the results in Figure 4D, by means of the presented pairwise comparisons.

The results for Gr. A showed that subjects exhibited the highest performance (i.e., the lowest piloting error rate) during the first flight on glass-cockpit simulator (Fl. 18) and the last flight on analogue simulator (Fl. 11). There was no significant difference between Fl. 11 and Fl. 18 and this initial result corresponds with the HRV results. Both measurements (Fl. 11 and Fl. 18) were statistically significant in difference from other measured flights. Further, it can be claimed that the first measurement on a simulator with analogue arrangement (Fl. 2) and the second measured flight (Fl. 17) manifested the same level of performance. Subjects exhibited the highest error rate during the first real flight (Fl. 12) and the flight with glass-cockpit arrangement (Fl. 19). Unlike for HRV results, where it was impossible to distinguish Fl. 2, Fl. 12, Fl. 17 and Fl. 19, in this case it was clear that glass-cockpit arrangement may have had significant influence on the piloting precision. This claim is supported by Gr. B results, which showed (see Figure 4D) that there was no statistically significant difference between real flight with glass-cockpit arrangement (Fl. 23), Fl. 11 and Fl. 22. This means that both real and simulated flight with glass-cockpit exhibited the lowest piloting error rate. The highest significant error rate was recorded during Fl. 12, which is no surprise given that for subjects, it was the first experience with real flying. Further, the highest error rate was exhibited during Fl. 2 (the very first measurement on simulator), followed by Fl. 17 (the second measurement on real flight).

These results indicated that change in cockpit ergonomics may have had negative impact on piloting precision. This claim is quite logical as there was a pattern change in flight instrument interpretation. It was very likely that Gr. B benefited from additional training, which supported increase in piloting precision during real flights.

4. Discussion

In aviation, a high level of flight safety is the main goal [56]. Humans are the most flexible element of modern aviation, but the flexibility of aviation professionals must be cultivated by adequate training, otherwise it can expose the system to unacceptable risk. The most demanding are the requirements for flight crews and their training. Important aspect in the process is establishment of adequate working conditions, which for pilots is the cockpit. Aircraft design and manufacturing accounted for cockpit ergonomics for a long time; adequate layout of flight, navigation, and other on-board systems instrumentation, including various switches and control elements etc., established the base requirement for achieving high levels of flight safety. In cockpit, safety is also tightly coupled with correct distribution of pilot attention, with subsequent processing and evaluation of the collected information. Important is the interaction between pilot and cockpit equipment since it is linked with acquisition of the information necessary for making optimal decisions about the flight [57]. Aviation is no exception in the development of visual digitization; besides analogue display of flight data, simultaneous digital display is often used [58]. Further, there are many aircraft types where analogue display was replaced with digital as a result of their modernization. This means that with the same aircraft type, a pilot may experience both classic (analogue) representation of data, so as the so-called glass-cockpit with modern LCD panels and overview displays, i.e., digital representation of data. For inexperienced pilot, such change in cockpit ergonomics may lead to inadequate distribution of attention, increased emotional and work stress, acquisition of only part of the information to ensure safety of flight and, consequently, to their insufficient processing, leading to inadequate decisions [21,22]. Acquisition of correct habits and patterns of identification, analysis and evaluation of flight information is part of the flight training. Although the aircraft used in flight schools are being progressively modernized from analogue to digital display of data, in many cases the practical training is still carried with analogue representation of data.

The results of the presented study fit the afore-mentioned context. First, however, it is important to realize that the concept of the study relied on hypothesis, where the training progress for both groups (Gr. A and Gr. B) in its common part (Fl. < 17) is the same. Change should occur during the transition to glass-cockpit display of flight, engine, and navigation data, to the benefit of Gr. B, which completed additional training for glass-cockpit. In ideal case, this should be manifested in workload and piloting precision during individual piloting tasks. This would mean that cockpit ergonomics has influence on both workload and piloting precision, where the two are in relation.

If we accept the generally known and used hypothesis that the human organism adapts to increased workload, this fact should quantitatively manifest through the autonomic nervous system (ANS). It should manifest in the regulation of heart activity, which can be quantified by means of HRV measures [59]. In this context, taking the above as a relevant fact, the presented study results show that the transition to a different readings display does not have to strictly pose any significantly increase in workload. This claim rests on the presented results, especially for Gr. A. and this findings are consistent with findings of Wright et al. [19], even though they used questionnaire based survey for workload evaluation.

Although the results show that there is statistically significant difference between the groups and it can be seen among the HRV parameters comparing Gr. A and Gr. B in individual flights, it cannot be claimed that this is caused by cockpit ergonomics change during the training. On the contrary, almost certainly this is due to inter-variability and inter-individual variability which is characteristic for HRV and, unfortunately, seldom considered in similar studies. As an example of inter-individual variability of HRV measures, see [60]. Due to that, it is necessary to understand the differences as a factor of groups consisting of subjects with different behavior and capability to react to stress factors, where these subjects are randomly selected from population. Consequently, the groups may exhibit difference and with sufficiently large population of research samples in both groups, there would likely be no difference between Gr. A and Gr. B. The results themselves may be influenced by other factors, such as human physical or mental condition. Fatigue is considered as one of the significant factors, often discussed in the aviation [61,62] besides other things due to its negative impact on the performance. In this study, however, efforts were spent to eliminate these factors to maximum possible extent, by setting a controlled experiment. Given the fact that no fatigue or any other negative physical and mental state were observed with the subjects, it can be assumed that these factors did not influence the study results.

Several other factors were recognized to generally affect workload and/or performance, however, all of these factors were eliminated and did not influence presented results: experience (subjects did not have any previous experience), manufacturer design philosophy (a single aircraft type was used) and advanced avionics features and user settings (the instructor ensured that no automation was used and default settings were selected).

In any way, it is possible to claim that HRV measures in this study address workload, but this is coupled with the training progress and is equivalent for both groups. Namely the HRV parameters fluctuation can be seen from the results e.g., when transiting to real flying, which indicates increase of stress. The transition from analogue representation to glass-cockpit, however, does not contribute significantly to workload increase, in the context of the carried measurements.

In the case of piloting precision, the preliminary assumption is that if pilots are not familiar with the cockpit ergonomics change and do not complete additional training with changed flight, engine and navigation display of data, there will be increased error rate during the transition. Gr. A performed only 1 h of simulated flying with changed cockpit ergonomics, where (similarly to Gr. B) this flying did not show performance decrease. During the transition to real flying, however, there was significant decrease in performance of Gr. A, manifested as increased error rate reaching the highest recorded value during the training. Wright et al. observed similar results regarding the performance evaluation [19]. In addition to this, our study was focused on the monitoring of more complex training together with the monitoring of psychophysiological condition. Based on our study, it is possible to claim that the positive influence of additional training on pilot performance is evident during the cockpit ergonomics change. The results show that if a pilot did not develop correct habits and sufficient experience with evaluating readings during a display change is not gathered, there may be degraded performance with absence of additional training and the change of analogue and digital representation of data. The results also indicate that workload increase is not necessarily followed by performance degradation if sufficient experience level is gained. From the experimental setup it also follows that in case of transition from analogue to glass-cockpit, 4 to 5 h of additional training should limit performance influencing factors related to this transition.

5. Conclusions

The paper dealt with the evaluation of cockpit ergonomics impact on workload and psychophysiological response to ad-hoc change in display of flight, navigation and engine data, with respect to performance assessment. Workload was assessed by means of monitoring heart rhythm variability, which reflects the level of autonomic nervous system involvement in reaction to a stressor. The presented study is fitted in the domain of general aviation. It recognized sudden change in the display of flight data during a training as a potential stressor, or at least a factor contributing to increased workload. The results, however, show that such change may not be considered as a factor influencing workload. On the other hand, such a change of cockpit ergonomy could lead to performance decrease, therefore transition between analog and glass-cockpit should be considered as performance influencing factor. The results also indicate that higher stress may not lead to performance degradation (in the case of this study to a change of piloting precision). The primary factors influencing the monitored sample of pilot subjects seem to be acquisition of piloting habits and familiarization with the cockpit representation by additional training. In case of the presented study, additional training that may significantly contribute to performance improvement, should include simulated training of 4–5 flight hours. Obviously, it would be useful to research detailed trends during such trainings, which is absent in the presented paper.

The study was carried with a sample of 20 participants with no previous experience with flying. By this, among other measures, the objective was to ensure research sample uniformity. Despite the efforts, this was not achieved as is shown by the inter-individual and inter-group variability. The research sample size may also have manifested in the inability to determine statistically significant differences in the measurements, especially for monitored HRV parameters. This fact can be considered the main limitation of the presented study. It is important to add, however, that the presented study is the first of its kind, which attempted to take an experimental attitude, with an approach close the real training. This approach is expensive and time-demanding, which was the main reason for not including more subjects.

As another limitation we consider data collection about piloting precision, which can be considered subjective. In this case, we rely on the instructor’s ethics, for whom such evaluation is a daily job. As already mentioned in the previous sections, the problem with exact evaluation of piloting precision lay with the inability to obtain flight recorder data from aircraft equipped with analogue cockpit. If these data were available, it would be possible to precisely evaluate all manoeuvres for piloting precision and correctness, or evaluate the whole flights against physiological parameters. We believe, however, that these limitations are minor considering the presented results, and that the study results present valuable contribution not only to the aviation domain.

For further research it would be interesting to collect other physiological parameters as well, such as EEG, respiratory curve, etc. Such collection should, however, be realized so that the applied devices do not limit the pilot in his or her activity, and influence the experimental setup. Currently it is possible to extend such measurements with subjective evaluation by means of structured questionnaires, which would provide information about subjective perception of performance and workload. This information could serve the comparison with objective evaluation methods. Interesting appears also the use of eye-tracking, which could help better understanding of attention distribution during the transition to other types of ergonomy cockpit. In this case, however, a wearable technology shall be used with implemented eye-tracking and camera system recording the environment, for its embedding into the scene view. It is apparent that from the perspective of data collection and objectivization of the problems related with the discussed topic, there are options for the next research. The presented study is a pilot experiment in this domain and may serve as a baseline for future research.