1. Introduction
Commercial aviation is one of the safest modes of transportation nowadays with less than one accident per million departures [
1]. This relatively low rate of accidents, which were 20–30 times higher in the 1960s, is mostly due to the technological progress of aeronautical systems, but also due to improvements in pilot training, flight crew and air traffic control procedures. Technical failure today is the cause of only about 10% of accidents, leaving a significant percentage as the implications of human factors. The exact values vary over years and sources and, as illustrated in
Figure 1, approximately 60 to 80 percent of aviation accidents involve human error [
2]. In particular, today’s low rate of overall accidents is due to the introduction of automation [
3]. However, it has also shifted the role of the crew from direct (manual) controllers to supervisors. Unfortunately, automation is not always fully understood nor surveilled correctly [
4] and this can generate complacency [
5]. This phenomenon can promote the failure of the crew to monitor flight instruments properly because of over-reliance on automation. Thus, the adequate, active visual monitoring of the flight parameters in the cockpit is an essential piloting skill and becomes one of the most critical issues for flight safety [
6].
The supervision of the flight deck is performed by two pilots, namely the Pilot Flying (PF) and the Pilot Monitoring (PM), whose roles are defined by standard operation procedures [
7]. The PF is responsible for controlling the flight (e.g., trajectory and energy) and gives orders to the PM. He fulfills these responsibilities by either supervising the auto-pilot and the auto-thrust when engaged, or hand-flying the aircraft. The PM is responsible for systems-related tasks and the monitoring of the flight instruments. He/she also performs actions requested by the PF such as radio communications, systems configurations and automatic mode selections. Eventually, the PM is responsible for monitoring the PF to cross-check his/her actions, to inform of any deviation of the flight parameters and backup if necessary.
However, maintaining an excellent monitoring performance is not easy. As Casner and Schooler [
8] showed in their recent study of airline pilots in a high-fidelity simulator, when pilots are free to monitor, i.e., there is not a particular defined task, monitoring lapses often occur. In this sense, the National Transportation Safety Board (NTSB) in their accident review of major flight-crew-involved accidents between 1978 and 1990 [
9] found that inadequate monitoring and cross-checking had occurred during more than 80% of accidents. However, even if the pilots are nowadays aware of the critical importance of adequate monitoring, accidents with monitoring lapses as contributing factors still occur. One recent example was the Asiana Airlines flight 214 accident in 2013. A change in the autopilot mode (which resulted in the deactivation of the automatic airspeed control) went unnoticed by the crew, as well as the subsequent drastic diminution of the speed. This lack of monitoring provoked a destabilized approach, with a below acceptable glide path and an excessive descent rate. The landing gear finally struck a seawall at San Francisco International Airport [
10]. Three passengers were fatally injured, with 49 other people receiving serious injuries; the airplane was destroyed by the impact and subsequent fire.
Recently, the Federal Aviation Administration (FAA) published a final training rule that requires enhanced pilot monitoring training to be incorporated into existing air careers training programs [
11], the compliance date being March 2019. Also, the Bureau Enquêtes-Accidents (BEA), the French investigation agency, recommended studying pilots’ monitoring with eye-tracking to improve piloting procedures [
12]. In a recent survey [
13], 75% of pilots answered that a publication of detailed information regarding the required visual patterns for the different phases of flight (for example, take-off, approach) could help them to enhance monitoring skills.
Consequently, the great challenge is to improve the pilot–aircraft interaction by considering the complex attentional and cognitive processes underlying piloting. Neuroergonomics [
14] proposes using physiological tools to find valid and robust measures of human behavior and cognitive processes, such as attention. For example, electroencephalography [
15], electrocardiography [
16] or functional infra-red spectroscopy [
17] are often used in flight/drive simulators to study cognitive and attentional states. However, none of these techniques can properly take measures remotely without direct contact with the human body. Meanwhile, the remote character of such a measurement device is one of the essential criteria to measure human attentional and cognitive behavior inside the cockpit, especially during emergency situations. To highlight the importance of this point, let us take the example of the tragic, well-known flight AF-447 [
18] from Rio de Janeiro to Paris that in 2009, after an aerodynamic stall, crashed over the Atlantic Ocean, taking 228 lives. As the captain interrupted his in-flight rest and returned in an emergency to the cockpit, it is difficult to imagine him calmly putting on some wearable device and proceeding to a calibration, while vital decisions were to be made. It is worth noting that there exist signal processing techniques that allow heart rate or body temperature to be estimated from video frames remotely. Nevertheless, these data are difficult to interpret and, especially, it is challenging to disentangle emotional reactions from the cognitive activity. There exist wireless alternatives, such as watches or smart shirts that can monitor these parameters. However, wireless data transfer may result in cybersecurity vulnerabilities and therefore is not recommended for transferring information relevant to flight safety.
Since eye-tracking devices can be directly embedded in the cockpit panel, one promising avenue is to consider this technology. Researchers have been publishing results of eye movements studies for over 60 years. However, the market of commercial eye trackers has been considerably democratized only during the last decade, and such devices today are non-invasive (can be remotely used without direct contact with the body), relatively inexpensive, reliable and, importantly, eye-tracking data can be available in near real time. All these advantages make the integration of eye trackers in modern flight decks realistic.
Due to the fact that “high definition” images are restricted to a small region called the fovea, humans need to shift their eyes constantly from one area to another to explore and monitor their environment. The raw data samples from an eye tracker can serve to detect basic oculomotor events [
19] such as fixations, during which we acquire useful visual information, and saccades, that serve to change the location of the visual focus. These events allow the monitoring of ocular behavior and give insights into the perceptual and cognitive processes [
20] underlying reading, free viewing, and decision-making (see, for example, [
21,
22]).
The use of eye-tracking in aeronautics is far from new. Starting with pioneer studies by Fitts and colleagues around 1950 conducted in a flight simulator [
23,
24,
25], many researchers have performed experiments on pilots equipped with an eye tracker system. During his work, based on forty pilots’ data, Fitts concluded that different flight instruments were checked with variable frequencies and required a different fixation length. He also found that more experienced pilots had a tendency to make shorter fixations. Researchers have reproduced these results since then, but it was the first time that the recording of eye movements was demonstrated to be a valuable method of evaluating pilots’ ocular behavior. At the time, researchers were confronted by the complexity of the recording set-up and elaborate manual analysis procedures. Nowadays, on the contrary, an eye tracker system can be easily embedded in a flight simulator or even a real aircraft. Thus, eye-tracking has already been successfully used to conduct high-fidelity flight simulator experiments [
26,
27,
28] and real flight experiments [
29,
30].
These recent and promising advances in eye-tracking demonstrate the need for conceptualizing its integration into the cockpit. In the following section, we identify and define a framework of four stages of eye-tracking integration in modern cockpits. These four stages are (I) Pilot Training, (II), On-board Gaze Recording, (III) Gaze-Based Flight Deck and (IV) Aircraft Adaptations. We support each stage with a description of relevant incidents or accidents and explain in what way the eye-tracking integration can enhance flight safety. An estimated milestone for the integration of each stage is also proposed together with a list of some implementation limitations.
2. Four Stages of Eye-Tracking Integration
In this section, we identify and define four stages of eye-tracking technology integration in the piloting activity. The first stage relates to pilot training on-ground, whereas the other three stages correspond to operational flight situations. We highlight the interest of the eye-tracking in each of the four stages (with examples of aeronautical incidents/accidents for stages III and IV), give an estimated integration date milestone in years, and consider some implementation and use limitations.
Figure 2 presents a flowchart of this eye-tracking integration, showing the positioning of each of the four stages.
2.1. Stage I: Pilot Training/Flight Performance Analysis
In line with the recent training rule published by FAA in 2013 [
11], which requires airlines to include a specific training program to improve monitoring by March 2019, one major axis of flight safety improvement is pilot training. In stage I of pilot training and flight performance analysis, eye-tracking data can be exploited in three different ways: as a flight expertise estimate, for incident debriefing and for example-based learning.
First of all, the eye-tracking data can be used to obtain an estimate of pilots’ monitoring (flight) skills. In their review of eye-tracking research in various domains (such as radiology, car driving, sports, and chess), Gegenfurtner and his colleagues [
31] concluded that experts (compared to less experienced professionals) have, in general, shorter fixation durations during the comprehension of visualizations. This echoes Fitts’ previous conclusion about experienced pilots and their shorter fixation durations [
23]. In the helicopter domain, during a simulated overland navigation task, Sullivan et al. [
32] found an estimate of expertise cost (or rather benefit) on scan management skill. Their model predicted that, on average, for every additional thousand flight hours, the median dwell time will decrease by 28 ms, and the number of transitions between zones of interest (out-the-window and navigational map) will significantly increase. In their review of the literature on eye-movements in medicine and chess, Reingold and Sheridan [
33] called it “superior perceptual encoding” of domain-related patterns. However, this perceptual advantage is not the only thing gained with expertise. It is closely related to the global processing advantage and knowledge of relevant visual patterns. Thus, Gegenfurtner et al. [
31] found, for example, that experts fixate more on task-relevant areas and spend less time fixating on relevant information. This visual strategy of “information-reduction” [
34] that helps to optimize the visual information processing by separation of task-relevant from task-irrelevant information, was also found to differentiate expert from novice pilots in aviation [
35,
36]. For example, Schriver and colleagues [
37] compared the distribution of visual attention using eye-tracking between experienced and novice pilots during problem diagnosis in a flight simulator. They found, in particular, that faster correct decisions by experts during trouble diagnosis were accompanied by a correct attending of cues relevant to that failure. Van Meeuwen et al. [
38] compared visual strategies of air traffic controllers with a different level of expertise and found that experts had more effective strategies, and that, furthermore, the scan paths were more similar between expert controllers compared with novices. More recently, a study of both PF and PM eye movements during the approach revealed that the PM’s attentional allocation was not optimal, especially during the short final. It corresponded to a high percentage of dwell time out of the window to the detriment of the processing of the flight parameters, such as the energy of the aircraft [
39]. Similar measurements performed by Lefrançois and colleagues [
13] showed that the gaze allocation of pilots, who failed to stabilize the approach, was not optimal compared to the pilots who succeeded. These results suggest that there is an optimal visual scan path for a given visual problem that can be acquired with years of expertise, but that can also be learned with eye-movement educational examples.
Secondly, recorded movie clips where the eye gaze position is superimposed on the visual scene could be used in training programs as example-based learning. Such eye movement modeling examples were already successfully applied during cooperative problem-solving [
40] and clinical visual observation [
41,
42]. Gaze following is a natural and innate tool of the learning [
43], including the causal one. It allows the learner to discover what information is relevant and how to guide their attention for a given goal by following the expert’s gaze. Jarodzka and colleagues [
44] argued that the attentional guidance by gaze following is not only effective at improving current performance, but also at fostering learning. Earlier, Nalanagula and colleagues [
45] also suggested that gaze following improves training for visual inspection tasks. Recently, Leff and colleagues [
46] showed that trainees’ performance is enhanced when experts’ gaze guides their visual attention in a surgery context.
Eventually, the movie clips of replayed pilots’ eye movements could be used for subsequent analysis and debriefing. Pilots do often request the recordings of flight parameters after a non-routine event to understand better what happened during an incident and how to prevent it in the future. It is impossible (and unnecessary) to remember all the eye movements we perform. Moreover, the majority of fixation patterns are automatic and unconscious. Therefore, a replay of the attentional behavior during a simulated session, or a real flight (in the case of later integration of eye-tracking systems), would allow the pointing out of attentional errors by pilots, such as an excessive focus on a particular flight parameter or, on the contrary, its disregard. Let us note, however, that the user-friendly and synthetic representation of eye movement recordings is not straightforward [
47], and that further research is needed to provide intuitive tools to support analysis and reviewing of the eye-tracking data. Thorough research is also needed to provide evidence that these eye movement examples can effectively foster pilots’ learning.
To conclude this stage I, we suggest that eye-tracking technology is fairly useful for pilots’ training on the ground because (1) we can debrief their behavior by studying gaze patterns; (2) visual scan paths can be learned by gaze following; and (3) monitoring skills can be estimated using eye-tracking data. The present stage can be already integrated and adopted by airline companies around the world by using relatively inexpensive head-mounted eye-trackers available today on the market or by incorporating a remote eye tracker system in their training flight simulator.
2.2. Stage II: On-Board Gaze Recording
Eye-tracking has potential, not only on-ground for pilot training and flight skills analysis, but also in operational settings. The first step of operational use is a recording of the crew’s points of gaze to facilitate incident/accident investigations. All modern aircraft are equipped with a black box comprising a cockpit voice recorder and a flight data recorder for recording information, required for crash investigation purposes. Obviously, pilots do not verbalize all their actions. Therefore, flight parameters and voice recordings are sometimes simply not enough to accurately reconstruct the course of events. Eye-tracking technology can enhance these devices by also recording the gaze data. First, the tracking of pilots’ gaze is useful for accident aftermath to verify at which flight instruments the pilots looked. The analysis of these data would help to deeply understand the failures in the human–system interaction (“Did the crew fail to perceive a critical warning? If so, why?”) that lead to an incident, and to avoid future repetition of the same circumstances. Secondly, pilots do not look exclusively outside or on the cockpit instrument panel, they also monitor one another’s activity. In particular, they observe one another’s hands when they press a button, move the engine lever or point at a display [
48]. Pointing in the airline cockpit is an important part of piloting; it guides one’s own and the other’s visual attention and helps to establish shared situation awareness [
49]. Thus, a recording of the pilots’ gaze could help investigation agencies to infer joint awareness and crew coordination. Moreover, these data would be beneficial for cockpit manufacturers, revealing the effectiveness of their user interfaces and leading to design improvement. Similar benefits for the design of operational procedures are expected.
However, although technologically feasible, we recognize that the process of introducing an on-board gaze recording might take some time. For example, the Transportation Safety Board of Canada indicates, in a recent reassessment of the responses to aviation safety, recommendations concerning cockpit image-recording systems [
50] that “since 1999, the NTSB has issued 14 recommendations to the FAA related to the installation of cockpit image-recording systems. To all but a single recommendation, the NTSB has assessed the FAA responses as unacceptable”. The recent NTSB’s safety recommendation [
51] requires that each aircraft be retrofitted with a crash-protected cockpit image recording system that should “be capable of recording, in color, a view of the entire cockpit including each control position and each action (such as display selections or system activations) taken by people in the cockpit.” Let us also note that, contrary to the cockpit image-recording systems, this stage does not necessarily interfere with the pilots’ privacy. The gaze recordings are impersonal and should record exclusively a gaze point within an established 3D-model of a given cockpit, or the name of the corresponding area of interest chosen from an exhaustive list of all the flight deck instruments and visual displays. Thus, the eye-tracking recording would not entail any privacy infringement (such as recording of pilots’ faces).
In the future, the cockpit voice recorder should be promoted to a human data recorder by also recording the gaze data. These data should comprise two channels to keep points of the gaze of both the pilot flying and the pilot monitoring. Such application is already possible today because it would not have any safety effects and, therefore, does not have to meet any failure probability requirement. A significant limitation of this stage is that the recordings should provide an accurate testimony of flight events. Therefore, it is mandatory to have an eye-tracking system of excellent quality, high precision and robustness. Such on-board gaze recordings would provide valuable documentation for safety boards to improve incident and accident investigations and may take the form of an additional column (e.g., “currently gazed area”) of data in the flight data recorder. These data can be used by the flight crew or by the airline companies for the post-flight debriefing. For example, if a particular event occurs during a flight, the flight crew or companies might want to review the visual circuits at the time of the problem. In any case, as for the training stage, there is a need for a tool that eases data analysis and reviewing. Such volumes of data are difficult to process immediately and further research is needed to handle the large size of eye-tracking data. Altogether, we speculate that the early adaptation of such gaze recording will emerge within 5 years, the principal requirement being high reliability, precision and accuracy. If such a device would necessitate a calibration procedure, the setup should be quick and simple (for example, during the preflight checklist).
2.3. Stage III: Gaze-Based Flight Deck Adaptation
The gaze recording is the simplest, but not the only, possible use of eye-tracking technology in operational settings. Numerous studies have successfully used eye-tracking to detect different degraded cognitive states such as spatial disorientation [
52], fatigue [
53,
54], attentional tunneling [
55], or automation surprise [
26]. Except for the adverse cognitive states, eye-tracking can also be used to infer one’s intentions. Thus, for example, Peysakhovich et al. [
56] showed that eye movements and pupil size could be predictive of upcoming decision-making in a simulated maritime environment. In the automobile domain, Zhou et al. [
57], proposed a method to infer the intention of a truck driver to change lanes based on eye-movements. Ha et al. [
58] showed that it is possible to guess the thoughts of a nuclear power plant operator using his or her eye movements. All these results let us imagine an operational support system that assists flight crews using human eye data as an input. Such an enhancement of the flight deck would take the human psycho-physiological state into account using the eye-tracking data collected in real-time. An aircraft would thus adapt itself explicitly using the information derived from the crew’s gaze. Note, nevertheless, that if such a support tool was considered as part of the aircraft system, safety objectives might be applied. Thus, if the failure of such a system would have a minor safety effect (minor safety condition may include a slight reduction in functional capabilities and a small increase in crew workload), then the aviation certification imposes at least 10
−3 allowable failure probability [
59]. As long as modern video-based eye-tracking systems cannot guarantee even the eye capture for 99.9% of the time (because of luminosity and geometry problems), it will take some time before the eye-tracking technology, and the corresponding market, will evolve and achieve sufficient reliability. Moreover, much high-quality research is to be done to detect critical and complex cognitive states properly. However, the eye-tracking can be used in a simpler manner to detect the presence or the absence of an eye fixation on relevant information in the cockpit.
To better understand how useful an eye-tracking system can be in operational flight situations, let us consider the following incident. An Air France Boeing 777-200 performing flight AF-471 was about to land at Paris Charles de Gaulle on 16 November 2011 [
60]. During the approach, while the aircraft was stabilized on the descent path, a Master Warning alarm indicated that the automatic landing mode had changed and this was called out by the relief pilot. According to the operator’s instructions, in this case, a go-around should be called out and initiated, as was done by the PM. Then, according to the protocol, the PF should handle, in particular, the thrust and the pitch, while the PM should manage the configuration change of the flaps. However, the PF erroneously and unintentionally pushed the auto-throttle disconnection switch instead of the Takeoff/Go-around switch that engages the go-around modes. Therefore, his nose-up inputs were contradictory with the auto-pilot system that was trying to keep the airplane on the descent trajectory. Meanwhile, the PM fully concentrated his attention on monitoring the retraction of flaps, a process which takes about ten seconds. The PM’s failure to properly monitor the aircraft’s attitude (position) and energy state had resulted in a brief loss of flight path and late adaptation of the go-around pitch. Thankfully, a third relief pilot was on board and made two deviation callouts of “pitch attitude” that led the pilots to apply the nose-up input. In the report’s conclusion, the BEA underlined that “the serious incident was due to the inadequate monitoring of flight parameters by the flight crew” [
60]. Indeed, without the relief pilot on board who called out the pitch attitude, such behavior in the ground proximity could have resulted in a controlled flight into terrain, the second most recurrent type of accident [
1].
One of the reasons for the temporary loss of control was the PM’s excessive focus on the flaps’ position indicator. Thus, he was ignoring the information on the aircraft’s attitude indicator (showing the artificial horizon with bank and pitch angles), flight mode annunciator (to verify if the automatic modes were consistent with actions) and engine parameters. Both the lack of pitch monitoring and the excessive focus on flap configuration could be detected with an eye-tracking system. Thus, the first would be expressed as an absence of fixation on the attitude indicator, while the latter would induce an excessively long fixation on flap control. Once such an event was detected, and, in particular, when an aircraft is about to exceed the nominal flight envelope, a counter-measure could be applied to the crew. For instance, a brief retraction of the display of flap configuration could disengage the PM from the flaps and make him allocate some attention to the energy state and pitch attitude. A discrete, but visible, highlight of these two latter parameters to attract the crew’s attention is also feasible.
Skippers Aviation de Havilland Dash 8-300 performing a charter flight to Laverton on 17 May 2012, is another example of an incident with degraded situation awareness as a contributing factor [
61]. While conducting a visual circling approach to the runway, a high descent rate triggered the aircraft’s warning system. Despite the alarm, the crew continued their approach, despite the operator’s procedures stipulating a go-around initiation in this situation. As noted in the safety board report [
61], “the crew’s monitoring of the aircraft’s rate of descent and altitude relative to the minimum stabilization height was secondary to their monitoring of their position in relation to the ground. This external focus degraded the crew’s overall situation awareness”. Thus, the crew was focused on descending through a break in the cloud and did not respond correctly to the sink rate alert and continued their approach. As in the previous situation, this unstable approach incident could have been avoided by re-engaging the captain and the first officer’s attention to the rate of descent and altitude monitoring. The first necessary step to do so is to detect the pilots’ excessive fixation on the runway environment. Next, a brief counter-measure could be applied to disengage the captain and the first officer’s attention from the outside. Such a counter-measure would reorient their attention to the rate of descent and the altitude, which are necessary parameters to determine whether or not the approach is stabilized.
When eye-tracking systems are reliable enough to capture the crew’s gazes in all conditions (different visual angles, head movements, turbulences, luminance), the stage III of gaze-based flight deck adaptation will be relatively easy to implement, at least at some basic level of logic. For example, for each instrument within the cockpit, with knowledge of the system and according to the flight phase (determined by real-time analysis of relevant flight parameters such as position, altitude and speed), one may define the following: (1) maximum allowed dwell time; and (2) maximum allowed time without monitoring. Then, if we detect that one of the rules is broken, a counter-measure could be applied. For instance, if the descent rate was not monitored (with a fixation) for some time by neither the Pilot Flying nor the Pilot Monitoring during the descent phase, an auditory alert may be played back. Alternatively, if we detect an extremely long fixation on flap configuration, for instance, the information can be retracted for a short period to re-engage the pilot’s attention. Nevertheless, before such an alerting system can be implemented, it is necessary to sort all the possible sources of information by priority order according to each flight phase and aircraft configuration. Indeed, information overload is one of the factors that promote pilots’ inadequate monitoring. Therefore, the system should carefully choose what information should be provided and verify that it is necessarily essential for the flight safety. It is also worth noting that gazing at an instrument does not guarantee that the information is processed, which is often referred to as “looked but failed to see”. However, it is possible to reduce the risk of not seeing the information and to prevent “not looked therefore failed” using the eye-tracking [
62].
Figure 3 illustrates this simple logic in the form of a flowchart of stage III. Taking all of this into account, we speculate that the early adaptation of such gaze-based support systems may occur within a decade.
2.4. Stage IV: Gaze-Based Aircraft Adaptation
Eventually, the final and most distant (because of immature technology) step of the eye-tracking integration in the cockpit, is the gaze-based aircraft adaptation. To palliate humans’ limited attentional and cognitive resources, different counter-measures and automation changes can be used and triggered using the pilot’s estimated attentional state. This stage can be combined with the previous stage III to provide the crew with some feedback about aircraft system changes. At stage IV, we propose to allow the automation to change the aircraft’s configuration or to take authority based on the crew’s gaze, and not only provide visualization enhancement and sonification as at stage III. For example, the automation could temporarily take authority over the crew in the case of incapacity of the human operator [
63,
64,
65]. Such authority policy, for instance, already exists in F-16 fighters and will be accepted in the next standard version of Dassault Rafale fighters. The aircraft automatically perform an auto-pull maneuver if the pilot does not properly react to avoid an upcoming controlled flight into terrain. In this example, however, the automatic maneuver is triggered at the last moment to leave more freedom for the pilot. However, eye-tracking can help to anticipate erroneous visual circuits that may precede degradation of the situation [
56]. Important safety issues from such pilot–aircraft interaction imply significant constraints on the probability of a failure condition. If such a system change would lead to hull loss and fatal injury of the flight crew, then the lower limit of the probability of a failure condition (classified as “catastrophic” in this case) should be between 10
−6 and 10
−9, depending on the aircraft class [
59].
Let us consider the following two examples to figure out how such gaze-based aircraft adaptation could improve flight safety. A Flybe de Havilland Dash 8-400 was performing flight BE-1794 to Exeter on 11 September 2010 [
66]. During the approach, the crew, being distracted by an unfamiliar avionics failure, forgot to monitor the flight path and descended below its cleared altitude. As states the report, “both pilots became distracted from the primary roles of flying and monitoring the aircraft and did not notice that “altitude select” and “vertical speed” modes were no longer engaged”. A terrain proximity warning system alerted the crew that the airplane was more than 700 ft below the selected altitude. After a difficult recovery, the aircraft continued to a safe landing.
Another similar incident occurred with the Flybe flight BEE247S on approach to Edinburgh Airport on 23 December 2008 [
67]. The aircraft descended below the glide slope, as the appropriate mode of the flight director was not selected. An approach controller noticed it before the flight crew. As contributory factors, the safety board report cited “an absence of appropriate monitoring of the flight path and the FMA (Flight Mode Annunciator)”. As mentioned earlier, this lack of flight parameter surveillance can be detected by an eye-tracking system. It would be expressed as an absence of fixations on the FMA, the descent rate and the altitude for some time. Within the integration stage III, an auditory verbal alarm can remind the crew of the required visual controls during the approach to palliate this oversight. In the case of stage IV, the difference is that the automation can take the initiative. For example, in the case of the BE-1794 flight, the captain encountered a failure of an Input–Output Processor and lost some of the primary flight display elements. He tried to regain the indications of the primary flight display by switching the air data computer that disengaged the “altitude select” and “vertical speed” modes. That did not help to recover the indications and, as the pilot did not verify the FMA status, he was unaware of the disengagement of these modes. One could imagine, that the aircraft re-engages the previously selected modes automatically (maybe with some auditory feedback) without the proper FMA verification neither from the pilot flying nor the pilot monitoring. This authority policy is an example of stage IV eye-tracking integration in the cockpit when the aircraft takes over the pilot input if degraded situational awareness was detected. Bearing in mind both the present insufficient reliability of the eye-tracking system and significant consequences of false detection of degraded situational awareness, we speculate that the early adaptation of this stage IV is not to be expected within the next 20 years.
Table 1 concludes this section by providing a few examples of possible eye-tracking usage within each of four integration stages.