Loss of Visual Reference in U.S. Aviation: An Analysis of 129 Accidents

Abstract

A “loss of visual reference” is a term used by the National Transportation and Safety Board investigators to describe a situation in which the pilot has lost reference to the horizon or ground. Previous research has largely focused on the conditions that are often associated with a loss of visual reference (e.g., spatial disorientation) instead of the term itself. The purpose of the present study is to examine the accident and pilot characteristics that are associated with a loss of visual reference and the extent to which both are associated with fatalities. A total of 129 accident reports from 2008 through 2020 were extracted from the NTSB online database in which the term loss of visual reference appeared in the event category. Fatal accidents, which made up more than half of the dataset, were more likely to have occurred during the en-route phase of flight compared to all of the other phases of flight combined. The reports in which “decision making” or “decision making/judgment” was present were more likely to be for fatal accidents compared to reports in which the terms were not listed. It is the pilot’s decision and lack of attention that are most commonly associated with accidents in which a loss of visual reference occurred. Decisions that are likely to cause a loss of visual reference include initiating or continuing in adverse weather or light conditions. The dangers of loss of visual reference need to be part of ongoing training of pilots as opposed to their initial training only.

1. Introduction

Loss of visual reference is a general term used by the National Transportation and Safety Board [1]. The term is used when the pilot has lost reference to the horizon or ground [2]. As a consequence of losing visual reference—either due to a loss of a natural horizon, an artificial horizon provided by onboard instruments, or if the visual system is affected by a degraded visual environment (DVE)—the pilot may lose control of the aircraft and experience an accident. Loss of visual reference may be caused by instrument meteorological conditions (IMC), surface phenomena, or night conditions, and may result in spatial disorientation, a condition that is associated with high fatality rates [3,4,5].

Previous research on accidents with aircraft flying into IMC [6,7,8] or in night conditions [9] and studies on spatial disorientation (SD) [3,4,5] have highlighted separately the dangers of these situations. However, not all flights into IMC or at night lead to a loss of visual reference and not all cases with a loss of visual reference lead to spatial disorientation. Therefore, it is important to examine the accidents associated with a loss of visual reference by NTSB investigators in a broader perspective in order to better understand the accident characteristics that are associated with it.

For instance, NTSB investigators have reported instances of loss of visual reference during a night landing pattern with a crosswind turn due to spatial disorientation on a moonless night with no visible horizon [1]. Other incidences include the pilot’s misidentification of the runway surface during landing or the pilot’s loss of sight of the runway due to sun glare [1]. The first of these three examples mentioned a loss of visual reference but was ultimately attributed to spatial disorientation by the NTSB investigators, while the other examples did not include SD, suggesting that the term “loss of visual reference” is used by NTSB investigators in a variety of situations [1].

Previous research includes both the precursors of a loss of visual reference and the possible spatial disorientation in which it may result [3,4,5,6,7,8,9]. A study on intentional and unintentional flight into IMC [10] suggests that pilots often underestimate the relationship between a loss of visual reference and subsequent loss of aircraft control. Additionally, while some pilots may accurately judge the association between loss of visual reference and loss of aircraft control, they overestimate their own ability to maintain control should they experience a loss of visual reference. It is possible to infer information about the specific situations and conditions that are associated with a loss of visual reference from such a study, but because it only focused on flights into IMC, the inferences are limited.

Spatial disorientation is an ongoing concern in aviation safety and has been investigated through both experimental studies and accident analysis [4,11]. Experimental studies help to understand the specific nature of SD, which is commonly defined as an occurrence in which a pilot has a false perception of the motion or orientation of their aircraft with respect to the Earth [11]. Alarmingly, almost 100% of SD-related aviation accidents, especially those in which the pilot is unaware of their SD, are fatal [1,12]. Moreover, one study that surveyed military pilots found that spatial disorientation was most frequently the result of “loss of horizon due to atmospheric conditions” [13] (p. 797). However, the association between SD and loss of visual reference is not always made clear. In a recent study that examined 30 years of accidents involving SD, the term “loss of visual reference” does not even appear [4].

A previous accident analysis study of general aviation (GA) accidents showed that most flight accidents that were operating under visual flight rules (VFR) but continued into IMC involved an intention on the part of the pilot to continue into adverse weather [8]. It was also reported that situation assessment, risk perception, and social pressure were the main underlying causes in such accidents. This study suggested that the underlying causes for a loss of visual reference are likely to include decision-making processes that are affected by several underlying causes.

In the present study, we extend the accident analysis study of general aviation described above and include air taxi and commuter together with agricultural operations for comparison. In particular, we examine the factors that explain why pilots entered a situation with a loss of visual reference, and note factors that could in part explain why accidents end up being fatal. We continue to concentrate on fixed-wing general aviation accidents where aviation safety has been a particular concern [14]. Our analysis focuses on general aviation accidents that occurred in the United States. We highlight weather and light conditions, geographical location, and phase of flight. Past research has identified that the state of Alaska is particularly dangerous and challenging [15,16], so our analysis of geographical location gives specific attention to that state. Weather and light conditions have been associated with loss of visual reference similar to specific geographical locations such as mountains. Previous research did not find a significant relationship between phase of flight and the occurrence of VFR–IMC accidents [8], but that study did not include all accidents with a loss of visual reference. Analysis of accident characteristics provides insight into the conditions that result in a loss of visual reference while also taking into account pilot characteristics such as experience (as defined by flight hours accrued), occupational versus nonoccupational pilots, and whether the pilots had an instrument rating. Additionally, because previous research has suggested that decision-making processes might play an important role in these accidents [8], we analyzed the NTSB reports for the specific kinds of decision-making processes that were highlighted by the investigators.

2. Materials and Methods

A total of 129 fixed-wing aviation accidents from the 13-year period from 1 January 2008 to 31 December 2020 were extracted from the US National Transportation and Safety Board (NTSB) online database using the Case Analysis and Reporting Online (CAROL) query tool [1]. These factual reports include a series of standardized items of information about the pilot(s), aircraft, airport, weather, and wreckage, together with an autopsy summary in case of fatal accidents. In addition, each report includes a narrative statement about the history of the flight, a list of findings, and a conclusion about the causes of the accident. These reports were selected only if they included the term “loss of visual reference” in the event category, pertained to fixed-wing airplanes, occurred in the region of the United States, and were classified as accidents. Additionally, only accident reports that were completed before 2021 were included.

The categorical information provided in each report was collated, including the list of findings for each accident. The findings may consist of causes, factors, and occurrences, and were all collated in separate columns for the final dataset. From these standardized sections of the reports, we organized information pertaining to pilot characteristics (e.g., flight hour accrued), geographical information (e.g., the specific state in which the accident occurred), and flight characteristics (e.g., weather conditions).

Additionally, the narrative statement in each report was coded by the authors and a research assistant for the presence of a discussion of social pressure by the NTSB investigator. The majority of the reports used in this study listed “decision making” and/or “spatial disorientation” as a finding(s), but the NTSB investigators listed a number of other determining factors as well (see

Table 1. Additional findings identified by the NTSB investigators.

Finding Category	Fatal	Nonfatal
Attention	33	38
Experience	3	0
Policy/Management Issues	0	2
Attention and Experience	10	3
Experience and Policy/Management Issues	0	2
Attention, Experience, and Policy/Management Issues	1	0
Other	23	14

). Therefore, we further categorized these additional factors as “psychological/attention”, which included factors such as monitoring the environment, incorrect sequence of actions, task monitoring, and aircraft control; “experience”, which included factors such as inadequate training, instrument experience, and qualification/certification; “policy/management issues”, which included factors such as aircraft malfunction and use of policy/procedure; and “other”, which included factors that were related to weather conditions and already addressed factors such as spatial disorientation. Although our insight in decision making is limited by the wording used in NTSB reports, these rubrics were used to assist in elucidating their relevance for loss of visual reference accidents.

The significance of differences was determined using Pearson’s chi-square analysis and least squares regression. Relations were considered significant if p-values were below 0.05. For analyses in which the expected cell frequencies were not all above 5, a Fisher’s exact test was used. Regression analysis was conducted using SPSS software (version 28).

3. Results

During the 13-year period between 2008 and 2020, there were 129 accident reports that listed “loss of visual reference” in the event category. The majority of the accidents occurred in 2015, 2017, and 2018 (see Figure 1). As can be seen from the least squares trend line in the figure, the total number of accidents per year does not appear to change. Figure 1 also displays the number of fatal accidents during the same time period. In this case, there is a very slight increase from 2008 to 2020, as can be seen from the least squares trend line.

Accidents occurred mostly under Part 91 (N = 111), which pertains to general aviation (GA), with fewer occurring under Part 135 (N = 11), which concerns air taxi and commuter operations, and Part 137 (N = 7), which relates to operations for agriculture.

Out of the 129 accidents, 70 (54%) reported a fatality, resulting in a total of 122 deaths. In all 129 accidents combined there were 236 passengers and crew with an average of 1.83 occupants per flight.

3.1. Analysis of Pilot Characteristics

The age of the pilots ranged from 20 to 92 years of age with a mean of 56.6 years. The pilots also varied considerably with regard to total experience as measured by flight hours accrued; the hours ranged from 0 to 30,000 h with a mean of 4084.71 h. In one accident, a private pilot without recorded flight hours was flying with an instructor. Both died as a result of the accident. The NTSB investigators determined it was the private pilot who was in control at the time of the accident, and the flight hours accrued was recorded as 0.

There were a total of 30 occupational pilots in our dataset. These pilots were operating under Part 91 (N = 12), Part 135 (N = 11), and Part 137 (N = 7). Occupational pilots were equally likely to be involved in a fatal accident (15 out of 30), i.e., an accident in which the occupant of the aircraft was fatally injured, compared to nonoccupational pilots (54 out of 96), p > 0.05. We did not find a significant difference when we examined the extent to which older pilots, defined as 60 years of age or older (33 out of 61) compared to younger pilots (37 out of 68) were associated with fatal accidents, p > 0.05. Additionally, we did not find evidence of pilot flight experience being associated with fatal accidents, p > 0.05. There were a total of 59 pilots in our dataset without instrument rating. These pilots were also not more likely to be involved in a fatal accident compared to their peers with an instrument rating, p > 0.05.

3.2. Analysis of Flight Conditions

A variety of broad phases of flight, as defined by the NTSB, were represented in the dataset (see

Table 2. Number of fatal and nonfatal accidents that occurred during each broad phase of flight.

Broad Phase of Flight	Fatal	Nonfatal
Approach	7	17
Climb	15	3
En-route	34	8
Landing	0	16
Maneuvering	12	7
Takeoff	2	6
Taxi	0	2

). Fatal accidents were more likely to have occurred during the en-route phase of flight (34 en-route fatal accidents out of 70 total fatal accidents) compared to all of the other phases of flight combined (8 en-route nonfatal accidents out of a total of 59 nonfatal accidents), χ² (1, N = 129) = 17.87, p < 0.0001.

IMC conditions were more likely to result in a fatality (39 out of 57) compared to VMC conditions (27 out of 68), χ² (1, N = 125) = 10.26, p < 0.001. The majority of the accidents occurred during daylight conditions (N = 83); however, these accidents were not less likely to result in a fatality (47 out of 83) compared to all other light conditions combined (23 out of 46), p > 0.05.

3.3. Analysis Specific to Geography

The accidents in this dataset occurred in 37 states with the majority occurring in Alaska (N = 21), and a smaller number of occurrences in California (N = 12), Florida (N = 8), and Texas (N = 8). There were 12 states in which only one accident occurred. Further examination of the 21 accidents that occurred in Alaska revealed that flat light, which is an optical illusion that causes pilots to lose their depth-of-field and contrast in vision (N = 4), self-induced pressure (N = 2), and insufficient training and/or risk assessment (N= 4) were described as contributing factors in the accident. Eight of the 21 accidents that occurred in Alaska resulted in fatalities. Fisher’s exact test revealed that these fatal accidents were more likely to have occurred if the flight was near mountainous terrain (6 out of 8) compared to the flights that were not near mountainous terrain (3 out of 13), p < 0.05. On the other hand, snowy or icy conditions in Alaska were not more likely to result in a fatal accident (4 out of 12) compared to other weather conditions (4 out of 9), p > 0.05.

3.4. Analysis of Findings Related to Decision Making and Spatial Disorientation

In their reports, NTSB investigators listed a number of findings that can, in part, explain the cause(s) of the accident. In our dataset, the investigators listed between 1 and 7 findings, with an average of 3.96 findings per accident. Across the 129 reports, a total of 156 unique findings were listed. NTSB investigators commonly listed “decision making” or “decision-making/judgment” as a finding in the reports. The reports in which either term was present were more likely to be for fatal accidents (40 out of 57) compared to reports in which the terms were not listed (30 out of 72), χ² (1, N = 129) = 10.42, p < 0.001. Of the 57 reports that listed these terms as a finding, the majority took place during daylight conditions (N = 39) and those accidents were more likely to result in fatalities (33 out of 39) compared to all the other light conditions combined (7 out of 18), χ² (1, N = 57) = 12.30, p < 0.001. Additionally, the majority of these accidents took place during the en-route flight phase and were more likely to result in a fatality (28 out of 32) compared to all other phases of flight (12 out of 25), χ² (1, N = 57) = 10.46, p < 0.01.

Table 3. The kinds of decision-making errors identified by the NTSB investigators.

Decision	Fatal	Nonfatal
Continue flying into adverse weather conditions	24	11
Initiate a flight into adverse conditions	12	3
Decision to consume alcohol or drugs	2	0
Other	2	3

displays the kind of decision and/or decision-making/judgment that was listed as part of the accident cause. Although the majority of the decisions was the pilot’s decision to continue to fly into adverse weather conditions, it was not more likely to result in a fatality (24 out of 35) compared to all the other decisions combined (16 out of 22), χ² (1, N = 57) = 0.11, p > 0.05.

As can be seen in Table 3, a total of 57 pilots were involved in accidents in which decision making was listed as a finding. Of these 57 pilots, 43 had accrued 1500 or fewer flight hours; however, these pilots were not more likely to be involved in a fatal accident (32 out of 43) compared to their more experienced peers (8 out of 14), χ² (1, N = 57) = 1.51, p > 0.05.

The NTSB investigators also frequently used the term “spatial disorientation” in their findings. The reports in which the term was present were more likely to be fatal (29 out of 35) compared to the reports in which the term was absent (41 out of 94), χ² (1, N = 129) = 15.82, p < 0.0001. The majority of the pilots involved in the accidents in which spatial disorientation was listed as a finding had 1500 flight hours or less; however, these pilots were not more likely to be involved in a fatal accident (20 out of 25) compared to their more experienced peers (9 out of 10), p > 0.05. There were 19 reports in which both “decision making” and “spatial disorientation” were listed as findings, the majority (N = 14) of which were fatal.

4. Discussion

In our study, it is shown that the number of accidents involving a loss of visual reference has remained steady over the years (see Figure 1), suggesting that this dangerous situation deserves more attention. The proportion of fatal accidents in these accidents is 54%, which is particularly disconcerting given that the proportion of fatal accidents for general aviation more broadly is 20% [14].

Most accidents occurred in Alaska, a region that, due to its challenging environment, is overrepresented for a diverse range of accidents [15]. Mountainous terrain in Alaska, a flying environment that is generally considered dangerous [17], appeared significantly more lethal and according to our dataset more so than weather conditions. In this dataset, accidents in IMC conditions were significantly more often fatal but, in contrast, night conditions were not, which follows the results of previous studies [9]. Additionally, in line with previous research, spatial disorientation when mentioned in the findings was more often fatal than other accidents reporting a loss of visual reference [12].

Most accidents occurred en-route, in which the proportion of fatal accidents was significantly higher than for any other flight phase. There were no fatal accidents reported during the landing phase, suggesting that a deadly loss of visual reference during that stage is less likely. This confirms previous research in which the percentage of accidents during the landing phase was much lower for VFR flights into IMC than in GA accidents in general [8]. However, the main type of decision contributing to loss of visual reference accidents is the decision to fly into adverse weather conditions, and this was the dominant cause of accident in the en-route flight phase but not during the take-off flight phase.

Past research has found that loss of visual reference is commonly part of multiple findings identified by the NTSB investigators [6,7,8,10]. The results of the present study confirm that loss of visual reference is not limited to IMC and light conditions. For instance, we encountered pilots blinded by sun glare or by a chemical substance covering the cockpit window. In those cases where “loss of visual reference” was found listed with “decision making”, the proportion of fatal accidents was higher even though they were in daylight conditions and while the aircraft was in the en-route flight phase. In addition to decision making, “attention” was the second most common finding. In these instances we found, for example, that the pilot failed to recognize in a timely manner that the aircraft had drifted below the approach path, as indicated by the visual approach slope indicator, or that they failed to perform the correct sequence of actions [1]. Contrary to previous studies we did not find social pressure as a frequent finding [9]. In that study, the authors were focused on VFR into adverse weather conditions, so it is possible that because our focus was broader, social pressure was no longer a significant factor.

5. Conclusions

As previous research has suggested, pilots often underestimate the dangers associated with loss of visual reference and/or they overestimate their ability to rectify a loss of visual reference situation [10]. Our findings point to specific recommendations for pilots that relate to ongoing and initial flight training. First, loss of visual reference is prominent for IMC and night conditions and often occurs during the en-route flight phase. Experience with those circumstances commonly occurs in the advanced stages of training and the dangers of a loss of visual reference are probably best simulated and trained in a simulator environment. Second, it is the pilot’s decision and lack of attention that are most commonly associated with an accident reporting a loss of visual reference. Decisions that are likely to affect a loss of visual reference include initiating or continuing in adverse weather or light conditions. Additional training that is specific to these conditions is recommended, possibly as part of scenario-based training already developed by the FAA [18,19,20]. Pilots in our sample ranged from inexperienced to very experienced. Therefore, the dangers of loss of visual reference need to be part of ongoing training of pilots as opposed to their initial training only.

Challenging environmental circumstances in Alaska create a particular concern for pilots and future research should concentrate on more effective mitigating strategies specific to this region. Several studies have concentrated on accidents in Alaska [15,16] and this study may assist in identifying the role of loss of visual reference as part of the broader concerns for this region.

Loss of visual reference encompasses several dangerous situations that have received attention in the aviation safety literature. This study may assist in developing a broader approach when it comes to training new and existing pilots. Although night and IMC conditions are obvious concerns, they are not the only circumstances that cause a loss of visual reference. Similarly, spatial disorientation is the most dangerous but not the only condition of the pilot that may result from a loss of visual reference.