Aerospace Mission Outcome: Predictive Modeling

1. Summary

“The reasonable man adapts himself to the world; the unreasonable one persists in trying to adapt the world to himself. Therefore all progress depends on the unreasonable man.”

—George Bernard Shaw, British playwright

Human-in-the-Loop (HITL)-related models can be applied in various aerospace vehicular problems, when human qualifications and performance are crucial and the ability to quantify them is therefore imperative; since nobody is perfect, these evaluations should preferably be done on a probabilistic basis. The suggested models can also be used in many other areas of applied science and engineering, not even necessarily vehicular engineering, when a human encounters an extraordinary situation and should possess a sufficiently high human capacity factor (HCF) to successfully cope with an elevated mental workload (MWL). The incentive for probabilistic predictive modeling and the rationale for such modeling is addressed in this article on the layman language level.

2. Introduction: Assuring Aerospace Mission Success and Safety in Conditions of Uncertainty

“If a man will begin with certainties, he will end with doubts; but if he will be content to begin with doubts, he shall end in certainties.”

—Sir Francis Bacon, English philosopher and statesman

Improvements in safety in the air and in space can be achieved through better ergonomics, better work environments, and other efforts of traditional avionic psychology that directly affect human behaviors and performance. There is also a significant potential for further reduction in aerospace accidents and in assuring aerospace mission success and safety through better understanding of the role that various uncertainties play in the planner’s and operator’s worlds of work, when never-perfect humans, never-failure-free navigation equipment and instrumentation, never-100%-predictable response of the object of control (air- or space-craft), and uncertain and often harsh environments contribute jointly to the likelihood of a successful outcome of the aerospace mission. By employing quantifiable and measurable ways of assessing the role and significance of various critical uncertainties and treating a human in the loop (HITL) as a part—often the most crucial part—of a complex man–instrumentation–equipment–vehicle–environment system, one could improve dramatically the state of the art in assuring the success and safety of an aerospace mission or an off-normal situation. This can be done by predicting, quantifying, and, if necessary, even specifying an adequate (low enough) probability of a possible accident.

Tversky and Kahneman [1] were the first who addressed various cognitive “heuristics and biases” when considering various inevitable uncertainties in human psychology in association with decision-making tasks and problems. However, being traditional psychologists, they discussed such problems from the qualitative viewpoint, while it is the importance of a quantitative approach, based on various probabilistic risk analysis (PRA) concepts, that is pursued here. The first attempt at doing that addressed the helicopter-landing-ship situation [2]. The CRC press book [3] considers and discusses adaptive probabilistic predictive modeling in human-in-the-loop situations in aerospace engineering, and emphasizes, as they say, the importance of a “new, powerful, flexible, and effective approach” to making mission and off-normal situations outcomes successful and safe.

3. Rationale behind a Probabilistic Risk Analysis (PRA) Incentive

“A pinch of probability is worth a pound of perhaps.”

—James G. Thurber, American writer and cartoonist

The rationale behind a PRA incentive/approach can be explained by the following more-or-less natural and even obvious reasoning. When the outcome of an aerospace mission or a situation is critical—and it typically is—the ability to predict and to quantify such an outcome is imperative. One could even argue that otherwise the favorable outcome of an anticipated effort simply could not be assured. Such a prediction/quantification should start at the design stage, and should be monitored and maintained during the fulfillment of the mission of interest [4].

Nobody and nothing is perfect, and, therefore, the prediction of the most likely outcome of an aerospace mission should be naturally based on probabilistic risk analysis (PRA) methods and approaches (see, e.g., [5,6]). The difference between a highly reliable system, failure-free human performance, or a successful outcome of a mission or a situation, on one hand, and insufficiently reliable ones, on the other, is “merely” in the level of their never-zero probability of failure.

In maritime engineering, for instance (I started my engineering carrier as a Naval Architect, and joined Bell Labs, basic research area, in 1982; when in 1998 they wrote an article of me in Bell Labs News, its subtitle was “an engineer moves from ships to chips”), it has been established, based on the projected strength of today’s ocean-going vessel, that the probability that her hull, when sailing in the Northern Atlantic for twenty years in a row, might break in half is about ${10}^{-7}–{10}^{-8}$. These numbers give a flavor of the upper limit of the probability of failure for an engineering object. In aerospace, human, and instrumentation/equipment engineering, the acceptable probability of failure might be considerably higher, especially having in mind that many possible failures, whether human- or equipment-related, are reversible.

4. The PRA Concept Is a Predictive (Prior) Effort, and Not a Statistical (Posterior) One

“We see that the theory of probability is at heart only common sense reduced to calculations: it makes us appreciate with exactitude what reasonable minds feel by a sort of instincts, often without being able to account for it.”

—Pierre-Simon, Marquis de Laplace, French mathematician and astronomer

While the traditional statistical (posterior) HITL-oriented approaches are based on experimentations followed by statistical analyses, our concept, on the contrary, is a predictive (prior) one. It is based on, and starts with, physically meaningful and flexible predictive modeling [7,8] followed, if necessary, by highly focused and highly cost-effective experimentations. Such experimentations are mostly of the failure-oriented accelerated testing (FOAT) type (testing is conducted until failure, whatever its definition might be) [9] and is geared to the chosen simple, easy-to-use, and physically meaningful governing model. This model, as Einstein put it, should be “as simple as possible, but not one bit simpler”. Some such models concerning mostly human performance and conducted on a simulator are addressed and briefly explained below. If the predicted probability of an outcome of a mission or an anticipated off-normal situation is not acceptable, then an appropriate sensitivity analysis based on the developed methodologies can be employed to improve the situation. One important aspect of this concept is bridging the existing gap between what the aerospace psychologists and system analysts do.

5. Mental Workload (MWL) versus Human Capacity Factor (HCF)

“The only real voyage of discovery consists not in seeing new landscapes, but in having new eyes.”

—Marcel Proust, French author

In the simplest application, the recently introduced human capacity factor (HCF) should be considered and compared with the mental workload (MWL) that the human (pilot) has to cope with [7,8]. It is the relative levels of the MWL (demand, “stress”) and HCF (capacity, “strength”) that determine in our approach the most likely outcome of an aerospace mission or a situation. The interrelated concepts of situation awareness and MWL are central to today’s aerospace psychology, and there is extensive published work devoted to understanding the role and measurement of the MWL in aviation, both military and commercial. The MWL factor has been widely recognized as a significant cause of human error. MWL is directly affected by the challenges that a navigator faces when controlling the air- or space-craft in a complex, heterogeneous, multitask, and harsh environment. Such an environment includes numerous different and interrelated concepts of situation awareness: spatial awareness (for instrument displays); system awareness (for keeping the pilot informed about actions that have been taken by automated systems); and task awareness (that has to do with attention and task management). The MWL depends on the operational conditions and on the mission or situation complexity, and has to do with the significance of the long- or short-term task. Measuring the MWL has become a key method of improving aviation safety. Current research efforts in measuring MWL use psycho-physiological techniques, such as electroencephalographic, cardiac, ocular, and respiration measures in an attempt to identify and predict MWL levels. Measurement of cardiac activity has been a useful physiological technique employed in the assessment of MWL, both from tonic variations in heart rate and after treatment of the cardiac signal.

While the MWL level is always important and is always considered when addressing and evaluating an outcome of a mission or a situation, the HCF is equally important: the same MWL can result in a completely different outcome depending on the HCF level of the individual(s) involved. In other words, it is the relative levels of the MWL and HCF that have to be considered and quantified in one way or another when assessing the likelihood of a mission’s or a situation’s success and safety. It is also important that these two criteria are not completely independent: the level of the MWL can be lower for individuals with high HCF.

MWL and HCF can be characterized by different means and different measures, but it is clear that both these factors have to have the same units in a particular problem of interest in order to meaningfully quantify their roles in the problem of interest. HCF includes, but might not be limited to, the following major qualities that would enable a professional human to successfully cope with an elevated off-normal MWL: psychological suitability for a particular task; professional experience and qualifications; education, both special and general; relevant capabilities and skills; level, quality, and timeliness of training; performance sustainability (consistency, predictability); independent thinking and independent acting, when necessary; ability to concentrate; ability to anticipate; self-control and ability to act calmly in hazardous and even life-threatening situations; mature (realistic) thinking; ability to operate effectively under pressure, and particularly under time pressure; ability to operate effectively, when necessary, in a tireless fashion for a long period of time (tolerance to stress); ability to act effectively under time pressure and make well-substantiated decisions in a short period of time; team-player attitude, when necessary; swiftness in reaction, when necessary.

These and other HCF-related qualities are certainly of different levels of importance in different HITL situations. It is clear also that different individuals possess these qualities in different degrees. Long-term HCF could be time dependent. In order to come up with suitable figures-of-merit (FOMs) for the HCF, one could rank, similarly to the MWL estimates for particular situations or missions, the above and perhaps other qualities on a scale from, say, one to four, and calculate the average FOM for each individual and particular task.

6. Possible Predictive Models

“Probability is too important to leave it to the mathematicians.”

—Unknown aerospace engineer

6.1. Convolution Model

“You can see a lot by observing.”

—Yogi Berra, American baseball player

“It is easy to see. It is hard to foresee.”

—Benjamin Franklin, American scientist and statesman

The convolution model was introduced and employed in the helicopter-landing-ship (HLS) situation, with consideration of the role of a human factor. This factor is important from the standpoint of the operation time that affects the likelihood of safe landing during the anticipated lull period in the sea condition. The operation time includes the time required for the officer-on-board and the helicopter pilot to make their go-ahead decisions, and the time of actual landing. It is assumed, for the sake of simplicity, that both these times could be approximated by Rayleigh’s law, while the lull duration follows the normal law with a high-enough ratio of the mean value to the standard deviation. Safe landing could be expected if the probability that it occurs during the lull time is sufficiently high. The probability that the helicopter undercarriage strength is not compromised is evaluated as a product of the probability that landing indeed occurs during the lull time and the probability that the relative velocity of the helicopter undercarriage with respect to the ship’s deck at the moment of encounter does not exceed the allowable level. This level is supposed to be determined for the helicopter-landing-ground situation. The suggested probabilistic model can be used when developing specifications for the undercarriage strength, as well as guidelines for personnel training. Particularly, the model can be of help when establishing the times to be met by the two humans involved in making their go-ahead decisions for landing and in actually landing the helicopter.

The convolution model was used in [10] to show that it was the exceptionally high HCF of the US Airways crew and especially that of its captain, Sullenberger, that made a reality what seemed to be, at first glance, a miracle. It was shown that the highly professional and, in general, highly qualified Swiss Air crew exhibited inadequate performance (quantified in our analysis as a relatively low HCF level) in the off-normal situation they encountered. Like in the helicopter-landing-ship problem, the probability of safe landing/ditching was evaluated in the miracle-on-the-Hudson event by comparing the random operation/“subjective” time (that consists of the decision-making time and the landing time) with the also random “available”/”objective” time needed for more-or-less safe landing. It is emphasized that the developed formalisms, after trustworthy input data are obtained (this could be done by using, e.g., flight simulators or by applying the Delphi method), might be applicable even beyond the avionic field, and even beyond the vehicular domain: they can be employed in various HITL situations when short-term failure-free human performance is critical and, therefore, the ability to quantify it is imperative. It was concluded that what was indeed a miracle in this famous event is not that safe landing occurred, but that the US Airways crew and especially Captain Sullenberger possessed an extraordinary high HCF level, and that such an individual turned out to be in charge in the situation in question. It was concluded also that, although the input and the obtained numerical data make physical sense, it was the approach, not the numbers, that was the main merit of the analysis.

The convolution model was employed also in the “anticipation in aeronautics” problem [11]. Two problems that have to do with uncertainties in an anticipation effort as an important cognitive resource for improved aeronautics safety were addressed on a probabilistic basis using analytical (mathematical) modelling: (1) evaluation of the probability that the random “subjective” (“internal”, pilot performance related) anticipation time is below the random “objective” (“external”, “available”, situation related) time in the anticipation process; and (2) assessment of the likelihood of success of the random short-term (extraordinary, off-normal) anticipation from the predetermined deterministic (ordinary, normal) long-term anticipation.

6.2. Double-Exponential Probability Distribution (DEPD) Model

“There are truths, which are like new lands: the best way to them becomes known only after trying many other ways.”

—Denis Diderot, French philosopher and writer

Different approaches can be used in the analysis and optimization of the interaction of the MWL and HCF. When the MWL and HCF characteristics are treated as deterministic ones, a safety factor $SF=\frac{HCF}{MWL}$ can be used. When both MWL and HCF are random variables, the safety factor can be determined as the ratio $SF=\frac{\prec SM\succ }{S_{SM}}$ of the mean value $\prec SM\succ$ of the random safety margin $SM=HCF-MWL$ to its standard deviation $S_{SM}$. If the capacity–demand (“strength–stress”) interference model is used, the HCF can be viewed as the capacity (strength) and the MWL as the demand (stress), and their overlap area could be considered as the potential (probability) of a possible human failure. The capacity and the demand distributions can be steady-state or transient, i.e., can move towards each other as time progresses, and/or the MWL and HCF curves can become spread over larger areas.

The recently suggested [12,13] double-exponential probability distribution (DEPD) models use a single distribution to account for the roles of both the HCF and MWL when these random characteristics deviate from (are higher than) their (deterministic) most-likely (regular) values. No sophisticated convolution effort is needed in such a case. In the simplest DEPD, the ratios of the random MWL and random HCF to their nonrandom specified/ordinary values are considered as the governing random variables, and the probability of human nonfailure is expressed through these ratios. The distribution makes physical sense, and the following conclusions can be made from the calculated data for the ratio of the probability of nonfailure in extraordinary (off-normal) conditions to the probability of nonfailure in normal (specified) conditions:

(1)

At normal (specified, most-likely) MWL level and/or at an extraordinary (exceptionally) high HCF level, the probability of human nonfailure is close to 100%;

(2)

When the MWL is extraordinarily high, the human will fail, no matter how high his/her HCF is;

(3)

When the HCF is high, even a significant MWL has a small effect on the probability of nonfailure, unless the MWL is exceptionally high;

(4)

For high HCFs, an increase in the MWL has a much smaller effect on the probabilities of nonfailure than for relatively low HCFs;

(5)

The probability of human nonfailure decreases with an increase in the MWL, especially at low MWL levels, and increases with an increase in the HCF, especially at low HCF levels;

(6)

An increase in the probability ratio above 3.0 has a minor effect on the probability of nonfailure. This means particularly that the navigator (pilot) does not have to be trained for an unrealistically high MWL, i.e., does not have to be trained by a factor higher than 3.0 compared to a navigator of ordinary capacity (skills, qualification). In other words, a pilot does not have to be a superman to successfully cope with a high-level MWL, but still has to be trained in such a way that, when there is a need, he/she would be able to cope with a MWL by a factor of 3.0 higher than the normal level, and his/her HCF should be by a factor of 3.0 higher than what is expected of the same person in ordinary (normal) conditions or of a person with ordinary skill. Of course, some outstanding individuals, like, e.g., Captain Sullenberger, might be characterized by an HCF ratio even higher than 3.0.

It has been shown [12,13] that the change in the probability of human nonfailure with the change in the HCF at its nominal (normal) level is proportional to the entropy of the DEPD and is inversely proportional to the nominal HCF. This explains the physics underlying this distribution. While the model described in this section is double-parametric, time was introduced in such a model when assessing the likelihood of a casualty, when one of the two pilots becomes incapacitated [14], and a DEPD model was introduced and analyzed to quantify the roles of human error and his/her state of health [15] as important factors that could have a considerable effect on the HCF and the MWL.

6.3. Probability Segmentation Model

“The truth is rarely pure and never simple.”

—Oscar Wilde, British dramatist, novelist, and poet, The Importance of Being Earnest

Let us consider this model in application to a vehicular mission’s success and safety [16]. Let, e.g., the mission of interest consists of a number of consecutive segments that are characterized by different probabilities of occurrence of a particular harsh environment or by other extraordinary conditions during the fulfillment of the particular segment of the mission; by different durations of these segments; and by different failure rates of the equipment and instrumentation. These failure rates may or may not depend on the environmental conditions, but could be affected by aging, degradation, and other time-dependent causes. In the simplified example [16] it has been assumed that the combined input of the hardware and the software, as far as the failure rate of the equipment and instrumentation is concerned, is evaluated beforehand and is adequately reflected by the appropriate failure rate values. These values could be either determined from the vendor specifications or could be obtained based on the specially designed and conducted failure-oriented accelerated testing (FOAT) and the subsequent predictive modeling. Based on the probabilistic segmentation method, the probability of mission nonfailure can be calculated as the sum of the products of the likelihood of the occurrence of the harsh environment of the anticipated severity at each segment of the vehicle’s route, the probability of nonfailure of the equipment, and the probability of nonfailure of the navigator(s). A detailed numerical example is given in Ref. [16].

6.4. Boltzmann–Arrhenius–Zhurkov (BAZ) Model

“Education is man’s going forward from cocksure ignorance to thoughtful uncertainty.”

—Don Clark, “Scrapbook”

The Boltzmann–Arrhenius–Zhurkov (BAZ) model [17,18] is used to predict the probability of nonfailure of equipment and instrumentation, both its hardware and software, for the subsequent employment of the obtained result in combination with the evaluated probability of human nonfailure to assess the roles and significance of human performance and instrumentation reliability in the outcome of a mission or a situation. BAZ is the heart of the Probabilistic Design for Reliability (PDfR) concept [6].

7. Conclusions

“There is nothing more practical than a good theory.”

—Kurt Zadek Lewin, German–American psychologist

HITL-related models can be applied in various aerospace vehicular problems, when human qualifications and performance are crucial. The ability to quantify these human qualities is therefore imperative. Since nobody is perfect, such a quantification should be done on a probabilistic basis. The suggested probabilistic models can also be used in many other areas of applied science and engineering, wherever a human encounters an extraordinary situation. He/she should possess a sufficiently high human capacity factor (HCF) to successfully cope with an elevated mental workload (MWL). It is the relative level of these two factors that determines the outcome of the particular mission or an extraordinary situation.