2.1.5. Summary of SEJE Components

In their guide on SEJE, Meyer and Booker [14] give an overview of methodology components and how to select among their available implementation options. The source emphasizes on different elicitation situations, response modes, dispersion measures, types of aggregation and documentation methods. Their summary is given in a form of a MM which could be used for the design of new SEJE processes, shown in Table 1. It combines the elements from [14] as well as other aspects from the literature mentioned earlier.

**Table 1.** Identified SEJE components and their implementation options in the form of a morphological matrix.


Based on the descriptions in the previous subsection, Table 2 summarizes the implementations of the SEJE components by the prominent methods found in the literature. It is necessary to underline that these methods have been shown according their original definition. Many of these have been further extended throughout the years and numerous

modifications can be found in the literature—e.g., by adding expert calibration or extending the elicitation to multiple variables [16,23,28].

**Table 2.** Summary of established SEJE methods.


#### 2.1.6. Bias as a Source of Uncertainty

One of the main reasons for the thorough structuring of SEJE approaches is the difficulty to obtain data reflecting expert knowledge and experience as exactly as possible. This is mostly due to the presence of multiple types of bias [14,15], often resulting in systematic errors when a person is asked to give objective scientific judgment based on knowledge and experience. The literature knows multiple descriptions and overviews of bias for the purposes of knowledge elicitation [14,15]. Instead of giving another review, the current work will use the bias definitions and guidance in order to construct a tailored SEJE method for the AMA aiming to minimize uncertainties.

Meyer and Booker [14] define two views on bias—motivational and cognitive. Bias can be considered motivational when the elicitation task aims at reflecting the expert's opinion as precisely as possible. This is for example the case when one's purpose is to understand the DM's way of thinking or problem solving. In such cases, ambiguous definitions or faulty elicitation processes could alter the expert's point of view and therefore be the source of motivational bias. The main types of motivational biases are social pressure (in group dynamics), misinterpretation (influence of sub-optimal elicitation methodology or questionnaires), misrepresentation (flawed modeling of expert knowledge) and wishful thinking (influence of one's involvement in the subject of inquiry) [14].

Meanwhile, the cognitive bias view should be preferred for the purpose of likelihood estimation or a mathematically/statistically correct quantification of given parameters [14]. In such cases, it is linked to the cognitive shortcuts people use to process information. Examples of cognitive bias are inconsistency (the inability to yield identical results to the same problem throughout time), anchoring (resolving a problem under the influence of a first impression), availability (vastly relying on a easier retrievable from memory event), and underestimation of uncertainty [14].

To the knowledge of the current article's authors, the majority of developed SEJE methods (originally) aim at eliciting probability distributions for physical values. In this context, an observation has been made that the literature pays more attention to the cognitive view of bias [15,20].

Although the mentioned motivational and cognitive biases might appear complimentary to each other, Meyer and Brooke [14] advise to take a single view of uncertainty depending on the purpose of the project. They bring forward the following justification. On the one hand, the reduction of motivational bias aims to help the data reflect the knowledge of the expert as well as possible by adjusting the methodology accordingly. On the other hand, the cognitive view states the inability of the expert to represent their opinion in an exact mathematical manner and tries to guide the DMs to express themselves in a correct statistical way [14].
