*2.1. Literature Overview*

The concepts of expert judgment elicitation originate from the fields of psychology, decision analysis and knowledge acquisition [14]. The literature suggests the following definitions for expert judgments and their elicitation:


In this context, one seeks the judgments from qualified professionals (experts) in their respective domains. According to the definitions, the elicitation process is not simply asking for the required information. It also implies the extraction of the required subjective knowledge, the filtration of biases and its possibly precise representation in a quantitative form—be it the evaluation of concrete physical (one might dare say "tangible") parameters or simply placing it on a qualitative scale.

One should note the various terminologies used throughout the literature to denote structured elicitation of expert opinions such as structured expert judgments, structured expert knowledge, expert knowledge elicitation or elicitation protocol. These in general stand for the principle of aiming to scientifically elicit expert knowledge. In the current paper, the abbreviation SEJE (structured expert judgment elicitation) will be used for this purpose. Similarly, the widely used notation of the participating experts as decision-makers (or DMs) is also adopted in this work.

The main advantage of formalized SEJE methods is the transparency and the possibility to review the process [14], therefore contributing to the scientific significance of the results. In this context, Cooke [19,20] advocates the achievement of rational consensus by adhering to the principles of Scrutability/Accountability (reproducibility of methods and results), Empirical Control (empirical quality control for quantitative evaluations), Neutrality (application of methods aiming to reflect true expert opinions) and Fairness (no prejudgment of the experts). However, it is necessary to underline that these requirements have been outlined during the definition of the Classical Model, which serves as a method for quantitative elicitation.

SEJE approaches have been developed starting from at least the 1950–60s or earlier [16,21]. In some cases, these are formalized into structured elicitation protocols and guidelines by/for authorities and companies [21], e.g., by the European Food Safety Authority [18] or the procedures guide in the field of nuclear science and technology [22]. Further applications can be found in the domains of human health, natural hazards, environmental protection [20], and provision of public services [17]. Some aeronautical disciplines have also profited from SEJE methods, which will be discussed in the next subsection.

Available SEJE methods establish the main components of a structured elicitation and present various possibilities to implement these. In order to identify and derive the appropriate SEJE process for the integration into the AMA methodology, the most prominent SEJE methods are briefly presented, summarized and compared—namely the Classical Method by Cooke [19], the The Sheffield Elicitation Framework [23], the Delphi method [24] and the IDEA protocol [25].
