Design for Societal Resilience: The Risk Evaluation Diversity-Aiding Approach (RED-A)

Abstract

The global impacts of disaster risks are on the rise. Moreover, evidence shows that the severity of damage will increase exponentially. In 2019, there were 395 natural disasters that caused 11,755 deaths. Literature and practice indicate that diversification of disaster risk management (DRM) approaches can make communities more resilient. One notable bottleneck in adopting diverse DRM approaches is the historical dominance of natural and technological sciences with little contribution from social sciences. Thus, a heterogeneous social-technical approach to DRM is rare and risk governance challenges are hardly understood. We conducted a systematic literature and practice review and extracted data to develop and answer five sub-questions. After that, we reviewed relevant information and selected eight risk evaluation approaches. We made comparisons and used the input to design the Risk Evaluation Diversity-aiding Approach (RED-A). The approach consists of 12 criteria and a checklist with 22 items. RED-A provides guidance to DRM researchers and practitioners when conducting socio-technical risk evaluations. It helps identify cognitive biases in the ongoing DRM process that may largely impact the quality of risk evaluation procedures. The goal of the 22-item checklist is to ensure that the 12 RED-A criteria are incorporated as much as possible to support the progressive transition towards a heterogeneous social-technical DRM approach. Finally, the RED-A criteria and checklist are applied in the Solotvyno municipality context (in Ukraine), to illustrate the use of the approach.

1. Introduction

Globally, disaster risks are increasing owing to anthropogenic activities and climate change effects [1]. In 2019, there were 395 natural disasters, 11,755 people died and 95 million were affected, resulting in damages worth USD 130 billion [2]. Recent research shows that the disaster risk impacts will exponentially increase [3] and urges policymakers to focus more on building societal resilience [4]. A growing body of literature and practice propose diversification of disaster risk management (DRM) strategies to make societies more resilient [5]. The Sendai Framework for Disaster Risk Reduction 2015–2030 calls for novel approaches to enhance risk governance [6]. van den Brink et al. 2011 [7] use the adaptive capacity wheel to assess climate change disaster preparedness levels, and identified a lack of diversity in approaches, strategies and policy options as a major challenge. Aerts et al. [8] develop a method to deal with uncertainty through diversification in flood risks in the Netherlands. Wardekker et al. [9] propose a resilience approach to diversify the risk management process and reduce the impacts of flood risks on low lying delta. Innocenti and Albrito [10] propose participatory dialogue between policymakers and scientists to infuse diversity into the risk management process, thus reducing disaster risks. Hegger et al. [11], based on the findings of the STAR-FLOOD project, concluded that diversification of DRM strategies is paramount for enhancing societal resilience. Wesselink et al. [12] conducted a study in six deltas that showed that a diverse mix of both ‘hard’ engineering, and non-structural ‘soft’ measures, reduced technological lock-in and increased community resilience.

One significant bottleneck to adopting diverse DRM approaches is the historical dominance of the natural and technical sciences [5]. There are enormous DRM advances in the engineering and natural sciences field with many ‘state-of-the art’ scientific tools. A common technological approach is developing hazard and exposure maps that delineate high, moderate and low risk zones. These maps are important for evaluating if a certain risk is acceptable. The maps also help prioritise risks and agree on risk reduction measures. However, scientists use remotely sensed data and expert knowledge to develop most of these maps, with little or no validation. Evaluation of disaster risks requires more information than the probability of occurrence [13,14] and the extent of the damage [13,14,15,16], as depicted in the maps. Further relevant information may entail:

• Explanation of missing scientific information and the assumptions made [13,14,15,16].
• Violation of equity—discrepancies between the risk bearers and the benefactors [13,15].
• The degree to which society has voluntarily accepted the risk, irrespective of contrary evidence [14].
• Divergences in the time-based spread of the probable damage [13,15,16].
• Perceived violation of socio-cultural and individual values and interests by risk-bearers [13,15].
• Whether the damage is reversible, and the environment can revert to its original state [13,15,16].
• Instances of delay in consequences after the occurrence of the risk event [13,14,15,16].
• Differences in the geographical spread of the probable damage [13,15,16].

DRM is broader than the technical construction of defences and development of early warning advisories [5]. The Sendai Framework for Disaster Risk Reduction 2015–2030 acknowledges diverse approaches to risk management that increase the social and technical understanding of the risk (priority 1), strengthen risk governance (priority 2) and enhance disaster risk preparedness (priority 4), [6]. A holistic DRM approach incorporates social and institutional barriers and opportunities for risk reduction measures. However, a social-technical heterogenous approach to DRM is “still rare and a clear understanding of the governance challenges is lacking” [5]. Renn and Klinke [17] propose risk evaluation approaches that embrace trans-disciplinarity, social diversity and flexibility.

Research affirms the significance of adopting a social-technical heterogenous approach to treat, assess, evaluate and manage disaster risks [18]. Lack of diversity may lead to risk maltreatment, weak decisions, escalation of controversies, deadlocks and lock-ins [19]. However, risk managers lack clear guidelines to support the adoption of heterogeneous risk evaluation approaches [15].

This article conducts a systematic literature and practice review of existing risk evaluation approaches and offers a new typology that distinguishes uncertainty, complexity, and ambiguity. Complexity dimension seeks to increase scientific accuracy in the treatment, assessment, evaluation, and management of risks. Uncertainty dimension integrates diversity in the risk approach, whereas the ambiguity dimension enhances socio-political viability. The primary question is: “does the literature and practice review guide the design of a Risk Evaluation Diversity-aiding Approach (RED-A) to support the progressive transition towards a heterogeneous social-technical DRM approach?” To answer the question, we developed the RED-A and applied it in the Solotvyno context. The results are useful for direct application, in Solotvyno. Moreover, researchers and practitioners can utilise RED-A to support adoption of holistic approaches to ensure scientific accuracy, respect for diversity and socio-political feasibility.

We structure the paper as follows. In Section 2, we define key terms of this study and provide the background and context to the problem we seek to address. Section 3 details the phases of the systematic literature and practice review. After that, we explain the search and categorisation results, explain what RED-A is including its criteria and checklist and finally explain the results of applying RED-A to the select case study. Section 5 discusses RED-A, its added value, explains RED-A’s progressive realisation and possible future research. The final section contains brief concluding remarks.

2. Definition of Terms and Context: Complex, Uncertain and Ambiguous Systemic Risks

Solotvyno (Ukrainian: Coлoтвинo) municipality is in Tyachev district, the Transcarpathian region, Ukraine. The name denotes salt, which form an enormous part of the municipality’s subsurface. Salt mining was the core activity for the Solotvyno community since the 1790s. The underground salt chambers served many purposes including cultural activities, music production and treatment of patients in two allergological hospitals. These hospitals provided a salt-saturated microclimate to manage and treat allergies. The mining company extracted the upper part of the salt-dome and removed the protective clay layer. In 1998, water filtrated the salt chambers, dissolving the salt materials. Upward mobility of the salt solution resulted in a karst technogenic reaction that led to subsiding of the peripheral land in 2000 [20]. In 2008, the formation of a massive 40-metre karstic hollow led to the closure of mine 9 [21]. Subsequently, the government closed mine 8 in 2010. Approximately 600 people working in the mines and factory lost jobs [22]. The two underground allergological hospitals were also closed [21]. Closure of the mines and hospitals affected the livelihoods of most Solotvyno residents [22]. In December 2010, the Transcarpathian Regional State Administration decision classified Solotvyno as being in a state of emergency. The 9th December 2010 Ministry of Emergency Situations expert report (No. 02-17292/165), approved the Regional State Administration’s decision. The report revealed a high probability of further land subsidence that may result in injuries, mortality, infrastructural damage and economic loss [21,23]. Thus, the Ministry classified Solotvyno as a high-risk zone [21].

Aven and Renn [24] define risk as “uncertainty about and severity of the consequences (or outcomes) of an activity with respect to something that humans value.” The basis for traditional risk treatment is the linear causal relation between two factors: the stimulus (event or human action) and the consequences. Reduction of simple risks involves a change of human actions or mitigation of effects [13]. However, studies show that framing all risks as a linear causal relation is “technocratic, decisionistic, and economic” centred [19]. The causal-effect risk model remains relevant for simple risks where the relationship between the stimulus and the consequences is linear, known, certain and unambiguous [25]. Past recurrent and consistent evidence is sufficient to predict the probable occurrence and effects of simple risks and support decision-making. For instance, the effects of a simple drug overdose risk are predictable and based on recurrent and consistent dose-effects medical data. The reverse of simple risks are systemic risks. Paton [26] explain that with reductions in natural hazard incidences, societal risks continue to increase, due to complex interactions between human actions (population, social, economic, infrastructure development) and their environment. Systemic risks have interdependencies between multitude stimuli and the spill-over effects cascade to unrelated risk clusters [27,28]. Systemic risks comprise three key elements: uncertainty, complexity and ambiguity [17,19].

A mono-causal risk model cannot represent complex risks, such as land subsidence in Solotvyno [29]. When the land subsided in the mining region, salt lakes emerged in the vicinity. Interdependencies between the high-risk salt mining area and the Salt Lake region increased the complexity of detecting and computing probable stimuli and intended effects. Designating low, medium and high-risk zones and quantifying the effects remains a major challenge. Moreover, since the mining zone is inaccessible, there are many unknowns that science cannot ascertain. Some of the unknowns include cascading effects of salt-intrusion into River Tisza and the impact of unregulated non-metered pumping of the brine to related and unrelated risk clusters [23].

Uncertainty denotes limited knowledge that impedes calculation of the likelihood of a risk occurring and the probable effects [30]. Doyle et al. [31] assess the effect of uncertainty on decision-making. The research found numerical probabilities to have more influence on evacuation decisions than verbal probabilities. Moreover, there was evidence of uncertainty-reduction amongst scientists who relied largely on the available knowledge, while non-scientists either suppressed or acknowledged uncertainty. When scientific knowledge is lacking, reducing uncertainty may lead to misleading predictions [19]. Solotvyno municipality has limited data on key risk factors, including: consistent temporal and spatial hazard exposure and vulnerability data; hydrological data on the contribution of groundwater to land subsidence; reversibility of the degraded land; and the prospect of future investments in salt mining [23]. There is a gap between the risk-bearers and the social groups enjoying the benefits. The few resorts that pump the brine from the collapsed mines have been able to maintain a booming tourist business every summer, while a large part of the community who face the consequences do not reap these benefits [32]. This makes the case exceedingly uncertain and complex.

Ambiguity signifies diverse, legitimate, competing and sometimes conflicting stakeholder perceptions that influence risk evaluation [17,19,33]. Risk evaluation is the third phase of the DRM process (after pre-assessment and risk appraisal), where stakeholders make acceptability and, tolerability judgments of given risks based on evidence and values. Acceptability occurs when there is no need to take risk reduction measures to ensure safety [34]. Tolerability refers to the readiness to co-exist with a particular risk under certain conditions [35]. Intolerable risks are avoided because the risk reduction costs outweigh the benefits. Conversely tolerable risks require treatment until they reach a level that the society considers acceptable. The final DRM phase (risk management) entails implementing risk reduction measures to treat tolerable risks until they reach acceptable levels [19].

Solotvyno has unique deep-rooted cultures, many nationalities and beliefs [36]. Apart from the Ukrainians, Solotvyno is the home for a lot of other nationalities. These includes Russians, Romanians, Hungarians, Roma, Germans, Slovaks and Czechs [36]. As a result, there are a lot of distinct cultures, norms and values. A suitable risk evaluation approach should be heterogeneous to incorporate the unique values and arguments from the diverse groups and adopt a transdisciplinary approach [13], (pp. 1071–1072). Highly disputed arguments include the effects of the booming tourist business and the recent issuance of new licences for open salt mining [23]. There are divergent risk perceptions within and between the community and experts, on whether the land is safe for habitation and which households should be relocated [21,23]. There is a major possibility that lack of consensus has led to inaction and delays in implementing risk reduction measures [23].

3. Materials and Methods

Kitchenham and Charters [37] guidelines provide the foundation for the systematic review of the literature. We adopted these guidelines to ensure that we conducted a holistic and methodical literature and practice review and enhanced the inclusion of relevant articles. The process comprises three phases: plan, conduct and report. Planning entails defining the situation, complication, research question (s) and solution in the form of an assessment procedure. We developed and implemented a search strategy in the second stage. It involves documentation of the search, selection of the study and extraction relevant data. The last stage involves the development and dissemination of a systematic review report. This section elaborates on the steps we undertook in each of the three phases.

3.1. Research Gap and Related Work

The core rationale of this study is to develop an approach that would infuse diversity into the current risk evaluation process to build societal resilience. However, there is need to undertake more research to identify the core elements that would guide the literature and practice review. The initial review identified three major focus areas, as explained in subsequent paragraphs:

• heterogenous risk evaluation approaches.
• approaches that facilitate tolerability and acceptability judgements; and
• approaches for complex, uncertain, and ambiguous systemic risks.

Individual risk-taking decisions depend on the risk and the societal context. A person may avoid, accept, transfer, mitigate or exploit a risk, based on their social context [38]. Kasperson [39] explain that the context a person is in affects their decision and preferences. Müller and Rau [38], conclude that the social context, the societal preferences and the level of heterogeneity of a particular risk affects the individual decision whether to take a risk. Therefore, the review focused on heterogeneous risk evaluation approaches.

The initial analysis relied on several studies to better understand uncertainty, complexity and ambiguity, in relation to disaster risks [17,19,30,40,41]. Renn [17] offer the definition and categorisation of complex, uncertain and ambiguous risks. Aven and Renn [30] develop a typology of risk challenges for simple and systemic risks. IRGC [19] provide the initial criteria to guide the search strategy (keywords and snowballing) and the detailed search (risk evaluation criteria).

Regarding practice, we looked for widely accepted international disaster risk indices. To identify these indices, we used the snowballing approach. The first index selected was the Index for Risk Management—InfoRM. Its methodological note identifies two other similar indices which we incorporated into the assessment [42]. Through the same snowballing approach, we identified the fourth and final index.

Finally, we conducted a search to identify literature and practice reviews on risk evaluation approaches. We found no literature that compares the different risk evaluation approaches to develop a new approach for complex, uncertain, and ambiguous risks. Therefore, this study’s scientific contribution is an in-depth analysis of risk evaluation approaches with the explicit of developing RED-A to support progressive transition into a socio-technical heterogenous DRM approach. This approach was specifically designed to improve the quality of tolerability and acceptability judgments, during the risk evaluation phase, in diverse contexts.

3.2. Research Questions

The primary research question is: do the select publications and practice guide the design of a RED-A to support the progressive transition towards a heterogeneous social-technical DRM approach?” The following research sub-questions guided the systematic literature and practice review and identified the ‘state of art’ in designing approaches for disaster risk evaluation.

• RSUBQ1: Does the publication offer a conceptualisation of risk evaluation criteria?
• RSUBQ2: Does the publication describe a risk evaluation approach to address uncertainty, complexity and ambiguity?
• RSUBQ3: Do the publications and practice guide the design of the RED-A typology of risk evaluation criteria and checklist for evaluating uncertainty, complexity and ambiguity?
• RSUBQ4: What result from applying RED-A to the select publications and practice?
• RSUBQ5: To what extent is the RED-A criteria applicable in Solotvyno municipality, Ukraine (select case study)?

The terms “disaster” OR “natural hazards” OR “hazards” ensured the inclusion of the relevant natural and man-made risks. The search strategy incorporates a broad scope of risk evaluation approaches. The search terms are: “risk evaluation approaches” OR “community-based risk evaluation” OR “community-based risk evaluation approaches” OR “risk evaluation methods” OR “criteria for evaluating risk.” We extrapolated data from the publications and practice to develop the RED-A criteria and checklist.

3.3. The Strategy for Searching for Risk Evaluation Approaches

We designed a strategy to support a consistent and holistic search. Category A search on the definition of risk used consistent terms [43]. The search terms were risk (A1), risk definition (A2), risk probability (A3), risk consequence (A4), define risk (A5) and what is a risk (A6). After deriving an acceptable definition of the term risk, the second search category (B) identified publications and practice that define and detail different community-based risk evaluation approaches. Category B comprises the search terms: risk evaluation approaches (B1), community-based risk evaluation (B2), community-based risk evaluation approaches (B3), risk evaluation methods (B4) and criteria for evaluating risk (B5). To narrow down the publications and practice, we combined category B search with the following terms: disaster, hazard or natural hazard.

We searched five databases to extract the results: Google Scholar, Scopus, ScienceDirect, SpringerLink and JSTOR. The select databases contain relevant scientific papers for the planned literature and practice review.

3.4. Selection of the Articles to Study

To make sure the articles are valid and reliable, we restricted the search to peer-reviewed articles. We limited the publications and practice to risk evaluation approaches, and not risk assessment approaches (prior phase of assessing a risk). To include an approach, it should consider the normative parts of a risk in making judgments on acceptability and tolerability [13]. We excluded non-English written publications and inaccessible, full copy articles (abstract only).

Table 1. Search strategies for evaluation approaches.

Keywords	Databases	Search Outcome	Last Date of Search
(“risk evaluation approaches,” OR “community-based risk evaluation,” OR “community-based risk evaluation approaches,” OR “risk evaluation methods,” OR “criteria for evaluating risk,” AND (“disaster,” OR “natural hazards,” OR “hazards”)	Google Scholar	1060	08/05/2020
	ScienceDirect	446
	SpringerLink	201
	Scopus	81
	JSTOR	5

summarises all the strategies used for finding evaluation approaches and the outcome. We obtained 1793 publications from searching electronic databases. Additionally, we extracted the practice information from existing internationally accepted disaster risk indices.

We followed the standard practice of developing exclusion and inclusion criteria [44].

Table 2. Inclusion and exclusion criteria.

Inclusion	▪ Publications must be peer-reviewed ▪ Publications that develop and detail a generic risk evaluation approach. ▪ Three internationally accepted disaster risk indices
Exclusion	▪ Publications on pre-assessment stage risk approaches ▪ Publications on risk assessment stage approaches ▪ Publications on concern assessment stage approaches ▪ Publications on risk characterisation stage approaches ▪ Publications that describe a DRM stage approach ▪ Non-English publications ▪ Publications accessible as an abstract only ▪ National disaster risk indices

presents the exclusion and inclusion criteria we used for the analysis. We reviewed all publications with no restrictions on the date of publication. The search date was 8th May 2020.

We filtered through the first set of publications based on the title. We only included peer-reviewed articles with relevant titles and chose seventy-two (72) articles based on their titles. The 72 abstracts were further reviewed to identify detailed risk evaluation approaches, as opposed risk appraisal approaches [45]. The final filter involved removal of articles that focused on a specific risk assessment and did not develop a generic and replicable risk evaluation approach. Figure 1 is a visualisation of the selection process.

After reading all the 72 abstracts and eliminating unsuitable publications based on the exclusion criteria, we included 12 articles.

3.5. Snowballing

We used the snowballing approach to identify key articles missed during the database searches. First, we used backward snowballing to identify articles that passed the exclusion-inclusion criteria stage. Thereafter, we used forward snowballing to check for articles that cited publications that had passed the initial assessment [46]. All the papers found through snowballing underwent the same assessment steps, as mentioned earlier in the section. We identified eleven articles in the initial snowballing exercise. While reading the select 23 articles, we subsequently identified another 4 articles. Therefore, 27 (23 + 4) articles passed the eligibility phase.

We also used snowballing to identify the practice-based risk evaluation approaches. Four internationally accepted disaster risk indices were identified, namely:

• The WorldRiskIndex scientifically managed and calculated by the Institute for Law of Peace and Armed Conflict (IFHV) at Ruhr University Bochum. The index comprises of 180 countries and since 2011 its annual WorldRiskReport is published by Bündnis Entwicklung Hilft. All the annual reports are easily accessed through the official webpage [47].
• The Index for Risk Management—InfoRM. InfoRM was initiated in October 2012 at the Joint Research Centre of the European Commission (JRC) workshop. It is a collaboration between the European Commission and the Inter-Agency Standing Committee Reference Group on Risk, Early Warning and Preparedness. European Commission JRC manages and is the technical lead of InfoRM. It is designed to support disaster risk prevention, preparedness and response decisions. This index is different from the other three since its outputs and data are updated regularly and are made readily accessible in different formats [42].
• The Global Urban Risk Index developed by the World Bank provides information on the high-risk cities to natural hazards and an analysis of temporal changes to risk levels. The index supports the prioritisation of disaster risks and guides the transition from managing emergencies to managing risks [48].

The United Nations Office for the Coordination of Humanitarian Affairs (UN OCHA), Global Focus Model (GFM) is a quantitative risk model, developed in 2007. It seeks to optimise humanitarian aid allocation and ensure equity, impartiality and transparency. The model has hazard, vulnerability, capacity and humanitarian focuses. Since its inception, the model is used internally with annual updates during the OCHA’s work planning cycles [49].

3.6. Quality Assessment of the Literature and Practice Review

To assess quality, we screened the select 27 articles and the four disaster risk indices, within the limits of the first four research sub-questions. Specifically, we assessed whether each publication and the index met the criteria. The response was no or yes, and guided by the sub-questions below:

• RSUBQ1: Does the publication/index offer a conceptualisation of risk evaluation criteria?
• RSUBQ2: Does the publication/index describe a risk evaluation approach to address uncertainty, complexity and ambiguity?
• RSUBQ3: Do the publications and practice guide the design of the RED-A typology of risk evaluation criteria and checklist for evaluating uncertainty, complexity and ambiguity?
• RSUBQ4: What result appears from applying RED-A to the select publications and practice?

We coded the yes or no responses as 1 or 0, respectively. The publications that met all the criteria scored 4 out of 4. The final publications scored 3 or higher. Publications and indices that scored 1 and 2 were first checked to ensure that the scoring was correct and thereafter excluded. Finally, we selected 16 articles to inform this study, as listed in Appendix A.

3.7. Extraction of Data

The data extraction phase involved the methodical assessment of the final 16 publications and 4 disaster risk indices. The research questions guided extracting data from the selected papers and indices. For instance, research question one guided the researchers to list all the risk evaluation criteria and provide an explanation of what each criterion represents. We left publications with insufficient information to address the research questions out of the analysis.

3.8. Analysis and Categorising Data

Research question 2-5 relied on Aven and Renn’s [30] criteria for determining complex, uncertain and ambiguous risks (see

Table 3. Typology of risk categorisation (modified from Aven and Renn [30]).

Risk Category	Strategy	Stakeholder Participation
Simple	Cause-effect risk-based strategy	Instrumental discourse • Involves relevant government agencies, enforcement personnel and groups directly affected.
2. Complex	Robustness risk-based strategy	Epistemological discourse • Resolves cognitive conflicts by applying the “state of art” probability and effects predictions.
3. Uncertain	Precaution-based strategy	Reflective discourse • Engages stakeholders in the process, safety boundaries agreements, consensus on risk avoidance actions and collective pact on balancing over and under protection.
	Resilience-based strategy
4. Ambiguous	Discourse-based strategy	Participative discourse • Open discussion of competing values, arguments, beliefs and objectives with the aim of consensus building, tolerance building for results and options and closure.

).

The typology of risk categorisation supported the systematic literature and practice review, design and testing of RED-A. We categorised the data collected from the publications and practice according to three categories (uncertainty, complexity and ambiguity), and the corresponding sub-criteria.

4. Categorisation of the Results

The section presents categorised outcomes in response to questions 1–4. The research used inductive reasoning and identified eight risk evaluation approaches that were used to answer the first three questions and informed the creation of a combined approach and its application in Solotvyno municipality. Subsequent subsections describe the categorisation results of research questions 1-5.

4.1. What Is the Conceptualisation of Risk Evaluation Criteria for Each of the Select Publications and Practice?

The aim of this sub-question is to analyse multiple approaches and select risk evaluation approaches that match the typology of risk categorisation (Table 3).

4.1.1. Risk Conceptualization in Selected Publications

We selected four approaches: German Advisory Council on Global Change (WBGU), Th. Plattner, Predictive Bayesian Risk Classification (PBRC) and Pollard [13,14,15,16].

WBGU [13] has nine criteria: extent of damage, probability of occurrence, reversibility, incertitude, persistency, ubiquity, violation of equity, delay effect and potential of mobilisation. Extent of damage are the negative consequences, for instance, mortality rate or number of persons injured. Probability of occurrence approximates the chance of given risk happening. Incertitude is the umbrella term for all uncertain components. Ubiquity shows the degree of the geographical spread of possible effects and persistency is the time-based spread of probable consequences. Reversibility assesses the extent in which the natural environment can be restored to its original state. Delay effect generally refers to slow-onset disasters where there is a long duration between when the event occurs, and when the consequences are experienced. Equity is violated when there are discrepancies between the risk-bearers and the persons enjoying the benefits. Potential of mobilisation refers to perceived violation of socio-cultural and individual values and interests by the groups of people in the community who must bear the risks. This might lead to social unrest and conflicts that need to be resolved.

Plattner [14] developed an evaluation approach to compute perceived and acceptable individual risk factors that focus on natural hazard risk perception at the individual level. These factors are voluntariness, reducibility, knowledge, endangerment, the extent of damage and frequency of an event.

Table A1. Th. Plattner evaluation criteria. Source Th. Plattner [14].

Evaluation Criteria	Represents
Voluntariness	Voluntariness
Reducibility	Reducibility Predictability Avoidability
Knowledge	Familiarity Knowledge about risk Manageability
Endangerment	Controllability, Number of people affected Fatality of consequences Distribution of victims Scope of area affected Immediacy of effects Directness of impact
Extent of damage	Extent of damage
Frequency of event	Frequency of event

in Appendix B show what each factor represents.

Kristensen and Aven [15] developed PBRC scheme, as a revised version of the WBGU approach. PBRC schema comprises nine criteria. The revised criteria are uncertainty about consequences and potential consequences. Potential consequence replaces the WBGU extent of damage. Uncertainty about consequences includes variability of the following factors: human, plant, and animal; technical systems; model results; and indeterminacy. Kristensen and Aven [15] included a new criterion: the difficulty in establishing appropriate performance measures to be measured using subjective rating scales.

The Pollard [16] approach comprises 17 attributes that include the natural environment risk and the societal risks.

Table A2. Explanations of the Pollard attributes. Source: Pollard [16].

Attribute	Explanation
Stock at Risk	Represents a valuation of the overall harm, either in economic terms of in terms of numbers of receptors.
Knock-on Effects	Reflects that harm to one receptor may affect the wellbeing of another receptor.
Spatial Extent	Denotes the overall area in which exposures to the hazard that causes environmental harm are experienced.
Heterogeneity	Reflects that within the overall area denoted by Spatial Extent, there may be heterogeneity in exposure to the hazard.
Sensitivity of Receptor	Reflects the proportion of receptors exposed that exhibit the harm.
Severity of Effect	Defines the general physical effect on an individual sensitive receptor only.
Temporal Extent	Denotes the time that the environmental harm will be experienced.
Latency	The period before the consequent environmental harm is realised during the hazard.
Accumulation	The changes in rate at which the harm progresses.
Reversibility	Considers both whether the effects of the harm are reversible and, if so, over what time scale.
Scarcity	Reflects the abundance of the receptor and is used to consider the loss of cultural resources, and physical environments.
Dread	Reflects that society can have an aversion to, or fear of, a harm that may be unrelated to its physical nature.
Unfamiliarity	Assesses concern that may arise out of low knowledge and understanding of the harm.
Notoriety	Reflects the potential for raised awareness of, and anxiety about, the harm via the media and other channels and information.
Unfairness	Reflects the discontent that may arise from the inequity or unfairness of a harm’s distribution.
Imposition	Measures the social value afforded by the degree of personal control over the harm.
Distrust	Captures the consequences of a lack of trust in the characterization of the harm or in those responsible for its management.

in Appendix B briefly explains each of the 17 attributes.

4.1.2. Risk Conceptualization in Practice (Select Indices)

We selected four international disaster risk indices: The WorldRiskIndex [47], Index for Risk Management—InfoRM [50], Global Urban Risk Index (GURI) [48] and the OCHA Global Focus Model (GFM) [49].

The WorldRiskIndex is calculated every year and a ranking report is released. The 2019 report ranked 180 countries and the theme was water supply (Figure 2). The country with the lowest disaster risk was Qatar. At the continent level, Europe had the lowest disaster risk and Africa had the highest societal vulnerability. Disaster risk hotspots were located in Central America, South East Asia, Oceania and Central and West Africa [47], (p. 6). The index is based on four components, namely exposure, susceptibility, coping and adaptive capacities. Exposure is limited to natural hazards, namely, floods, sea-level rise, droughts, earthquakes and cyclones. Susceptibility components calculate the likelihood that food supply systems, infrastructure and economic framework conditions will suffer harm when a risk event occurs. Coping component assesses the capacity to reduce the disaster impacts within healthcare, governance, material security and social systems. Adaptive component assesses the capacity of societal changes and long-term strategies to adapt to future natural disaster events, and climate change [47], (p. 15). Crucially, the index only admits quantifiable data. Possible data quality variations may occur if the data is solely collected by government agencies. The report contains theme specific qualitative data to complement the ranking results. Key limitations of the approach are complexity-reduction to individual values, missing values, and difficulty of ascertaining whether islands or overseas territories are included [47], (p. 50).

InfoRM disaster risk conceptual framework is based on four components hazard, exposure, vulnerability and coping capacity [50] (p. 13). Vulnerability is based on Cardona [51] holistic approach to assess and manage disaster risks. The holistic approach calculates vulnerability on three levels: physical exposure and vulnerability (hard risk); socio-economic system fragility (soft risk); and weak coping and recovery capacity (soft risk). The InfoRM model combines hazard and exposure into one dimension and then designates socio-economic vulnerability and lack of coping capacity as two separate dimensions. Therefore, even though the model comprises of four components, they are assessed as three dimensions (Figure 3). The model is designed to counterbalance the relationship between exposure and hazard on one side of the scale and vulnerability and lack of coping capacity on the other side of the scale [50] (p. 16). Each dimension has a number of categories that vary depending on the user needs [50] (p. 17). Key methodological limitations are: (1) complexity reduction by use of a deterministic approach; (2) dimension interactions not included in the model; and (3) use of proxies to address uncertainty [50] (p. 44). Data limitations are: (1) slow and sudden onset hazards are not included; (2) biological and technological hazards are not included; (3) there is possible subjectivity of the DRR component because more than 70 percent of the countries submit self-assessments; (4) probability of distortions due to missing values; and (5) even though the model is designed to provide real-time data, there constraints to this aspect of the index due to the reporting time lag and the requirement for validation before updating the model [50] (pp. 44–45).

The Global Urban Risk Index (GURI) comprises of four main components hazard (frequency and severity), exposure (population and GDP), vulnerability (estimated damage and mortality) and calculation of loss. Hazard is limited to floods, landslides, cyclones and earthquakes. Hazards are determined using global datasets. Cyclones and floods are derived from the UNEP/GRID-Europe and Dartmouth Flood Observatory databases. Landslide data is derived from the Norwegian Geotechnical Institute and the International Centre for Geohazards. The Global Urban Risk Index exposed elements are people, communities, economics, buildings and infrastructure. The population of the city is determined by the “Henderson city” and the Centre for International Earth Science Information Network (CIESIN) databases. The CIESIN database is known as the Global Rural-Urban Mapping Project (GRUMP) raster data. Vulnerability is disaggregated into social (access to resources/political power), environmental, economic and physical (age of building, materials used and type of construction) categories. Vulnerability data is extracted from the Centre for Research on the Epidemiology of Disasters (CRED) Emergency Events Database (EM-DAT), [48].

The OCHA Global Focus Model is designed to provide three output indices: risk, humanitarian, and focus. Risk is calculated by multiplying hazard (natural and man-made), vulnerability and capacity. Capacity includes an assessment of three areas: institutional, economic and infrastructure. Vulnerability contains four aspects, namely poverty, livelihood, dependency and the environment. The humanitarian component is a standalone component that objectively assesses whether the risk falls within the OCHA mandate. Focus indicates the likelihood of OCHA intervening in the humanitarian action to reduce or manage the disaster risk. The model is designed for the external audience to understand the severity of the disaster risk and identify whether it falls within the OCHA mandate. Internally OCHA uses the model to prioritize the risks in which OCHA is expected to intervene [49].

4.2. Do the Select Risk Evaluation Approaches Address Uncertainty, Complexity, and Ambiguity?

This sub-question assesses whether the select publications and practice address uncertainty, complexity and ambiguity. The process involved understanding the definition and application of each criterion in the four publications and practice and grouping all of them according to the typology of risk categories (Table 3). Most of the eight approaches contained complexity-based, uncertainty-based and ambiguity-based risk evaluation criteria.

Table 4. Risk evaluation categories and corresponding criteria proposed by select approaches.

	Objective/Function	Criteria	Approach
Complexity	Identify and ensure there are adequate, effective and efficient risk reduction measures Enhance agreement on causal relationships	Extent of damage, Probability of occurrence, Delay effect	Klinke and Renn [13]
		Endangerment, Extent of damage, Frequency of event	Plattner [14]
		Potential consequences, Delay effect and the difficulty in establishing appropriate performance measures	Kristensen, Aven and Ford [15]
		Stock at Risk, Knock-on Effects, Sensitivity of Receptor, Severity of Effect, Accumulation, Scarcity, Latency	Pollard, Davidson and Yearsley [16]
		Exposure (Earthquakes, Cyclones, Floods, Droughts and Sea-level rise); Susceptibility (public infrastructure, environmental status)	WRI [47]
		Hazard and exposure (natural—earthquake, flood, tsunami, tropical cyclone, drought and epidemics)	InfoRM [42]
		Hazard (frequency and severity); Exposure (population and GDP); and environmental and physical vulnerability	GURI [48]
		Hazard (natural and man-made)	GFM [49]
Uncertainty	Resilience building. Ensure fair distribution of risk, damage, and associated benefits (equity)	Incertitude, Ubiquity, Persistency, Reversibility	Klinke and Renn [13]
		Reducibility, Knowledge	Plattner [14]
		Uncertainty about consequences, Ubiquity, Persistency, Reversibility	Kristensen, Aven and Ford [15]
		Spatial Extent, Temporal Extent, Heterogeneity, Unfamiliarity, Unfairness, Reversibility	Pollard, Davidson and Yearsley [16]
		Susceptibility (education, gender equality, adaptation strategies, poverty and dependencies, economic capacity and income distribution, housing conditions, nutrition) Coping capacity (government and authorities, services, social networks, insurance)	WRI [47]
		Socio-economic vulnerability and vulnerable groups, Lack of institutional coping capacity (DRR and governance), Infrastructure coping capacity, Investment	InfoRM [42]
		Economic and social vulnerability (access to resources and political power)	GURI [48]
		Capacity (institutional, economic, and infrastructure), Vulnerability (poverty, livelihood, dependency, environment)	GFM [49]
Ambiguity	Social acceptance of decisions. Resolve conflicting values and judgments. Fair process to align competing objectives and concerns.	Violation of equity, Potential of mobilisation	Klinke and Renn [13]
		Voluntariness, Knowledge	Plattner [14]
		Violation of equity, Potential of mobilisation	Kristensen, Aven and Ford [15]
		Dread, Unfamiliarity, Notoriety, Unfairness, Imposition, Distrust	Pollard, Davidson and Yearsley [16]
		Human conflict (current and projected);	InfoRM [42]

shows the risk evaluation approaches encountered during the literature and practice review, their respective objective or function, the identified criteria in each of the approaches and the references.

4.3. What Is the Typology of RED-A Criteria for Evaluating Uncertainty, Complexity, and Ambiguity?

The purpose of this sub-question is to develop the typology of RED-A criteria and distil it further into a checklist. RED-A provides guidance to DRM researchers and practitioners on adopting a diverse and holistic approach to DRM. It helps them to recognize cognitive biases in the current DRM cycle which can have a direct effect on the efficiency of future DRM procedures.

RED-A is particularly useful at the third phase of DRM—the risk evaluation phase. This is a critical phase because judgments are made whether a certain risk is acceptable, tolerable or intolerable. According to literature and practice, these judgements cannot be made solely on evidence, societal values are equally critical in the decision making process [45]. During this phase, knowledge regarding the disaster risk has already been collected in the pre-assessment (problem framing, early warning, screening and determining scientific conventions) and the risk appraisal phases. At the risk appraisal phase, the scientists conduct risk (hazard, exposure and vulnerability) and concern (perceptions, social concerns and socio-economic impacts) assessments. However, since the first two phases are mainly conducted by natural and technological scientists, there is a wealth of heterogenous information that is left out which is mainly data on vulnerability and concern assessments. Therefore, RED-A is an important tool that helps DRM experts to holistically incorporate useful information in the DRM process before proceeding to evaluate risks.

RED-A consists of a three-stage process that is fundamentally iterative and adaptable. First, the DRM researchers check whether complexity is sufficiently addressed in the content and process. Second, they assess whether uncertainty is adequately conceptualised, assessed, evaluated and managed in available DRM evidence and procedures. Finally, they check whether ambiguity is adequately conceived and managed throughout the DRM process. Each RED-A stage is guided by a set of criteria. In total, RED-A comprises of 12 criteria: three to address complexity, six for uncertainty-based risks and three for ambiguity. Figure 4 provides the detailed list of RED-A criteria.

Based on the results of sub-question 2, we reduced the criteria in Table 4 to twelve criteria. The reduction exercise involved removing repeated criteria represented under similar or different names in the three categories. In addition, we eliminated criteria that were not relevant for the risk evaluation phase.

Table 5. Typology of Risk Evaluation Diversity-aiding Approach (RED-A) criteria.

Category	Identified Need	Criteria	References
Complexity	Shift from simple cause-effect rules to generative predictive rules that enable the systematic adaptation of societies to their environments [52].	Magnitude of damage [53]	[13,14,15,16,42,47,48,49]
		Probability of occurrence	[13,14,42,47,48,49]
		Delay Effect [Latency]	[13,14,15,16]
2. Uncertainty (epistemic)	Shift from a single measurable dimension of uncertainty to multiple dimensions [natural systems, social systems, technical systems, temporal variability, spatial variability, frequency] [40]	Uncertainty about consequences [Incertitude]	[13,14,15,16]
		Ubiquity [Intragenerational justice]	[13,15,16]
		Persistency [Intergenerational justice]	[13,15,16]
		Reversibility	[13,15,16]
		Vulnerability	[42,47,48,49]
		Coping capacity	[42,47,49]
3. Ambiguity	Shift from single interpretations to variability of interpretations of risk, effects and benefits [13]	Degree of voluntariness	[14]
		Violation of equity	[13,15,16]
		Potential of mobilisation	[13,15,42]

shows the twelve select criteria, and the identified need that each of the three categories and twelve criteria are seen to address.

To support the RED-A criteria, we developed a checklist of 22 items (

Table 6. Risk Evaluation Diversity-aiding Approach (RED-A) Checklist.

WE HAVE EVALUATED THE FOLLOWING …
The magnitude of the damage/harm a natural or man-made disaster risk may cause #C1.
2. The potential consequences (economic, injured, mortality, damage to critical infrastructure and disruption of basic services) #C1.
3. The discrete or continuous estimates of relative frequency #C2.
4. The probability function robust to holistically predict systemic risks #C2.
5. The time of latency between the initial event and actual impact #C3.
6. The impact of the time of latency on risk severity and perceptions #C3.
7. Taken account of limited or no knowledge in modelling and predicting risks #U1.
8. The maximum level of risk exposure to individuals when there is limited or no data #U1.
9. The geographical dispersion of potential damages (ubiquity) #U2.
10. The temporal extension of potential damages (persistency) #U3.
11. Whether the damage can be reversed/the possibility of restoration to the original state #U4.
12. The costs of risk reduction and what amount should be invested by the society #U4.
13. Extent of environmental, physical, infrastructural and economic vulnerability #U5.
14. Extent of susceptibility (education, gender equality, adaptation strategies, poverty and dependencies, economic capacity and income distribution, housing conditions, nutrition), social vulnerability (access to resources and political power) and the vulnerable groups #U5.
15. Level of institutional coping capacity (DRR and governance), infrastructure coping capacity, investment #U6.
16. Level of societal resilience (government and authorities, services, social networks, insurance) #U6.
17. If some risks are perceived to be less serious due to opinions of high societal value and control #A1.
18. Whether there is a consistent risk profiling by society based on semantic patterns #A1.
19. Whether there is a discrepancy between those who enjoy the benefits and the risk-bearers #A2.
20. Whether risk-benefit and cost effectiveness analyses have been conducted #A2.
21. If there are inequities and injustices associated with risk and benefit distributions #A3.
22. If the risk generates stress, discomfort, conflict, public pressure and spill-over effects #A3.

) to ensure that there is an objective assessment of the application of RED-A, based on the twelve RED-A criteria. The purpose of the 22-item checklist is to ensure that all 12 RED-A criteria are integrated extensively to support the progressive transition to a heterogeneous social scientific DRM approach.

The RED-A Checklist addresses unconscious biases and expands the comfort zones from simple to systemic risk approaches. The checklist comprises mainly of two questions that a person preparing for a risk evaluation process should check as a pre-requisite for each of the RED-A criteria. RED-A checklist may help scientists and practitioners to be better prepared for a risk evaluation process that can effectively make judgments on systemic risks in a heterogeneous and inclusive manner.

4.4. What Result from Applying Risk Evaluation Diversity-Aiding Approach (RED-A)?

The purpose of this sub-question is to apply RED-A to the eight risk evaluation approaches and tabulate the results. Sub-question 4 relied on the twelve RED-A criteria to manage complex, uncertain and ambiguous risks (see Figure 4). We will first explain the ten risk evaluation criteria (minus vulnerability and the coping capacity) in relation to the publications only and later briefly explain the added value of the indices.

Table 7. Applying the Risk Evaluation Diversity-aiding Approach (RED-A).

	Criteria	WBGU	Th. Plattner	PBRC Scheme	Pollard Approach	WRI	InfoRM	GURI	GFM
Complexity	Magnitude of damage (C1)
	Probability of occurrence (C2)
	Delay Effect [Latency] (C3)
Uncertainty	Uncertainty about consequences (U1)
	Ubiquity (U2)
	Persistency (U3)
	Reversibility (U4)
	Vulnerability (U5)
	Coping capacity (U6)
Ambiguity	Degree of voluntariness (A1)
	Violation of equity (A2)
	Potential of mobilisation (A3)

shows the results of applying RED-A to the approaches in the select publications and practice. According to the results WBGU was the most aligned approach, with all of its nine criteria represented. The additional criteria not included in the WBGU approach is the Th. Plattner’s degree of voluntariness criterion. The PBRC results are almost like the WBGU results because it adopted most of the WBGU criteria with a few changes. The Pollard approach has seven criteria in the same way as the RED-A criteria. The least aligned criteria are the Th. Plattner, which only have five criteria represented and all of them except for the degree of voluntariness criterion fall within the complexity-based category.

Fully represented criteria are the magnitude of damage, delay effect, and uncertainty of consequences. They scored 4 out of 4. The least represented is degree of voluntariness, followed by probability of occurrence.

Two important contributions from most of the practice-based risk evaluation approaches are vulnerability and coping capacity. Though these two elements are supposed to be included in the calculation of risk, most DRM experts do not include them in their analysis because their models are not designed to handle the increased complexity, uncertainty and ambiguity [19]. However, the four indices focused on the basic definition of risk (hazard x exposure x vulnerability x coping capacity), with little focus on other social and political dimensions of risk. All the indices did not address delay effect, uncertainty about consequences, ubiquity, persistency, reversibility, degree of voluntariness and violation of equity. Since the indices are quantitative, it is difficult to include the missing criteria into their model, as most of the data are qualitative and heterogenous. Ambiguity was not addressed in most of the indices, except for InfoRM, which slightly incorporated aspects of potential for mobilisation through introducing a human conflict criterion.

4.5. What Result from Applying RED-A to Solotvyno Municipality?

The purpose of this sub-question is to apply RED-A to the select case study: Solotvyno municipality. The RED-A criteria and checklist (Table 5 and Table 6) guided the systematic assessment of whether the case study had complied with the RED-A criteria.

Table 8. Application of RED-A to Solotvyno municipality, functions, strategies and instruments.

	Criteria	Function	Strategies	Instruments
Complexity	Magnitude of damage	Agreement on the extent of land subsidence damage and measures	• Reduce the potential of mortality, injuries, economic loss, job loss, damage to infrastructure and disruption to basic services.	• Hazard, exposure, and vulnerability assessments. • Risk estimation • Cost-effectiveness • Risk-benefit analysis
	Probability of occurrence	Agreement on the probability of land subsidence and measures	• Plans to limit overall risk levels, restoration, and relocation plans	• Risk mapping • Early warning advisories
	Delay Effect [Latency]	Treatment of latency between initial event and the actual impact	• Short-medium-long-term risk and land management plans	• Latency predictions • State of art risk reduction measures
Uncertainty	Uncertainty about consequences [Incertitude]	Build system and societal resilience as a buffer from uncertain consequences	Resilience building and embracing diversity	• Develop substitutes/scenarios/pathways
	Ubiquity [Intragenerational justice]	Avoid an inequitable geographical dispersion of potential damages	• Trade-off analysis • Limiting the spatial range of effects	• Spatial containment of high-risk zones (close mining area)
	Persistency [Intergenerational justice]	Avoid inequitable temporal extension of potential damages	• Implementation of sustainability ‘state of art’ standards	• Time containment • Create environment protection regulation
	Reversibility	Avoid irreversibility	• Stop salt mining • Stop brine pumping • Stop tourism • Restore the land	• Develop mining and tourism substitutes • Land restoration action planning
	Vulnerability	Reducing vulnerabilities and the number of vulnerable groups	• Social insurance, Livelihood diversification • Green tourism and businesses • Establish programmes for vulnerable groups	• Vulnerability assessments • Investment mapping • Development planning and implementation • Land use planning • Investment planning
	Coping capacity	Strengthen coping capacity (institutional, infrastructural, environmental)	• Enhance DRR capacity • Improve governance • Enhance stakeholder communication • Strengthen the capacities of infrastructure	• Coping capacity assessments • Capacity enhancements
Ambiguity	Degree of voluntariness	Incorporate diverse risk perceptions	• Seek consensus • Analyse voluntariness	• Risk perception mapping
	Violation of equity	Address discrepancies between benefit-sharing and risk-bearing.	• Develop a socially accepted benefit and risk sharing formula	• Household surveys • Citizen panels • Advisory committees
	Potential of mobilisation	Resolve conflicting values and judgments and competing objectives and concerns.	• Seek consensus on competing values, objectives and concerns.	• Multi-criteria decision analysis • Value tree analysis • Negotiated rulemaking

The grey highlights represent the criteria, function, strategies, and instruments that have been implemented by the Ukrainian central and regional government. The other parts are either works in progress or have not yet been addressed.

shows the results of applying RED-A to Solotvyno, the respective objective, the identified strategies for each criterion and the probable instruments.

The highlighted parts (Table 8) are the criteria, function, strategies and instruments that have been implemented by the Ukrainian central and regional government [21,23]. The government and stakeholders have not yet addressed the other criteria. Based on the results, there is a bias toward risk-based strategies and less on precaution, resilience and discourse-based strategies. There is no attempt to implement resilience and discourse-based approaches, though there was a limited adoption of precaution-based criteria.

5. Discussion: Risk Evaluation Diversity-Aiding Approach

In this study, we identified eight risk evaluation approaches that form the foundation for RED-A [13,14,15,16]. Thereafter, we applied RED-A to Solotvyno. In this section, we explain the contribution and added value of RED-A and then discuss the results of its application for Solotvyno. Also, we highlight the identified biases and needs, discuss the concept of embodied uncertainty to address the identified needs and conclude with future research.

5.1. Overview of the RED-A Design and Application Process

The study found little guidance in literature on how scientists and practitioners shift to socio-technical heterogeneous DRM approaches that incorporate uncertainty, complexity, and ambiguity reduction strategies. In addition, there is an unconscious bias towards risk-based strategies with little or no focus on precaution, resilience, and discourse-based strategies. Thus, developing a diversity-aiding approach is particularly useful.

The primary question is: “does the literature and practice review guide the design of RED-A to support the progressive transition towards a heterogeneous social-technical DRM approach?” We designed RED-A to infuse diversity into the prevailing risk evaluation approaches and support the shift from simple to systemic risk approaches that incorporates theories, methodologies and practices from diverse disciplines. We tested RED-A in the case of Solotvyno, and concluded that the RED-A can infuse diversity into the risk evaluation process for a given case study.

The design process entailed 4 steps (Figure 5): a literature and practice review, selecting priority criteria, developing the criteria and checklist and implementing RED-A to a specific case study to draw some insights. Figure 5 provides a better understanding of the process.

First, we performed a literature and practice review. By defining a set of search terms that describe risk evaluation approaches, we conducted a systematic search for articles. We selected relevant articles based on the exclusion and inclusion criteria. One major inclusion-exclusion criterion was the separation of risk evaluation publications from the risk assessment articles. To do this, we considered whether the aim of the approach was to support acceptability and tolerability judgments or the estimation of stimuli and consequences.

After applying the inclusion-exclusion criteria, we found several publications on risk evaluation, indicating a bias towards the pre-assessment and risk appraisal risk governance phases over the risk evaluation phase. To ensure that we included most of the relevant publications, we adopted the snowballing method. The last step of the initial analysis phase was the selection of the actual approaches to undergo detailed analysis and support the answering of the sub-questions.

We developed five sub-questions in the initial analysis phase to guide the categorisation of the results so that the subsequent discussion is systematic. In addition, we developed a three-pronged typology of risk categories and applied it consistently throughout Section 4. Section 4.1 explained the conceptualisation of risk evaluation in the select approaches. Section 4.2 applied the typology of risk categories to the select publications and practice, and confirmed that the four risk evaluation approaches seek to address uncertainty, complexity and ambiguity. However, the indices sought to reduce complexity, uncertainty and ambiguity to be able to support the functionality of the respective models. Section 4.3 reduced the number of criteria for the RED-A to twelve by removing the repeated criteria or items that fall within the pre-assessment and risk appraisal phases.

We outlined the RED-A criteria and checklist in Section 4.3 and compared the RED-A criteria with the eight risk evaluation approach items in Section 4.4. WBGU and PBRC approaches are more aligned to the RED-A. Three key additions are the degree of voluntariness criterion, introduced by Th. Plattner approach and two practice related criteria, vulnerability, and coping capacity. The Degree of voluntariness criterion is particularly important for Solotvyno, where certain societal groups have demonstrated high degrees of voluntariness and refused to relocate to Tereblya due to various reasons. Therefore, the risk acceptance levels amongst these groups may be extremely high even though they are exposed to an intolerable land subsidence risk [23]. It is important to note that even though vulnerability, and coping capacity are part of the risk appraisal process, many scientists eliminate them from the risk assessment process because their models cannot incorporate the inherent complexity, uncertainty and ambiguity. Thus, in RED-A, they are included as exclusive criteria to ensure that future DRM processes take them into account.

5.2. Added Value of RED-A in Unveiling Implicit or Cognitive Biases

The traditional cause-effect approach of treating, assessing, evaluating and managing risks is counterproductive for systemic risks [29]. However, making the shift from simple cause-effect rules to generative robustness-based rules is difficult because there was no approach that confronted existing implicit or cognitive biases that are unconsciously made. Therefore, the first contribution of RED-A is the creation of an inclusive risk evaluation environment through heightened consciousness. RED-A assesses whether scientists and practitioners are adopting approaches that incorporate uncertainty, complexity and ambiguity or perpetuate traditional risk approaches that exclude systemic risks. RED-A helps scientists and practitioners to stay open, pay attention and be intentional to make targeted shifts. These shifts should increasingly lead to more use of generative rules to manage complexity, embody multiple dimensions of uncertainty and embody a variability of interpretations of risks, effects, costs and benefits.

RED-A can unveil implicit or cognitive biases and increase understanding of the need to create more inclusive risk evaluation environments. Based on the results, we noted the following biases:

•

Risk-based category: the use of simple risk-based mono cause-effect strategies as opposed to complex and robustness risk-based strategies (Table 3 and Table 8).

•

Risk-based category: a focus on simple and complex-based categories with little or no focus on the scientific uncertainty-based category and no focus on the ambiguity-based category (Table 3 and Table 8).

•

Risk category strategy: the use of risk-based strategies as opposed to precaution, resilience and discourse-based strategies (Table 3 and Table 8).

•

Stakeholder participation: a focus on instrumental discourse with little or no focus on epistemological discourse and no focus on reflective and participative discourse (Table 3 and Table 8).

•

Objective | function: a focus on risk reduction measures and agreement on a causal relationship as opposed to resilience, equity and social acceptance objectives (Table 4 and Table 8); and

•

Risk-based criteria: more focus on complexity-reduction criteria and less on uncertainty-reduction criteria, with no evidence of ambiguity reduction criteria in the case study (Table 4 and Table 8).

RED-A is also valuable in identifying needs. We identified the following needs based on the application of RED-A and checklist to the Solotvyno case study:

•

Complexity-based risks: a shift from simple cause-effect rules to generative predictive rules that enable the systematic adaptation of societies to their environment (Table 5 and Table 8).

•

Uncertainty-based risks: a shift from a single measurable dimension of uncertainty to multiple dimensions [natural systems, social systems, technical systems, temporal variability, spatial variability, frequency] (Table 5 and Table 8); and

•

Ambiguity-based risks: a shift from single interpretations to variability of interpretations of risk, effects, costs and benefits (Table 5 and Table 8).

The RED-A criteria and checklist can support deeper reflection by infusing diversity related thinking into the risk evaluation process. The checklist supports reflection through providing a list critical content and process-oriented pre-requisites to the risk evaluation process.

5.3. Embodied Uncertainty: Towards Diversity in Risk Evaluation

At the start of this paper, we defined “risk as uncertainty about and severity of the consequences (or outcomes) of an activity with respect to something that humans value” [24]. ISO 31000:2018 [54] defines risk “as the effect of uncertainty on objectives,” whether it derives a loss or a gain. Uncertainty is at the core of any risk and is the reason why scientists and practitioners invest time and resources to understand, treat, assess, evaluate and manage a risk. This sub-section explains the linkages between the RED-A and embodied uncertainty.

Sword-Daniels et al. [40] explain that risk perceptions and responses are influenced by “a suite of personal and interpersonal characteristics and relationships that affect how we individually and collectively embody uncertainty.” [40]. The paper argues that objective risk is not holistic. The theory of embodied uncertainty explains the progressive shift from understanding the complexity, accepting and incorporating it rather than reducing it. Risk acceptance lies in a continuum from scientific uncertainty, towards decision-making and communication of uncertainty, and finally to embodied uncertainty (Figure 6).

Embodied uncertainty is the foundation of RED-A. RED-A promotes shifts from reducing uncertainty, complexity and ambiguity towards accepting and embracing these challenges. It supports the progressive acceptance of the persistency of uncertainty, complexity, and ambiguity in many heterogeneous forms. RED-A helps scientists and decision-makers to recognise their unconscious biases. RED-A embodies uncertainty by intentionally making what seemed uncertain and intangible–visible, measurable, tangible and actionable. Constant application of RED-A in each context may expose implicitly ignored complexities, uncertainties and ambiguities. It should be a constant reminder of systemic risks: heterogeneously complex, uncertain and ambiguous. Importantly, the progressive realisation of embodied uncertainty provides an opportunity for further analyses and adoption of transdisciplinary knowledge to bring about changes in the risk governance domain.

The shift may require further research in various domains. The first domain is research on institutional changes to support the transformation to resilience-based strategies and embodied uncertainty [26,55,56]. The second is studies to understand how decision are made under uncertainty and to support cognitive changes to embody uncertainty [31,40,57]. The third is studies to test social cognitive models that motivate the community in hazard preparedness and support relational changes necessary to implement RED-A [58,59], including trust-formation [60,61]. Heterogeneous risk perception studies and respect for diversity to support the shift to embodied uncertainty [62,63]. Finally, we propose research to catalyse changes in values and beliefs [64,65], and help manage the transition to systemic risk governance [33].

RED-A can also be the basis for the design of risk evaluation tools that embody uncertainty. We used RED-A to develop tools that promote diversity in the risk evaluation process of the European Union ImProDiReT (Improving Disaster Risk Reduction in Transcarpathian, Ukraine) project. One such tool is the community-based risk evaluation approach (CREA) household survey (see Supplementary Materials 1). To increase inclusivity, we translated the survey into Ukrainian (see Supplementary Materials 2), Hungarian (see Supplementary Materials 3) and Romanian (see Supplementary Materials 4). We implemented the household survey in September to December 2019 to collect risk evaluation data. The data provided input for the community action plan to reduce risks associated to the collapse of the land near the salt mines. We collected data on the community perceptions of the land subsidence risks, household, communal disaster risk readiness and the place attachment of the diverse groups.

6. Conclusions

Research establishes that disaster risk effects will exponentially increase unless there is a heavy investment of resources to build societal resilience. One key proposal for enhancing community resilience is to diversify disaster risk management (DRM) strategies. The Sendai Framework for Disaster Risk Reduction 2015–2030, the overarching international framework for DRM, calls for a diversified holistic DRM approach. However, a key bottleneck to adopting diverse DRM approaches is the historical dominance of the natural and technical sciences. DRM has largely been within the domain of the technology-related disciplines with little involvement of social scientists. This has led to weak risk governance. The DRM stage that has largely suffered because of the lack of involvement of social political scientists is the risk evaluation phase.

The risk evaluation phase mainly determines the tolerability and the acceptability of given risks using knowledge from evidence and societal value judgements. Tolerability and acceptability judgments are critical when deciding whether there is need for risk reduction measures. These judgments are made by considering available evidence and societal values. This study seeks to enrich the risk evaluation process by designing an approach to encourage transdisciplinary social-technical risk assessment, appraisal, evaluation and management.

DRM scientists and practitioners mainly focus on the pre-assessment and the risk appraisal phases. Sometimes the risk evaluation phase is by-passed. The danger with such an approach is the adoption of scientific evidence on the probability of a risk event and its consequences without considering other critical elements of a risk. Two key elements that are left out of traditional risk appraisal are incorporation of uncertainty and risk perceptions. Therefore, risk evaluations assess the acceptability and tolerability of a risk-based on the evidence and values judgments. However, evidence shows that the content of the scientific evidence is not sufficient because most risk appraisal experts are still using the simple cause-effect model to extract the evidence. This has led to a reduction of complexity, and the production of unreliable predictions. In addition, current risk appraisals do not incorporate uncertainty and ambiguity.

To address this gap, we conducted a systematic literature and practice review of 16 publications and 4 indices that applied risk evaluation approaches in DRM. The primary question is: “does the literature and practice review guide the design of a Risk Evaluation Diversity-aiding Approach (RED-A) to support the progressive transition towards a heterogeneous social-technical DRM approach?” To answer the question, we developed the RED-A and applied it in the Solotvyno (Ukraine) context.

At the initial phase of the literature and practice review, we developed a typology of risk categorizations: simple, uncertain, complex and ambiguous. Since our focus was on systemic risks, we removed simple risks in subsequent phases of the literature and practice review. From these publications and practice, we selected eight approaches for in-depth analyses. We analysed the eight publications and practice, and the input formed the basis for the design of a Risk Evaluation Diversity-aiding Approach (RED-A) criteria and its supporting checklist.

RED-A is designed to support iteration and flexibility. The approach comprises three stages. First, a scan to ensure that complexity is adequately conceptualised and managed. Then the same process is repeated for uncertainty. Finally, the scan is conducted for ambiguity. To undertake the scan, the DRM scientist or practitioner is guided by 12 criteria and a 22-item checklist.

After designing RED-A, we applied RED-A to the Solotvyno municipality land subsidence risk case study. The study confirmed that even though the land subsidence risk is a systemic risk, scientist reduced its complexity at the risk appraisal stage and represented it by maps. They barely addressed uncertainty and ambiguity in the risk appraisal model. Based on the results, we noted a bias toward risk-based strategies and less on precaution, resilience and discourse-based strategies. There was no attempt for resilience and discourse-based approaches with limited adoption of precaution-based criteria.

The results also identified three needs. First, risk experts need to shift from simple cause-effect rules to generative predictive rules to address complex risks. Second, the need to shift from a single measurable dimension of uncertainty to irreducible multiple dimensions, also termed as embodied uncertainty. Last, practitioners need to shift from single interpretations to variability of values, opinions, judgments, beliefs, perceptions and objectives.

The suggested changes call for the adoption of a social—technical DRM approach and heterogeneous strategies with different modes of stakeholder participation. When faced with complex-based risk categories, we recommend the adoption of robustness risk-based strategies (to generate more options and scenarios) and epistemological discourse (to resolve cognitive conflicts). For uncertain risk categories, we recommend precaution as well as a resilience-based and reflective discourse. Precaution and resilience-based strategies enhance the speed and strength of disaster response. Reflective discourse strategies support consensus building on safety margins, risk acceptance levels and risk avoidance actions. With ambiguous risk categories, we propose discourse-based strategies (to increase social acceptance) and participative discourses (to discuss conflicting values and judgments and agree to a socially acceptable decision).

The goal of the 12 RED-A criteria and the 22-item checklist is to support the progressive transition towards a heterogeneous social-technical DRM approach. RED-A promotes shifts from reducing uncertainty, complexity and ambiguity towards accepting and embracing these challenges. RED-A is the basis for the development of risk evaluation tools that embody uncertainty. RED-A helps scientists and decision-makers to recognise their unconscious biases. RED-A embodies uncertainty by intentionally making what seemed uncertain and intangible–visible, measurable, tangible and actionable. Finally, the application of RED-A opens numerous opportunities for transdisciplinary research where social, political, natural and technological scientists can work together to design solutions that enhance societal resilience.