Development and Simulation of Cyberdisaster Situation Awareness Models

Abstract

Cyberdisasters require an organization’s disaster team to be prepared. Disaster events are difficult to predict, but the impact of this risk on an organization is large. However, organizations sometimes struggle in being prepared for disaster situations. Here, awareness of disaster situations when analysing priority disasters (e.g., earthquakes and pandemics) and how to mitigate them can help an organization’s preparedness. Mitigation scenarios need to be determined and simulated so that a disaster team is ready to face disaster. Using Endsley’s situational awareness model and a tabletop exercise, this study aimed to help a disaster team determine cyberdisaster risk priority and assess a team’s preparedness for dealing with a cyberdisaster. The situation awareness model was divided into two stages: awareness of cyberdisaster situations and tabletop evaluations. Awareness of a disaster situation was carried out by determining the highest priority for disaster risk using the fuzzy failure modes and effects analysis (FMEA) method. The results of the first study show that the high-risk category contains ransomware attacks during pandemics and earthquakes. The second study performed a tabletop simulation questionnaire survey of earthquakes and ransomware attacks during a pandemic for several disaster teams with 152 respondents. The results of the survey evaluation of the earthquakes and ransomware attacks simulation survey show that the effect factors of cyberdisaster simulation decisions are 95% system capability (p < 0.05), 90% knowledge (p < 0.05), and 90% awareness of a disaster situation (p < 0.05); these factors show the effect of a disaster team’s decision during a tabletop simulation. The novelty of this research lies in building a model for how an organizational process determines the priority of a cyberdisaster tabletop simulation and the factors that contribute to increasing a disaster team’s awareness in dealing with cyberattacks.

1. Introduction

Because of numerous countries’ policies for preventing COVID-19 transmission, many organizations have been carrying out teleworking activities during the pandemic. Teleworking is the use of information and communication technology, such as smartphones, tablets, laptops or desktop computers, for work outside an organization [1]; teleworking can be performed remotely at home or in a public place with facilities.

Teleworking can affect internet usage, hence amplifying the threat of cyberattacks. Indonesia is a country with the fourth rank of internet users in the world [2]. According to the Indonesian National Cyber and Crypto Agency, Indonesia ranks second in the number of cyberattacks when compared with other countries. In 2020, the number of cyberattacks in Indonesia increased 2.7 times compared with 2019, and the most common cyberattacks are phishing, ransomware and malware [3].

The conditions during the COVID-19 pandemic disaster have generated the threat of organizational cyberattacks, which can cause problems such as the disruption of electronic services because of viruses, data theft, disruption of access to electronic services and theft of an organization’s intellectual property. Cyberattacks can also damage an organization’s reputation, causing financial losses and service disruptions.

Changes from normal to catastrophic conditions require organizations to take precautions and ensure that the risk from cyberattacks and vulnerabilities is effectively controlled. Disasters are rare events that have extreme effects on organizations. Organizational readiness to face disasters, especially during the COVID-19 pandemic, can be problematic, because people’s disaster preparedness tends to be low since many activities are carried out by teleworking. As a solution, the current paper proposes a new situation awareness model concept in cyberdisaster management, from determining disaster risk using the fuzzy failure modes and effects analysis (FMEA) method to evaluating cyberdisaster situation simulations using a tabletop method.

According to the World Health Organization (WHO), a disaster is an event that disrupts normal conditions and causes a level of suffering that exceeds the adaptive capacity of those communities affected by the disaster [4]. Based on this definition, an information technology disaster disrupts critical business processes and services, reducing service levels to where the financial and operational impacts on an affected organization become unacceptable as it could result in disruption or interruption of business services. Some threats or hazards that result in disasters are natural (e.g., floods, pandemics, fires and earthquakes), human (e.g., terrorism, cyberattacks and riots) and/or technological (e.g., transportation accidents, infrastructure accidents and information system outages) [5].

Disaster management is a holistic strategy for reducing disaster risk, and disaster preparedness, response and recovery can together reduce the impacts of disasters. The stages of a disaster management process are predisaster (preventive action) and postdisaster (corrective action). Predisaster actions involve mitigation and preparedness, whereas postdisaster actions focus on response and recovery [6]. Disaster risks threaten organizations; hence, organizations need to implement effective risk management systems to ensure business continuity [7]. Therefore, a model that can increase the awareness of cyberattack risks and serve as a basis for decision makers to ensure business continuity needs to be developed.

One preparedness and mitigation action lies in disaster risk analysis and emergency simulation testing. An emergency simulation aims to evaluate the effectiveness of equipment, people and systems according to those policies for dealing with cyberdisaster situations. Postdisaster actions, for example, involve backing up data, gaining access to damaged facilities, restoring system data and/or operating alternative equipment and systems successfully [8].

Several studies exist on the awareness of disaster situations, such as research that has examined the combined sociodemographic data and geotagged Twitter data to understand disaster situation awareness from a social justice perspective [9]. Another study examined disaster situation awareness through social media text classification in real time by comparing the support vector machine and logistic regression methods [10]. The present research focuses on using situation awareness in understanding disasters through social media.

Another study examined a pragmatic approach by analysing disaster risk management insights on school emergency preparedness in Khyber Pakhtunkhwa Province, Pakistan [11]. One study discussed disaster risk reduction education programmes as an important part of increasing awareness of the factors driving disaster risk. In addition, longitudinal studies have been conducted to compare the perceptions of groups of students over time regarding disaster risk [12]. In this context, the current research analyses the perceptions of disaster risk management.

Several studies related to disaster simulation and situation awareness simulation have focused on varying areas, such as on learning emergency response preparedness using game design [13], use of virtual web-based tabletop exercises [14], simulations using the Situation Awareness Global Assessment Technique [15] and simulations using the Situation Awareness Rating Technique [16]. The present research focuses on simulation methods for disaster preparedness and the simulation assessment of situation awareness.

The state of the art in this line of research is to combine the above research, namely, the use of a situation awareness model as a conceptual framework for the process from a disaster risk assessment to cyberdisaster simulation assessment. Cyberdisaster risk assessment uses the fuzzy FMEA method, because this method can help the organization in determining disaster risk priorities. With this model, it is hoped that the organization’s business continuity will be more resilient in the face of cyberdisasters.

Cyberdisaster simulation activity is an ongoing organizational plan for serving secure IT to customers. Here, organization sustainability is an organization’s ability to survive and be competitive in the face of economic, social, environmental, ethical and technological impacts both now and in the future [17]. Customer needs require organizations to take sustainability actions [18]. One of the objectives of the current research is to develop a concept of a cyberdisaster simulation model that can be used by organizations to determine simulation priorities based on the risk value of the impact of cyberdisasters.

Based on the literature, the following problems are faced when conducting disaster simulations:

• The need to develop an effective method of testing a team’s level of awareness regarding disaster response;
• Insufficient data and inaccurate calculation of risk impact values for determining priority actions to deal with disaster risk;
• The conditions of the COVID-19 pandemic disaster resulting in changes in carrying out disaster simulations.

To address the above problems, the current paper combines several research methods (i.e., aspects of situational awareness and cybersecurity disaster risks). This cybersecurity disaster awareness risk analysis uses the fuzzy FMEA method, and the results can determine the priority of disaster preparedness. In the current study, action scenarios for managing the highest disaster risks were determined, and simulations were conducted using the online tabletop method because of the COVID-19 pandemic. This simulation is focused on testing the readiness of the organization’s cyberdisaster team in Indonesia.

We propose a situation awareness model to assist the tabletop simulation process in responding to cyberdisaster management. The main contributions of the current paper are as follows:

• Provide a situational awareness framework model to be used for tabletop simulation in facing cyberdisaster preparedness if an earthquake occurs and pandemic conditions with ransomware attacks were to occur;
• Introduce a concept of how to prioritise cyberdisaster risk values using the FMEA fuzzy method;
• Assist organizations in planning business continuity in the face of cyberdisasters to improve organizational sustainability.

The current paper is structured as follows: Section 2 presents the theoretical framework; Section 3 presents the methods for implementing the fuzzy FMEA and tabletop simulation; Section 4 presents the evaluation data from a tabletop survey; and Section 5 provides a brief discussion and conclusion.

2. Theoretical Framework

2.1. Risk of Cyber Situation Awareness

Situational awareness in cyber environments refers to the ability to perceive and understand the meaning of risk and predicting a future status to support decisions for dealing with cybersecurity conditions. To ensure that awareness of cyber conditions is comprehensive, other information besides knowledge of the cyber world is required [19].

The cyber situation awareness model provides a holistic approach to understanding cybersecurity threats and weaknesses and for projecting potential future cybersecurity performance [20]. Cyber situation awareness can be divided into three levels: environmental awareness, risk awareness and decision-maker awareness [21]. Cybersecurity risk awareness can use a network security cybersecurity risk assessment framework [22], and situation risk assessment cybersecurity awareness is a part of information security, which is necessary to overcome the lack of risk management assessment leading to poor and inappropriate information security decisions [23]. The novelty of the current research is in providing a risk assessment solution using fuzzy FMEA to evaluate the risk of cyber situation awareness in disaster conditions, which can make the assessment more objective and accurate.

The research basis for situational awareness is influenced by Endsley’s model [24], as shown in Figure 1.

In Endsley’s situational awareness model, decisions are generally influenced by the following:

• Situation awareness factors;
• Information-processing mechanisms based on one’s ability, experience and training (knowledge); and
• System capabilities, interface design, workload, stress and complexity (system capabilities).

In this study, the Endsley situation awareness model is used as a framework model for determining cyberdisaster risk and evaluating cyberdisaster simulations. According to ISO 31000:2018, risk is the effect of uncertainty on objectives. Risk levels can also be categorized as low risk, medium risk, high risk and catastrophic risk. Based on this definition, the risks associated with this research were the impact of cyberattacks because of uncertainty regarding the possibility of a catastrophic event; to analyse cyberdisaster risk quantitatively, by conducting a risk assessment that refers to the ISO 27005 standard. ISO 27005 involves determining the context of information security and carrying out risk identification, risk analysis and risk evaluation [25], as shown in Figure 2.

Contextual determination is performed by examining objectives, scopes, boundaries and risk criteria. A risk identification process identifies the sources or causes of risk, and risk analysis is a process for determining the likelihood of an event, estimating its impact, identifying risk controls and determining risk ratings. The effectiveness of risk controls can change the inherent risks into residual risks. Risk evaluation is used to determine the priorities for dealing with existing risks. Risk assessment techniques can be based on ISO 31010 and use either quantitative or qualitative methods. The results of a risk assessment are employed to mitigate risks until they reach acceptable risk values. The types of risk treatment include avoiding and reducing risks, eliminating their sources, changing their impacts, assessing various risks for other parties and accepting risks with certain considerations.

2.2. Fuzzy FMEA

FMEA is a reliable and systematic technique for identifying potential failure modes, causes of failure and the impact of a failure of a system or process. FMEA can reduce or eliminate the likelihood of failures and is used in various industries, such as the automotive, construction, aerospace, electrical, computer, safety and maintenance industries.

Some benefits of FMEA fuzzy risk assessment in assessing the effectiveness of corrective actions can be applied in industry [26], health [27] and information systems [28]. The calculation of the FMEA fuzzy risk priority number (RPN) is more accurate and objective than the conventional FMEA RPN calculation method [29].

The calculation of risk assessment in fuzzy FMEA can be addressed using the following equation:

RPN = S × L × D

(1)

According to FMEA, risk priority values can be calculated by multiplying three factors: the impact of failure (S), the possibility of failure (L) and the ease of detecting failure (D) [30], which can be formulated as Equation (1).

Based on several traditional FMEA studies, a scale was developed to measure the levels of impact, likelihood and detection using the number range of 1–10 [28], as shown in

Table 1. Impact scale values. Adapted from Ref. [28].

Rating	Description	Definition
1	No danger	Failure causes no injury and has no impact on the system
2	Slight danger	Failure could cause no injury and the customer is unaware of the problem; however, the potential for minor injury exists. There is little or no effect on the system
3 4	Low to moderate danger	Failure could cause a very minor or no injury but annoys customers and/or results in minor system problems that can be overcome with minor modifications to the system or process
5	Moderate danger	Failure could cause a minor injury with some customer dissatisfaction and/or major system problems
6 7	Dangerous	Failure could cause a minor to moderate injury with a high degree of customer dissatisfaction and/or major system problems requiring major repairs or significant re-work
8 9	Very dangerous	Failure could cause a major or permanent injury and/or serious system disruption with interruption in service, with prior warning
10	Extremely dangerous	Failure could cause the death of a customer (patient, visitor, employee, staff member, business partner) and/or total breakdown, without any prior warning

,

Table 2. Likelihood of occurrence scale values. Adapted from Ref. [28].

Rating	Description	Potential Failure Rate
1	Remote probability of occurrence	Failure almost never occurs; no one remembers the last failure
2	Low probability of occurrence	Failure occurs rarely, or failure occurs about once per year
3 4	Moderate probability of occurrence	Failure occurs occasionally, or failure occurs once every 3 months
5 6	Moderately high probability of occurrence	Failure occurs approximately once per month
7 8	Very high probability of occurrence	Failure occurs frequently, or failure occurs about once per week
9	Failure is almost inevitable	Failure occurs predictably, or failure occurs every 3–4 days
10	Certain probability of occurrence	Failure occurs at least once a day, or failure occurs almost every time

and

Table 3. Detectability scale values. Adapted from Ref. [28].

Rating	Description	Definition
1	Almost certain chance of detection	There are automatic “shut-offs” or constraints that prevent failure
2	Very high chance of detection	There is 100% inspection of the process, but it is automated
3 4	High chance of detection	There is 100% inspection or review of the process, but it is not automated
5	Moderate chance of detection	There is a process for double-checks or inspections, but it is not automated and/or is applied only to a sample and/or relies on vigilance
6 7	Remote chance of detection	The error can be detected with a manual inspection, but no process is in place, so that detection left to chance.
8 9	Very remote/unreliable chance of detection	The failure can be detected only with a through inspection, and this is not feasible or cannot be readily performed
10	No chance of detection	There is no known mechanism for detection the failure

.

The impact value scale ranges from 1 (a harmless risk impact) to 10 (an extremely dangerous risk that can disrupt systems).

As shown Table 2, The likelihood scale values range from 1 (the possibility of an event rarely or never happening) to 10 (the probability that it is likely to occur at any time).

As shown in Table 3, the values for the ease of detection scale ranged from 1 (a failure that is almost certainly easily detectable) to 10 (a failure with zero probability of being detected).

The technical stages of the risk priority assessment are shown in the flowchart in Figure 3.

From the FMEA flowchart process above, the RPN value will be generated for deciding on action decisions to overcome the highest risks.

Fuzzy logic provides a basis for decision making by approximating the degree of membership in a certain set. Fuzzy member values range from 0 to 1, which is in contrast to binary logic, which has values of only 0 or 1. Fuzzy set A is the degree of membership of element x in the interval [0, 1] and can be stated as follows [30]:

A:X→[0, 1]

(2)

One form of fuzzy membership degree is a triangle. The value of the fuzzy triangle members (a, b and c) and the µ_A (x) membership function can be expressed as follows:

$$\left(x:a,b,c\right)=\left\{\begin{array}{c}0 if x \le a,\\ \frac{\left(x-a\right)}{\left(b-a\right)} if a \le x\le b,\\ \frac{\left(c-x\right)}{\left(c-b\right)} if b \le x\le c,\\ 0 if x\ge c\end{array}\right.$$

(3)

The stages of a Mamdani fuzzy outage are as follows:

• Fuzzification;
• Establishment of a fuzzy knowledge base (fuzzy rule);
• Implementation of the implication function using a minimum and inter-rule composition using the maximum; and
• Defuzzification using a centroid method with the following equation:

$$\mathrm{COG}=\frac{\int {\mu }_A\left(x\right).xdx}{\int {\mu }_A\left(x\right)dx}$$

(4)

From the FMEA algorithm and fuzzy rules above, the cyberdisaster FMEA fuzzy flow is shown in Figure 4 [31].

2.3. Disaster Simulation

In principle, simulation training aims to demonstrate and develop individual competencies to face an emergency [32]. Team skills are needed in dealing with emergency conditions because of high risk, time pressure and complexity; here, a model is needed that can simulate personal responses during emergencies [33]. Developing an integrated simulation involving related departments can increase knowledge and skills in a complex and stressful environment [34]. Besides personnel’s ability and awareness, a simulation can describe the improvement of an emergency response system’s resilience if a disaster were to occur [35].

In the current study, we examined the effects of decision level on three simulation result factors: personnel knowledge, situation awareness and the ability of a resilience system in a disaster situation.

A tabletop exercise is a discussion-based simulation exercise in which the participants meet in a class or group setting to discuss their roles during an emergency and their responses to a particular emergency; this is one way that a simulation can prepare a team for a disaster [36,37]. A tabletop test exercise tool can save costs [38] and evaluate responses through discussion without involving the support of communication equipment [39].

In the present study, a tabletop exercise was used to test the awareness of a disaster team in dealing with cyberattacks and disaster conditions. In the context of the current study, a tabletop exercise was a discussion session between users dealing with certain issues, such as pandemic disasters, ransomware or earthquakes in Indonesia. The discussion session comprised members discussing their respective roles in raising awareness of risk management for dealing with cybersecurity incidents and emergencies.

Cybersecurity incidents and disasters can disrupt and paralyse service processes, either partially or completely, resulting in organizational losses. Understanding risk awareness and the cooperation of various parties is important for solving problems quickly and precisely so that the losses suffered by an organization can be minimised. Each involved party needs to know the tasks and structured communication flows for handling cyberattack incidents, respond appropriately and ensure effective disaster recovery.

As a stage of managing cybersecurity disasters, a simulation can refer to the ISO 22301:2019 business continuity management system. The business continuity management system is a standard management system that refers to ISO; this management system was developed for business continuity in the face of acceptable or unacceptable disturbances in an organization. The operational requirements include the stages of the disaster management process that are used to prevent a business from experiencing disasters or disruptions. The stages according to ISO 22301:2019 are shown in

Table 4. Operational requirements for business continuity ISO 22301:2019.

No.	Requirements	Description of ISO 22301:2019
1	8.2.	Business impact assessment and risk assessment
2	8.3.	Determination of business continuity strategies and solutions
3	8.4.	Business continuity planning
4	8.5.	Exercise program
5	8.6.	Document evaluation and business continuity capabilities

[40].

The stages of the ISO 22301:2019 operational process are the basis for current research, namely, developing an awareness model of disaster risk situations and disaster management simulation. The development of this model is new to the field.

2.4. Validation Process

Validation can be defined as the ability of an instrument to measure the level of accuracy of a particular measurement [41]. A good instrument has high validity, and for the current study, an instrument being tested is the result of input from a disaster team. The validation process for the questionnaire is shown in Figure 5 [42].

A validation study with a point-biserial formula was conducted by Yigal and Tamar, who explained how to use the point-biserial formula to calculate the validation value of a multiple-choice questionnaire [43]. In the current study, the validation test of the survey results as a measurement instrument was conducted using the following point-biserial correlation coefficient equation:

$$P{B}_c=\frac{Mc-M}{S}\sqrt{\frac{Pc}{1-Pc}}$$

(5)

where

• PB_c = point-biserial correlation coefficient;
• Mc = average value calculated for true items;
• M = average value of the total value;
• S = standard deviation of the total value; and
• Pc = proportion of respondents who answered and agreed to the items being tested for item validation.

The next stage of our research is the reliability test. Several studies related to reliability have been conducted using the Cronbach’s alpha method via a questionnaire to assess the internal consistency. A good Cronbach’s alpha acceptance value is above 0.70 [44]. The reliability testing method using Cronbach’s alpha can be expressed as follows [45]:

$$\alpha =\frac{K}{K-1}\left[1-\frac{{{\displaystyle \sum }}_{i=1}^k{\sigma }_i^2 }{{\sigma }_T^2}\right]$$

(6)

where

• $\alpha$ = Cronbach’s alpha reliability coefficient;
• K = number of questions;
• ${\sigma }_i^2$ = variance of the score for each question item;
• ${\sigma }_T^2$ = total score variants.

In testing the validation of the present research, we have used data point-biserial. For reliability testing, Cronbach’s alpha was used because this study used a multiple-choice questionnaire with a dichotomous value score (i.e., yes/no questions).

3. Materials and Methods

In the current paper, the process of determining disaster stages refers to the operational section of the business continuity management system ISO 22301:2019. The relationship between the stages of a cyberdisaster recovery planning process and operational requirements of the business continuity management system ISO 22301:2019 is described in

Table 5. Correlation between the clauses of ISO 22301:2019 and process of the cyberdisaster recovery planning stages.

No.	Requirements	Description of ISO 22301:2019	Cyber Disaster Planning Process
1	8.2.	Business impact assessment and risk assessment	Risk Analysis
2	8.3.	Determination of business continuity strategies and solutions	Disaster management scenario
3	8.4.	Business continuity planning	Determination of Priority for Disaster Management
4	8.5.	Exercise program	Simulation Program
5	8.6.	Document evaluation and business continuity capabilities	Evaluation of simulation implementation

.

Based on the above table, the following are the stages of the cyberdisaster recovery planning process:

• Cyberdisaster risk analysis using the FMEA fuzzy method;
• Disaster management scenario;
• Determining disaster management priorities;
• A simulation programme using the tabletop method; and
• Evaluation of the implementation of a simulation using a survey.

These stages form the basis for developing a cyberdisaster situation awareness model. The purpose of this research model is to increase disaster teams’ awareness in dealing with cyberdisasters so that the impact of cyberdisaster risks can be reduced. In this study, the fuzzy FMEA method was used to improve the accuracy of the risk assessment and disaster risk prioritisation so that it is easier to determine decisions for dealing with cyberdisaster risk.

In this study, we conducted a questionnaire survey among respondents after simulating a cyberdisaster using the tabletop method. The measurement of a tabletop simulation survey results can validate the influence of certain factors, such as the ability of organizational systems, a disaster team’s knowledge and awareness of a disaster situation assessment, to increase a disaster team’s awareness in making effective decisions when a cyberdisaster occurs.

The model developed in the current study—the cyberdisaster situation awareness model with the fuzzy FMEA method for disaster tabletop simulation during the COVID-19 pandemic—is shown in Figure 6.

The cyberdisaster situation awareness model using the fuzzy FMEA method for tabletop simulation is divided into two parts: the development of cyberdisaster risk awareness using the fuzzy FMEA method and determining mitigation scenarios and cyberdisaster tabletop simulations.

3.1. Development of Cyberdisaster Risk Awareness

Developing cyberdisaster risk awareness involves identifying the risk of critical assets against cyberdisasters, analysing the vulnerability and threat of cyberattacks to cyberdisaster risks as a whole, analysing the impact of cyberdisaster risk and determining the RPN. In a disaster recovery plan’s responsibility structure, the disaster recovery risk team is responsible for risk assessment.

At the stage of determining the value of cyberdisaster risk, it is necessary to define a rating scale of possible events, the impacts of these events and their ease of detection. The stages for determining the RPN refer to the fuzzy FMEA algorithm using MATLAB assistance, as follows:

• Table 6. Performance levels S, L and D fuzzy membership.

  Fuzzy Membership Degree Value Level 3 Performance Factors(S,L,D)	Performances Level
  [0, 0, 1.25]	Very Low (VL)
  [0, 1.25, 2.5]	Low (L)
  [1.25, 2.5, 3.75]	Medium (M)
  [2.5, 3.75, 5]	High (H)
  [3.75, 5, 6.25]	Very High (VH)

  presents a definition of a rating scale of likelihood (L), impact (S), ease of detection (D) and risk value and a design of fuzzy membership degrees for likelihood, impact and ease of detection.

Based on the performance table above, there are five levels of performance which are categorized as follows: very high, high, medium, low, and very low.

The RPN value on the fuzzy membership degree scale is shown in

Table 7. Description of performance levels S, L and D fuzzy membership.

Description
Performances Level	Severity	Likelihood	Detection
Very Low (VL)	The threat event could be expected to have a negligible adverse effect on organizational operations, organizational assets, individuals other organizations, or the Nation.	Adversary is highly unlikely to initiate the threat event. Error, accident, or act of nature is highly unlikely to occur; or occurs less than once every 10 years. If the threat event is initiated or occurs, it is highly unlikely to have adverse impacts.	It is very certain to be able to detect attacks and cyber security vulnerabilities
Low (L)	The threat event could be expected to have a limited adverse effect on organizational operations, organizational assets, individuals other organizations, or the Nation. A limited adverse effect means that, for example, the threat event might: (i) cause a degradation in mission capability to an extent and duration that the organization is able to perform its primary functions, but the effectiveness of the functions is noticeably reduced; (ii) result in minor damage to organizational assets; (iii) result in minor financial loss; or (iv) result in minor harm to individuals.	Adversary is unlikely to initiate the threat event. Error, accident, or act of nature is unlikely to occur; or occurs less than once a year, but more than once every 10 years. If the threat event is initiated or occurs, it is unlikely to have adverse impacts.	The possibility of an attack and cyber security can be detected
Medium (M)	The threat event could be expected to have a serious adverse effect on organizational operations, organizational assets, individuals other organizations, or the Nation. A serious adverse effect means that, for example, the threat event might: (i) cause a significant degradation in mission capability to an extent and duration that the organization is able to perform its primary functions, but the effectiveness of the functions is significantly reduced; (ii) result in significant damage to organizational assets; (iii) result in significant financial loss; or (iv) result in significant harm to individuals that does not involve loss of life or serious life-threatening injuries	Adversary is somewhat likely to initiate the treat event. Error, accident, or act of nature is somewhat likely to occur; or occurs between 1–10 times a year. If the threat event is initiated or occurs, it is somewhat likely to have adverse impacts	A moderate possibility of detecting attacks and cyber security vulnerabilities
High (H)	The threat event could be expected to have a severe or catastrophic adverse effect on organizational operations, organizational assets, individuals, other organizations, or the Nation. A severe or catastrophic adverse effect means that, for example, the threat event might: (i) cause a severe degradation in or loss of mission capability to an extent and duration that the organization is not able to perform one or more of its primary functions; (ii) result in major damage to organizational assets; (iii) result in major financial loss; or (iv) result in severe or catastrophic harm to individuals involving loss of life or serious life-threatening injuries	Adversary is highly likely to initiate the threat event. Error, accident, or act of nature is highly likely to occur; or occurs between 10–100 times a year. If the threat event is initiated or occurs, it is highly likely to have adverse impacts.	It is unlikely to be able to detect attacks and cyber security vulnerabilities
Very High (VH)	The threat event could be expected to have multiple severe or catastrophic adverse effects on organizational operations, organizational assets, individuals, other organizations, or the Nation.	Adversary is almost certain to initiate the threat event. Error, accident, or act of nature is almost certain to occur; or occurs more than 100 times a year. If the threat event is initiated or occurs, it is almost certain to have adverse impacts	Cannot detect potential attack failures and cyber security vulnerabilities

. The values of the risk scale refer to NIST 800–30 [46].

Based on

Table 8. Values of the risk scale.

Fuzzy Membership Degree Value Risk Level	Performances Level	Description
[93.75,125,156.3]	Very High Risk (VHR)	Very high risk means that a threat event could be expected to have multiple severe or catastrophic adverse effects on organizational operations, organizational assets, individuals, other organizations, or the Nation
[62.5, 93.75, 125]	High Risk (HR)	High risk means that a threat event could be expected to have a severe or catastrophic adverse effect on organizational operations, organizational assets, individuals, other organizations, or the Nation
[31.25, 62.5, 93.75]	Medium Risk (MR)	Moderate risk means that a threat event could be expected to have a serious adverse effect on organizational operations, organizational assets, individuals, other organizations, or the Nation.
[0, 31.25, 62.5]	Low Risk (LR)	Low risk means that a threat event could be expected to have a limited adverse effect on organizational operations, organizational assets, individuals, other organizations, or the Nation.
[0, 0, 31.25]	Very Low Risk(VLR)	Very low risk means that a threat event could be expected to have a negligible adverse effect on organizational operations, organizational assets, individuals, other organizations, or the Nation.

, the value of the risk scale can be classified into five levels: very high, high, medium, low and very low. The higher the risk value of the RPN, the higher the priority of corrective action being taken. This risk performance scale uses MATLAB, as shown in Figure 7.

As shown in Table 6, Table 7 and Table 8, the value of the risk RPN can be calculated as a result of multiplying the impact of the failure mode, the likelihood of an event occurring and the ease of its detection. The RPN with the fuzzy method can be calculated using MATLAB. The RPN is an output factor of FMEA, and the impact (S), likelihood (L) and detection (D) are the input factors of FMEA, as shown in Figure 8.

2.

The fuzzy FMEA rule is carried out by a team of experts or disaster specialists who share their knowledge. MATLAB can be a tool for calculating the RPN output factor value based on the input factor value for multiplying the factors of severity, likelihood and detection. An example of calculating fuzzy FMEA rules is shown in

Table 9. Example of calculating a fuzzy FMEA rule.

No.	Severity (S)	Likehood (L)	Detection (D)	RPN
1	VL	VL	VL	VLR
2	VL	L	L	VLR
3	VL	M	M	LR
4	VL	H	H	MR
5	VL	VH	VH	MR
6	L	VL	VL	VLR
7	L	L	L	LR
8	L	M	M	MR
9	L	H	H	MR
10	L	VH	VH	HR
11	M	VL	VL	VLR
12	M	L	L	LR
13	M	M	M	MR
14	M	H	H	HR
15	M	VH	VH	HR
16	H	VL	VL	VLR
17	H	L	L	LR
18	H	M	M	MR
19	H	H	H	HR
20	H	VH	VH	VHR
21	VH	VL	VL	LR
22	VH	L	L	MR
23	VH	M	M	HR
24	VH	H	H	HR
25	VH	VH	VH	VHR
26	VH	VL	L	VLR
27	VH	VL	M	VLR
28	VH	VL	H	VLR
29	VH	VL	VH	LR
30	VH	L	VL	VLR

.

From Table 9, the RPN can be assessed using the MATLAB IF-THEN fuzzy inference function (i.e., the AND calculation relationship).

In the current study, there were 125 calculation rules for risk assessment, including the three factors of likelihood (L), impact (D) and detection (D). The results of an example MATLAB simulation calculation are shown in Figure 9.

Figure 9 shows the MATLAB rule evaluation viewer, providing an example for an impact factor (severity) value of 2.63, a likelihood value of 2.58, a detection value of 2.39 and an RPN output value of 34.

3.

The next stage was to determine the RPN fuzzy FMEA assessment with cases of cyberdisasters during the COVID-19 pandemic in Indonesia. This RPN value was determined using the Delphi method. The results of the RPN fuzzy FMEA analysis are shown in Appendix A 

Table A1. Cyber risk assessment for disaster conditions using fuzzy FMEA.

Risk Number	Asset	Failure Potential		Impact of Failure (Confidentiality, Availability, Integrity)	Caused of Failure	Current Risk				Current Scenario Decisions
		Attack	Vulnerability			Level of Severity	Level of Likehood	Level of Detection	Fuzzy Risk Priority Number
R1	Data Center Building	Fire	The building does not have a fire protection system and the fire emergency team is unprepared	Data not available, data cannot be accessed	Failure in the selection or design of the data center building does not have a fire protection or handling system, the addition of another data center in a different location and the competence of the fire emergency handling team	H	M	L	MR	Developing fire management scenarios and simulations, Provision of fire safety systems such as fire extinguishers, provision of emergency telephone lists, emergency teams and data back up in a disaster recovery center
R2	Data Center Building	Flood	The building lacks a flood protection system and the flood emergency team is unprepared	Data not available, data cannot be accessed	Failure in the selection or design of the data center building does not have a flood handling system, the addition of another data center in a different location and the competence of the flood emergency handling team	H	M	M	MR	Developing flood management scenarios and simulations, providing drainage systems, providing emergency telephone lists, emergency teams and data backup in a disaster recovery center
R3	Data Center Building	Earthquake	The building does not have a disaster protection system and an earthquake emergency team is unprepared	Data not available, data cannot be accessed	failure in the selection or design of the data center building does not have an earthquake handling system, the addition of other data centers in different locations and the competence of the earthquake emergency handling team	H	M	H	HR	Developing scenarios and simulations for handling earthquakes, providing anti-earthquake systems, providing emergency telephone lists, emergency teams and backing up data in a disaster recovery center
R4	Data Center Room	Fire	The data center does not have a fire protection system and the fire emergency team is unprepared	Data not available, data cannot be accessed	The absence of a fire protection system and the provision of data center locations in different locations	H	M	L	MR	Developing fire management scenarios and simulations, Provision of fire safety systems such as fire extinguishers, provision of emergency telephone lists, emergency teams and data back up in a disaster recovery center
R5	Data Center Room	Flood	The data center does not have a flood protection system and the flood emergency team is unprepared	Data not available, data cannot be accessed	The absence of a good drainage system, and the provision of data centers in different locations that are flood free	H	M	M	MR	Developing flood management scenarios and simulations, providing drainage systems, providing emergency telephone lists, emergency teams and data backup in a disaster recovery center
R6	Human Resource	Infectious/pandemic diseases	Risk analysis and information security tools for access during teleworking are still inadequate, so there is potential for malware, ransomware, phishing attacks.	The absence of competent human resources to perform operations	There is no business continuity plan procedure related to pandemics or infectious diseases, and the provision of teleworking devices to prevent attacks	H	M	H	HR	Developing scenarios and simulations for handling pandemic conditions or infectious diseases and providing teleworking tools and mechanisms to prevent attacks from malware, ransomware, phishing

The example of an RPN assessment in Appendix A Table A1 shows that the high-risk categories are R3 (a pandemic ransomware attack) and R6 (an earthquake disaster threat). Hence, it is necessary to test the levels of cyberdisaster situational awareness regarding ransomware handling during disasters such as COVID-19 and earthquakes.

3.2. Tabletop Simulation

After determining the priority of disaster risk—namely, during an earthquake or a ransomware attack during a pandemic—simulations were conducted using the tabletop method. In the tabletop panel, the discussion is moderated by the facilitator and followed by users involved in the risk management process and decision making in handling cyberattacks and disaster threat management and recovery. The scenarios vary and the members discuss them to determine the actions to be taken in dealing with cyberdisasters. The decision of the disaster team in dealing with cyberdisaster risk will be tested for effectiveness based on the results of a questionnaire survey distributed to the team involved in the tabletop simulation.

Examples of tabletop scenarios related to the threat of earthquakes and ransomware attacks during the pandemic on data centre access are as follows:

• When a ransomware attack occurs on a user’s computer, it also attacks certain servers, resulting in blocked services and data access to the server that cannot be used. This ransomware attack needs to update the patch immediately. The scenarios carried out temporarily stop activities and inform disaster events, provide quarantine/lockdown on network areas infected with ransomware, disconnect internet connections for data centre servers, activate disaster recovery centres, check for updated data backups and divert service directions directly to data centres.
• If an earthquake occurs, the disaster team will first save itself. Then, it will activate the disaster recovery centre to run the service, check the updated data backup and then divert the direction of the service directly to the data centre.

After developing these scenarios, a simulation was performed using direct methods or online tabletop methods. To determine the extent to which the effectiveness of a simulation method can increase a team’s awareness in dealing with ransomware and earthquake disasters, a survey was conducted using a questionnaire.

Based on the situational awareness of the Endsley model in decision making, this questionnaire considered the factors affecting team awareness, such as team knowledge, existing ability systems and team mentality. This questionnaire involved 12 questions with the answer “yes” for a value of 1 and “no” for a value of 0. The results helped form the basis for deciding which simulation method would be effective in building team awareness to handle disaster situations based on predetermined scenarios.

The study sample was a disaster team of 152 people, and most respondents had between three and six years of experience carrying out disaster recovery plan activities. All 152 respondents stated that they had performed simulations using either direct methods in the field or in the form of a tabletop simulation.

The questionnaire survey process to evaluate the effectiveness of the simulation in increasing awareness of ransomware and earthquake disasters was carried out via validation testing, as shown in Figure 10. The validation process involves the following steps:

• Using the Delphi method to gather input from the cyberdisaster recovery team, we designed a questionnaire based on the factors that affect situational awareness.
• Based on the questionnaire, we validated the questionnaire results to determine measurement accuracy.
• We then performed a reliability process to determine the reliability and consistency of the questionnaire as a measuring tool.

Testing the validity of each questionnaire item was used as a biserial-point correlation because the score was 1 or 0.

4. Results

4.1. Development of a Survey Questionnaire for Simulation Evaluation

The earthquake and ransomware attack disaster scenarios during a pandemic were developed, and tabletop simulations were conducted using an online system. To determine the extent to which the effectiveness of the simulation method can increase team awareness in dealing with ransomware and earthquake disasters, a questionnaire survey was conducted. This research survey questionnaire uses yes and no questions (point-biserial) and consists of 12 question items. The survey was conducted among as many as 152 cyberdisaster team responders from several organizations in Indonesia. The questionnaire involved four variables and 12 questions, as shown in

Table 10. Development of a simulation questionnaire for disaster situation awareness.

No.	Variable	Question
1	Decision Making Level (Y)	1	During the simulation, the decision in the response team to handle or act on the threat of a ransomware or earthquake disaster was relatively fast in under 2 h)?
		2	Did the team make a decision to evaluate the simulation results of the ransomware and earthquake disasters?
		3	Did the team make any corrective action decisions after evaluating the simulated ransomware and earthquake disasters?
2	System Capabilities (X1)	4	Does the direct method simulation increase awareness and understanding more than the remote method?
		5	Is the team provided with infrastructure in handling or communication during the simulation?
		6	Does the team provide and perform a data backup facility to save data during disaster simulations?
3	Knowledge (X2)	7	Does the team have sufficient training on how to deal with an earthquake or ransomware disaster before the simulation is carried out?
		8	Are there any disaster management procedures, whether earthquake or ransomware?
		9	During the disaster simulation, the team responds to disaster management in accordance with earthquake handling procedures or ransomware attacks?
4	Disaster Situation Awareness (X3)	10	Does the response team care to understand what is being done when simulating the threat of an earthquake and ransomware?
		11	Are ransomware and earthquakes a threat to cybersecurity?
		12	Is the simulation able to increase the awareness of the team in responding to a disaster?

.

The following are the four questionnaire variables:

• Level of decision making: Decision-making levels were used to evaluate the purpose and extent of the disaster team’s effectiveness in dealing with cyberdisaster actions during a simulation. This included evaluating post simulation actions to determine whether the team had improved the simulation results. This was an output variable.
• System capabilities: These were related to effective simulation methods and infrastructure to support cyberdisaster management simulations. This was an input variable.
• Knowledge: This included training, experience and procedures. This was an input variable.
• Disaster situational awareness: An evaluation was carried out to assess the extent of the disaster team’s concern about the dangers of cyberdisasters, especially ransomware attacks during the COVID-19 pandemic and earthquakes. The participants had to determine whether ransomware was a threat and whether simulations could help improve responses to cyberdisasters. It was considered an input variable simulation.

Figure 11 indicates that the level of awareness in decision making is based on system capabilities, knowledge and awareness of disaster situations.

4.2. Validation and Reliability Test for the Survey Results

Appendix A 

Table A2. Validation test of the survey results.

No.	Variable	Question		Mc	M	S	Pc	1-Pc	PBc (Count)	PBc (Table)	Validation (PBc Count > PBc Table)
1	Decision Making Level (Y)	1	During the simulation, the decision in the response team to handle or act on the threat of a ransomware or earthquake disaster was relatively fast in under 2 h)?	11.58	10.93	1.96	0.86	0.14	0.82	0.304	Valid
		2	Did the team make a decision to evaluate the simulation results of the ransomware and earthquake disasters?	11.23	10.93	1.96	0.93	0.07	0.56	0.304	Valid
		3	Did the team make any corrective action decisions after evaluating the simulated ransomware and earthquake disasters?	11.35	10.93	1.96	0.88	0.12	0.59	0.304	Valid
2	System Capabilities (X1)	4	Does the direct method simulation increase awareness and understanding more than the remote method?	11.26	10.93	1.96	0.90	0.10	0.53	0.304	Valid
		5	Is the team provided with infrastructure in handling or communication during the simulation?	11.02	10.93	1.96	0.98	0.02	0.31	0.304	Valid
		6	Does the team provide and perform a data backup facility to save data during disaster simulations?	11.02	10.93	1.96	0.98	0.02	0.31	0.304	Valid
3	Knowledge (X2)	7	Does the team have sufficient training on how to deal with an earthquake or ransomware disaster before the simulation is carried out?	11.34	10.93	1.96	0.90	0.10	0.65	0.304	Valid
		8	Are there any disaster management procedures, whether earthquake or ransomware?	11.30	10.93	1.96	0.88	0.12	0.51	0.304	Valid
		9	During the disaster simulation, the team responds to disaster management in accordance with earthquake handling procedures or ransomware attacks?	11.34	10.93	1.96	0.90	0.10	0.65	0.304	Valid
4	Disaster Situation Awareness (X3)	10	Does the response team care to understand what is being done when simulating the threat of an earthquake and ransomware?	11.46	10.93	1.96	0.88	0.12	0.74	0.304	Valid
		11	Are ransomware and earthquakes a threat to cybersecurity?	11.38	10.93	1.96	0.88	0.12	0.63	0.304	Valid
		12	Is the simulation able to increase the awareness of the team in responding to a disaster?	11.13	10.93	1.96	0.95	0.05	0.45	0.304	Valid

presents the validation results obtained from the point-biserial correlation coefficient equation.

From Appendix A Table A2, it can be seen that the PBc value (table) is at a significance level of 5%, with a PBc coefficient value (table) of 0.304. The survey questionnaire distributed to respondents consisting of 12 question items was declared valid because, from the results of the calculation of the questionnaire data, the point-biserial correlation coefficient PBc (count) > PBc (table).

The next test conducted in the validation process was reliability testing, which was used to determine the consistency of the quality of the questionnaire measurement instrument. In the current study, Cronbach’s alpha was used to evaluate the internal consistency of the questionnaire items for a cyberdisaster awareness simulation with 152 respondents. Cronbach’s alpha is declared reliable if the reliability value is 0.75. The calculation results show that the questionnaire survey data are reliable because they range between 0.70 and 0.95.

The next step was to test the correlation between the input and output values of the survey results obtained from the cyberdisaster recovery simulation. We used logistic regression because the survey data are binary; hence, the significance value of the variables x1, x2 and x3 with variable y was obtained. A significance value of x1 (0.00), x2 (0.06), and x3 (0.06) above 0.05 means that variable x1 is the ability of the system, x2 is knowledge, and x3 is awareness of the disaster situation. Then, the value of the variables x1, x2 and x3 significantly affects the variable y (that is, awareness in decision making).

5. Conclusions and Future Lines of Research

5.1. Conclusions

The current paper has presented a process model for increasing the awareness of disaster teams for making decisions in response to cyberattack disasters, particularly ransomware attacks during pandemics and earthquakes. The process of increasing a disaster team’s awareness and assessment of disaster risks using a fuzzy FMEA method can determine which risk priorities need to be addressed first and can do so in an accurate and objective manner. The results indicate that ransomware attacks during earthquakes and or pandemics are a priority risk in terms of cyberdisaster threats with a high risk value.

The next process to increase the awareness of a disaster team is to determine which simulation methods are effective in responding to cyberdisaster situations, especially ransomware attacks during pandemics and earthquakes. The survey conducted with a sample of 152 respondents showed that system capability (p < 0.05), knowledge (p < 0.05) and awareness of a disaster situation (p < 0.05) significantly influence decision making on the tabletop simulation.

The main contribution of this research is to be able to contribute to the development of new models, namely, the situation awareness model which is applied in cyberdisaster simulations and the use of the fuzzy FMEA method used to determine cyberdisaster risk priorities. The result of determining the highest risk will be the main priority of the cyberdisaster simulation.

With this model, a cyberdisaster team can carry out a systematic analysis starting from the risk of cyberdisasters to evaluating the results of simulations of awareness of cyberdisaster situations, especially ransomware disasters during pandemics and earthquakes in Indonesia.

5.2. Future Research

The agenda for future research related to the development of a cyberdisaster simulation model for the sustainability of an organization needs to be mapped between cyberdisaster risk and situation awareness, namely, in the following ways:

• Simulation with a single cyberdisaster case on individual situation awareness;
• Simulation of multi-cyber-hazard cyberdisaster cases on system situation awareness; and
• Development of risk assessment methods in determining risk, such as machine learning methods, game theory, Markov chains and others.