Identification of Contributing Factors to Organizational Resilience in the Emergency Response Organization for Nuclear Power Plants

Abstract

Resilience engineering is a new approach to safety, focused on systems for coping with complexity and balancing productivity with safety. Since the early 2000s, several studies have been conducted on the application of resilience to various industries. However, the nuclear industry has yet to harness the full potential of the resilience concept. The International Atomic Energy Agency (IAEA) gave an inkling of the use of this concept in its report on the human and organizational factors related to the Fukushima nuclear power plant (NPP) accident. Although the ability of emergency response organizations (EROs) to reduce the radiation risks to the public in the case of accidents is crucial, no method has been developed so far to evaluate ERO resilience in NPPs. This paper aims to determine the factors that contribute to the resilience of EROs in NPPs. This work commenced by providing a systematic review of the literature on resilience factors as applied in several domains within the last two decades, including general domains, healthcare, transportation, infrastructure, process plants, and business. Based on the review, and the application of additional procedures like resilience analysis grid filtering, ERO applicability assessment, and merger/reclassification, the resilience factors are determined. Fifty-two factors contributing to the resilience of EROs in NPPs are proposed. The identified contributing factors are expected to aid efforts to develop resilience strategies and to measure the resilience of EROs in NPPs.

1. Introduction

Resilience engineering (RE) is a new approach to safety focused on systems for coping with complexity and balancing productivity with safety. RE aims to provide tools for addressing the proactive management of risk, inherent complexity of systems, and corresponding need(s) for performance variability [1]. Resilience can be defined as the ability of a system to adjust its functioning prior to, during, or following changes and disturbances so that it can sustain the required operations under both expected and unexpected conditions [1]. Resilience is considered a “Safety-II” concept focused on success and how things often go well, in contrast to a “Safety-I” concept focused on failures and analysis of their causes [1,2].

According to E. Hollnagel [1], D. Woods [3], and N. Leveson [4], resilience research has been conducted since the early 2000s. Resilience studies have been reported in the aviation [5], medical [6], space [7], disaster management [8], and nuclear domains. Some studies have been conducted to assess reliability based on the concept of resilience. For instance, A. V. Lee et al. [9] developed a survey tool for organizations in order to identify their strengths and weaknesses and to develop and evaluate the effectiveness of their resilience strategies and investments. A. Azadeh et al. [10] conducted research on a quantitative assessment of RE, based on nine resilience indicators in the process plant field. R. Burch [11] presented a method for the aerospace field for calculating resilience values to represent an expected value of the retained capability for a particular threat scenario (or over several threat scenarios). All of the aforementioned are not possible without a previous determination of the contributing factors to resilience in each specific case.

Reviews of the literature in relation to RE and its contributing factors have been conducted from several different perspectives. D. Van der Beek et al. [12] reviewed studies to identify evidence for the construct validity of E. Hollnagel’s four resilience abilities, based on team resilience literature for the business sector. M. Iflaifel et al. [13] conducted reviews of the literature on resilient health care (RHC), aiming to identify the research methods and tools commonly used to study RHC, and ultimately determined factors for enhancing RHC in the medical sector. E. Barasa et al. [14] performed a thematic review of 34 empirical studies in order to study the concept of resilience and its usefulness in the health care sector. The authors also recommend a further study to test the resilience features identified for the health sector. D. Kantur et al. [15] reviewed organizational resilience and its contributing factors, with the aim of developing a synthesized framework to improve the progress in both theoretical and empirical research on organizational resilience for strategic management. While there are many pieces of research that aim to identify the contributing factors to resilience via surveys of the literature, no such attempt has been directed toward the resilience of the emergency response organizations (EROs) in nuclear power plants (NPPs) until now.

Nuclear safety has the key objective of protecting people and the environment from ionizing radiation associated with the operation of nuclear installations [16], and it has developed over the years, albeit reactively in many cases. Lessons learned from early nuclear-related fatalities due to radiation excursions, such as those of Louis Slotin and Harry Daghlian in nuclear facilities [17,18], meant that safety became a prominent topic. Initially, safety concerns were mainly technical in nature and were addressed through the defense in depth (DID) strategy and deterministic safety assessments [19,20]. However, due to the accidents in the early 1970s, the classical probabilistic safety assessment (PSA) techniques were developed. More so, the Three-mile Island (TMI) NPP accident triggered the inclusion of human (operators and maintenance personnel) aspects into the safety concepts [20] and analysis methods. The IAEA report on the Chernobyl accident [21] drew the attention of the nuclear community to the influence of organization and management on the safety of NPPs. Most recently, the Fukushima NPP accident has brought numerous changes to the safety regulations in NPPs; most visible among them is the Diverse and Flexible Coping Strategies (FLEX) developed by the U.S. Nuclear Energy Institute (adopted by several countries with different names and or national conditions), which is an extension of the DID concept to external events, i.e., beyond-design basis external events [22]. Additionally, in light of the Fukushima Daiichi NPP accident, the International Atomic Energy Agency (IAEA) published a report on the human and organizational factors in nuclear safety [23]. The report stated that a new safety paradigm was needed for NPPs, as the current nuclear safety paradigms are based on compartmentalized and linear approaches aimed at identifying and correcting each weakness separately, i.e., PSAs and deterministic safety assessments. The report argued that the safety paradigm should shift to resilience concepts, including those for systems, structures, and organizations.

In the nuclear domain, J. Park et al. [2], J. Kim et al. [24], and F. Kamanja et al. [19] studied the safety II concepts by using the emergency operating system (EOS) developed by Électricité de France (EDF) to apply resilience concepts. However, those studies have two major limitations: (1) the EDF factors are limited and do not give all possible factors that can affect the resilience of emergency organizations; (2) the studies use the OPIS database to formulate resilience models. However, the OPIS reports cannot reveal all resilience-related problems, as they are retrospective and only focus on specific failures, i.e., reactor trips. Their focus on failures means that they do not inherently conform to the resilience concept, which looks at both failures and successes. The focus on reactor trips and related events also means the resilience factors that can be deciphered from these reports are limited. Although the studies are related to the safety II concept in NPPs, none of the previous studies are specific to the resilience of EROs in NPPs. Thus, evaluation factors and methods for clearly assessing the resilience of NPP EROs have not been developed.

This study aims to identify contributing factors for resilience evaluation as part of the development process of an ERO resilience evaluation method for NPPs. Accordingly, this study conducted a systematic review of the literature on resilience as applied in several industries, aiming to identify all the factors contributing to the resilience of EROs in NPPs. The remainder of this paper is organized as follows. Section 2 provides a review of the literature on resilience application cases. Section 3 introduces the EROs and the four capabilities required for resilience as proposed by Hollnagel. Section 4 describes a classification process for the contributing factors derived from the results of the review of the literature. Section 5 proposes the contributing factors for ERO resilience in NPPs. Section 6 discusses the results and proposes steps for future research. Finally, Section 7 is the concluding statement.

2. Review of Resilience in Several Domains

The publications used in this study were selected from Google, ScienceDirect, and Google Scholar. In order to properly review the appropriate literature within each website, the search was conducted with the following keywords: “Resilience,” “Resilience Analysis Grid,” “High Reliability Organization,” “Organizational Resilience,” and “Resilience Evaluation.” The inclusion criteria for papers to be considered in this review were as follows: (1) the paper was published in English or Korean; (2) the paper indicated the resilience contributing factors with corresponding descriptions; and (3) the paper focused on organizational resilience, rather than on individuals.

After reading the abstracts and overall structures of the retrieved studies, irrelevant studies were excluded from the review, so as to give the reader a broad yet focused context of resilience in safety-critical fields, and to ultimately identify and define the factors contributing to organizational resilience (i.e., “contributing factors”). Approximately 166 studies were initially retrieved for this review. However, after considering the purposes of the papers, their definitions of resilience, and the contributing factors to resilience mentioned therein, only 63 papers were fully reviewed for this study.

The fully reviewed papers applied to several fields, but were grouped into larger related fields. A total of seven categories were created, and brief reviews thereof are presented in the following subsections.

2.1. Research on Resilience in the General Domain

The “general” domain can be classified as general methodology studies or studies applicable to numerous fields. A total of 13 papers were reviewed for this domain. In this context, E. Hollnagel [1,3] proposed four capabilities of RE required for resilient performance. A total of 38 evaluation factors were presented. Salisu et al. [25] reviewed four methods used to evaluate regulations (the Resilience Scale for Results, Connor–Davidson Resilience Scale, Connor–Davidson Resilience Scale 10, and Brief Resilience Scale). Righi et al. [26] conducted a review of the literature with the aim of presenting a research agenda for RE. Kantur et al. [15] developed a framework for enabling a better understanding of the concepts and relationships in organizational resilience-level analyses. Pillay [27] developed a framework for future research in RE as a safety-management strategy. Nemeth et al. [28] provided scientific and industrial communities with an opportunity to present current RE work diagrams, and to conduct research to identify the progress and contributions in RE research and applications. The study presented inspiring questions regarding the future of RE. Lee et al. [9] developed a survey to enable organizations to understand their strengths and weaknesses, and to develop and evaluate the effectiveness of their resilience strategies and investments. Amir et al. [29] conducted research regarding the understanding of organizational resilience and reinforcement of regulations through a review of growth-based compositions and measurements. Kim [30] studied how the effects of family resilience support the successful adaptation of remarried families. McManus et al. [31] identified the key contributors to organizational resilience with case studies from a wide range of industry sectors in New Zealand, and suggested ways to strengthen them. Mendonca et al. [32] proposed experience-based criticism and contributing factors to supplement existing contribution factors, based on the 2011 New York power loss incident. Salanova et al. [33] conceptualized and evaluated the “Health & Resilient Organization” model.

Table 1. Summary of identified contributing factors from the general domain.

Reference	Main Topic	Contributing Factors
		No. of Factors	Examples
E. Hollnagel et al. [1,3]	Capabilities of RE for resilient performance.	38	Respond, Monitor, Anticipate, Learn, Event List, Background, Limitation, Response List, Duration, and others.
I. Salisu et al. [25]	Review four methods used to evaluate regulations.	70	Respond, Monitor, Anticipate, Learn, Event List, Verification, Delay, Communication, Culture, Strategy, and so on.
A. Righi et al. [26]	A review of the literature with the aim of presenting a research agenda for RE.	28	Self-esteem, Self-efficacy, Self-looking, Hope, Determination, Extraversion, Social Adeptness, Cheerful mood, Ability to Initiate Activities, and so on.
D. Kantur et al. [15]	Developing a framework for enabling a better understanding of the concepts and relationships in organizational resilience.	25	Perceptual Stance, Contextual Integrity, Strategic Capacity, Strategic Acting, Robustness, Redundancy, Resourcefulness, Rapidity, Recovery, Renewal, and so on.
M. Pillay [27]	Developing a framework for future research in RE as a safety-management strategy.	6	Top-level Commitment, Awareness, Learning Culture, Just Culture, Flexibility, and Preparedness.
C. Nemeth et al. [28]	Providing scientific and industrial communities with an opportunity to present current RE work diagrams, and to identify the progress and contributions in RE research and applications	6	New Ways to Organize, Coping with Reality, Firsthand Witness, Placing RE in Context, Managing Within Limits, and Understanding as it is Done.
A. Lee et al. [9]	Developing a survey to enable organizations understand their strengths and weaknesses, and develop and evaluate the effectiveness of their resilience strategies and investments.	42	Devolved and Responsive Decision-making, Commitment to Resilience, Network Perspective, Minimization of Silos, Internal Resources, Staff Engagement and Involvement, Information and Knowledge, Leadership, Innovation and Creativity, Decision-making, Situation Monitoring and Reporting, Planning Strategies, Participation in Exercises, Proactive Posture, External Resources, Recovery priorities, and so on.
M. Amir et al. [29]	Conducting research on understandings of organizational resilience and reinforcement of regulations through a review of growth-based compositions and measurements	4	Perseverance, Commitment to Growth, Positive Emotion, and Meaning Making
Y. Kim [30]	Studied how the effects of family resilience support the successful adaptation of remarried families.	11	Giving Meaning, Positive Outlook, Spirituality, Flexibility, Connectivity, Mobilization of Social Resources, Communication, Collaborative Problem Solving, and so on.
S. McManus et al. [31]	Identifying the key contributors to organizational resilience with case studies from a wide range of industry sectors in New Zealand.	30	Succession, Staff Welfare, Backup, Privacy/Protection, Training/Review, Risk Management, Continuity Planning, Cash Flow, Market/Brand Knowledge, Insurance/Aid, Interconnectedness, Statutory Compliance, and so on.
D. Mendonca et al. [32]	Proposing experience-based criticism and contributing factors to supplement existing contribution factors, based on the 2011 New York power loss incident.	24	Improvisation of the Action, Applicability of the Organization’s Prior Experience, Quality of the Response, Flexibility/Stiffness, Buffering Capacity, Margin of the System, Tolerance, Cross-scale Interaction, and so on.
M. Salanova et al. [33]	Conceptualized and evaluated the “Health & Resilient Organization” model.	17	Kindness, Expressing Gratitude, Forgive, Sharing Positive news, Taking Care of Social Relationships, Reflecting on the Positive, Practicing Mindfulness, Setting Personal Goals, Relishing, A Positive Audit, and others.

gives a summary of the identified contributing factors in the general domain, along with the corresponding references.

2.2. Research on Resilience in the Process Plant Domain

The process plant domain classification includes NPPs, chemical plants, and industrial domains. In this context, 11 studies were reviewed. Hollnagel et al. [34] demonstrated how RE can replace industrial safety understanding in situations where safety systems pre-equipped in NPPs operate properly, and vice versa. Park et al. [2] developed a Safety-II model for safety in unexpected reactor shutdown situations at NPPs, based on the “Model of Resilience in Situation” developed by Électricité de France. Kim et al. [24] developed a quantitative evaluation model for unexpected scenarios. Kamanja et al. [19] characterized resilience in an NPP EOS, aiming to gain an understanding of the various resilience properties. Labaka et al. [35] developed a comprehensive resilience framework for considering both internal and external organizational regulations and the resilience level of a plant. Gauthereau et al. [36] conducted a study aiming to provide a general view of emergency response plans in the event of an accident at a Swedish NPP. Shiral et al. [37] conducted a quantitative evaluation of RE based on nine resilience factors.

Azadeh et al. [38] conducted research to strengthen the scientific platforms for petrochemical plants and identify the performance of the corresponding human and safety resources by considering a new understanding of the relationship between the performance assessment and resilience level. Rabbani et al. [39] proposed a novel optimization algorithm for evaluating the RE culture and performance in an industry based on graph theory, a matrix approach, and statistical methods for analysis of safety cultures. Shiral et al. [40] conducted a quantitative evaluation of RE in the process plant industry, based on six RE factors, including Top Management Commitment, Just Culture, Learning Culture, and Awareness and Opportunity. Morales et al. [41] identified the factors contributing to organizational resilience when an organization performs particular tasks, and presented a differentiation from the existing RE. Gomes et al. [42] conducted simulations to determine the influence of a system’s resilience in real-world disaster situations. Azadeh et al. [43] introduced the main factors contributing to petrochemical plant resilience and conducted research to improve the safety of petrochemical plants by calculating the effects of these contributing factors on the system.

Table 2. Summary of identified contributing factors from the process plant domain.

Reference	Main Topic	Contributing Factors
		No. of Factors	Examples
E. Hollnagel et al. [34]	Demonstrating how RE can replace industrial safety understanding in situations where safety systems pre-equipped in NPPs operate properly, and vice versa.	4	Monitor, Anticipate, Respond, and Learn.
J. Park et al. [2]	Developing a Safety-II model for safety in unexpected reactor shutdown situations at NPPs based on the “Model of Resilience in Situation” developed by EDF.	18	Training, Procedure, Organization Culture, Human Resource, Human–System Interface, Decision-making, Verification, Reconfiguration, Teamwork, Adaption, and so on.
J. Kim et al. [24]	Developing a quantitative evaluation model for unexpected scenarios.	17	Training, Procedure, Organization Culture, Human Resource, Human–System Interface, Execution, Verification, Reconfiguration, Teamwork, Robustness, and so on.
F. Kamanja et al. [19]	Characterizing the resilience in an NPP EOS, aiming to gain an understanding of the various resilience properties.	20	Prescription, Training, Human–machine Interface, Human Resource, Safety Culture, System Verification, System Reconfiguration, Execution, Experience, Communication, and so on.
L. Labaka [35]	Developing a comprehensive resilience framework for considering both internal and external organizational regulations and the resilience level of a plant.	45	Safety System, Redundancy, Simplicity, Loose Coupling, Audits (External and Internal), Preventive Maintenance, Corrective Maintenance, Data Acquisition Equipment, Information Monitoring Equipment Management Procedures, Coordination Procedures with External Stakeholders, and so on.
V. Gauthereau et al. [36]	Aiming to provide a general view of emergency response plans in the event of an accident at a Swedish NPP.	16	Message from Plant/information, Explaining Situation, External Communication, Repeating Known Information, Questioning, Improvisation, Order to Other Team Members, Coordination, Approving and Taking the Decision, Request for Silence, and others.
G. Shirali et al. [37]	Quantitative evaluation of RE based on nine resilience factors.	9	Buffering Capacity, Margine, Tolerance, Cross-scale interactions, Learning Culture, Flexibility, Anticipation, Attention, and Response.
A. Azadeh et al. [38]	To strengthen the scientific platforms for petrochemical plants and identify the performance of the corresponding human and safety resources.	10	Management commitment, Learning Culture, Reporting Culture, Awareness, Preparedness, Flexibility, Teamwork, Redundancy, and Fault-tolerant.
M. Rabbani et al. [39]	Proposing a novel optimization algorithm for evaluating the RE culture and performance in industry based on graph theory, a matrix approach, and statistical methods for analysis of safety cultures.	10	Top Management Commitment, Reporting Culture, Learning Culture, Awareness, Preparedness, Flexibility, Self-organization, Fault-tolerant, Teamwork, and Redundancy.
A. Shirali et al. [40]	A quantitative evaluation of RE in the process plant industry, based on six RE factors.	6	Top Management Commitment, Just Culture, Learning Culture, Awareness and Opacity, Preparedness, and Flexibility.
S. Morales et al. [41]	Identifying the factors contributing to organizational resilience when an organization performs particular tasks, and presented a differentiation from existing RE.	23	Vision Sharing, Leadership, Management of Change, Perspective network, Commitment and Involvement, Management of Change, Vision Sharing, Network Perspective, Commitment and Involvement, Functions and Responsibilities, and so on.

gives a summary of the identified contributing factors in the process plant domain along with the corresponding references.

2.3. Research on Resilience in the Business Domain

Five papers pertaining to the application of the concept of resilience to businesses and companies are reviewed in this subsection. Annarelli et al. [44] presented a methodology for assessing the effectiveness of organizational resilience strategies in the service industry. Azusa et al. [45] identified the psychological factors corresponding to transient trauma or distress from unexpected serious events, aiming to provide quicker recovery. Beek et al. [12] developed the “Analyzing and Developing Adaptability and Performance in Teams to Enhance Resilience” (i.e., ADAPTER) organizational resilience evaluation model. Ljungberg et al. [46] applied the “Resilience Analysis Grid” (RAG) approach to analyze a Swedish Air Traffic Management navigation service company, e.g., with regard to its ability to respond to unexpected incidents and disturbances. Bruggy et al. [47] emphasized the need for organizational resilience, a model for organizational resilience, and methods for improving organizational resilience. A summary of the identified contributing factors in the business domain, along with the corresponding references, is presented in

Table 3. Summary of identified contributing factors from the business domain.

Reference	Main Topic	Contributing Factors
		No. of Factors	Examples
A. Annarelli et al. [44]	Presenting a methodology for assessing the effectiveness of organizational resilience strategies in the service industry.	9	Continuous Monitoring, Anticipation Ability, Redundancy, Simulation, Initial Vulnerability, Focus on Minor Aspect, Learning from Mistakes, Internal Communication, and Improvisational Capabilities.
K. Azusa et al. [45]	Identifying the psychological factors corresponding to transient trauma or distress from unexpected serious events, aiming to provide quicker recovery.	15	Orientation for Completing Tasks, Orientation for Interpersonal Relation, Job Directions, Concern for Interpersonal Relations, Information Sharing, Clarification of Task, Monitoring and Coordination, Mutual Feedback, Organizational Resilience Evaluation, Intrinsic Commitment, and so on.
D. Beek et al. [12]	Developing the “ADAPTER” organizational resilience evaluation model.	13	Team Responding Behavior, Collective (Learning) Behavior Team, Psychological Safety Team, Preoccupation with Failure, Situation Assessment, Heedful Interrelating, Team Factors, Monitoring, Cooperation with Other Department, Shared Leadership, and so on.
D. Ljungberg et al. [46]	Applying the “Resilience Analysis Grid” (RAG) approach to analyze a Swedish Air Traffic Management navigation service company	4	Respond, Monitor, Anticipate, and Learn.
P. Brouggy et al. [47]	Emphasizing the need for organizational resilience, a model for organizational resilience, and methods for improving organizational resilience.	16	Leadership, Decision-making, Staff Engagement, Situation Awareness, Innovation and Creativity, Effective Partnerships, Internal Resources, and so on.

.

2.4. Research on Resilience in the Medical and Healthcare Domain

Seven papers related to the application of resilience to hospital emergency rooms and healthcare, in general, were reviewed. Santomauro et al. [6] reviewed the second victim phenomenon for both symptom studies and organizational responses in healthcare. Goncalves et al. [48] applied a benchmark resilience tool in a Spanish environment in order to identify safety aspects and relationships. Iflaifel et al. [13] identified and analyzed elements for developing and enhancing RHC through conceptualization thereof for research, identification, and analysis methods. Barasa et al. [14] conducted a systematic review, aiming to apply the concept of resilience to the healthcare domain. McEwen et al. [49] developed a team resilience measurement method called “Resilience at Work” to measure individual and organizational regulations, and measured the results and independent contributing factors. Ayed [50] determined the impacts of strategic human resource management in an organizational resilience context. Chuang et al. [51] identified factors for improving the performance of emergency room personnel, based on the four RAG capabilities proposed by Erik Hollnagel. A total of 36 resilience contributing factors were considered, including Event List, Background, and Relevance. A summary of the identified contributing factors in the medical and health care domain along with the corresponding references is presented in

Table 4. Summary of identified contributing factors from the medical and healthcare domain.

Reference	Main Topic	Contributing Factors
		No. of Factors	Examples
C. Santomauro et al. [6]	Reviewing the second victim phenomenon for both symptom studies and organizational responses in healthcare.	4	Shame, Self-blame, Loss of Sleep, and Self-doubt.
L. Goncalves et al. [48]	Applying a benchmark resilience tool in a Spanish environment to identify safety aspects and relationships.	22	Goal-oriented Solutions, Avoidance or Skepticism, Critical Understanding, A System of Roles, Commitment of Management, Fair Culture, Learning Culture, Awareness, Planning, Adaptive Capacity, and so on.
M. lflaifel et al. [13]	Identifying and analyzing elements for developing and enhancing RHC through concepts for research, identification, and analysis methods.	7	Teamwork, In-situ Practical Experience, Exposure to Diverse Views and Perspectives on the Patient’s Situation, Trade-offs, The Value of Using Protocols and Checklists, System Design, and Workarounds.
E. Barasa et al. [14]	A review of applying the concept of resilience to the healthcare domain.	8	Preparedness and Planning, Information Management, Collateral Pathways and Redundancy, Governance Process, Leadership Practice, Organizational Culture, Human Capital, and Social Networks Collaboration.
K. McEwen et al. [49]	Developing a team resilience measurement method called “Resilience at Work” to measure individual and organizational regulations, and measuring the results and independent contributing factors.	7	Resourcefulness, Robustness, Perseverance, Self-care, Capability, Connected, and Alignment.
S. Ayed [50]	Determining the impacts of strategic human resource management in an organizational resilience context.	12	Continuous Developmental Opportunities, Group Incentives, Cross-Functional Work Assignments, Broad Job Descriptions, Employee Suggestions, Cross-Departmental Employees, Results-Based Evaluations, Empowerment, Employee–Customer Networks, and so on.
S. Chuang et al. [51]	Identifying factors for improving the performance of emergency room personnel, based on the four RAG capabilities proposed by Erik Hollnagel.	36	Event List, Background, Relevance, Threshold, Response List, Speed, Response Capability, Stop Rule, Duration, Verification, Indicator List, and so on.

.

2.5. Research on Resilience in the Transportation Domain

The transportation domain refers to, e.g., aerospace, railway, and public transportation sectors. In total, 11 papers covering resilience concepts in these sectors were reviewed. Burch et al. [11] proposed an evaluation measure for resilience, aiming to allow a system to maintain its functionality in certain threat scenarios. Owen et al. [52] proposed a key method for resilience to improve readiness for global aviation and shipping transport, and discussed its consequences. A report from the Homeland Defense and Global Security Office [7] (the organization in the USA that defines the Office of the Secretary of Defense’s policy perspective and conceptual origins of an actionable taxonomy for missions in space) identified factors contributing to resilience in the aerospace context. Patriarca et al. [53] developed the RAG-State Assessment Tool to combine Safety-I and Safety-II principles. Gabriel [54] tested whether the organizational citizenship behavior in Nigeria, especially in the domestic aviation sector, could lead to resilience. Woltjer et al. [55] successfully applied both resilience and agility to real-world models. Heese et al. [5] developed the “Inventory of Behavioral Resilience in Aviation” for monitoring and evaluating the daily behaviors of groups in the aviation industry, and conducted research to validate it. Azadeh et al. [38] evaluated the resilience performance of a railway transport system. Omer et al. [56] presented metrics to measure the resilience of an intelligent transportation system. Huber et al. [57] suggested contributing factors to represent an organization’s ability to cope with interference. Research was also conducted on how this metric could be measured. Saurin et al. [58] introduced a framework for representing the sources of analyses of resilience vulnerabilities, without limiting the identification process to specific systems within socio-technical systems or to specific units of analysis.

Table 5. Summary of identified contributing factors from the transportation domain.

Reference	Main Topic	Contributing Factors
		No. of Factors	Examples
R. Burch et al. [11]	Proposing an evaluation measure for resilience, aiming to allow a system to maintain its functionality in certain threat scenarios.	21	Countermeasures, Deterrence, Mobility, Maneuverability, Covertness, Active Redundancy, Over Capacity, Excess Margin, High Damage Thresholds, Passive Redundancy, and so on.
D. Owen et al. [52]	Proposing a key method for resilience to improve readiness for global aviation and shipping transport.	4	Respond, Monitor, Anticipate, ane Learn.
Homeland Defense and Global Security office [7]	Identifying factors contributing to resilience in the aerospace context.	6	Disaggregation, Distribution, Diversification, Protection, Proliferation, and Deception.
R. Patriarca et al. [53]	Developing the RAG-State Assessment Tool to combine Safety-I and Safety-II principles.	15	Normal System and Environmental Variability, Routine Abnormal Situation, Unusual Abnormal Situation, Unanticipated Situation, Potential Performance, Current Safety Performance, Past Performance, Consequences of Innovation, Consequences of Changes, and so on.
J. Gabriel [54]	Testing whether the organizational citizenship behavior in Nigeria, especially in the domestic aviation sector, could lead to resilience.	3	Situation Awareness, Keystone Vulnerability, and Adaptive Capacity.
Woltjer et al. [55]	Applying both resilience and agility to real-world models.	14	Responsiveness, Versatility, Flexibility, Innovativeness, Adaptability, Rebound, Robustness, Graceful Extensibility, Network Architectures for Sustained Adaptation, and others.
A. Azadeh et al. [38]	Evaluating the resilience performance of a railway transport system.	3	Teamwork, Redundancy, and Self-organization.
M. Omer et al. [56]	Presenting metrics to measure the resilience of an intelligent transportation system.	5	Leadership, Awareness, Flexibility, Preparedness and Emergency Planning, and Culture.
G. Huber et al. [57]	Suggesting contributing factors to represent an organization’s ability to cope with interference.	4	Awareness, Efficiency, Commitment, and Adaptability.
M. Heese et al. [5]	Developing the “Inventory of Behavioral Resilience in Aviation” for monitoring and evaluating the daily behaviors of groups in the aviation industry, and validation studies.	13	Goal Trade-offs, Coordination, Timing/Pacing/Synchronization, Approximate Adjustments, Actual Practice/techniques, Buffering Capacity, Tolerance, Adaptive Capacity, Complexity/Procedures, Underspecification, and so on.
T. Abreu et al. [58]	A framework for representing the sources of analyses of resilience vulnerabilities, without limiting the identification process to specific systems within socio-technical systems or to specific units of analysis.	4	Top Management Commitment, Learn from Both Incident and Normal Work (Learning), Increase Flexibility (Flexibility), and be Aware of System Status (Awareness).

gives a summary of the identified contributing factors in the transportation domain along with the corresponding references.

2.6. Research on Resilience in the Infrastructure Domain

A total of 14 papers applying the concept of resilience to construction, disaster, and socio-technical systems were reviewed. Rehak et al. [59,60] conducted a study to verify the “Critical Infrastructure Elements Resilience Assessment” methodology. They also researched the strengths of organizational resilience, and how such resilience could be measured in critical infrastructure systems. Francis et al. [61] reviewed risk analysis and resilience analysis to provide guidance to infrastructure system engineers. Renschler et al. [62] identified and measured disaster resilience for organizations on a variety of scales, and developed the “Population and Demographics, Environmental/Ecosystem, Organized Government Services, Physical Infrastructure, Lifestyle and Community Competence, Economic Development, and Social-cultural Capital” (i.e., “ PEOPLES”) resilience framework. Patriarca et al. [63] developed an approach to measure resilience abilities after developing a questionnaire based on the four capabilities of RAG proposed by Erik Hollnagel. Rioli et al. [64] conducted a study on the theory of stress and resilience in the information system field. Lee et al. [65] analyzed existing domestic and international disaster management evaluation systems and developed a model for evaluating disaster management using RE. Yang et al. [66] proposed a method for self-evaluating the cultural heritage disaster management capabilities of cultural heritage administrations, and for managing vulnerable areas based on a regeneration disaster management competency assessment. Rodrigues et al. [67] proposed a systematic approach to evaluating the usefulness of a water sector analysis method by applying the four capabilities of the RAG proposed by Erik Hollnagel, and evaluating the organizational resilience in each water sector. Nam et al. [52] proposed measures for strengthening resilience by minimizing the impacts of climate change on social and economic activities in coastal areas and maximizing the ability to recover. Sungho Hong [8] researched the need for an organization dedicated to national disaster management, and for improvements in resilience capacities and systems to enhance the effectiveness of such an organization. Gwangwu Cho [68] analyzed the impacts on and vulnerability of Korea’s coast, owing to rising sea levels and climate change in terms of community planning and adaptation methods, and presented corresponding policy directions. Sung Cho [69] analyzed the impacts of disaster preparedness activities on the disaster management effects of the public officials in charge of disaster management. Stewart et al. [70] analyzed national disaster-related resilience at the community level. A summary of the identified contributing factors in the infrastructure domain, along with the corresponding references, is presented in

Table 6. Summary of identified contributing factors from the infrastructure domain.

Reference	Main Topic	Contributing Factors
		No. of Factors	Examples
D. Rehak et al. [59,60]	Verifying the “Critical Infrastructure Elements Resilience Assessment” methodology.	80	Recovery Planning, Evaluation/Audit of the Security and Risk Analysis, Backup Data Center, Backup Control Workplaces, Available Redundant Capacity, Auditing Event in Systems, IDS/IPS/SIEM System, Firewall/Demilitarized Zone, Incident Reporting, Asset Monitoring, and so on.
R. Francis et al. [61]	Reviewing risk analysis and resilience analysis to provide guidance to infrastructure systems engineers.	4	Absorptive, Adaptive, Recovery, and Restorative.
C. Renschler et al. [62]	Identifying and measuring disaster resilience for organizations on a variety of scales, and developing the PEOPLES resilience frame-work.	24	Distribution, Socio-economic Status, Air Quality, Biomass, Biodiversity, Legal and Security Services, Health services, Lifelines, Financial, Production, and so on.
R. Patriarca et al. [63]	Developing an approach to measure resilience abilities after developing a questionnaire based on the four capabilities of RAG proposed by Erik Hollnagel.	33	Internal Protocols, Resources Availability in an Expected Situation, Resources Availability in Unexpected Situation, Experience, Discretionary Power, Teamwork, Roles Division, List of Questions for Preoperative Examination, Preoperative Specialist Support, Identification of Intraoperative Complications, and so on.
L. Riolli et al. [64]	A study on the theory of stress and resilience in the information system field.	7	Community, Competence, Connections, Commitment, Communication, Coordination, and Consideration.
J. Lee et al. [65]	Analyzing existing domestic and international disaster management evaluation systems, and developing a model for evaluating disaster management using RE.	44	Time Horizon, Responsibility, Acceptability of Risks, Culture, Efficiency, Background, Selection Criteria, Response List, Verification, Threshold, and so on.
H. Yang et al. [66]	Proposed a method for self-evaluating the cultural heritage disaster management capabilities of cultural heritage administrations.		Responsibility, Acceptability of Risks, Culture, Efficiency, Indicator List, Relevance, Indicator Type, Validity, Delay, Measurement Type, and so on.
M. Rodriguez et al. [67]	Proposing a systematic approach to evaluating the usefulness of a water sector analysis method by applying the four capabilities of the RAG proposed by Erik Hollnagel, and evaluating the organizational resilience in each water sector.	20	Assumption About the Future, Acceptability of Threats, Time Horizon, Learning Basis, Data Collection, Implementation and Communication, Frequency, Event List, Background and Relevance, Response List, and others.
J. Nam et al. [52]	Proposing measures for strengthening resilience by minimizing the impacts of climate change on social and economic activities in coastal areas and maximizing the ability to recover.	32	Forecast, Emergency Measures are in Place, Compliance With Legal Standards, Human Resources, Facility Relocation and Purchase, Regular Maintenance, Supplies and Materials, Road Waste Disposal Capacity, Comprehensive (Strategic) Plan Established, Damage Reduction Plan Establishment Status, and so on.
S. Hong et al. [8]	Researching the need for an organization dedicated to national disaster management, and for improvements in resilience capacities and systems to enhance the effectiveness.	27	Classification, Formalization, Training, Learning style, Resources, Learning target, Implementation, Frequency, Communication, Strategy, and so on.
K. Cho [68]	Analyzing the impacts on and vulnerability of Korea’s coast, owing to rising sea levels and climate change, in terms of community planning and adaptation methods, and presenting corresponding policy directions.	17	Direct experience, Communication network, Support system, Regulation, Purchase and transfer, Insurance, Building regulations, Human resources, Education, Maintenance of facilities, and so on.
S. Cho et al. [69]	Analyzing the impacts of disaster preparedness activities on the disaster management effects of public officials in charge of disaster management.	11	Planning, Equipment (resource), Mobilization, Education, Training, Cooperation System, Social Capital, Economic Capital, Human Capital, and Institutional Capital.
T. Geoffrey et al. [70]	Analyzing national disaster-related resilience at the community level.	5	Information Linkage, Operational Linkage, Legal Bonds, Cooperative Norms, and Relationship-Specific Adaptations by the Buyer or Seller.

.

2.7. Research on Resilience in Other Domains

The studies not applicable to the already defined domains, or with only a few examples of resilience-related content, were classified into “Other Domains.” Four papers were reviewed. Gonzalo et al. [71] proposed a method for assessing RE in response to a military disaster by identifying the optimal state of resilience. The study also evaluated the current state of organizational resilience in that sector. Johns et al. [72] reviewed the specific resilience contributing factors affecting the overall reliability of any organization. Woods [73] proposed an overall RE approach and also proposed some resilience contributing factors. Wicker et al. [74] investigated the organizational resilience of common sports clubs in the aftermath of natural disasters.

Table 7. Summary of identified contributing factors from other domains.

Reference	Main Topic	Contributing Factors
		No. of Factors	Examples
J. Gonzalo et al. [71]	Proposing a method for assessing RE in response to a military disaster by identifying the optimal state of resilience and level of recent organizational resilience.	11	Management Commitment, Reporting, Learning, Fault Tolerance, Redundancy, Teamwork, Self-organization, Flexibility, Preparedness, Awareness, and Just Culture.
R. Johns et al. [72]	Reviewing the specific resilience contributing factors affecting the overall reliability of any organization.	7	Psychological Safety, Cohesion, Procedural Justice, Identification, Learning Goal Orientation, Leadership, and Organization Resilience.
D. Woods [73]	Proposing an overall RE approach.	6	Margin, Tolerance, Downward, Buffering Capacity, Flexibility/Stiffness, and Upward.
P. Wicker et al. [74]	Investigating the organizational resilience of common sports clubs in the aftermath of natural disasters.	25	Robustness, Redundancy, and Resourcefulness.

gives a summary of the identified contributing factors in other domains along with the corresponding references.

3. Emergency Response Organizations and Capabilities of Organizational Resilience

This section may be divided by subheadings. It should provide a concise and precise description of the experimental results, their interpretation, and the experimental conclusions that can be drawn.

3.1. Emergency Response Organizations

Typically, emergency response organizations (EROs) are set up for nuclear power plant emergencies and may also be mandated by law. For example, the Nuclear Emergency Preparedness Act of Korea has stated that the nuclear utility has an obligation to secure personnel and organizations exclusively in charge of work in preparation for radioactive disasters or mitigating the consequences of such disasters. In the event of a nuclear plant accident, the ERO performs several activities, including initial response to the accident, prevention of accident escalation, accident recovery and decontamination activities, and accident cause investigation. The ERO in Korean plants consists of the Technical Support Center (TSC), Operating Support Center (OSC), and Emergency Operation Facility (EOF). The following describes the roles of the three EROs.

3.1.1. Emergency Operation Facility (EOF)

The role of the EOF is to supervise emergency response activities of power plants, cooperate with external disaster prevention agencies, monitor environmental emission of radioactive materials, measure and evaluate the radiation (ability), and recommend residents’ expected exposure dose and protective measures.

3.1.2. Operating Support Center (OSC)

The role of the OSC is to perform emergency maintenance, fire extinguishing and relief activities, and to serve as a waiting place for emergency maintenance support personnel, to establish a cooperation system with the MCR and EOF. The head of the OSC shall operate the OSC in general, keep emergency maintenance consultation with the TSC, and cooperate with related organizations related to fire extinguishing and medical treatment response.

3.1.3. Technical Support Center (TSC)

The TSC performs the functions of the EOF until the EOF is established, in order to prevent confusion among the MCR operators and to provide technical and administrative support necessary for responding to accidents. The purpose of this action is to ensure that MCR operators are free from peripheral tasks that are not directly related to reactor and system operations. The Director of TSC directs the emergency response before the EOF is established, and when the EOF is established, the Director of EOF shall report the results of the emergency situation and make a final decision on important matters. If an EOF is established, the head of TSC proposes to the head of EOF a final decision on the results of the emergency analysis and countermeasures, as well as the overall important matters of critical incident management.

3.2. Capabilities of Organizational Resilience

If resilience is defined as described in the Introduction, then the purpose of organizational resilience is to create resilience in an organization. In this context, organizational resilience represents the ability of an organization to adjust its tasks to both unexpected and expected conditions. Hollnagel [75] proposed a RAG concept that requires systems to have four abilities to achieve a high level of resilience. The four capabilities proposed in the RAG are as follows: (1) responding (knowing what to do, being capable of doing it), (2) monitoring (knowing what to look for), (3) anticipating (identifying and knowing what to expect), and (4) learning (knowing what has happened) [1]. The four capabilities of the RAG concept are described further as follows.

• Responding: This refers to the ability to know what to do in expected or unexpected situations (especially unexpected situations), to respond by performing regular or irregular actions (or changes thereto), to avoid confusion, to perform already prepared actions for opportunities, or to adjust the approach to work.
• Monitoring: This refers to knowing what one is looking for or being able to observe changes (positive or negative) affecting a system’s performance in a short period of time. Monitoring includes observation of the environment within and outside the system boundary, and provides the opportunity to cope with near-future events.
• Anticipating: This refers to knowing what to expect, as well as forecasting more distant future developments such as potential disruptions, new demands, new constraints, new opportunities, and/or changes in the working environment. Although it denotes the ability to predict the future, such as in monitoring, it specifically denotes the ability to predict the future for a long time, not a short time.
• Learning: This refers to knowing what happened, experiencing the right things from a particular incident, and learning the right things.

These four capabilities are typically associated with NPPs. For example, an NPP will be designed based on other industries or previous experience (learning), and the NPP will have the necessary safety systems to cope with accidents and possible accident scenarios from the NPP at that stage (anticipation). In the plant operation phase, the direct and indirect acquisition of NPP variables will be monitored to cope with current conditions (monitoring), and when events occur, they will respond with NPP stabilization based on monitoring (responding).

All of the aforementioned tasks are currently conducted in NPPs by organizations (including humans). Thus, this study applies the four RAG capabilities as high-level factors to identify NPP organizational resilience factors.

4. Methodology for Identification and Classification of Contributing Factors to Organizational Resilience in Nuclear Power Plants (NPPs)

A total of 1282 contributing factors were derived from the review of the literature discussed above. However, it is difficult to directly apply all of these contributing factors to the EROs at NPPs. This is because some of the contributing factors in the literature are domain-specific, and not relevant to NPPs. Hence, this study developed a process to identify the specific factors contributing to the resilience of NPP EROs. The process consisted of three steps, as shown in Figure 1.

4.1. Step 1: Is It a Contributing Factor That Can Be Classified According to the Resilience Analysis Grid (RAG)?

Based on the review of the literature, this study determined whether the preselected contributing factors were applicable to the RAG concept proposed by Hollnagel [1,3]. As mentioned above, a total of 1282 contributing factors were identified in Step 1. Of these, 650 were rejected, because they were deemed inapplicable to the four RAG capability definitions. For example, Yoojung Kim [30] includes “Belief Systems” as a proposed contributing factor. This is an ambiguous factor, and was deleted because it did not correspond to any of the capabilities of the RAG. Another example is in Rehak et al. [59,60], which suggested “Allocation of Financial Resources.” The definition of this contributing factor is “allocation and management of financial capital to help an organization achieve its strategic goals.” In the case of individuals, finance can be a significant factor. However, in a case where the nuclear utility is managed by the government, it was judged that issues relating to finance need not be presented. The ultimate goal of Step 1 was to define which contributing factors could be classified as contributing factors within the RAG’s four capabilities. Hence, this study compared and analyzed all contributing factors, along with the definition of RAG capabilities.

4.2. Step 2: Is the Contributing Factor Applicable to Emergency Response Organizations (EROs) in NPPs?

In Step 2, the remaining factors (results from Step 1) were scrutinized in order to verify that the resilience contributing factors applicable to other industries were also applicable to EROs at NPPs. In this phase, some criteria were applied to determine the applicability to the EROs at NPPs. A total of 632 contributing factors were considered in Step 2. Of these, 219 were rejected, because they were deemed inapplicable to the ERO activity.

The criteria were initially reviewed based on the Korean Atomic Energy Act on Protection of Nuclear Facilities, the Act on Countermeasures against Radioactive Disaster Prevention, the Enforcement Decree, and the Enforcement Rule. For example, the context for the declaration and report of a radioactive disaster under Article 23 of the Act is as follows: “in the event of a radioactive disaster, a radioactive disaster must be declared without delay.” This content allows other organizations or the public to be informed regarding radioactive disasters, and should be related to the reports on the relevant factors. Accordingly, a contributing factor called “report” was considered to be applicable.

The literature related to nuclear power and safety was also reviewed. Human reliability analysis (HRA) is a major part of the PSAs for NPPs, as it considers the organizational aspects of safety. As such, the PSA literature [76,77,78,79,80], nuclear and nuclear safety literature [81,82,83,84], and safety culture literature [85,86,87] were scrutinized for resilience contributing factors. At this stage, the basic concept was to identify the contributing factors suggested for HRA via performance shaping factors (PSFs). For example, the definition of execution is “The measure of whether the person performs the intended actions correctly.” This contributing factor passed Step 1 as a possible contributing factor corresponding to Response in RAG Capability. However, according to HRA, ‘execution’ is related to the direct manipulation of equipment. In the case of EROs of NPPs, they do not directly operate (manipulate equipment or control) the NPPs. EROs only provide MCR operators with appropriate decision-making results. Accordingly, Execution was rejected in Step 2.

The reason why the PSFs were used as the basis for the HRA is that they are used in reliability evaluation methods applied to humans currently working in NPPs. Thus, it was expected that many PSFs would be applicable to EROs. For example, the definition of the “Procedure” PSF in the “Standardized Plant Analysis Risk-Human Reliability Analysis” method (an HRA method) refers to the existence and use of pre-written steps/guidelines for the situation under consideration. It determines whether incorrect or inappropriate information is provided based on the characteristics and sequences of the procedure [76]. Such procedures represent a very important factor for NPPs. In fact, the Korean Nuclear Law stipulates on “matters related to procedures such as operation and repair.” In addition, the IAEA has discussed Procedure as an important factor in determining whether expected outcomes are met [88], and the importance of ensuring the appropriate qualification of procedure writers [89].

4.3. Step 3: Merge and Classify Identified Factors

In Step 3, a merger and re-classification of the remaining contributing factors was performed based on whether the screening tasks of Steps 1 and 2 are performed. At this stage, redundant contributing factors were eliminated, and the proposed contributing factors were reclassified. A total of 413 contributing factors were identified from Step 2. Finally, a total of 52 contributing factors were identified for the ERO.

This was necessary because the extensive literature review showed that various authors used different words to refer to the same context. For instance, both Park et al. [2] and Kim et al. [24] used a contributing factor called “Teamwork.” The definition of Teamwork was “how operators interact with each other to exchange information, coordinate behavior, and maintain social order.” In another study, Salisu et al. [25] used a contributing factor called “Cooperation”, and the definition represented “cooperation, including information transfer with other organizations.” Thus, these factors were judged as the “Communication in Level 2” to have the same meaning, as they both denoted the interactions of operators or different organizations. In this study, the four capabilities proposed by RAG were each denoted as a “Level 1 RAG Capability,” and Level 2 Factors were identified as contributing to Level 1. For example, in the process of merging, Cooperation was selected as a Level 2 factor, whereas Teamwork was used as a factor for evaluating Cooperation. There were 35 Level 3 factors identified as sub-contributing factors to the Level 2 factors. The Level 3 contributing factors were necessary because the Level 2 factors had relatively broad scopes. More specific assessment items will likely ease future evaluations of EROs at NPPs.

5. Contributing Factors for Organizational Resilience in NPPs

A total of 52 factors contributing to the resilience of EROs at NPPs were identified according to the three aforementioned steps. The contributing factors comprised three levels. The structure of the contributing factors is shown in Figure 2. A detailed description of each factor is given in the following subsections.

5.1. Responding Capability for the ERO of NPP

A total of 19 contributing factors were considered for the response capability. Five contributing factors were considered as Level 2 factors, and 14 contributing factors were considered as Level 3 factors. The definitions of each contributing factor (Level 2 and Level 3) are shown in

Table 8. Definitions and contributing factors for responding capability.

Level 1	Level 2	Definition (Level 2)	Level 3	Definition (Level 3)
1. Responding	1.1 Responding Procedure [2,5,24,35,63]	Is it possible to respond with the relevant emergency procedures?	1.1.1 Responding Preparedness [2,10,14,19,24,26,27,37,38,39,40,56,59,60,71]	Is there a corresponding response procedure in place?
			1.1.2 Action List [13,51,67]	Is a list of actions provided in the procedures?
			1.1.3 Background [51,65,66,67]	Is there sufficient evidence for a response?
			1.1.4 Revision [13,35]	Is the latest version of the procedure used?
	1.2 Performance [13,53,63]	How effectively can an organization respond to emergencies?	1.2.1 Speed [51,65,66]	Can ERO respond quickly to the situation?
			1.2.2 Duration [51,65,66]	How long can an effective response last?
			1.2.3 Effectiveness [26,34,55,59,60]	Are effective and accurate responses taken (or possible)?
	1.3 Staffing [42,49,59,60]	Are there adequate members of the organization for emergency response?	1.3.1 Number of Personnel [2,10,14,19,24,30,59]	Is there an adequate number of personnel in place?
			1.3.2 Qualification [9,14,15,38,59]	Is the qualification/professionalism of the workforce sufficient?
			1.3.3 Role and Responsibility [9,15,43,53,59,63]	Are the roles and responsibilities of the workforce defined?
	1.4 Adaptability [2,9,15,19,24,25,26,31,43,56,57,60,61,70]	How flexibly can emergency response organizations respond to situations?	1.4.1 Stopping Rule [2,19,24,51,65,66]	Are there criteria in place for (1) discontinuing if the strategy one is currently applying is invalid, or (2) terminating if completed?
			1.4.2 Alternative Strategy [3,10,14,15,25,27,32,37,38,39,40,43,48,71,73]	Is there an alternative, if the strategy currently applied is not valid?
	1.5 Tool and Equipment [26,35,44,53,63]	Are equipment and tools in place to respond to emergencies?	1.5.1 Availability [5,15,53,59,60,63]	Are equipment and tools available?
			1.5.2 Adequacy [2,19,24,26,35,59]	Does ERO have the right tools to respond to the situation?

. For each of the contributing factors to responding capability, the relevant literature has also been indicated.

5.2. Monitoring Capability for the ERO of NPP

For the monitoring capability, 10 contributing factors were considered. Three contributing factors were considered as Level 2 factors, and seven contributing factors were considered as Level 3 factors. The definitions of each contributing factor (Level 2 and Level 3) are shown in

Table 9. Definitions and contributing factors for monitoring capability.

Level 1	Level 2	Definition (Level 2)	Level 3	Definition (Level 3)
2. Monitoring	2.1 Monitoring Procedure [1,2,19,24,35,44,46,75]	Are procedures properly in place for monitoring?	2.1.1 Monitoring Preparedness [2,3,12,23,24,35]	Is there a procedure for monitoring?
			2.1.2 Monitoring List [2,12,19,24,35,44,46]	Does the procedure describe the targets to be monitored?
			2.1.3 Decision-Making Criteria [12,44,51,75]	Are criteria provided for decision-making as a result of monitoring?
	2.2 Information [2,9,14,19,24,31,35,44,45,70]	How effectively can an organization respond to emergencies?	2.2.1 Availability [2,14,17,18,47,53,54,57]	Is the information required for monitoring available in a timely manner?
			2.2.2 Adequacy [14,31,70]	Is the information required for monitoring provided in an appropriate manner?
	2.3 Support [25,53,59,60,63,65,66,67,74]	Are there adequate members of the organization for emergency response?	2.3.1 Support System [2,10,19,24,35,37,38,40,41,43]	Is there a system in place to provide or share information necessary for monitoring or decision-making?
			2.3.2 External Support [3,10,14,15,25,27,32,37,38,39,40,43,48,71,73]	Are there external experts available for assistance if necessary?

. For each of the contributing factors to monitoring capability, the relevant literature has also been indicated.

5.3. Learning Capability for the ERO of NPP

For the learning capability, 10 contributing factors were considered. Two contributing factors were considered as Level 2 factors, and eight contributing factors were considered as Level 3 factors. The definitions of the contributing factors (Level 2 and Level 3) are shown in

Table 10. Definitions and contributing factors for learning capability.

Level 1	Level 2	Definition (Level 2)	Level 3	Definition (Level 3)
3. Learning	3.1 Training [2,19,24,26,31,35,37,40,51,53,56,59,60,63,65,66,67]	Is education/training necessary for an emergency response to be properly conducted?	3.1.1 Objective [18,20,47,57]	Is the purpose of education/training clearly presented?
			3.1.2 Trainer [53,54]	Are the qualities of a trainer/instructor sufficient?
			3.1.3 Method [2,19,24,59,60,65,66]	Are educational/training methods appropriate for the purpose (e.g., practice, teaching)?
			3.1.4 Facility [31,35,56,60,65]	Are suitable facilities (e.g., simulators, actual facilities) used to achieve training/training objectives?
			3.1.5 Evaluation [35,37,40,59,60]	Are the evaluation criteria, evaluation methods, feedback methods, etc., of education/training specifically presented?
			3.1.6 Frequency [51,65,66,67]	Is the frequency of training appropriate?
	3.2 Experience Dissemination [2,13,19,24,32,34,52,53,63,67]	Are experience cases properly disseminated to Members?	3.2.1 Reporting [9,10,25,32,37,53,54,65]	Does the ERO have procedures and systems in place to report and manage good and bad experiences?
			3.2.2 Dissemination [13,32,53,63,67]	Do you have procedures or systems for disseminating experience cases?

. For each of the contributing factors to learning capability, the relevant literature has also been indicated.

5.4. Anticipating Capability for the ERO of NPP

For the anticipating capability, a total of 13 contributing factors were considered. Three contributing factors were considered as Level 2 factors, and ten contributing factors were considered as Level 3 factors. The definitions of each contributing factor (Level 2 and Level 3) are shown in

Table 11. Definitions and contributing factors for anticipating capability.

Level 1	Level 2	Definition (Level 2)	Level 3	Definition (Level 3)
4. Anticipating	4.1 Communication [2,9,10,19,24,31,35,38,42,43,44,51,53,56,57,63,64,64,65,66,67]	Is there a means of communication for information exchange?	4.1.1 Clarity [9,10,31,35,44,51,53,56,57,63,67]	Are information providers, targets of provision, information exchanged, etc. clearly presented in manuals, protocols, etc., for communication?
			4.1.2 Communication Equipment [2,19,24,44,56,64,65]	Does the ERO have tools for communication?
	4.2 Planning [2,9,14,19,23,24,26,31,42,44,48,53,56,57,59,60,63]	Is there a proper plan for accident management?	4.2.1 Plan Establishment [2,9,14,19,24,53,56,57,59,60,63]	Is there a radioactive disaster prevention plan and a manual for on-site measures in preparation for accidents?
			4.2.2 Scope [59,60]	Is the scope of accidents under consideration in the accident management plan appropriate?
			4.2.3 Time Horizon [59,60,65,66,67]	Is the time range considered in the accident management plan appropriate (e.g., 72 h after the accident)?
			4.2.4 Basis [51,65,66,67]	Is the technical background of the accident management plan presented?
	4.3 Safety Culture [10,24,33,34,35,37,38,40,43,48]	Is the awareness of safety culture appropriate for workers?	4.3.1 Level [33,34,40]	Is the level of safety culture appropriate for workers?
			4.3.2 Cooperative Norm [49,70]	Are cooperative measures, such as governance and responsibility, established with related agencies in preparation for accidents?
			4.3.3 Risk Factor Recognition [59,60,63,65,66,67]	What is the level of awareness among members of the organization regarding the risk of nuclear accidents?
			4.3.4 Trust and Engagement of Participants [9,31,35]	Are major partners (power plants, regulators, government departments) and networks (consultative bodies, workshops, etc.) established as strategies to enhance disaster management expertise?

. For each of the contributing factors to anticipating capability, the relevant literature has also been indicated.

6. Discussion

6.1. Discussion on Results

The results from this study are absolutely essential to any efforts to evaluate the resilience of EROs in NPPs. These factors, as presented in the results, would be the starting point for developing a method for ERO resilience analysis.

For the ERO responding capability, factors such as Speed, Responding Preparedness, Action List, Background, Revision, Duration, Effectiveness, Number of Personnel, Qualification, Role and Responsibility, Stopping Rule, Alternative Strategy, Availability, and Adequacy, should be used for evaluation. For example, the definition of Speed is “Can ERO respond quickly to the situation?” This means how quickly appropriate required action can be performed in an actual emergency situation. In the case of such situations, ERO staff must perform the appropriate actions that eventually prevent the situation from evolving into a severe accident.

Similarly, for the monitoring capability, the factors such as Monitoring List, Decision-making Criteria, Monitoring Preparedness, Availability, Adequacy, Support System, and External Support can be used at the lower levels for evaluation. For example, the definition of a Monitoring List is “Does the procedure describe the targets to be monitored?” This checks whether the procedure mentions the appropriate monitoring variables for the ERO in an actual emergency response situation. In actual emergency situations, when EROs are launched, MCR provides the current state or variables of the plant to the ERO without a decision-making authority. The ERO receives necessary information from the MCR to make decisions. This is a limitation of the current strategy. Thus, procedures that enable ERO to identify the required variables, i.e., a monitoring list, can be an important requirement for resilience at this stage.

Factors such as Trainer, Method, Objective, Facility, Evaluation, and Frequency can be used to evaluate the learning capability at the lower levels. For example, the definition of a trainer is “Are the qualities of a trainer/instructor sufficient?” This is a factor that represents the instructor’s ability to train in order for the ERO to properly handle emergency situations in real emergency response situations, including those situations that are unprecedented. ERO is a large organization made up of employees in various fields. Thus, instructors who can properly train the varied members of this organization are important contributors to ERO resilience.

In addition, factors such as Communication Equipment, Clarity, Plan Establishment, Scope, Time Horizon, Basis, Level, Cooperative Norm, Risk Factor Recognition, and Trust and Engagement should be used for the lower level evaluation of the Anticipating capability of EROs. For example, the definition of Communication Equipment is “Does the ERO have tools for communication?” This is a factor that identifies whether ERO has an appropriate communication device to handle an emergency situation in an actual emergency response situation. In the case of ERO, communication with other organizations, such as TSC and EOF, is important, starting with the MCR. In addition, inadequate communication in emergency situations will not adequately stabilize the plant and can be the starting point for serious accidents. Accordingly, Communication Equipment is an important contributing factor.

6.2. Limitations and Future Work

This study provides readers with a clear idea of the broad applications of the resilience concept in various domains, and based on an extensive review of the literature, a systematic approach is used to derive the factors contributing to the resilience of EROs in NPPs. However, during the practical application of these factors to determine the resilience of EROs in NPPs, some caution must be exercised.

First, when the contributing factors derived from this study are applied to an actual ERO of an NPP, they must be evaluated by considering factors appropriate to the characteristics of each ERO. For example, among the factors contributing to the response, the “execution” may be assessed as contributing to the main control room (MCR), but in contrast, it is difficult to apply “execution” to a technical support center, operations support center, emergency headquarters, etc. The reason for this is that the reactor operation of the NPP is mostly conducted in the MCR; therefore, there is no need to evaluate the top ERO for “execution.” As all of the contributing factors currently proposed have been derived for all organizations responsible for NPP emergency responses, each of the necessary factors for a detailed ERO needs to be selected. For this reason, future studies should add procedures for deriving the applicable resilience contributors for each ERO.

In addition, the interactions between the organization and contributing factors must also be considered, but were not considered in the literature review for this work. The interaction between an organization and a factor should be considered, because it is to determine to what extent the level 3 items can affect the level 2 and 1, and also to consider the effects of the items (or factors) between those on the level 1 on the level 2 and 3, so these interactions should be considered in future studies. Two procedures are proposed in this work for considering the interactions between contributing factors. First of all, it is necessary to identify the correlations between each ERO. Next, as mentioned earlier, the factors specifically contributing to each ERO and their various weightings should be reflected appropriately. The two limitations presented above need to be addressed in future research.

7. Conclusions

This study reviewed several research works related to resilience in a wide variety of domains within the last two decades, including general domains, healthcare, transportation, infrastructure, process plants, business, and other domains, leading to the identification of several organizational resilience factors. This study applied three additional procedures: RAG filtering, ERO applicability assessment, and merger/re-classification. As a result, 52 factors (level 2 and level 3) contributing to the resilience of EROs in NPPs were proposed. The identified contributing factors would be useful in measuring the resilience of EROs in NPPs, e.g., to indicate the robustness of the NPP emergency response system. Moreover, this study can be used as a technical basis for improving the robustness of radiation emergency response strategies. A subsequent study will include expert judgments to test the identified resilience factors, and an analysis of the interactions between the factors. Furthermore, the various weightings of the factors as contributors to the resilience of EROs in NPPs should be determined before practical application. As the reliability of EROs plays a very important role in incident response, the results from this study may indicate a direction for overall NPP organizational safety assessments in the future.