A Risk Identification Method for Ensuring AI-Integrated System Safety for Remotely Controlled Ships with Onboard Seafarers

Abstract

The maritime sector is increasingly integrating Information and Communication Technology (ICT) and Artificial Intelligence (AI) technologies to enhance safety, environmental protection, and operational efficiency. With the introduction of the MASS Code by the International Maritime Organization (IMO), which regulates Maritime Autonomous Surface Ships (MASS), ensuring the safety of AI-integrated systems on these vessels has become critical. To achieve safe navigation, it is essential to identify potential risks during the system planning stage and design systems that can effectively address these risks. This paper proposes RA4MAIS (Risk Assessment for Maritime Artificial Intelligence Safety), a risk identification method specifically useful for developing AI-integrated maritime systems. RA4MAIS employs a systematic approach to uncover potential risks by considering internal system failures, human interactions, environmental conditions, AI-specific characteristics, and data quality issues. The method provides structured guidance to identify unknown risk situations and supports the development of safety requirements that guide system design and implementation. A case study on an Electronic Chart Display and Information System (ECDIS) with an AI-integrated collision avoidance function demonstrates the applicability of RA4MAIS, highlighting its effectiveness in identifying specific risks related to AI performance and reliability. The proposed method offers a foundational step towards enhancing the safety of software systems, contributing to the safe operation of autonomous ships.

1. Introduction

Maritime transportation is undergoing a paradigm shift driven by two main forces: stricter environmental regulations and advancements in ICT technologies. Since 2005, the IMO has been promoting e-navigation with the goal of ensuring safer and cleaner seas. This initiative aims to enhance safety, security, and marine environmental protection by using electronic means to harmonize the collection, integration, exchange, display, and analysis of marine information between ships and shore throughout the entire voyage, from departure to arrival [1]. In June 2017, the IMO defined MASS as vessels capable of operating independently on the sea surface without human intervention at various levels of automation and began related research [2,3]. As a new paradigm for addressing challenges in the maritime sector, such as enhancing maritime safety and logistics efficiency, MASS aim to initially operate under remote human supervision and gradually reduce human oversight, ultimately achieving full autonomous operation.

To ensure the safe navigation of MASS, the IMO approved the development of the MASS Code at the 105th session of the Maritime Safety Committee (MSC) in April 2022. The regulation development process began with this approval, targeting the completion of a non-mandatory MASS Code by the 108th MSC meeting in May 2024 [4,5]. Following an experience-building period, the mandatory MASS Code is expected to be developed starting in 2028, with an intended enforcement date of 1 January 2032. The MASS Code will encompass the purpose and principles of the regulation, key principles and functions of MASS, standards for remote operations, certification and inspection, approval procedures, risk assessment, operating environment, system design, and software principles.

On the technological front, efforts are being made to reduce human error and improve operational efficiency by replacing traditional crew roles with remote control and autonomous navigation, ship monitoring, and collision avoidance systems based on AI technology. However, since AI will perform crucial tasks for the safe operation of ships, it will carry risks of physical damage, economic loss, and bodily injury, along with potential functional malfunctions and logical errors [1,5,6,7,8]. Therefore, it is essential to ensure and verify the safety of AI operations.

The MASS Code, which ensures the safety of autonomous ships, involves identifying and evaluating potential risks within the system, specifying safety requirements, and verifying that the system is developed correctly according to these requirements. In light of this, this paper proposes a method for risk assessment, the first step in ensuring the safety of software systems, as AI technology is increasingly applied across various systems in the maritime domain with the introduction of MASS. This paper introduces RA4MAIS, a method designed for identifying risks in AI-integrated maritime systems. RA4MAIS provides a systematic approach to discover unknown risk situations by considering various kinds of factors, including system functional failures, human errors, environmental conditions, and AI-specific concerns such as performance and trustworthiness.

2. Background

2.1. IMO Requirements for Safety of Software Systems Applied in Autonomous Ships

Through the Regulatory Scoping Exercise (RSE), the IMO has defined four degrees of autonomy for MASS:

• Degree 1: involves ships with automated systems and decision support, where seafarers are on board to operate and oversee shipboard functions. Automation may handle some operations independently, but crew members are always present to take control if necessary.
• Degree 2: pertains to ships that are remotely controlled but still have seafarers on board. Although the ship is mainly operated from a remote location, crew members are available to manage and control onboard systems when needed.
• Degree 3: describes remotely controlled ships without any crew on board. All operations are conducted remotely from another location, and no seafarers are present on the vessel itself.
• Degree 4: refers to fully autonomous ships where the onboard operating system independently makes decisions and performs actions without human intervention.

For Degree 2 and higher levels of MASS, addressing system functional failures, human errors, and environmental conditions becomes particularly crucial. At these levels of autonomy, the system must not only ensure software reliability under diverse conditions but also manage potential human errors that may occur during complex interactions between operators and AI systems. Identifying risks related to functional failures, human errors, and environmental influences early in the design and operational phases is essential for ensuring safe and effective operations. A robust risk identification process helps detect vulnerabilities in software performance and human–system interactions, enabling proactive measures that enhance safety and reliability in these semi-autonomous systems.

The IMO is developing the MASS Code to provide a comprehensive regulatory framework ensuring the safe operation of autonomous ships. This Code will address both remotely controlled and fully autonomous ships, focusing on maintaining safety, security, and environmental protection standards equivalent to those of conventional vessels.

A significant aspect of the MASS Code relates to the safety of software systems used in autonomous ships. The Code includes specific provisions to ensure that software functions essential for ship navigation, communication, and control are designed, implemented, and maintained to minimize risks [5]. The MASS Code emphasizes the following software safety-related elements:

• Software Reliability and Robustness: The MASS Code requires that all software components, especially those responsible for critical functions such as collision avoidance, navigation, and emergency response, must be reliable and robust. The software must be able to function correctly under various conditions, including adverse environmental factors like rough seas, fog, or high traffic density.
• Risk Management for Software Systems: The Code integrates risk management principles to assess potential hazards associated with software failures or malfunctions. It mandates the identification and evaluation of software-related risks throughout the lifecycle of autonomous ships, from design and development to operation and maintenance. This involves defining safety requirements, conducting rigorous testing, and implementing fail-safe mechanisms to prevent accidents caused by software errors or vulnerabilities.
• Cybersecurity and Data Integrity: Given the reliance on software for communication and control, the MASS Code stresses the importance of cybersecurity measures to protect against unauthorized access or data breaches that could compromise ship safety. This includes regular software updates, secure communication protocols, and redundancy in critical software functions to prevent single points of failure.
• Human Oversight and Interface Requirements: While autonomous ships reduce the need for human intervention, the MASS Code highlights the importance of human oversight capabilities. Software systems must be designed with user-friendly interfaces that allow remote operators to monitor, control, and intervene if necessary. The Code also requires that software provides clear, real-time information about the ship’s status and any potential safety risks.
• Continuous Monitoring and Software Updates: The Code mandates continuous monitoring of software performance and regular updates to address identified vulnerabilities or improve functionality. This includes implementing systems for automated diagnostics, error detection, and software patch management to enhance overall safety and reliability.
• Validation and Verification of Software Systems: To ensure safety, the MASS Code requires thorough validation and verification processes for all software used in autonomous ship systems. This includes testing under various simulated and real-world conditions, formal verification methods, and ongoing assessment of software performance and reliability.

The MASS Code aims to establish a structured framework that integrates these safety measures into the design and operation of autonomous ships. By emphasizing software reliability, risk management, cybersecurity, and human oversight, the Code seeks to mitigate risks associated with the software systems essential for safe and effective autonomous ship operations.

2.2. Safety of AI-Integrated System

AI-integrated systems are most observed in the automotive industry. Advanced Driver Assistance Systems (ADAS) play a crucial role in autonomous vehicles. However, as seen in accidents involving Uber and Tesla, issues that threaten safety have begun to emerge due to limitations in the performance of components like sensors and processors or due to environmental factors, even in the absence of systematic failures or random failures. For instance, in 2018, an Uber self-driving test vehicle struck and killed a pedestrian in Arizona. The vehicle’s sensors detected the pedestrian, but the software misclassified her as a non-hazard, leading to a failure to brake in time. Similarly, Tesla’s autopilot system has been involved in several fatal accidents, including a 2016 crash in Florida where the system failed to recognize a white truck against a bright sky, resulting in a collision. These accidents highlight the challenges posed by AI technology, such as misinterpretation of sensor data and inadequate responses to unexpected environmental conditions, emphasizing the need for robust safety verification.

To address these issues, the ISO 21448 standard [7], known as Safety of the Intended Functionality (SOTIF), was established. This standard focuses on safety risks arising from the performance limitations of the intended functionality, even when there are no system faults [6,7]. It identifies external factors that can affect system performance, such as backlighting, low light, sudden changes in brightness, rain, snow, fog, contamination, and dust storms, which can lead to obstacles being missed or misidentified. These problems can cause unintended risks that are independent of software or hardware failures.

Additionally, ISO JTC 1/SC 42 addresses the reliability and risk management of AI across various standards. ISO/IEC 22989 [8] was developed as a foundational standard to understand AI systems from an IT industry perspective and to create future standards using a common language. This standard provides general classifications of AI and defines all common concepts used in the field of AI. It specifically defines the trustworthiness of AI as the ability of a system to meet the expectations of its stakeholders and identifies nine characteristics, as presented in

Table 1. Characteristics of trustworthiness.

Characteristics	Description
Robustness	The ability to maintain performance levels despite external interference or harsh environmental conditions.
Reliability	The ability to perform required functions without failure for a specified period under given conditions.
Resilience	The ability to quickly restore operational status after an incident.
Controllability	The ability for external agents to intervene and take control.
Explainability	The ability to explain why the AI system made certain decisions.
Predictability	The ability to enable humans to make trustworthy assumptions about the AI system’s outcomes.
Transparency	The degree to which the data used to create the system, and how they were collected and used for training, are disclosed.
Fairness	The degree to which different groups are not discriminated against.
Jurisdiction	The difference in regulations that apply when the operational region (jurisdiction) of the AI system changes.

, that can be used to verify trustworthiness.

ISO/IEC 23894 is a standard for managing potential risk factors associated with the development and implementation of AI or intelligent services [9]. It provides general guidelines and frameworks for AI risk management, including risk management procedures, objectives, and causes, as well as methods for managing risks throughout the AI lifecycle. This standard categorizes the potential sources of risk into seven types, as shown in

Table 2. Potential risk sources in AI systems.

Risk Sources	Description
Level of automation	AI is often used in system automation, and the level of automation can impact risks, especially in human–AI handover situations.
Lack of transparency and explainability	Without clear disclosure of AI development and training information, or if AI decisions are not explainable, trust is compromised.
Complexity of environment	AI is used to handle complex environments, which can introduce additional risks compared to simpler settings.
System life-cycle issues	AI poses risks due to differences in the system development life cycle, such as inadequate verification methods.
System hardware issues	Hardware or sensor failures may cause service disruptions or inaccurate data, and insufficient system performance or bandwidth can lead to risks.
Technology readiness	Using immature AI algorithms or models in real-world tasks can present risks.
Risk sources related to machine learning	AI development relies heavily on machine learning; poor data quality or improper training processes can lead to risks.

.

According to the seven potential risk sources of AI systems identified in ISO/IEC 23894, the first risk is that AI is often used for system automation, which may require collaboration with humans depending on the level of automation. Therefore, human–system collaboration itself can become a risk factor. The second risk is that if humans cannot verify the learning process or decision-making basis of AI, trust in the AI may be compromised, leading to risks. The third risk is that complex and diverse environments can affect the performance of AI, causing additional risks. The fourth risk is that the development methods for AI differ significantly from those of traditional systems. Applying conventional development methods directly may not adequately verify AI-integrated system, resulting in risks. Unlike traditional software, AI development often involves iterative training and optimization processes rather than predefined coding and testing steps, making it challenging to predict and control AI behavior. Additionally, AI systems are highly dependent on data quality and variability, requiring continuous updates and validation cycles that traditional development methods are not designed to handle effectively, leading to potential verification gaps and unanticipated risks. The fifth risk is that handling various situations requires significant computation, necessitating high-performance hardware and diverse sensors; therefore, risks can arise from insufficient system or sensor performance. The sixth risk is that prematurely using AI algorithms that are not yet technologically mature or fully verified can also pose risks. The seventh risk is that the performance of AI can vary significantly depending on its training, and risks may arise if the quality of the data required for training is poor or if the training process is inappropriate.

2.3. Formal Safety Assessment (FSA) in Maritime Safety

FSA is a structured and systematic methodology developed by the IMO to enhance maritime safety and aid in the decision-making process for regulatory development. FSA is designed to provide a transparent, evidence-based approach to evaluating risks, costs, and benefits associated with safety measures, making it a crucial tool in assessing the impact of new or revised regulations in the maritime sector.

The primary objective of FSA is to identify potential risks in the maritime sector and evaluate the effectiveness of risk control measures. By systematically analyzing hazards, FSA supports the development of balanced and cost-effective regulations that improve safety while considering economic implications. This approach ensures that safety regulations are both practical and justified, providing a sound basis for the IMO’s regulatory framework. FSA consists of a five-step process that guides the comprehensive evaluation of maritime risks, as shown Figure 1.

• Hazard Identification: The first step involves identifying all potential hazards that could impact maritime safety. Techniques such as FMEA (Failure Modes and Effects Analysis), HAZID (Hazard Identification), HAZOP (Hazard and Operability Study), and System-Theoretic Process Analysis (STPA) are commonly used to systematically detect and list possible risks [10,11,12].
• Risk Assessment: This step evaluates the likelihood and potential impact of identified hazards. By quantifying risk levels, this phase helps prioritize risks that require attention, providing a clearer understanding of which safety measures are most critical.
• Risk Control Options (RCOs) Development: In this phase, various options for controlling or mitigating risks are developed. Each option is analyzed to determine how effectively it can reduce the identified risks and whether it addresses the safety concerns adequately.
• Cost-Benefit Assessment: The effectiveness of each risk control option is further analyzed in terms of costs and benefits. This step is essential in determining the economic viability of proposed safety measures, ensuring that the solutions are both efficient and financially sustainable.
• Recommendations for Decision-Making: Based on the assessments, recommendations are made for implementing the most effective risk control options. These recommendations guide the IMO and other regulatory bodies in making informed decisions about maritime safety regulations.

FSA has been widely applied in maritime transportation to support the development of regulations related to ship design, operations, and safety equipment. For example, FSA significantly contributed to the adoption of double-hull requirements for oil tankers, which enhanced their structural integrity to prevent oil spills. It has also influenced safety standards for passenger ships and guided the implementation of navigational safety measures.

However, as the maritime transportation increasingly integrates AI technologies into its systems, the limitations of the traditional Hazard Identification methods used in FSA become more apparent. While FSA provides a structured framework for assessing the safety of AI-driven applications, it relies on conventional techniques that may not fully capture the unique risks associated with AI, such as algorithmic failures, data quality issues, and the complexities of human–AI interaction. Therefore, a more advanced approach to Hazard Identification is needed to address these challenges effectively. This need becomes even more critical as the industry moves towards higher levels of autonomy, including the implementation of MASS.

2.4. Literature Review

Recently, with the IMO’s implementation plan for MASS, there has been active research in academia and industry to ensure the safe operation of MASS. Risk assessments have been performed from a system perspective to ensure the safety of operations for safety-critical systems. Consequently, there has been an increased focus on conducting risk assessments for these systems and developing measures to mitigate identified risks.

However, despite the growing attention to risk assessment in this area, there are relatively few studies that propose entirely new risk assessment methods. Nonetheless, some research efforts have emerged that aim to address this gap, exploring innovative approaches to risk assessment that consider the complexities of autonomous ship operations, such as human–machine interaction, environmental variability, and the integration of advanced AI systems.

Wang, X. et al. [13] present a new method for assessing risks in human evacuation from cruise ships, relevant to both conventional and autonomous maritime systems. The authors critique traditional methods like FMEA for their limitations in dealing with the complexities of maritime evacuations. To address these gaps, they propose a model that combines FMEA with Bayesian Networks (BN) and Evidential Reasoning (ER), enhancing the accuracy of risk quantification and prioritization in complex scenarios influenced by human behavior and environmental conditions. Their study highlights the need for advanced risk assessment techniques to improve safety in maritime applications, particularly for autonomous operations.

Jovanović, I. et al. [14] review the research landscape in autonomous and unmanned shipping, highlighting technological advancements and key challenges, including regulatory barriers, cybersecurity risks, and the need for effective risk management. The authors identify critical gaps in research, such as the lack of standardized safety protocols and comprehensive risk assessment models for autonomous operations. They emphasize the need for future research to develop adaptive risk management strategies that incorporate AI-specific factors to enhance system resilience.

Feng, Y. et al. [15] propose an advanced machine learning approach for predicting the severity of marine accidents, improving the accuracy of maritime risk assessments. The authors highlight the limitations of traditional statistical methods in handling complex, nonlinear accident factors and develop a model that integrates diverse maritime data, such as weather, ship characteristics, and operational behaviors. Using feature selection and optimization algorithms, the model enhances predictive accuracy, offering a deeper understanding of accident severity. Their research underscores the importance of data-driven, advanced machine learning techniques in enhancing risk prediction and mitigation in maritime operations, including autonomous shipping.

Bolbot, V. et al. [16] review small unmanned surface vessels (USVs) and their impact on the safety of larger autonomous ships. They highlight small USVs as key testing platforms for developing safety protocols for larger vessels, addressing concerns like collision avoidance, system reliability, and human–automation interaction. The study emphasizes the importance of robust safety assurance frameworks and explores how insights from USV operations can inform safety measures for larger ships, particularly in regulatory compliance and risk management. The authors note significant gaps in safety assurance and stress the need for ongoing development of safety frameworks to enhance the reliability of autonomous maritime systems.

Li, W. et al. [17] utilized a hybrid framework combining STPA and Hidden Markov Models (HMM) to assess risks in the navigation of MASS. The authors highlighted that conventional risk assessment methods like FSA and FMEA face challenges in accounting for nonlinear interactions and system complexities in autonomous ship systems. To address this, they proposed using STPA to model safety control structures and identify Risk Influencing Factors (RIFs), which are then analyzed using HMM to infer hidden risks.

Hoem Åsa Snilstveit [18] examined the characteristics of five risk assessment methods through eight different publications on MASS. The study concluded that there are limitations in applying existing risk assessment methods to MASS due to the limited knowledge of past operational data or system structures. Furthermore, this paper highlighted that the importance of operational cooperation between humans and autonomous systems, depending on the level of MASS, poses a challenge for conventional risk assessment methods that fail to fully consider these interactions.

Chang, C.-H. et al. [19] pointed out that the FMEA method determines the Risk Priority Number (RPN) based on severity and likelihood but does not account for the weights of variables or the interrelationships among risk factors. They proposed a method that categorizes risk factors into six types, assigns weights to these factors through expert surveys, and uses a Bayesian network to prioritize risks.

Fan, C. et al. [20] proposed a framework called 4P4F to identify the RIFs that impact the safe operation of MASS. The framework identifies four operational stages—voyage planning, berthing and unberthing, entering and leaving port, and maritime navigation—and four types of elements: human, ship, environment, and technology. Through literature review and expert consultation, they defined 23 human-related factors, 12 ship-related factors, 8 environmental factors, and 12 technological factors that affect the safe operation of ships.

Lee, C., and Lee, S. [21] proposed a combination of AI risk causes and metrics for evaluation by referencing standards developed or being developed by the ISO/IEC JTC1/SC42 AI subcommittee and applying them to classification society guidelines from Det Norske Veritas (DNV) and the American Bureau of Shipping (ABS). Their paper noted that while the ABS and DNV guidelines specify requirements for role changes according to the level of autonomy, external environment, and hardware failures from a ship operation and maritime service perspective, they do not explicitly specify requirements for verifying AI performance and reliability or consider the unique characteristics of AI. The authors emphasized the importance of ensuring reliability with regard to AI’s specific characteristics.

Reviewing the existing literature shows that research on software safety in the maritime domain is increasing, particularly due to the emergence of MASS, highlighting the growing need for studies on both system safety and human–system interaction safety. Recent studies, including those by Wang, X. et al. [13], commonly argue that traditional risk assessment methods are inadequate for thoroughly considering the risks associated with MASS.

3. Proposed Method: RA4MAIS

To ensure the safety of systems with AI software in the maritime domain, it is essential to first conduct a risk assessment to identify potential risks and evaluate their severity. Among conventional risk assessment methods, HAZOP has the advantage of easily identifying risks using process variables and guide words; however, it is limited in providing detailed descriptions of risk scenarios. Conversely, STPA is effective in detailing risk scenarios due to its development for complex and safety-critical systems like defense systems, but its procedures are complex and challenging to apply to maritime transportation.

With the development of AI-integrated systems for realizing MASS, the concept of SOTIF has become essential to manage risks when intended functions are not properly executed due to hardware limitations or environmental changes. Additionally, to ensure safe navigation for MASS, it is necessary to consider risk factors related to AI from the risk assessment phase when creating safety requirements for AI-integrated systems. Therefore, this paper proposes a new risk assessment method, RA4MAIS, which integrates the concepts and advantages of four methods and standards: STPA, HAZOP, SOTIF, and AI, as illustrated in the conceptual diagram in Figure 2.

RA4MAIS adapts the risk factors presented in ISO 21448, originally developed for autonomous vehicles, to the maritime context. For example, the terminology used in the automotive sector differs from that in the maritime sector, and these terms have been adapted accordingly. While risks in the automotive sector often arise from misrecognition or failure to recognize road conditions, in the maritime sector, risks can emerge from environmental forces like waves or currents, interactions with various systems, and the control and maneuvering characteristics of ships due to their large mass. The risk factors also account for issues arising not only from environmental factors but also from AI performance deficiencies, lack of reliability, or improper training, as outlined in IEC 23894 (AI Risk Management). These selected risk factors are used to create a risk assessment support module by applying the concepts of process variables and guide words from the HAZOP method, which is then used to identify risk scenarios. The risk assessment support module offers a collection of terms for stakeholders, system functions, AI functions, behavioral characteristics, environment, and conditions by analyzing various literature and standards to help infer potential risks in AI systems in the maritime domain.

Using the risk assessment support module, risks are identified from a system function perspective, such as internal errors or human mistakes, and from an AI function perspective, such as AI characteristics or data quality. The two types of identified risks are further supplemented with environmental factors and situational conditions using the SOTIF concept from ISO 21448. Throughout this risk assessment process, the STPA method is used to systematically analyze and evaluate risks arising from interactions between components due to internal failures, environmental factors, human elements, AI characteristics, and data quality.

Until now, we have operated systems without fully understanding the potential risk factors that could cause accidents. However, by performing a risk assessment using the RA4MAIS method, we can identify as many causes of accidents as possible, helping to ensure the safe operation of the system. Figure 3 illustrates the risk assessment procedure developed according to the RA4MAIS concept, which integrates the SOTIF concept into the baseline procedures of the STPA method. It also shows how the risk assessment support module, incorporating the HAZOP concepts of process variables and guide words, can be used to analyze various AI-related risk factors.

The RA4MAIS method consists of four main steps. First, regulations are collected, and potential harms and hazards are defined. The second step involves identifying risk factors by diagramming the control structure using an AI-integrated basic control structure module and generating a list of potential risks using the risk assessment support module. The support module provides five types of modules, including the stakeholder module, which can be used selectively depending on the characteristics of the system being analyzed. In the third step, hazard actions are identified from either the system function or AI function perspectives. These two perspectives comprehensively cover the unique risks associated with traditional system operations and AI-specific factors. The system function perspective focuses on identifying hazards related to human error, internal functional failures, and operational inconsistencies commonly seen in conventional systems, ensuring that foundational safety aspects are addressed. Meanwhile, the AI function perspective captures additional AI-specific hazards, such as those related to AI reliability, performance variability, data quality, and lifecycle management, which are critical for understanding the unique challenges posed by AI technologies. The combination of system and AI function perspectives is an effective approach to capturing critical and distinct areas of risk, providing a balanced and hazard identification process. If the target system does not include AI, the hazard identification process from the AI function perspective can be omitted. The identified hazard actions are then combined with potential environmental factors or contextual conditions to generate interaction-based unsafe actions, which are subsequently analyzed for their causes. Finally, in the fourth step, the unsafe actions are evaluated for their risk degrees, and safety requirements are specified. The RA4MAIS procedure is detailed as follows.

3.1. Identify Expected Hazards

In Step 1.1 of RA4MAIS, relevant regulations that have been applied to the existing target system and those necessary for AI are collected. Then, in Step 1.2, the harms and hazards that may arise when AI is integrated to the existing target system are identified based on the collected regulations.

3.2. Identify Hazard Factors

In Step 2.1 of RA4MAIS, the control structure of the target system is diagrammed. The concept of a control structure may be unfamiliar and is often mistaken for a system structure diagram that shows the hardware or software components of the system, or a block diagram of components or functions. However, a control structure is not simply a combination of system components; it focuses on diagramming the control relationships between the objects issuing control commands and the objects receiving them. Therefore, the control structure must first identify and diagram the objects issuing commands and those receiving them, while also indicating the messages exchanged between them. The scope of the control structure should not be limited to the system’s hardware and software but should also include elements such as people, external systems, and environmental conditions that affect the system.

Next, in Step 2.2, a set of terms needed to hypothesize potential hazards and the situations in which they might occur, known as the risk assessment candidate pool, is generated. This process is similar to selecting process variables and guide words in HAZOP. Just as HAZOP facilitates hazard identification by combining process variables and guide words, Step 2.1 uses the diagrammed control structure of the target system and the risk assessment support module to generate the risk assessment candidate pool.

The risk assessment support module consists of five sub-modules: Stakeholder module, System function module, AI function module, Behavior type module, and Environment/Context module. The System function module includes characteristics related to the internal functions of the system, while the AI function module includes characteristics related to AI, such as performance, reliability, data quality, and lifecycle. The Behavior module includes actions of objects, such as providing and judging. The Environment/Context module includes various environmental conditions and contexts in which risks may occur. Depending on the system’s characteristics, all or some of these modules may be used. It is recommended to use all five modules for systems that include AI. For intelligent systems that do not include AI, the four modules excluding the AI function module are recommended.

3.3. Analyze Unsafe Actions

In this step, the control structure and risk assessment candidate pool are used to identify hazard actions from either the system function perspective or the AI function perspective. In other words, the identification of hazard actions involves determining “who is performing what action.”

Following Step 3A.1, system function-based hazard actions are identified by combining (i.e., concatenating) candidates from the Stakeholder candidate pool, System function candidate pool, and Behavior type candidate pool generated in Step 2.2, as shown in Figure 4. Stakeholders can be specific systems, equipment, or individuals, while actions can refer to the functions of systems or equipment or to human behaviors.

In Step 3B.1 of RA4MAIS, AI function-based hazard actions are identified by combining candidates from the Stakeholder candidate pool, AI function candidate pool, and Behavior type candidate pool, as illustrated in Figure 5. In this step, stakeholders are often represented by AI models, while actions refer to deficiencies in or misuse of AI characteristics such as performance and reliability. If the target system is an intelligent system without AI, this step can be omitted.

In Step 3C.2, not all hazard actions lead to accidents; accidents occur only under specific conditions. Thus, as shown in Figure 6, the system-based and AI-based hazard actions identified earlier are combined with multiple environmental and context factors from the Environment/Context candidate pool generated in Step 2.2 to derive interaction-based unsafe actions (UAs). This step enables the identification of unsafe actions that could occur even without internal functional errors, and it further specifies unsafe actions based on the environmental conditions or contexts in which they may arise.

Lastly, in Step 3C.3, the causes of unsafe actions that trigger harms are analyzed. The causes of unsafe actions can be intuitively identified based on the control structure and can result from interactions or environmental conditions.

3.4. Assessing Degree of Risk for Unsafe Actions

After analyzing interaction-based unsafe actions, Step 4.1 involves determining the risk grade. Various methods can be used to determine the risk grade, but it is generally calculated using a risk score derived from the likelihood and impact of the risk. Based on the risk score, a risk grade is assigned. Once the risk grade is assigned, Step 4.2 involves deriving safety requirements by comprehensively considering the risk grade of the unsafe actions and the budget allocated for system development.

4. Case Study

While it would be ideal to conduct a case study of the RA4MAIS on a real-world system where AI is integrated to perform its functions, this is challenging because such systems are often considered confidential by companies. Therefore, for this case study of RA4MAIS, we assume that the ECDIS includes a collision avoidance function as part of the MASS implementation, specifically corresponding to autonomous degree 2 of MASS RSE, where AI and humans interact, reflecting a human–machine interaction context. We further assume that AI technology is integrated to recognize objects and avoid collisions. Despite these assumptions, the data necessary for the case study were collected from prior research reports, academic publications, and interviews with developers from companies that develop ECDIS. The authors took a leading role in conducting the case study, working closely with these industry developers to apply the proposed method.

In this section, we will refer to this system as ECDIS with CA (Collision Avoidance). Although the actual structure of an ECDIS with CA would be highly complex, we will simplify its structure and functions for the purposes of this case study.

4.1. Identifying Expected Hazards

Following Steps 1.1 and 1.2 of the procedure, one of the regulations relevant to collision avoidance for ships—specifically one regulation from the International Regulations for Preventing Collisions at Sea (COLREGs) [22]—is identified, and one harm and two hazards are identified, as shown in

Table 3. Harms and hazards for ECDIS with CA identified in Step 1.2.

Rules	Harm	Hazard	Hazard Identifier
COLREGs Rule 5: Every vessel shall at all times maintain a proper look-out by sight and hearing as well as by all available means appropriate in the prevailing circumstances and conditions so as to make a full appraisal of the situation and of the risk of collision.	People are injured and the vessel is damaged due to a collision.	Failure to detect another vessel.	H1
		Failure to maintain a continuous lookout.	H2

[23,24].

According to Rule 5 of the COLREGs, a proper lookout must be maintained at all times to assess collision risks during ship operations. Failure to comply with this rule can lead to injuries to people and damage to the ship due to a collision. The causes of this harm were identified as either a failure to detect another vessel or failure to maintain a proper lookout continuously.

4.2. Identifying Hazard Factors

In Step 2.1 of the procedure, the control structure of the ECDIS with CA is diagrammed using the AI-integrated basic control structure module, as illustrated in Figure 7 [25,26,27].

The system collects navigational information using Global Navigation Satellite System (GNSS), gyro, speed log, and Automatic Identification System (AIS), and captures the surrounding environment using a camera. These data are transmitted to the controller via the sensor interface. The situation awareness model identifies objects based on the information provided by the controller and returns the results to the controller. The controller displays the electronic chart data, collected sensor data, and identified objects to the crew via the user interface. While an ECDIS with CA would typically require much more equipment and inter-system integration, this example simplifies the control structure to illustrate the use of RA4MAIS.

Next, in Step 2.2, a risk assessment candidate pool is generated using the risk assessment support module, as shown in

Table 4. Risk assessment candidate pool for ECDIS with CA generated in Step 2.2.

Category	Hazard Assessment Candidates
Stakeholder candidate pool	Crew, Camera, Situation Awareness Model
System function candidate pool	User Alerts, Data Measurement
AI function candidate pool	Data Quality—Completeness, Validity AI Performance—Precision, Recall AI Reliability—Controllability, Fairness
Behavior type candidate pool	Did not provide, Did not understand, Misunderstood, Misjudged, Failed to perceive due to lack of concentration, Intentionally ignored, Too low
Environment/Context candidate pool	Weather—Clear, Rain, Snow, Sleet, Fog State of Own Vessel—Acceleration State of Other Vessels—Approaching, Crossing, Overtaking Sea Objects—Other Vessels (Fishing Boats)

. This step is crucial, as it requires a detailed identification of system functions and environmental/context factors based on the diagrammed control structure. For the purposes of this case study, the system function candidates of the ECDIS with CA have been simplified, and easily understandable candidates have been selected to create the risk assessment candidate pool.

4.3. Analyzing Unsafe Actions

Following Step 3A.1 of the procedure, hazard actions are identified from the system function perspective using the control structure and the risk assessment candidate pool, as shown in

Table 5. Hazard actions based on system functions for ECDIS with CA identified in Step 3A.1.

Hazard Identifier	Stakeholder	Action
H1	Crew	Did not understand the user alert.
H1	Crew	Misunderstood the user alert.
H1	Crew	Misjudged the user alert.
H1	Camera	Did not provide data measurement.
H2	Crew	Failed to perceive the user alert due to lack of concentration.
H2	Crew	Intentionally ignored the user alert.

, based on the hazard factors selected in Step 2.2. When identifying hazard actions from the system function perspective, stakeholders are typically humans or devices/controllers performing specific functions, and actions generally involve commands or feedback.

For the hazard where a collision occurred due to a failure to detect another vessel (H1), system function-based hazard actions could include the crew misunderstanding a user alert or the camera failing to provide measurement data. Similarly, for the hazard where a collision occurred due to a failure to maintain a proper lookout (H2), hazard actions might involve the crew failing to recognize an alert due to lack of concentration or deliberately ignoring the alert.

Next, in Step 3B.1, unsafe actions from the AI function perspective are derived, as shown in

Table 6. Hazard actions based on AI functions for ECDIS with CA identified in Step 3B.1.

Hazard Identifier	Stakeholder	Action
H1	Situation Awareness Model	Precision of performance is too low.
H1	Situation Awareness Model	Recall of performance is too low.
H2	Situation Awareness Model	Completeness of data is too low.
H2	Situation Awareness Model	Validity of data is too low.
H2	Situation Awareness Model	Controllability is too low.
H2	Situation Awareness Model	Fairness is too low.

. In this context, stakeholders are typically AI-related models or systems, and the actions relate to deficiencies in AI performance and trustworthiness.

For the hazard where a collision occurred due to a failure to detect another vessel (H1), AI function-based hazard actions might involve the situation awareness model having too low a precision or recall. Similarly, for the hazard where a collision occurred due to a failure to maintain a proper lookout (H2), hazard actions might involve the data quality elements of completeness or validity being too low, or the controllability or fairness of the AI’s trustworthiness being inadequate.

Subsequently, in Step 3C.2, the results of the system function-based and AI function-based hazard actions are combined, and environmental/context factors in which the harms might occur are added, resulting in interaction-based unsafe actions (UAs), as shown in

Table 7. Unsafe actions based on interactions for ECDIS with CA identified in Step 3C.2.

Hazard Identifier	Stakeholder	Action	Condition	UA Identifier
H1	Crew	Did not understand the user alert.	Acceleration, Approaching	UA1
H1	Crew	Misunderstood the user alert.	Acceleration, Crossing	UA2
H1	Crew	Misjudged the user alert.	Acceleration, Overtaking	UA3
H1	Camera	Did not provide data measurement.	Fog, Crossing	UA4
H1	Situation Awareness Model	Precision of performance is too low.	Rain, Snow, Sleet, Fog, Other Vessels (Fishing Boats)	UA5
H1	Situation Awareness Model	Recall of performance is too low.	Clear	UA6
H2	Crew	Failed to perceive the user alert due to lack of concentration.	Acceleration, Approaching	UA7
H2	Crew	Intentionally ignored the user alert.	Acceleration, Crossing	UA8
H2	Situation Awareness Model	Completeness of data is too low.	Approaching	UA9
H2	Situation Awareness Model	Validity of data is too low.	Crossing	UA10
H2	Situation Awareness Model	Controllability is too low.	Fog, Approaching	UA11
H2	Situation Awareness Model	Fairness is too low.	Crossing	UA12

.

Unsafe actions occur either under specific contexts or when exposed to particular environments. For example, a misunderstanding of an error indication by the crew does not always lead to an accident. However, if the crew misunderstands the error indication while accelerating toward an approaching vessel, an accident may occur, which constitutes an unsafe action. Similarly, in adverse weather conditions like rain, snow, sleet, or fog, the situation awareness model may fail to detect an unregistered fishing vessel using a camera, leading to an accident; this is also considered an unsafe action.

In Step 3C.3, the causes of the 10 identified unsafe actions are analyzed, as shown in

Table 8. Causes of unsafe actions for ECDIS with CA analyzed in Step 3C.3.

UA Identifier	Cause of Unsafe Actions
UA1	Part of the alert information was presented as a code, making it incomprehensible.
UA2	The alert content was not communicated using clear terminology.
UA3	The impact of the alert was not properly communicated.
UA4	Objects were not captured due to fog.
UA5	Objects were not clearly detected due to rain, snow, sleet, or fog.
UA6	Objects were misidentified due to sea surface reflection on a clear day.
UA7	There is no method to convey important alerts when concentration is diminished.
UA8	The crew became desensitized to alerts due to repeated notifications, including those of lower importance.
UA9	Insufficient data were provided for learning about the side views of other vessels or docks encountered during docking.
UA10	The refinement of data for generating evasive paths in crossing situations was not correctly performed.
UA11	There is no function for the crew to forcibly intervene in decision-making when objects are not detected in fog and the risk situation is not assessed.
UA12	The AI was incorrectly trained to assume that small vessels are more likely to be pirate ships.

. For example, the reason for the crew misunderstanding the error indication was that the alert information was displayed as an error code rather than in a language understandable by the crew. Similarly, the reason the situation awareness model failed to perceive the situation was due to rain, snow, sleet, or fog, which obscured the camera’s view of other vessels.

4.4. Assessing Degree of Risk for Unsafe Actions

Following Step 4.1 of the procedure, risk scores and risk degrees are calculated by assigning likelihood and impact values to the unsafe actions, as shown in

Table 9. Degree of risk for unsafe actions for ECDIS with CA estimated in Step 4.1.

UA Identifier	Likelihood	Impact	Risk Score	Degree of UA
UA1	3	2	6	Medium
UA2	4	2	8	Medium
UA3	4	2	8	Medium
UA4	4	4	16	High
UA5	4	4	16	High
UA6	4	3	12	High
UA7	4	3	12	High
UA8	4	4	16	High
UA9	3	4	12	High
UA10	4	4	16	High
UA11	3	5	15	High
UA12	3	5	15	High

. The values for likelihood and impact are indicative and may vary depending on the system’s importance or the objectives of the group conducting the risk assessment. In this case study, the cybersecurity guidelines for ships issued by the Baltic and International Maritime Council (BIMCO) were referenced [28].

Likelihood quantifies the probability of an event occurring, with scores ranging from 1 (highly improbable) to 5 (frequent occurrence). Impact represents the severity of the damage caused by an event, with scores ranging from 1 (no significant impact) to 5 (catastrophic impact, including fatalities or severe damage). The risk score is calculated by multiplying the likelihood and impact values. Based on the risk score, the risk degree is categorized as follows: a score of 1–5 indicates low risk, 6–10 indicates medium risk, 11–19 indicates high risk, and 20–25 indicates extreme risk.

For example, for the unsafe action where the crew misunderstood an error indication (UA1), the likelihood was rated at 3 because it could occur due to occasional equipment failure or human error, while the impact was rated at 2 due to the potential for minor injuries. The risk score, obtained by multiplying likelihood by impact, was 6, which places this UA at a medium risk degree.

Next, in Step 4.2, based on the assessed risk degree, it is determined whether the unsafe actions should be translated into safety requirements. Although this decision depends on the project budget and circumstances, for illustration purposes, all unsafe actions were converted into safety requirements, as shown in

Table 10. Safety requirements for ECDIS with CA identified in Step 4.2.

Hazard Identifier	UA Identifier	Safety Requirements
H1	UA1	Alert information should be conveyed as a message, and error codes for developers should be stored/displayed separately.
H1	UA2	The content of alerts should use clear terminology and be expressed in a way that avoids misunderstandings.
H1	UA3	Alerts should include information on potential impact and damage to help the crew assess the importance of the alert.
H1	UA4	An alert message should be displayed to allow camera settings to be adjusted in crossing or heavy fog situations.
H1	UA5	Sufficient model training should be conducted for conditions such as rain, snow, sleet, and fog.
H1	UA6	Image processing to remove reflections and model training on reflections should be conducted.
H2	UA7	Important alerts should be displayed visually and audibly to attract user attention and remain visible until acknowledged.
H2	UA8	The importance of alerts should be determined, and the level of alert display should be set according to the time of day.
H2	UA9	Model training should include not only the general appearance of vessels but also the side views of nearby vessels and docks.
H2	UA10	When generating training data, the scope and format of data required for path avoidance in crossing situations should be defined and processed accordingly.
H2	UA11	A function should be provided that allows an authorized crew member to intervene and determine whether a situation is normal or abnormal if there is an error in situation assessment.
H2	UA12	The characteristics of pirate ships should be learned not by the appearance of the vessel, but by the shape of the crew or the items they are carrying.

.

For the unsafe action where the crew misunderstood an error indication (UA1), a safety requirement was derived to display error information in a user-friendly language instead of an error code. For the unsafe action where the situation awareness model failed under severe weather conditions (UA5), a safety requirement was derived to ensure sufficient model training for conditions such as rain, snow, sleet, and fog.

By applying safety requirements from UA1 to UA6 in the design and implementation phases, the risk of collision due to the failure to detect another vessel (H1) could be significantly reduced. Similarly, considering and implementing safety requirements from UA7 to UA12 could reduce the risk of collisions due to the failure to maintain a proper lookout (H2), consequently decreasing the loss of life and property damage caused by violations of Rule 5 of the COLREGs.

5. Limitations

While the RA4MAIS method proposed in this paper offers a structured approach to identifying risks in AI-integrated maritime systems, it has certain limitations. One significant limitation is that the effectiveness of RA4MAIS has not yet been verified in real-world field operations. As with many methodologies in software engineering, validating its impact quantitatively remains challenging because the effectiveness of risk assessment is largely dependent on human judgment and interpretation.

Conventional risk assessment methods like HAZOP have been validated over time through extensive empirical studies and application across a wide range of systems. These studies have established their effectiveness in identifying and mitigating risks through consistent use in diverse industrial contexts. RA4MAIS, being a relatively new approach, has not yet undergone the same level of empirical validation. To build similar credibility, RA4MAIS must be tested through empirical studies that apply the method to real systems, particularly those currently under development, such as MASS in Korea and other AI-integrated maritime technologies.

To objectively assess the effectiveness of RA4MAIS, it is essential to conduct more empirical studies involving actual applications in various AI-integrated systems. These studies should aim to apply RA4MAIS to real-world scenarios, including those described in the literature review, to gather evidence of its practical benefits and limitations. By doing so, the method can be systematically refined and further developed to address any identified shortcomings. As the method is applied to more systems and its use becomes more standardized, it will be possible to refine its modules and processes, enhancing its overall reliability and effectiveness.

Another limitation of the RA4MAIS method is its focus on the functional safety of AI-integrated maritime systems, which does not encompass the equally critical aspect of cybersecurity. While RA4MAIS provides a structured approach to identifying risks related to system operations, human interactions, and environmental factors, it does not address cybersecurity threats, which are increasingly significant for autonomous ships. Cybersecurity involves a complex set of challenges, including protecting data integrity, securing communication networks, and preventing unauthorized access, all of which are essential to the overall safety of AI-integrated systems. Expanding the scope of RA4MAIS to include cybersecurity considerations would be an important area for future research, as it would provide a more comprehensive risk assessment framework that addresses both functional and cyber-related risks. Therefore, while this study focuses on system functionality, we recognize that integrating cybersecurity measures is necessary to fully ensure the safety of AI-integrated maritime systems.

While RA4MAIS offers a systematic approach to identifying a broad range of risks, including system functional failures, human errors, environmental conditions, and AI-specific concerns such as performance and trustworthiness, it is still in the early stages of development and requires further empirical validation to fully establish its effectiveness. To realize its potential as a reliable risk identification tool, RA4MAIS needs to be tested extensively in real-world applications, particularly under the complex and variable conditions faced by AI-integrated maritime systems. This will involve expanding the scope of case studies and engaging in collaborations with industry stakeholders to evaluate the method’s performance in actual operational environments. Through ongoing empirical testing, refinement, and comparison with established methods such as HAZOP and FMEA, RA4MAIS can evolve into a more comprehensive tool that addresses both functional and cybersecurity risks.

6. Conclusions

The goal of the RA4MAIS method proposed in this paper is to efficiently identify risks that may arise when advanced automation systems or AI-integrated systems are implemented on ships due to the MASS paradigm shift. RA4MAIS aligns with the risk identification step of FSA, addressing limitations in traditional methods by providing an enhanced approach to identify risks specific to AI technologies, such as algorithmic failures and human–AI interaction challenges. RA4MAIS provides a comprehensive framework for identifying potential risks in AI-integrated maritime systems by incorporating the SOTIF concept into the STPA approach. This integration allows for the specific identification of situations where risks may occur, taking into account factors such as human interactions, environmental conditions, and AI-specific characteristics. Additionally, RA4MAIS incorporates elements from the HAZOP method, which helps guide users—particularly those with limited experience in risk assessment or IT—to systematically identify AI-related risks and potential hazardous scenarios.

The RA4MAIS approach was demonstrated through a case study on the collision avoidance function of an ECDIS. This case study, set in a scenario of Degree 2 or Degree 3 autonomy, illustrated how RA4MAIS could be used to identify risks related to AI performance issues such as precision and recall, as well as reliability concerns like controllability and fairness. It also highlighted specific situations where risks could emerge, such as vessel encounters or challenging environmental conditions like rain and fog. The case study served as a practical demonstration of the RA4MAIS method, showcasing its applicability in identifying potential risks, but it does not yet fully validate the effectiveness of the method in field operations.

It is important to note that RA4MAIS has not yet been fully validated through empirical studies in real-world environments. Due to the developmental stage of AI-integrated systems and confidentiality concerns within companies, it remains difficult to conclusively demonstrate the effectiveness of RA4MAIS. Future research should focus on conducting further case studies and empirical testing on actual systems, such as those under development in Korea for MASS, as well as other AI-integrated systems like propulsion and power management systems. Applying RA4MAIS to these real-world systems, including those highlighted in the literature review, will provide valuable insights and data for its empirical validation.

RA4MAIS has the potential to be integrated into the software development lifecycle of AI-integrated maritime systems, providing a structured way to translate identified risks into safety requirements that can influence system design and implementation. By incorporating RA4MAIS early in the development process, developers could proactively address identified risks, embedding safety measures into the system design rather than dealing with them as afterthoughts. This integration would support a more systematic approach to managing potential hazards, ultimately contributing to the development of safer and more reliable AI-integrated maritime systems.