*3.2. Improvement in the RPN Technique and Improvement of the Evaluation Method Using Kendall's Concordance Coe*ffi*cient*

To improve the problems that occur in RPN evaluation using FMEA and derive objective results, as shown in Figure 3b, this study precisely identified the potential failure types matching the characteristics of the fuel cell-based hybrid power system for ships and analyzed the RPN evaluation criteria. Figure 3a shows the process for determining the existing RPN evaluation items, and Figure 3b shows the process for determining the RPN evaluation items applied in this study.

**Figure 3.** (**a**). The process for determining the existing RPN evaluation items. (**b**). The process for determining the RPN evaluation items applied in this study.

Minimizing differences between the results of various evaluators can increase RPN evaluation reliability. To increase the reliability of the evaluation results, team members with a certain amount of experience in specialty fields were selected for the FMEA team in this study. They performed a system analysis by function. The FMEA team is aware of the problems with existing FMEA because it has been working in the field for a certain period of time and selected experts with basic experience in FMEA evaluation. Therefore, we understand the importance of FMEA evaluation criteria and item setting.

The composition of the FMEA team and the criteria for selecting experts are as follows:


Based on the functional analysis of potential failures, this study designed clear evaluation criteria for S, O, and D. The existing effects of potential failures were identified, then the RPN evaluation criteria were created, and an evaluation was immediately performed. However, when creating the RPN evaluation criteria, this study identified the effects of the potential failures of S, O, and D. The reliability of the evaluation criteria were then confirmed, and the criteria were established using the following procedure.

First, the evaluation items for S, O, and D were established, after which the following research hypothesis for the evaluation items was set: "the evaluation scores by item of the evaluators will be similar.". The FMEA team then performed its own internal evaluation, confirming the significance probability results for the reestablished evaluation items and validating the research hypothesis. Next, the team RPN internal evaluation results were compared with Kendall's concordance coefficient to determine the reliability of each evaluation item. In this paper, Kendall's coefficient of consensus mentioned to verify the reliability of the evaluation items is one of the methods used in nonparametric statistics to analyze the relationship between phenomena measured on the sequence scale [35]. Kendall's coincidence coefficient is typically used for attribute agreement analysis, with coefficient values ranging from 0 to 1. The higher the value of the coefficient, the stronger the association. If the coefficient is greater than 0.9, the relevance is considered very high and the high or significant Kendall's coefficient means that the evaluators apply essentially the same standard when evaluating the sample [36]. Applying the same criteria decreases the ambiguity of the evaluation items, removing arbitrariness and encouraging objectivity. Then, the significance probability of the evaluation criteria items and the results of Kendall's concordance coefficient were determined. If the reliability of the evaluation criteria was lower than the threshold, then the process returned to the previous steps to identify the effect of potential failures; once the reliability of the evaluation criteria reached the threshold, the evaluation criteria was confirmed.

This final evaluation criteria were then used as the basis to assess the external evaluators. Finally, by comparing the results with the existing evaluation criteria, this study numerically confirmed the high reliability of the reestablished evaluation criteria.

#### **4. FMEA Methodology of This Study**

#### *4.1. FMEA Procedure of This Study*

According to the IEC 60812 standard, the FMEA procedure can be divided into three steps: the preparation, performance, and finishing [37].
