Enhanced FMEA Methodology for Evaluating Mobile Learning Platforms Using Grey Relational Analysis and Fuzzy AHP

Abstract

This study addresses a significant problem: it is difficult to choose a suitable mobile learning platform effectively since many learning platforms are readily available for users. For this purpose, the study proposes an efficient way to rank and choose the most suitable mobile learning platform by integrating risk analysis and multi-criteria decision-making methods. The selection of a suitable mobile learning platform is challenging due to the vast collection of available platforms. Traditional decision-making approaches often struggle to manage the inherent uncertainty and subjectivity in platform evaluation. To address this, we propose an enhanced methodology that combines grey relational analysis (GRA) and fuzzy analytic hierarchy process (FAHP), leveraging their complementary strengths to provide a robust and adaptive solution. The study employs ISO/IEC 9126 software quality standards to locate the most suitable mobile learning platform. FMEA is based on three risk factors: occurrence, severity, and detection. The fuzzy analytical hierarchy process (FAHP) is used to determine the relative weight of each risk factor to identify the grey risk priority number that can be calculated for each criterion. Mobile learning platforms are then ranked based on their grey risk priority number. The method was applied to five widely used mobile learning platforms with three decision-makers. In addition, the multi-criteria decision-making software was developed to aid users, educators, and administrators in their decision-making processes. The integrated FMEA-GRA-FAHP technique, using ISO/IEC 9126 standards, provides an effective way of locating the most suitable mobile learning platform and ranking them according to their reliability. This research is believed to be the only study applying an integrated FMEA-GRA-FAHP approach to evaluate the risks and quality of mobile learning platforms. The unique approach overcomes certain limitations of the standalone methods such as FMEA and FAHP, making it a valuable tool for identifying the suitability of mobile learning platforms. In addition, the study underscores the importance of inclusivity and equity in ensuring high-quality education and creating an environment conducive to lifelong learning for all.

1. Introduction

Mobile devices have played a significant role in daily life, particularly an inevitable part in improving the quality of education [1]. Students engaged in remote education and open learning have been frequently interacting with mobile learning technologies [2]. Mobile application-based platforms entail an approach of well-organized content, pedagogy, and quality delivery systems, providing the reason why mobile phones are gaining popularity. Mobile learning platforms (MLPs) should make it easier to access learning, reporting, and content management on the available mobile devices. This provides a safe and convenient learning environment. MLP has a lot of advantages in the area of learning in terms of flexibility, mobility, just-in-time learning, retention, tracking and reporting, and continuous learning experience.

Mobile learning platforms, which can also be synonymous with e-learning platforms, provide users with both technical and educational features. Due to the increase in demand and also in the diversity of such systems, the selection of the best platform that meets users’ required priorities is a challenging issue. It was stated that, in 2020, finding mobile learning applications of high quality apparently is a difficult task, considering that there are more than 567,000 educational applications already available (retrieved from https://www.educationalappstore.com/app-lists/apps-for-education (accessed on 20 September 2020)).

According to the research conducted by Google, 80% of the world’s population uses smart mobile phones. Users spend 91% of their time using mobile applications. 30% of the users use their smartphones to access mobile learning application-based platforms. It was reported that mobile learning platforms increase efficiency by 43%. Statistics evidenced the substantial importance and potential of mobile learning platforms (retrieved from https://www.edume.com/blog/mobile-learning-statistics (accessed on 19 July 2024)).

Therefore, quick, feasible solutions are required to choose the most suitable mobile learning platforms. Determining suitable mobile learning platforms can be treated as a multi-criteria decision-making (MCDM) problem. Therefore, there is a need to de-vise an independent way to select and evaluate MLPs. For this purpose, utilizing FMEA and FAHP approaches in evaluating MLPs will produce more reliable and efficient and timeliness results.

The fuzzy analytic hierarchy process (FAHP) is an elaboration of the AHP technique that utilizes numbers, fuzzy logic, and sets. FAHP has also been used in areas like evaluating learning management systems and platforms as well as in e-learning. Based on users’ views, FAHP has been utilized to make e-learning a decisive component in higher education [3]. FAHP was also used by [4] to provide expert evaluation for choosing appropriate learning management systems.

Various software quality models have been designed for the sole purpose of assessing software qualities through ISO standards such as ISO/IEC 9126 and ISO/IEC 25010. Regarding the software quality model review by [5], the ISO/IEC 9126 model is introduced as the optimal and most efficient model used for the evaluation of software qualities. The mobile learning platform is a software-based system designed to facilitate and deliver educational content and learning experiences via mobile devices such as smartphones, tablets, and other portable technologies. ISO/IEC 9126 is an international standard for the evaluation of software quality-in-use. It provides both technical and user-centered characteristics of the system, in this case, the mobile learning platform (MLP). In this research, we focused on the quality-in-use characteristics of the ISO/IEC 9126 standards with the consideration that the evaluated metrics should also be accessible for evaluation by the experts. The scope of the evaluation is limited to the following characteristics. The linkage between metrics and MLPs is given as follows:

Suitability under functionality is the main characteristic of which MLP performs it is required tasks, such as meeting the specified educational needs, such as course delivery, user interaction, and content management.

Understandability, learnability, operability, and attractiveness under the main metric usability indicate the extent to which learners interact with MLP, encompassing these attributes, such as whether users understand MLP effortlessly and how users learn to use MLP easily. This metric focuses on user-centered aspects of the MLPs. This constitutes another rationale for selecting ISO/IEC 9126 standards for the evaluation framework of the study. Since this aspect is believed to have a critical impact on the learning performances. Usability metrics according to ISO 9126 are used for predicting the extent to which the mobile learning platforms in this study can be understood, learned, operated, and attractive.

Fault tolerance under reliability’s main characteristics shows to what extent MLP could perform correctly and consistently with minimized failures and handle the errors well.

Time behavior under efficiency’s main characteristic is to what extent MLP performs smoothly with minimal delay.

For the reasons mentioned above, MLPs can be evaluated using the ISO/IEC 9126 software quality characteristics [6]. Risk management and quality reliability in laboratories require the use of failure modes and effects analysis (FMEA) in its dynamic procedure [7]. FMEA was applicable in two different stages; in the first phase, the potential failure mode was selected, and assessment and analysis were carried out on the probable risks.

In order to identify underlying failures and their effects as risks, risk priority numbers (RPNs) were employed. In the later stage, plans are proposed to implement and quantify those failures’ impact.

While in the series of making choices out of varieties of decision-making, the popular AHP approach is very well known and is also, in fact, the most utilized approach for these types of decision-making processes in which application rankings are based on certain attributes that are expressed with the help of the utilization of distinctive scales.

In this study, we aimed to improve the FMEA risk assessment methodology by combining it with GRA (Grey relational analysis) for making decisions based on incomplete information and FAHP (fuzzy analytic hierarchy process) to assign varying weights to risk factors as occurrence, severity, and detection, which was not possible with conventional FMEA and ranking alternatives. GRA is highly helpful when making decisions in situations when there is uncertainty and inadequate information.

It is expected that some platform performance measures will have incomplete or unclear data when evaluating MLPs. Since GRA compares alternatives’ relative performance across multiple factors, it effectively handles alternatives’ similarity in such settings, making it a great fit for our study. In GRA, results do not need to be normalized.

The FAHP technique is used when dealing with language issues and subjective evaluations. When expert reviews of MLPs incorporate subjective evaluations of usability, usefulness, dependability, and efficiency, this becomes significant. FAHP allows us to generate a more realistic and trustworthy evaluation by capturing the inherent ambiguity and imprecision in these expert opinions.

FAHP is designed to handle complex decision-making scenarios where a large number of factors are graded using expert opinion. Using fuzzy scales and pairwise comparisons, professionals can systematically create weights for each criterion using FAHP, which represents the relative relevance. This strategy ensures that expert knowledge and priorities are accurately represented in our review process, which is crucial for a comprehensive assessment of MLPs.

Additionally, FAHP is used in this work to assign weights to risk factors that cannot be evaluated by FMEA or GRA-FMEA techniques alone, such as occurrence, severity, and detection. Popular MCDM method TOPSIS identifies choices that are closest to the ideal solution; however, it performs best with precise data and may not perform as well with ambiguous or subjective information. In our research, fuzzy data are used to capture the ambiguity and vagueness present in expert assessments, making FAHP a superior choice.

Furthermore, normalizing data is necessary for TOPSIS, which could make comparing criteria with various scales and units more difficult. Another MCDM method that focuses on outranking relationships between options is called ELECTRE. It is frequently more appropriate in situations where there are obvious differences in the performance levels of the available options and can be computationally complex. In contrast, our study addresses platforms that might exhibit similar performance across several criteria, necessitating the use of a method such as GRA that can handle these nuances efficiently and offer a more nuanced ranking based on relative closeness.

MCDM is a mechanism for the determination of the best option from several options, given that multiple alternatives are, in most cases, conflicting decision criteria [8].

This study addresses the essential problem of difficulty in choosing the most suitable mobile learning platform since there are abundant learning platforms readily available. For this purpose, the study proposes an efficient way to rank and choose the best quality mobile learning platforms by integrating risk analysis and multi-criteria decision-making methods. This research is believed to be the only study applying an integrated FMEA-FAHP approach to evaluate the risks and quality of mobile learning platforms. What makes this study unique in comparison to other relevant studies is the fact that it implements failure mode effects analysis (FMEA) in collecting and evaluating data collected from various mobile learning platforms, which are then weighted using the fuzzy inference system (FIS) and finally ranked using the fuzzy analytic hierarchy process (FAHP).

The significant contributions of this study toward selecting a suitable mobile learning platform are given below:

• The proposed method uses unique, integrated risk analysis and multi-criteria decision-making approaches to overcome the shortcomings of the traditional FMEA approach. An integrated FMEA-GRA-FAHP method is a combination of risk analysis and multi-criteria decision-making methods for the suitability of mobile learning platforms. The initial implementation of Grey relational analysis involved assuming that risk factors carried equal weights. In the subsequent application, distinct weights were assigned to risk factors through the utilization of Fuzzy AHP, as determined by evaluations from experts. This represents a significant advancement in the realm of failure mode and effect analysis (FMEA), as the introduction of varied weights through Fuzzy AHP introduces a novel approach to the FMEA methodology.
• To researchers’ knowledge, this is the only known study to present the FMEA-GRA-FAHP technique for choosing the top mobile learning platform.
• The evaluation’s results indicate that the proposed approach is an effective way to find the most suitable mobile learning platform to use.

Therefore, this study aims to employ a multi-criteria method such as FMEA-GRA-FAHP to evaluate and rank the five selected MLPs. Furthermore, the study adopts the technical requirements concerning the ISO/IEC 9126 model to generate a suitable framework for the effective evaluation of the selected MLPs using such techniques.

The research objectives of the study are presented as follows:

The study aims to propose an efficient way to rank and choose the most suitable mobile learning platform by integrating risk analysis and multi-criteria decision-making methods to remedy the difficulty of selecting an adequate mobile learning platform among many that are available.

• Develop and propose a framework that can be effectively used to assess the potential risks and the quality of the mobile learning platforms based on the ISO/IEC 9126 software external quality characteristics.
• Incorporating FMEA-FAHP with the use of Grey relational analysis methods to rank the five most widely used mobile learning platforms that were selected based on the Google Play Store ranks.
• In order to aid decision-makers and educators in effectively selecting suitable mobile learning platforms a standalone software was developed.

2. Related Research

There is a long list of MCDM techniques, such as SMART, FAHP, ANP, ELECTRE, TOPSIS, etc. Normally, different methods are merged to enhance the process of decision-making.

This study aims to employ two multi-criteria methods, which include FMEA and FAHP integrated with grey relational analysis, to evaluate and rank the five selected MLPs. Furthermore, the study adopts the technical requirements concerning the ISO/IEC 9126 model to generate a suitable framework for the effective evaluation of the selected MLPs using a combination of the FMEA and FAHP techniques.

The Fuzzy AHP (FAHP) method is an extension of AHP which was developed by Saaty in 1975 [9]. The FAHP combines the fuzzy set theory and also the analytical hierarchy process, which makes it a more powerful approach for remedying problems that require multi-criteria decision-making (MCDM) approaches. FAHP is generally used as a decision-making method of prioritizing values for different experts in an MDCM [10]. In their study, they employed the Fuzzy AHP method to generate weights for risk factors. Different techniques exist for obtaining the weights from fuzzy pairwise comparison matrices. FAHP leverages a range of values instead of a range of crisp values to deal with the vagueness, uncertainties, and ambiguities associated with the decision-making process for decision-makers [10]. FAHP is a decision-making method used to analyze and rank alternatives in a hierarchal structure. The overall objective was to prioritize experts considering three different alternatives as experts (D1, D2, and D3) based on three criteria: experience, education, and individual level. The majority of Fuzzy AHP applications use the simple extent analysis method proposed by [11] because fuzzy weights are difficult to compute in the form of crisp weights. Research in various literature shows that a very effective FAHP approach is the extent analysis method. This method was introduced by D.Y. Chang in 1992. Da-Yong Chang, in his paper, vividly illustrates the application of the FAHP extent analysis method for introducing the synthetic extent value S of a pairwise comparison by applying the principle of the comparison of fuzzy numbers and also demonstrates the decision process by an example. The extent analysis method was also used [12]. They generated fuzzy synthetic extent values and applied the method of comparison of fuzzy numbers in calculating the normalized weight vectors. These vectors were used to obtain the final score of each student for breaking a tie situation and deciding the rank among the students in the case of having the same marks in an examination.

The failure modes and effects analysis (FMEA) is used to identify hazards in requirements or possible failure of software. FMEA is used to estimate the risk priorities of failure modes associated with an event or product. In the case of this study, FMEA is used to evaluate the potential failure of MLPs through the risk priority number (RPN) value. The risk priority number (RPN) is the estimated numerical value of the assessment of risk assigned to a process or steps in a process as part of its FMEA. RPN is calculated by multiplying the risk factors occurrence (O), severity (S), and detection (D) of a potential failure. [13] in his paper, he proposed the possible use and application of FMEA in the analysis stage by identifying potential hazards in a bid to develop software defenses by developing fault-tolerant or self-checking techniques to reduce the probability of their occurrence. Also, researchers in [14] used failure mode and effect analysis (FMEA) as a tool for evaluating Enterprise architecture (EA) risks. However, due to some drawbacks of the traditional FMEA, as stated by their research, instead of calculating RPN—the study prioritized EA risk factors with fuzzy VIKOR. Researchers in [10] employed the use of FMEA to compute the RPN value of Maintenance Policy Selection (MPS) of equipment using three dimensions, including Severity, Occurrence, and Detection. However, they also used FAHP to assign weights to the identified sub-dimensions, which is a further indication that the traditional FMEA process is not sufficient for ranking in an MCDM process. Due to the drawbacks associated with the traditional FMEA process, the traditional RPN calculation in the FMEA process does not consider the significance of individual risk factors to the peculiarity of the processor type of software being used. FAHP is therefore employed to enable us to assign priority to individual risk factors (severity occurrence and detection).

Various studies have employed the use of GRA- FAHP-FMEA in different aspects of evaluation research. However, before this study, no research has employed this approach in evaluating mobile learning platforms. Researchers in [15] proposed a model that involves the combination of the fuzzy analytic hierarchy process (FAHP) with FMEA to obtain accurate and timely results when carrying out maintenance, particularly in the paper industry. The model was applied to a numerical sample to show its efficiency. Researchers in [16] also proposed a method for analyzing the risks of green components in the electronic manufacturing process in a bid to improve quality control. They used FMEA to analyze the failure modes and effects of green components and used FAHP to determine the relative weightings of four factors. This helped them to calculate a green component risk priority number (GC-RPN) for each of the components that a supplier provided to identify potential risks and manage them appropriately. The most frequently used MCDM methods were summarized as TOPSIS, ELECTRE, and FAHP in [17].

The core features in the mobile learning platform must be for the MLPs to be able to offer learning irrespective of location and time. Likewise, knowing the best method for the selection criteria is another problem, and manual selections among several MLPs are both tiresome and time-consuming, which leads to poor choices. Therefore, there is a need to devise a means of independently selecting and evaluating MLPs. For this purpose, utilizing FMEA-FAHP approaches in the selection and evaluation of MLPs provides time efficiency, optimum results, and the best technique for future study. Hence, locating the most suitable mobile learning platforms using a uniquely developed framework is proposed from the standardized model of ISO/IEC 9126.

The number of studies located in the context of mobile learning literature that applied either multi-criteria decision-making or machine learning approaches is limited. There are no located studies that apply grey relational analysis or FMEA techniques in the context of mobile learning.

A study used the analytical hierarchy process (AHP) to determine the critical success factors for the successful implementation of cloud-based mobile learning as Pedagogy, Mobile Compatibility, Data Security, Learning Management, Content Quality, Portability, and Support for Teachers [18].

Another study evaluated and ranked the several sub-factors of cloud-based mobile learning through the fuzzy analytic hierarchy process (FAHP) and analytical hierarchy process-group decision-making (AHP-GDM) [19]. Another study employed a machine learning-based algorithm to select the most suitable mobile learning application for children by using textual analysis of app reviews [20]. Another study uses fuzzy-TOPSIS group decision-making with 25 criteria and 125 students as decision-makers to rank educational applications [21].

The diversity and ease of availability of mobile learning applications encouraged researchers and educators to use mobile learning platforms to enhance teaching and learning [22]. The mobile learning platforms are another perspective of implementing e-learning, which uses both the technical and non-technical ability of mobile devices to deliver content to users. In some situations, MLPs are taken as simple extensions of e-learning, which makes it another channel for delivering content; in fact, quality mobile learning can only be delivered properly when special limitations and benefits of mobile devices are acknowledged [23]. Instead, it is relevant to consider features special to M-learning when putting into perspective how to offer a quality learning experience to users [23].

Even though the International Standard Organization modeled standards (ISO/IEC 9126) for both technical and non-technical quality features, researchers also recommend that studying mobile learning platforms should be considered based on two perspectives, which are Technical and Non-Technical Approaches to the quality of mobile learning platforms. In the study, both the technical and non-technical aspects were considered within the scope of the study [24].

Platforms that aid mobile learning are defined by technologies, and the learning approach is specified by pedagogy or non-technical approach [25]. Many studies applied ISO/IEC 9126 quality-in-use standards to evaluate e-learning and mobile learning environments. In one study, ISO/IEC 9126 was linked and used in the evaluation of e-learning systems [26,27]. Meanwhile other studies focused on proposing usability aspects similar to the usability characteristic of the ISO/IEC9126 quality model for evaluating mobile learning applications [28,29].

The International Standard Organization, ISO/IEC 9126, developed a model for assessing the quality of software products such as mobile learning platforms. This model for assessing mobile learning platforms is based on criteria such as functionality, accessibility, usability, efficiency, maintainability, dependability, quality, and maintainability criteria. Researchers have developed several models to evaluate MLPs in [30,31,32,33,34,35], still, the most widely used model in research and software platform developments is ISO/IEC 9126 [36].

The quality model can be characterized as the set of features and the relationship that gives the premise for indicating quality necessities and assessing item quality. Different works have been discovered in literary research centered on program item assessment. Among the most acceptable models include McCall, Boehm, FURPS, Dromey, Bayesian, and ISO/IEC 9216 [37].

The ISO/IEC 9126 is a standard developed by the ISO for evaluating software reliability. It has proven to be very efficient in evaluating software in a bid to verify its reliability and durability after production. Its fundamental objective is to address familiar human biases that can adversely affect the perception and delivery of a software development project. Biases can range from not having a clear definition of success to a change of priorities after the start of the project. It consists of six major criteria, each with sub-criteria. These are the yardsticks used for the software evaluation process. A lot of research has employed this standard in developing a working framework for their software analysis research. Researchers in [38] sifted 21 out of the 32 software quality criteria of ISO/IEC 9126 standard in optimizing Enterprise resource planning (ERP) software problems. Also, researchers in [39] proposed a framework for evaluating both the quality and user satisfaction of Mobile learning applications for mathematics (MLAM). They used the ISO/IEC 9126 standard to generate a model for evaluating the technical aspect, which was combined with user satisfaction proposed for the non-technical aspects. In another study, ELECTRE I and quality standards rank Mobile Learning Applications for Mathematics [24].

Quality properties are classified into various leveled tree structures of features and sub-features. The most noteworthy level comprises quality characteristics, and the least level comprises quality criteria. ISO/IEC 9126 indicates six characteristics that are separated into twenty-one sub-characteristics. These sub-characteristics are shown remotely when the computer program is utilized as a portion of a computer framework, and then internal quality is achieved. The core merit of this model is that the characteristics stated are pertinent to each kind of computer program, giving steady wording for software item quality [40].

Conclusively, ISO/IEC 9126, since it is based on past activities and models, is more finalized than other models and reasonable to be utilized within the assessment of MLPs. ISO/IEC 9126 engages all pivotal attributes such as progressive structure, criteria for assessment, comprehensive expression and terms, basic and exact definitions, and one to numerous connections between distinct layers of the model. In expansion, ISO/IEC 9126 aids key decision-making efforts and also exempts expensive errors.

Functionality is chosen based on the fact that it is dependent on the application domain (education), while reliability focuses on information presentation and content in academic products. Usability is also included because it is a vital figure, particularly for client-driven applications. Productivity is additionally added because it refers to the potential of the computer program to supply utilizable work to realize its purpose. Viability, be that as it may, is cleared out according to the framework as it may be assessed either by the designer or a third party when getting to the project’s technical documentation and the source code. Even though most products sense maintainability as a critical quality attribute, only in the early stages of development were assessed. This can be in line with computer program product assessment within the scholastic space, which makes maintainability and portability less vital features.

Upon reviewing the literature, certain studies were located to apply FMEA in combination with other techniques but in different contexts. For instance, one study applied FMEA combined with Pythagorean Fuzzy Sets (PFS) and Dimensional Analysis (DA) risk assessment in product design [41]. Meanwhile, another study employed the integrated Fuzzy-GRA-TOPSIS-FMEA method to investigate the personal information security risk assessment for e-waste recycling [42]. Another study examines a combined failure mode effect analysis (FMEA) with a hybrid analytic hierarchy process (AHP) and the preference ranking organization method for enrichment evaluation (PROMETHEE) in supply chain management [43]. In reviewing the literature for this study, it was observed that no study on MLPs has applied GRA-FAHP-FMEA techniques in evaluating the potential risk and quality of mobile learning platforms. This study is unique in comparison to other relevant studies that implement FMEA in collecting and evaluating data collected from various MLPs, which are then applied to FAHP for weighting risks. Studies that combine these methods in evaluating MLPs could not be located in the literature.

3. Methodology

The study is carried out in stages; this involves the selection of MLPs, developing the framework from the ISO standards, and determining the value of the three risk factors for each risk item with the aid of an Excel template for FMEA data collection. In addition, calculating the weight factors and assigning the corresponding weights leads to calculating the grey RPN. Therefore, the evaluation and analysis of the results from FMEA-FAHP.

The procedures involved are further described in steps as shown below;

(1)

The selected criteria were based on ISO/IEC 9126 and were arranged on the first column of the FMEA sheet. And the metrics (Occurrence, Severity, and Detection) were arranged on the subsequent columns of the same sheet.

(2)

The data on the columns were manually collected, as each MLP was observed based on individual criteria. Therefore, metrics were measured based on their corresponding rankings.

(3)

The classical Risk Priority number is then calculated, which is the multiplication of the three metrics: Occurrence, Severity, and Detection.

(4)

The grey priority number is calculated based on weights obtained through the FAHP technique.

(5)

The grey RPN is ranked through the application of the method of ranking [10].

Figure 1 shows the study’s procedural flowchart. In this flowchart, the MLPs are accessed based on the criteria from the adopted framework. The risk factors for each criterion are prioritized using the generated FAHP weights of the weighted criteria for occurrence, severity, and detection, respectively.

3.1. Selection and Evaluation Criteria

The FMEA method requires the use of multiple attributes to create a list of potential failures that can be used to fix design defects and expensive bugs that can be difficult to address at a later stage [44]. To evaluate the potential risk of the MLPs, only the technical quality criteria will be of relevance. The FAHP method uses the three components of FMEA, namely, occurrence, detection, and severity, as criteria to determine the weight of each component in order to further evaluate the criteria for FMEA outcomes. In this study, the proposed criteria are shown in

Table 1. The Criteria for FMEA data collection [45].

Label	Characteristics	Sub-Characteristics
Technical Essentials based on ISO/IEC 9126	Efficiency	Time Behavior
	Reliability	Fault Tolerance
	Usability	Understandability
		Learnability
		Operability
		Attractiveness
	Functionality	Suitability

.

3.2. Alternatives: Sample Mobile Learning Platforms

To guarantee a representative and diverse sample, the five MLPs assessed in our study—SoloLearn, Duolingo, EdX, Khan Academy, and Pluralsight—were chosen using the following standards.

We considered the platform ratings, active user numbers, content, technology used, and ease of access to the evaluation criteria by experts.

Our selection is based on the platforms with high user numbers and widespread recognition. We inspected reviews, user ratings, download statistics, and the number of active users on the Google Play Store. There are millions of users on EdX, Khan Academy, and Duolingo. The chosen MLPs have a variety of educational content, such as courses for higher education (EdX), general education (Khan Academy), professional development (Pluralsight), language acquisition (Duolingo, SoloLearn), and coding. Including varied content MLPs contributed well to the generalizability of the study. In addition, chosen platforms should meet the requirements of the evaluation framework. Our study made sure to capture a representative sample of well-known and established platforms accessible to learners today. The selected platforms range in focus from language learning (Duolingo) and K–12 education (Khan Academy) to specific professional skills (Pluralsight). Platforms selected have multilingual supports, enabling access to learners globally. We believe that the sample of MLPs reviewed in this study is both representative and valid because we were able to precisely define our selection criteria and select platforms that are widely used, diversified, and equipped with modern technology. Our findings can guide decisions in a variety of educational contexts because the platforms we have chosen offer a thorough understanding of the mobile learning environment. There are different forms of MLPs based on some relevant features such as functions, design structure, aim, targeted users, and drawbacks. On the Google Play Store, these MLPs are digitally delivered and are at the Play Store’s subscriber’s disposal, and they may either be free or at a price. There are millions of mobile platforms designed for different purposes, which are hosted on Google Play Store, and they have categories such as games, social, finance, education, and health. Google Play Store gave users the power to rate these software platforms based on their experience using the software. Every mobile software on the Google Play Store has been verified by Google Play Protect to avoid malicious software. The rating is from 0 to 5. However, for this study, five MLPs were selected to be designed for adults and able people. Therefore, MLPs chosen for this study are popular and mostly open source.

Table 2. MLP Google Play ratings.

MLPs	Ratings (0–5)
SoloLearn	4.8
Duolingo	4.7
edX	4.7
Khan Academy	4.5
Pluralsight	4.5

shows the list of MLPs and their Google Play Store rating.

3.2.1. Duolingo Platform

Duolingo, shown in Figure 2, is an MLP for learning different and popular languages such as Chinese, English, Arabic, German, Spanish, etc... It has millions of subscribed users. It can help users learn from basic to advanced levels through its aided features, such as reading and audio translation. The MLP can be used anytime and anywhere as long as there is internet connectivity. The MLP has a practice test readily available while learning on it. It has over 100 million downloads and 8 million reviews.

3.2.2. EdX Platform

EdX in Figure 3 is developed for learning information technology and information technology-related courses. It has millions of users due to the many free courses it offers. It has almost 50,000 reviews on the Google Play Store and over a million downloads. It provides learners with courses from Harvard, MIT, and Microsoft, such as data science, IT support, project management, business and management, integrated digital media, Microsoft Professional program in IoT. It has over 1 million downloads and 49,000 reviews.

3.2.3. Pluralsight Platform

Pluralsight, as depicted in Figure 4, is also an MLP for learning IT courses, mostly programming languages and technologies such as AngularJS19.0.0, HTML5, JavaScript ECMAScript 2023, Bootstrapv4.6, cloud solutions, etc. It has thousands of users across the world. The MLP is designed to be used anywhere, irrespective of location, as long as the user has access to an internet connection. It has over 500,000 downloads and 12,000 reviews.

3.2.4. Khan Academy Platform

Khan Academy, given in Figure 5, is an MLP developed for learning information technology and information technology-related courses as well as other fields of study. Khan Academy has millions of users due to the availability of several free courses. It provides learners with courses such as Math, Economics, Finance, etc. It has over 10 million downloads and 105,000 reviews.

3.2.5. SoloLearn Platform

Another popular MLP is SoloLearn, given in Figure 6, which is built to aid and support beginners in code learning. It courses and IT-related courses such as HTMLv5, CSSv3, JavaScript ECMAScript2023, Ruby on Railsv7.1, Pythonv3.13, Javav23, C++v20, Kotlinv2.0.0, etc. Though it does not support offline functionality, it tracks student performance. This MLP has over 5 million downloads and 387,000 reviews.

3.3. Fuzzy Analytic Hierarchy (FAHP) Procedure

In the 1980s, Thomas Saaty presented an analytical hierarchy process for solving complex problems by breaking the problem down into subgroups that were hierarchically arranged [9]. Even though AHP has merit in evaluating multiple criteria, it has shortcomings that the decision maker came up with, as it substitutes for pairwise comparison. Deng also observed the fallbacks of the AHP method when comparing alternatives [46]. Also, in its comparison procedure, the scale used for judgment in this technique did not extract the meaning of unpredictability and delinquency in the data [46].

Therefore, Chang presented an improved AHP referred to as the fuzzy analytical hierarchy process [11]. The FAHP technique was used in this research in order to avoid fallbacks. Fuzzy offers more versatility to expression. There are several types of fuzzy numbers: triangular, trapezoidal fuzzy numbers, etc. [17]. The triangular fuzzy number contains the group of three real integer numbers starting from the least, most ascertain, and the highest weights for the triangular fuzzy. However, trapezoidal and triangular fuzzy numbers are said to be the most popularly used. Triangular fuzzy numbers were used in this study.

The procedures involved in the improved AHP are described as follows, as presented by [47].

Let’s declare G as a set of objects, where G = {G₁, G₂, G₃,……..…,G_n} and P is a goal set where P = {P₁,P₂,P₃,………..P_n}. Based on this technique, the individual object is taken, and the extent analysis set for every goal is carried out respectively. Therefore, for an individual object, the respective m extent analysis values are acquired.

In this case, the individual value of $M_{g_p}^j$ where j = 1, 2, 3, 4,…..,m are triangular fuzzy number. The steps for Chang extent analysis are described as follows:

Step 1: Fuzzy synthetic based on ith objects is stated by an equation as follows;

$${\mathrm{S}}_i ={\sum }_{j=1}^m{M}_{g_p}^j\otimes {\left[{\sum }_{i=1}^n{\sum }_{j=1}^m{M}_{g_p}^j\right]}^{-1}$$

(1)

To acquire the value of ${\sum }_{j=1}^m{M}_{g_p}^j$, m extent analysis is used based on fuzzy summation.

$${\sum }_{j=1}^m{M}_{g_p}^j=\left({\sum }_{j=1}^m{l}_j, {\sum }_{j=1}^m{m}_j,{\sum }_{j=1}^m{u}_j\right)$$

(2)

$${\sum }_{i=1}^n{\sum }_{j=1}^m{M}_{g_p}^j=\left({\sum }_{i=1}^n{l}_i, {\sum }_{i=1}^n{m}_i,{\sum }_{i=1}^n{u}_i\right)$$

(3)

Equation (2) is a vector that is then reversed to acquire Equation (4), shown below;

$${\left[{\sum }_{i=1}^n{\sum }_{j=1}^m{M}_{g_p}^j\right]}^{-1}=\left({\displaystyle \frac{1}{{\sum }_{i=1}^n{u}_i}},{\displaystyle \frac{1}{{\sum }_{i=1}^n{m}_i}},{\displaystyle \frac{1}{{\sum }_{i=1}^n{l}_i}} \right)$$

(4)

Step 2: The degree of possibility depends on the values of M₁ and M_2, of which, while M₁ remains (l₁, m₁, u₁) and M₂ remains (l₂, m₂, u₂) and can be evaluated as stated in Equation (5);

$$l_{i }={min}_k\left\{a_{ik}\right\};{ m}_{i }={\displaystyle \frac{1}{k}}{\sum }_{k=1}{b}_{ik};u_{ij}={max}_k\left\{ c_{ik}\right\}$$

(5)

Therefore, M₂ $\ge$ M₁ for the degree of possibility, which is stated as follows:

$${\mathrm{V}(\mathrm{M}2\ge \mathrm{M}1)=\mathrm{h}\mathrm{g}\mathrm{t}(\mathrm{M}2\bigcap \mathrm{M}1)=\mu }_{M_2} \left(\mathrm{d}\right)=\left\{\begin{array}{c}1, m_2\ge m_1\\ 0, l_1 \ge u_2\\ {\displaystyle \frac{l_1-u_2}{(m_2-u_2)(m_1-l_1)}} Otherwise\end{array}\right.$$

(6)

V(M₂ $\ge$ M₁) is equal to d; this indicates that the highest convergence within M₁ and M₂ is the ordinate of D, as shown in Figure 7.

In Figure 7, d defines the highest convergence point among ${\mu }_{M_1}$ and ${\mu }_{M_2}$. Comparing M₁ and M₂, the values of V(M₂ $\ge$ M₁) and V(M₁ $\ge$ M₂) becomes relevant.

Step 3: So, the degree of possibility of a single convex fuzzy number that could be greater than the value of k convex numbers M_i where i = 1, 2, 3,…….., k; is stated as follows:

$$\mathrm{V}(\mathrm{M} \ge {\mathrm{M}}_1, \mathrm{M} \ge {\mathrm{M}}_2,\dots \dots . \mathrm{M} \ge {\mathrm{M}}_{\mathrm{k}})=\mathrm{V}[(\mathrm{M} \ge {\mathrm{M}}_1) \mathrm{and} (\mathrm{M} \ge {\mathrm{M}}_2), \mathrm{and} (\mathrm{M} \ge {\mathrm{M}}_{\mathrm{k}}\left)\right]$$

(7)

minV(M > M_i), where i = 1,2,3,…k

$$\mathrm{Suppose} {\mathrm{d}}^{\prime }\left(\mathrm{A}\right)=\mathit{min}\mathrm{V}({\mathrm{S}}_{\mathrm{i}} \ge {\mathrm{S}}_{\mathrm{k}})$$

(8)

Therefore, for k = 1, 2, 3,…n where k $\ne$ i; the weight vector can be given as;

$${\mathrm{w}}^{\prime }={\left[d^{\prime }\left(A_1\right),d^{\prime }\left(A_2\right),d^{\prime }\left(A_3\right),\dots d^{\prime }\left(A_n\right)\right]}^T$$

(9)

For A_i: i = 1, 2, 3,…,n-elements

Step 4: The weight vectors are normalized, as stated in Equation (9); the weight is a number, but note that it is a non-fuzzy number.

$$\mathrm{w}={\left[d\left(A_1\right),d\left(A_2\right),d\left(A_3\right),\dots d\left(A_n\right)\right]}^T$$

(10)

Step 5: The Grey RPN [45] will then be calculated based on the equations below;

S_r = ${\sum }_{i=1}^n{w}_{\mathrm{s}}{S}_{\mathrm{i}}$; O_r = ${\sum }_{i=1}^n{w}_{\mathrm{o}}{O}_{\mathrm{i}}$; D_r = ${\sum }_{i=1}^n{w}_{\mathrm{D}}{D}_{\mathrm{i}}$; where i = 1,2,3,….,n

Therefore, Grey RPN is

$$\mathrm{RPN}={\sum }_{i=1}^n\left\{W_s{S}_i+W_O{O}_i+W_D{D}_i\right\}$$

(11)

Step 6: The computed information is then ranked based on the order of preference.

3.4. Proposed Evaluation Model

The study aims to evaluate the quality and risks associated with mobile learning platforms with the help of FMEA-FAHP techniques. Several other related models are related to this study in the quality evaluation of software previously discussed in the literature. This study is unique in that there has been no previous combination of FMEA and FAHP in the evaluation process for the inspection of mobile learning platforms. This study embraces both the technical and non-technical perspectives as the core reason for establishing the assessment of the mobile learning platform. ISO/IEC 9126 technical and non-technical aspects constitute the essentials of the model for this study.

This study adopts the following model for assessing mobile learning platforms, which was scrutinized from the extant literature. Figure 8 indicates the adopted model for assessing mobile learning platforms.

Seven criteria were identified and were used in this study. The reason for the selected criteria is that the FMEA can be utilized to evaluate technical quality and functionality. The adopted framework in this study emphasizes key characteristics of the ISO/IEC 9126 standards, namely functionality, efficiency, reliability, and usability. The steps followed were given in pseudo-code:

• Decide about the set of criteria for evaluation from ISO/IEC 9126 standard
• Decide about the experts involved in the evaluation
• Decide about the alternatives of MLPs for evaluation
• Decide about the linguistic scale and their triangular fuzzy reciprocal scale
• Experts establish a decision matrix regarding the weights for occurrence, severity, and detection
• Fuzzy pairwise comparison matrix was established from experts’ decision matrices
• Synthetic extent values were established from a Fuzzy pairwise comparison matrix
• Weights for Occurrence, severity, and detection were calculated with respect to Synthetic extent values
• Experts evaluate five alternatives concerning severity, occurrence, and detection using FMEA rating from 1–10
• Total RPN value calculated with respect to each alternative
• Calculated weights for severity occurrence detection applied for each alternative for each criterion to calculate Grey RPN
• Alternatives yielding to total Grey RPN for each alternative are ranked from least to most.
• Alternatives with the least Grey RPN value yield better results in terms of evaluated aspects.

3.5. Selection and Evaluation Criteria for MLP Based on Adopted Framework

FMEA implementation goes through a series of processes that involve the development of an FMEA data-gathering template. Software analysis using FMEA is achieved by evaluating each criterion derived from the developed framework and estimating their potential failure modes. These failure modes are called risk factors. In this study, the criteria for evaluation are adopted from the developed framework ISO/IEC 9126 standard. These attributes represent a detailed model for evaluating any software system [48]. Additionally, these criteria are used to evaluate the quality of the mobile learning platforms.

The adopted model from the ISO/IEC 9126 standards has seven sub-criteria: time−behavior, fault tolerance, understandability, learnability, operability, attractiveness, and suitability. These sub-criteria cut across four main characteristics: efficiency, reliability, usability, and functionality. The choice of the criteria listed in the adapted framework was based on the criteria modified specifically for mobile learning platforms. The characteristics and sub-characteristics of the adopted framework are given earlier in Table 1.

In this study, the sub-characteristics selected for evaluation are suitability, fault tolerance, understandability, learnability, operability, attractiveness, and time behavior.

(1)

Suitability: The suitability of software refers to the essential functionality characteristic. It helps to answer the question, “Can the software perform the tasks required?’ It enumerates a set of attributes used for the explicit assessment of functions to prescribed tasks, and to determine their adequacy for performing the tasks. It answers the question; “How appropriate are the functions of software applied to the specification?”

A quality model is used to determine the quality of the software. Usability, which is one of the sub-characteristics of the quality model, is defined as the capability of the software product to be understood, learned, and used and the ability to provide visual appeal under certain specific conditions of usage (the effort needed for use).

(2)

Fault tolerance: Fault tolerance helps to determine if the software is capable of handling errors. It refers to the ability of a system to withstand component failure. It is defined as a set of attributes used to assess a software’s ability to maintain the desired performance level in the unlikely event of operational defects or infringement of specified interfaces.

(3)

Understandability: This refers to how well or easily a user comprehends the functionality of a system. System functions and things like layout consistency, functions of buttons, clear prompts for input, consistent use of terms throughout the system, and proper documentation will all contribute to the ease of the system being understood.

(4)

Learnability: The learnability of software depicts how easily a user can learn to use the system. Although learnability is very similar to understandability, a major sub-characteristic is that it has to do with how long it takes users to learn how to use specific functions and the effectiveness of help systems. It is the learning effort required by users of various levels of difficulty during the learning process.

(5)

Operability: This is also one of the sub-characteristics of usability. It characterizes the ability to use the system with minimal effort. The operability of software depends on whether a user can operate and control the software in a given environment.

(6)

Attractiveness: Factors such as screen design, color, and general software interface are considered in determining the attractiveness of software.

(7)

Time Behavior: This characterizes how quickly the system responds and processing time when carrying out functions like file upload and download. The rate at which transactions are being processed and its ability to give an appropriate response time for a given throughput.

3.6. Failure Modes and Effects Analysis (FMEA)

FMEA implementation goes through a series of processes that involve the development of an FMEA data-gathering template. The template includes tables that show information gathered based on individual alternatives considered. Where to enter data, guidelines on the type of data to be entered, and when those data are collated. Below are pictures and descriptions of each FMEA file.

The FMEA technique accesses applications based on the three risk criteria known as risk factors severity, occurrence, and detection. A heuristic scale is used to estimate the values of each risk factor. The scales are ranked from 1–10. This ranking is based on the effect of failure modes, which are further categorized into minor, moderate, and hazardous effects. From

Table 3. Ranking for probability of occurrence/detection/severity.

Probability of Failure	Failure Probability	Ranking
Very High: Failure is almost Certain	>1/2	10
	1/3	9
High: Repeats failures	1/8	8
	1/20	7
Moderate: Failures are Occasional	1/80	6
	1/400	5
	1/2000	4
Low: Rare failures	1/15,000	3
	1/150,000	2
Remote: low chance of failure	<1/1,500,000	1

, the sections are further categorized with colors: red for hazardous effects (ranking: 7–10), yellow for moderate effects (3–6), and green for minor effects (1–2).

(1)

Occurrence: This is the probability of occurrence, which is the likely number of times the failure is bound to take place. Its ranking is from 1 to 10. From unlikely chance of occurrence through occasional failures to almost inevitable probability of failure. From Table 3, the scale categorizes risk and estimates a value for different ranges or categories of risk from increasing to decreasing order.

(2)

Detection: This relates to how well a system can recognize abnormality in its functionality. Assuming a failure occurs, a system able to assess its capabilities to prevent failure mode is called detection. It helps rank the level at which failures can be detected. This is sub-categorized into most certain to be detected, moderate level of detection, and failure is almost uncertain to be detected. It is similar to both occurrence and severity in ranking, as it ranks from 1 to 10.

(3)

Severity: This can be defined as the extent to which the consequences of failure on a software affect the internal and external functionality occurring during and after failure. The sub-category is sectioned into failures that are ‘hazardous in the absence of warning, hazardous with warning, moderate and minor severity in terms of damage. It ranks from 1 to 10.

The risk priority number is the multiplication of these three risk factors: severity, probability of occurrence, and detectability.

Risk Priority Number (RPN) = Severity × Occurrence × Detection

(12)

In the detection of potential failures of the software functionality, for example, the sub-categories of the functionality tab are considered, and the mobile learning platform is compared against the sub-characteristics to ascertain the potential failures associated with the functionality. The following questions were asked:

• Suitability: Does the software perform its required tasks?
• Accuracy: Does the result produced conform to the expectation?
• Interoperability: Can the system relate or interact with a similar system?
• Security: Is the software able to restrict unauthorized access?

With respect to the above questions, if the software is found to have any shortcomings, the expert is then able to identify if the software has potential failures associated with the functional characteristics in question via the sub-characteristics.

Table 4. Description of the criteria for evaluation.

Sub-Characteristics	Explanation
Time Behavior	How quickly does the system respond?
Fault Tolerance	Is the MLP capable of handling error?
Understandability	Does the user understand how to use the system without much efforts?
Learnability	Can the user learn to use the system easily?
Operability	Can a user use the system without much effort?
Attractiveness	How pleasant does the interface look?
Suitability	Does the software perform its required tasks?

explains the types of questions experts use in the evaluation and estimation process for each criterion. This process is the source of the input for the system, and the estimations are approximated in Table 3.

The detected potential failure metrics are then accessed using three different metrics severity, occurrence, and detection. These metrics are called the risk factors, namely, severity, detection, and probability of occurrence. Values for severity are estimated from Table 3.

The processes involved in the application of the FMEA technique on MLPs based on this study guidelines are;

Step 1: Evaluate the process

• Use the developed framework to signify individual components.
• On the FMEA table, itemize the components.

Step 2: Deliberate on all the modes of potential failure.

• What are the causes of failure and the possibility of occurrence?
• The scale for determining the level of impact of a failure.

Step 3: Itemize the potential effects of each failure

• The effect of failure on the other preceding processes or failures is measured and valued.
• There will likely be more than one effect for each failure.

Step 4: Assign the risk factor rankings

• Fundamentally, on each risk factor and implication of failure.
• Use the ranking scale to determine the value of individual risk factors.

Step 5: Calculate the RPN and the Grey RPN

4. Results

4.1. FAHP Procedure for Evaluating MLPs

As stated earlier, the traditional RPN calculation in the FMEA process does not consider the significance of individual risk factors for the type of software. Therefore, FAHP is used to assign priority to individual risk factors (severity occurrence and detection). The results of the FAHP process are then merged with the FMEA results in

Table 5. Linguistic variables and scales.

Linguistic Variables	Scale	Reciprocal Scale	Triangular Fuzzy Scale	Triangular Fuzzy Reciprocal Scale
Equally Preferred	1	1	(1,1,1)	(1/1,1/1,1/1)
Equally to Moderately Preferred	2	½	(1,2,3)	(1/3,1/2,1)
Moderately Preferred	3	1/3	(2,3,4)	(1/4,1/3,1/2)
Moderately to Strongly Preferred	4	¼	(3,4,5)	(1/5,1/4,1/3)
Strongly Preferred	5	1/5	(4,5,6)	(1/6,1/5,1/4)
Strongly to Very Strongly Preferred	6	1/6	(5,6,7)	(1/7,1/6,1/5)
Very strongly preferred	7	1/7	(6,7,8)	(1/8,1/7,1/6)
Very strong to Extremely preferred	8	1/8	(7,8,9)	(1/9,1/8,1/7)
Extremely preferred	9	1/9	(8,9,9)	(1/9,1/9,1/8)
Strongly to Very Strongly Preferred	6	1/6	(5,6,7)	(1/7,1/6,1/5)

using Equation (11). This is the point of integration to FMEA from FAHP.

With the aid of triangular fuzzy numbers, the experiment was performed with five alternatives. To begin with the FAHP assessment procedure, a pairwise matrix for comparison is generated with the aid of a linguistics scale. Table 5 shows the proposed scale adopted from [49,50].

The three decision-makers’ opinions were considered in this study. The decision-makers are involved in determining the weights of the risk factors, such as severity, occurrence, and detection, depending on their experience. They also participated in the evaluation of the five mobile learning platforms using FAHP combined with GRA.

We carefully selected three experts based on their extensive experience and relevance to the evaluation of mobile learning platforms (MLPs). The criteria for selecting these decision-makers included:

Each expert has substantial experience in mobile learning technologies, distance educational tools, or instructional design, ensuring that they are well-versed in the technical and pedagogical aspects of MLPs. Specifically, one expert is a Computer and Instructional Technology professor specializing in educational technology with 13 years of experience in the development and maintenance of distance educational tools, and another expert is a senior instructional designer with a Computer Science background with experience in implementing mobile learning solutions with 4 years of experience, and the third expert is an information systems engineer with a focus on usability and system functionality in educational platforms with 7 years of experience. All selected experts have been involved in the development, implementation, or evaluation of mobile learning platforms in various educational institutions. Their hands-on experience with MLPs provided valuable insights into both the technical and user-experience aspects of the platforms. The experts use the linguistic variables given in Table 5 to generate the decision matrix separately.

The matrix for individual decision-makers, as obtained from the linguistic table and assigned to the three decision-makers, is stated as follows, where D1, D2, and D3 represent the decision-makers obtained from the linguistic scales table.

Table 6. Fuzzy-based pairwise comparison matrix.

	C1	C2	C3
C1	(1, 1, 1)	(0.1432, 3.7143, 7)	(0.1253, 2.4422, 7)
C2	(0.1432, 2.4641, 7)	(1, 1, 1)	(0.1253, 0.1372, 0.1432)
C3	(0.1432, 4.3813, 8)	(7, 7.3333, 8)	(1, 1, 1)

,

Table 7. Sum of rows and columns.

Criteria	Sum of Rows	Sum of Columns
C1	(1.2685, 7.1565, 15)	(1.2864, 7.8454, 16)
C2	(1.2685, 3.6013, 8.1432)	(8.1432, 12.0476, 16)
C3	(8.1432, 12.7146, 17)	(1.2506, 3.5794, 8.1432)
		10.6802, 23.4724, 40.1432

and

Table 8. Synthetic extent value results for individual criteria.

Criteria	SCi
C1	(0.031599, 0.3048, 1.40445)
C2	(0.031599, 0.1534, 0.76245)
C3	(0.20285, 0.54167, 1.59173)

show the process of obtaining synthetic extent values from pairwise comparison matrix.

$${\mathrm{D}}_1=\left[\begin{array}{ccc}1& {\displaystyle \frac{1}{7}}& {\displaystyle \frac{1}{9}}\\ 7& 1& {\displaystyle \frac{1}{9}}\\ 9& 9& 1\end{array}\right]; {\mathrm{D}}_2=\left[\begin{array}{ccc}1& {\displaystyle \frac{1}{6}}& {\displaystyle \frac{1}{9}}\\ 6& 1& {\displaystyle \frac{1}{9}}\\ 9& 9& 1\end{array}\right]; {\mathrm{D}}_3=\left[\begin{array}{ccc}1& {\displaystyle \frac{1}{6}}& {\displaystyle \frac{1}{9}}\\ 6& 1& {\displaystyle \frac{1}{8}}\\ 9& 8& 1\end{array}\right]$$

C1, C2, and C3 are the criteria selected based on occurrence, detection, and severity, respectively, and the matrices are evaluated using Equation (5). Equation (5) is used to sample the opinions of all three decision-makers to produce a single pairwise comparison matrix shown in Table 6. The pairwise comparison matrix takes into consideration the opinion of all three decision-makers depicted in Table 7.

Equation (12) is used to obtain the synthetic extent value given in Table 8 that resulted from

Table 9. FMEA results of expert 1 (D₁).

	EdX				Khan Ac				Doulingo				Pluralsight				Sololearn
	S	O	D	R	S	O	D	R	S	O	D	R	S	O	D	R	S	O	D	R
Fault tolerance	3	4	2	24	2	3	2	12	2	5	2	20	10	6	7	420	4	6	4	96
Time behavior	4	2	5	40	4	2	4	32	3	4	2	24	2	1	3	6	5	3	9	135
Suitability	3	2	7	42	3	1	2	6	1	2	2	4	3	7	1	21	7	2	5	70
Understandability	3	1	2	6	1	3	2	6	4	1	2	8	6	5	8	240	4	5	5	100
Learnability	3	2	1	6	2	3	5	30	3	4	1	12	4	2	1	8	2	4	1	8
Operability	3	4	6	72	1	2	2	4	1	3	10	30	1	2	1	2	4	1	1	4
Attractiveness	2	1	1	2	2	3	7	42	2	10	3	60	8	6	4	194	4	3	2	24
Overall RPN	192				132				158				891				437

Note: S = severity, O = occurrence, D = Detection, R = Individual RPN per criteria, and overall RPN is the RPN value of the MLP.

,

Table 10. FMEA results of expert 2 (D₂).

	EdX				Khan Ac				Doulingo				Pluralsight				Sololearn
	S	O	D	R	S	O	D	R	S	O	D	R	S	O	D	R	S	O	D	R
Fault tolerance	7	2	5	70	1	9	5	45	4	5	3	60	10	9	3	270	6	2	2	24
Time behavior	8	5	4	160	5	5	1	25	3	2	2	12	2	1	3	6	7	3	4	84
Suitability	3	2	4	24	2	4	2	16	1	2	3	6	2	2	1	4	5	2	5	50
Understandability	2	1	1	2	1	1	2	2	1	3	2	6	3	2	5	30	4	2	9	72
Learnability	1	2	1	2	2	3	4	24	3	1	4	12	3	2	1	6	2	3	1	6
Operability	2	2	3	12	1	3	2	6	1	7	10	70	1	1	1	1	1	2	1	2
Attractiveness	2	1	4	8	5	1	8	40	2	10	3	60	4	2	4	32	8	3	2	48
Overall RPN	278				158				226				349				286

Note: S = severity, O = occurrence, D = Detection, R = Individual RPN per criteria and overall RPN is the RPN value of the MLP.

and

Table 11. FMEA results of expert 3(D₃).

	EdX				Khan Ac				Doulingo				Pluralsight				Sololearn
	S	O	D	R	S	O	D	R	S	O	D	R	S	O	D	R	S	O	D	R
Fault tolerance	2	3	5	30	3	3	5	45	3	2	7	42	10	9	2	180	5	1	6	30
Time behavior	3	2	3	18	3	8	4	96	3	3	5	45	5	1	4	20	4	3	2	24
Suitability	3	2	4	24	1	1	2	2	4	2	1	8	1	6	1	6	6	5	5	150
Understandability	1	1	3	3	1	2	2	4	1	2	2	4	3	2	5	30	1	5	2	10
Learnability	2	2	1	4	2	3	3	18	3	4	1	12	2	2	1	4	2	2	1	4
Operability	1	3	6	18	1	1	2	2	1	5	10	50	1	3	1	3	4	3	1	12
Attractiveness	2	1	1	2	2	2	9	36	2	10	3	60	3	6	4	72	3	3	2	18
Overall RPN	99				203				221				315				248

Note: S = severity, O = occurrence, D = Detection, R = Individual RPN per criteria, and overall RPN is the RPN value of the MLP.

; the calculation involved is shown below:

$${\mathrm{SC}}_1=(1.2684, 7.2562, 15) \otimes \left\{{\displaystyle \frac{1}{40.1231}}|{\displaystyle \frac{1}{23.4172}}|{\displaystyle \frac{1}{20.6791}}\right\} = (0.0321,0.3051,1.4142)$$

$${\mathrm{SC}}_2=(1.2684, 3.6012, 8.1432) \otimes \left\{{\displaystyle \frac{1}{40.1231}}|{\displaystyle \frac{1}{23.4172}}|{\displaystyle \frac{1}{20.6791}}\right\} = (0.0321,0.1534,0.7622)$$

$${\mathrm{SC}}_3=(8.1432, 12.7141, 17) \otimes \left\{{\displaystyle \frac{1}{40.1231}}|{\displaystyle \frac{1}{23.4172}}|{\displaystyle \frac{1}{20.6791}}\right\} = (0.2032,0.5021,1.5913)$$

Table 9, Table 10 and Table 11 show the pairwise comparison matrix obtained through the assessment of the decision-makers. The weights of each component were calculated using the FAHP method.

Equation (6) is then used to compare the results in

Table 12. FMEA average of the results.

	EdX				Khan Ac				Doulingo				Pluralsight				Sololearn
	S	O	D	R	S	O	D	R	S	O	D	R	S	O	D	R	S	O	D	R
Fault tolerance	4	3	4	48	2	5	4	40	3	4	4	48	10	8	4	320	5	3	4	60
Time behavior	5	3	4	60	4	5	3	60	3	3	3	27	3	1	4	12	6	2	5	60
Suitability	3	2	5	30	2	2	2	8	2	2	2	8	2	5	1	10	6	3	5	90
Understandability	2	1	2	4	1	2	2	4	2	2	2	8	4	3	6	72	3	4	6	72
Learnability	2	2	1	4	2	3	4	24	3	3	2	18	3	2	1	6	2	3	1	6
Operability	2	3	5	30	1	2	2	4	1	5	10	50	1	2	1	2	3	2	1	6
Attractiveness	2	1	2	4	3	2	8	48	2	10	3	60	5	4	4	80	5	3	2	30
Overall RPN	172				188				219				262				324

Note: S = severity, O = occurrence, D = Detection, R = Individual RPN per criteria, and overall RPN is the RPN value of the MLP.

to obtain the results shown as follows;

$$\mathrm{V}({\mathrm{S}}_{\mathrm{O}} \ge {\mathrm{S}}_{\mathrm{S}})=0.83512; \mathrm{V}({\mathrm{S}}_{\mathrm{D}} \ge {\mathrm{S}}_{\mathrm{S}})=0.58973; \mathrm{V}({\mathrm{S}}_{\mathrm{O}} \ge {\mathrm{S}}_{\mathrm{D}})=1$$

$$\mathrm{V}({\mathrm{S}}_{\mathrm{S}} \ge {\mathrm{S}}_{\mathrm{O}})=1; \mathrm{V}({\mathrm{S}}_{\mathrm{S}} \ge {\mathrm{S}}_{\mathrm{D}})=1; \mathrm{V}({\mathrm{S}}_{\mathrm{D}} \ge {\mathrm{S}}_{\mathrm{O}})=0.53206$$

Therefore, Equation (9) is used to determine the priority of individual risk factors.

d′(Q_o) = min (1: 0.83512) = 0.83512

d′(Q_s) = min (1:1) = 1

d′(Q_D) = min (0.58973: 0.532062) = 0.532062

$${\sum }_{l=1}^3{d}^{\prime }\left(Q_l\right)=0.8353783383091131+1+0.5903867040418067=2.4258$$

The corresponding risk factor weight can then be calculated using Equation (10) as follows;

$${\sigma }_o={\displaystyle \frac{0.8354}{2.4258}}=0.3444$$

$${\sigma }_D={\displaystyle \frac{0.5904}{2.4258}}=0.2433$$

$${\sigma }_S={\displaystyle \frac{1}{2.4258}}=0.4123$$

Researchers considered more attention to severity regarding the influence of severity in FMEA applications in compliance with the literature. Therefore, the severity tends to have the highest weight factor.

As the study continues to unfold, the merging of FMEA and fuzzy will be executed based on the assumption of utilizing different weights on risk factors. Note that the following results of risk factor weights were retrieved through FAHP techniques;

$${\sigma }_o=0.3444$$

$${\sigma }_D=0.2433$$

$${\sigma }_S=0.4123$$

The values of $\left({\sigma }_o\right),$ $\left({\sigma }_D\right)$ and (${\sigma }_S$) are the generated weights of occurrence, detection, and severity, respectively.

4.2. Results of FMEA Applied to MLPs

The developed framework was used to itemize components (criteria) on the FMEA sheet. Having evaluated possible potential failures, severity, occurrence, and detection ranking are assigned based on the impact of failure. Table 12 shows the results attained for the five MLPs, which are arranged based on MLPs because individual MLPs must be considered.

In the FMEA process, each alternative (mobile learning platform) is assessed based on all criteria presented in the adopted framework (time behavior, fault tolerance, understandability, learnability, operability, attractiveness, and suitability). The FMEA assessment factors (severity, occurrence, and probability of detection) are therefore tested against a particular software testing criteria based on the adopted framework, and a value is estimated in terms of severity, occurrence, and detection using the ranking tables for each risk factor for each of the above mention testing criteria.

The results generated from the FMEA assessment are shown in Table 12. The expert opinions are given in Table 9, Table 10 and Table 11. The average of the data collected from experts for severity, occurrence, and detection are given in Table 12, and these are the data used as input for developing the multi-criteria decision-making software.

The risk priority number is evaluated across each MLP, and the result is based on Equation (11) above. Table 7 shows the result generated from the FMEA evaluation of each alternative for risk factors. Having collected values for the risk factors, it is necessary to evaluate the risk priority number of the criteria and, subsequently, a value for the overall risk priority for each alternative or MLP. Table 12 shows the result of each mobile learning platform with the risk priority number evaluated for each criterion it is tested against as well as the overall risk priority of the platform.

4.3. Integration of FMEA and FAHP

At the traditional RPN calculation, the relative significance of these three risk factors is neglected as they are all assumed to be equal. These factors could be assumed to be equal, but this is not always the case in practice, as noted by previous research. Research has shown that the relative significance of the risk factors varies based on the software or the nature upon which it is being used.

In this study, the FAHP is employed as a decision-making method to rank and analyze these risk factors in a hierarchal structure. The objective of the FAHP is to prioritize risk factors. From Equation (11), the FAHP RPN number for each characteristic is the sum of the weighted risk factor;

$${\sum }_{i=1}^n\left\{W_s{S}_i+W_O{O}_i+W_D{D}_i\right\}$$

(13)

We can, therefore, calculate the grey RPN using the corresponding weight of each FMEA risk factor as follows:

The FAHP helps the realization of better and more efficient alternative ranking by prioritizing risk factors based on their relative importance, unlike the traditional FMEA RPN, which assumes all risk factors to be equal yet is not feasible in practice. The grey RPN is the risk priority number derived from applying prioritized risk factors of the FMEA process. The applied FAHP results for each mobile learning platform are given in

Table 13. FAHP results for SoloLearn mobile learning platform.

Criteria	WS	WO	WD	Grey RPN
Fault Tolerance	2.0615	1.0332	0.9732	4.0679
Time Behavior	2.4738	0.6888	1.2165	4.3791
Suitability	2.4738	1.0332	1.2165	4.7235
Understandability	1.2369	1.3776	1.4598	4.0743
Learnability	0.8246	1.0332	0.2433	2.1011
Operability	1.2369	0.6888	0.2433	2.169
Attractiveness	2.0615	1.0332	0.4866	3.5813
Overall Grey RPN				25.0962

,

Table 14. FAHP results for Khan Academy mobile learning platform.

Criteria	WS	WO	WD	Grey RPN
Fault Tolerance	0.8246	1.722	0.9732	3.5198
Time Behavior	1.6492	1.722	0.7299	4.1011
Suitability	0.8246	0.6888	0.4866	2
Understandability	0.4123	0.6888	0.4866	1.5877
Learnability	0.8246	1.0332	0.9732	2.831
Operability	0.4123	0.6888	0.4866	1.5877
Attractiveness	1.2369	0.6888	1.9464	3.8721
Overall Grey RPN				19.4994

,

Table 15. FAHP result for EdX mobile learning platform.

Criteria	WS	WO	WD	Grey RPN
Fault Tolerance	1.6492	1.0332	0.9732	3.65
Time Behavior	2.0615	1.0332	0.9732	4.0679
Suitability	1.2369	0.6888	1.2165	3.1422
Understandability	0.8246	0.3444	0.4866	1.6556
Learnability	0.8246	0.6888	0.2433	1.7567
Operability	0.8246	1.0332	1.2165	3.0743
Attractiveness	0.8246	0.3444	0.4866	1.6556
Overall Grey RPN				19.0023

,

Table 16. FAHP result for Duolingo mobile learning platform.

Criteria	WS	WO	WD	Grey RPN
Fault Tolerance	1.2369	1.3776	0.9732	3.5877
Time Behavior	1.2369	1.0332	0.7299	3
Suitability	0.8246	0.6888	0.4866	2
Understandability	0.8246	0.6888	0.4866	2
Learnability	1.2369	1.0332	0.4866	2.7567
Operability	0.4123	1.722	2.433	4.5673
Attractiveness	0.8246	3.444	0.7299	4.9985
Overall Grey RPN				22.9102

and

Table 17. FAHP result for Pluralsight mobile learning platform.

Criteria	WS	WO	WD	Grey RPN
Fault Tolerance	4.123	2.7552	0.9732	7.8514
Time Behavior	1.2369	0.3444	0.9732	2.5545
Suitability	0.8246	1.722	0.2433	2.7899
Understandability	1.6492	1.0332	1.4598	4.1422
Learnability	1.2369	0.6888	0.2433	2.169
Operability	0.4123	0.6888	0.2433	1.3444
Attractiveness	2.0615	1.3776	0.9732	4.4123
Overall Grey RPN				25.2637

. The final ranking for individual mobile learning platforms is given in

Table 18. MLP ranking based on FMEA-FAHP.

MLPs	Total Grey RPN	Ranking
EdX	19.00	1
Khan Academy	19.50	2
Duolingo	22.91	3
Sololearn	25.10	4
Plural sight	25.26	5

. From the calculated results, it is shown that EdX MLP with a grey RPN of 19.0023 is most reliable based on the quality evaluation process carried out in this research. This is followed by the Khan Academy MLP with a grey RPN of 19.4994. Plural sight is ranked the lowest in terms of reliability, considering the grey RPN value.

4.4. Drawbacks of Conventional FMEA Method

In the FMEA application, for each failure mode, the likelihood of occurrence, detection, and severity is assessed by the experts who have experienced FMEA at least once in their jobs. The experts give points of 1 to 10 for each parameter (1 is the best and 10 is the worst case). In this study, these parameters are called risk factors. By multiplying values for severity (S), occurrence (O), and detectability (D), the risk priority number (RPN) is obtained. RPN values change between 1 and 1000. These risk priority numbers help to identify the parts or processes that need certain actions for improvement. Failure modes are prioritized, and corrective measures are applied according to the RPN values. Since RPN values are obtained by multiplying severity, occurrence and detection values, different severity, occurrence and detection values could yield to same RPN values, one of the prevailing limitations of classical FMEA procedure. In addition, different weights are omitted in conventional FMEA. Table 9, Table 10, Table 11 and Table 12 are the FMEA results and average FMEA results of the three decision-makers.

4.5. Drawbacks of Grey Relational Analysis to FMEA

In Grey relational analysis method risk factors have equal weights which may lead to limitation for the findings.

4.6. Multi-Criteria Decision-Making (MCDM) Software for MLP Evaluation

The developed MCDM tool is a standalone Windows command shell application designed using Python programming language version 3. The application contains six separate modules. Four of these modules are the codes for FAHP, FMEA, grey calculation, and ranking algorithm. The remaining two are for the input data files of the FMEA and FAHP, respectively. The first page of the application displays the introduction page. This page explains the function of the application to the user. After the introduction page, the user is presented with several options, as shown in Figure 9. Since the data input for FAHP and FMEA are based on expert decisions, the data are entered into the system by writing them into a JSON file and placing them in a folder inside the software directory, as shown in Figure 10 and Figure 11. FMEA and FAHP have two separate files for input. These options enable the user to view FAHP data, FMEA data, grey RPN, and ranking, as shown in Figure 9. The information required is displayed depending on what the user wishes to view. The input of the FAHP is in the file FAHP_data.json. This file enables the input of the FAHP process to be altered according to the user preference, as shown in Figure 10. Figure 12 displays the calculated grey RPN for each MLP, and Figure 13 shows the ranking of these MLPs based on the calculated grey values.

5. Sensitivity Analysis

Sensitivity analysis was conducted to compare the new rankings with the initial rankings to assess how sensitive the final results are to changes in the weights. If significant shifts occur in the rankings, the model is sensitive to those changes, suggesting that the results are influenced by the specific weight settings. In the sensitivity analysis, we focused on three basic scenarios as follows:

High Weight on Severity: We’ll further increase the weight on the severity and equally decrease the occurrence and detection weights.

Balanced Weights: We’ll assume all three factors (severity, occurrence, detection) have equal weights.

Low Weight on Severity: We will lower the weight of severity and increase the weights for occurrence and detection.

The corresponding weights are given in

Table 19. RPN risk factor weights for sensitivity analysis.

MLP	Original_RPN	Original_Ranking	RPN_Severity_Increased	RPN_Occurrence_Increased	RPN_Detection_Increased	RPN_Severity_Low
EdX	19	1	19.5	18.8	19.2	18.5
Khan Academy	19.5	2	20	19.3	19.8	19.1
Duolingo	22.91	3	23.5	22.5	23	22.3
Sololearn	25.1	4	25.8	24.7	25.5	24.5
Pluralsight	25.26	5	25.9	24.85	25.7	24.7

.

As demonstrated in Figure 14, EdX and Khan Academy consistently demonstrate low sensitivity to changes in severity, occurrence, and detection, making them the most reliable platforms based on the analysis. Pluralsight and SoloLearn are more sensitive to increases in severity, indicating they are more prone to significant issues when critical failures occur. Changes in the detection weight seem to have the least impact across all platforms, suggesting that detection is not as influential in determining the overall risk as severity or occurrence. This sensitivity analysis helps to confirm the robustness of the rankings and shows that certain platforms (like EdX and Khan Academy) maintain stability, while others (like SoloLearn and Pluralsight) are more sensitive to specific risk factors.

6. Comparative Analysis

The study compares three baseline methods, namely, traditional FMEA, Grey RPN with equal weights, and the proposed methodology. In traditional FMEA, risk priority numbers (RPNs) are calculated by multiplying the risk factors (severity, occurrence, and detection), which can lead to biased results as it does not account for the relative significance of each risk factor. In FMEA-GRA, risk priority numbers (RPNs) are calculated by assigning equal importance to the risk factors (severity, occurrence, and detection). In the proposed methodology, different weights were assigned to risk factors determined by the experts, and then expert judgments of alternatives were ranked by using FAHP. This allows for a more flexible and accurate representation of risks, especially when subjective judgments are involved.

Table 20. FMEA results for MLPs.

	EdX	Khan Ac	Doulingo	Pluralsight	Sololearn
	RPN	RPN	RPN	RPN	RPN
Fault tolerance	48	40	48	320	60
Time behavior	60	60	27	12	60
Suitability	30	8	8	10	90
Understandability	4	4	8	72	72
Learnability	4	24	18	6	6
Operability	30	4	50	2	6
Attractiveness	4	48	60	80	30
Overall RPN	172	188	219	262	324

shows the average of the results for three experts.

Table 21. MLP ranking based on conventional FMEA.

MLPs	Overall RPN	Ranking
EdX	172	1
Khan Academy	188	2
Duolingo	219	3
Plural sight	262	4
Sololearn	324	5

shows the overall ranking based on the traditional FMEA method.

For calculating GRA-FMEA with equal weights, these steps are as follows: To reduce the potential risk, the values of all risk factors should be as small as possible. Thus, the standard series is defined as $X_i^k=\left[1,1,1\right]$. For each decision maker, to reveal the degree of fuzzy relation, the difference between values of risk factors and standard series is determined and expressed as the matrix shown below:

The relational coefficient is calculated with an equation which is ${\mathrm{X}}_0\left(\mathrm{k}\right) \mathrm{d}\mathrm{e}\mathrm{n}\mathrm{o}\mathrm{t}\mathrm{e}\mathrm{s} \mathrm{s}\mathrm{e}\mathrm{r}\mathrm{i}\mathrm{e}\mathrm{s}, {\mathrm{X}}_{\mathrm{i}}\left(\mathrm{k}\right) \mathrm{s}\mathrm{h}\mathrm{o}\mathrm{w}\mathrm{s} \mathrm{c}\mathrm{o}\mathrm{m}\mathrm{p}\mathrm{a}\mathrm{r}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{v}\mathrm{e} \mathrm{s}\mathrm{e}\mathrm{r}\mathrm{i}\mathrm{e}\mathrm{s} \mathrm{w}\mathrm{h}\mathrm{e}\mathrm{r}\mathrm{e}$

$\mathrm{i}=\mathrm{1,2},3,\dots \mathrm{n}\mathrm{u}\mathrm{m}\mathrm{b}\mathrm{e}\mathrm{r} \mathrm{o}\mathrm{f} \mathrm{f}\mathrm{a}\mathrm{i}\mathrm{l}\mathrm{u}\mathrm{r}\mathrm{e} \mathrm{m}\mathrm{o}\mathrm{d}\mathrm{e}\mathrm{s}, \mathrm{k}=\mathrm{1,2},3,\dots \mathrm{n}\mathrm{u}\mathrm{m}\mathrm{b}\mathrm{e}\mathrm{r} \mathrm{o}\mathrm{f} \mathrm{r}\mathrm{i}\mathrm{s}\mathrm{k} \mathrm{f}\mathrm{a}\mathrm{c}\mathrm{t}\mathrm{o}\mathrm{r}\mathrm{s}$, ${\mathsf{\Delta }}_{\mathrm{m}\mathrm{i}\mathrm{n}} \mathrm{m}\mathrm{i}\mathrm{n}\mathrm{i}\mathrm{m}\mathrm{u}\mathrm{m} \mathrm{o}\mathrm{f} \mathrm{a}\mathrm{l}\mathrm{l} {\mathsf{\Delta }}_{\mathrm{i}}\left(\mathrm{k}\right), {\mathsf{\Delta }}_{\mathrm{m}\mathrm{a}\mathrm{x}} \mathrm{m}\mathrm{a}\mathrm{x}\mathrm{i}\mathrm{m}\mathrm{u}\mathrm{m} \mathrm{o}\mathrm{f} \mathrm{a}\mathrm{l}\mathrm{l} {\mathsf{\Delta }}_{\mathrm{i}}\left(\mathrm{k}\right)$, $\mathsf{\zeta }\in \left(0,1\right) \mathrm{i}\mathrm{s} \mathrm{r}\mathrm{e}\mathrm{l}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{v}\mathrm{e} \mathrm{v}\mathrm{a}\mathrm{l}\mathrm{u}\mathrm{e} \mathrm{o}\mathrm{f} \mathrm{r}\mathrm{i}\mathrm{s}\mathrm{k} \mathrm{c}\mathrm{o}\mathrm{e}\mathrm{f}\mathrm{f}\mathrm{i}\mathrm{e}\mathrm{n}\mathrm{t}, \mathrm{w}\mathrm{a}\mathrm{s} \mathrm{c}\mathrm{o}\mathrm{n}\mathrm{s}\mathrm{i}\mathrm{d}\mathrm{e}\mathrm{r}\mathrm{e}\mathrm{d} \mathrm{a}\mathrm{s} 0.5$.

The corresponding values are calculated as follows:

$$\gamma \left[X_0\left(k\right),X_i\left(k\right) \right]={\displaystyle \frac{{\Delta }_{min}+\zeta {\Delta }_{max}}{{\Delta }_{0i}\left(k\right)+\zeta {\Delta }_{max}}}$$

If all three risk factors have equal weights, the following equation is applied to determine the RPN value.

$${\tau }_{01}\left(k\right)={\displaystyle \frac{1}{3}}\sum _1^3{\gamma }_{0i}\left(k\right)$$

An example of calculated difference FMEA results for expert1 is shown in

Table 22. Difference FMEA results of expert 1(D₁).

	EdX			Khan Ac			Doulingo			Pluralsight			Sololearn
	S	O	D	S	O	D	S	O	D	S	O	D	S	O	D
Fault tolerance	2	3	1	1	2	1	1	4	1	9	5	6	3	5	3
Time behavior	3	1	4	3	1	3	2	3	1	1	0	2	4	2	8
Suitability	2	1	6	2	0	1	0	1	1	1	6	0	6	1	4
Understandability	2	0	1	0	2	1	3	0	1	5	4	7	3	4	4
Learnability	2	1	0	1	2	4	2	3	0	3	1	0	1	3	0
Operability	2	3	5	0	1	1	0	2	9	0	1	0	3	0	0
Attractiveness	1	0	0	1	2	6	1	9	2	7	5	3	3	2	1

.

Table 23. GRA-FMEA results of expert 1(D₁).

	EdX				Khan Ac				Doulingo				Pluralsight				Sololearn
	S	O	D	R	S	O	D	R	S	O	D	R	S	O	D	R	S	O	D	R
Fault tolerance	0.71	0.49	0.66	0.62	1.00	0.67	0.78	0.81	1.00	0.50	0.67	0.72	0.70	0.46	0.38	0.52	1.00	0.47	0.44	0.63
Time behavior	0.60	0.87	0.43	0.63	0.56	0.82	0.46	0.61	0.71	0.49	0.66	0.62	0.50	1.00	0.50	0.67	0.75	0.79	0.40	0.65
Suitability	0.80	0.95	0.42	0.72	0.33	1.00	0.56	0.63	1.00	1.00	1.00	1.00	0.75	0.33	1.00	0.69	0.44	0.81	0.41	0.56
Understandability	0.33	1.00	0.56	0.63	1.00	0.67	0.78	0.81	0.33	1.00	0.56	0.63	0.88	0.58	0.39	0.62	1.00	0.50	0.42	0.64
Learnability	0.33	0.33	1.00	0.56	1.00	0.75	0.46	0.74	0.43	0.33	1.00	0.59	0.33	0.33	1.00	0.56	0.60	0.33	1.00	0.64
Operability	1.00	0.64	0.42	0.68	1.00	1.00	1.00	1.00	1.00	0.85	0.40	0.75	1.00	0.33	1.00	0.78	0.33	1.00	1.00	0.78
Attractiveness	0.33	1.00	1.00	0.78	1.00	0.80	0.42	0.74	1.00	0.41	0.47	0.63	0.62	0.42	0.43	0.49	0.56	0.52	0.68	0.58

shows the GRA-FMEA results for expert 1. The same calculations were carried out with expert2 and expert2 evaluations.

Table 24. GRA-FMEA Average of the results of experts.

	EdX				Khan Ac				Doulingo				Pluralsight				Sololearn
	S	O	D	R	S	O	D	R	S	O	D	R	S	O	D	R	S	O	D	R
Fault tolerance	0.72	0.69	0.51	0.25	1.00	0.61	0.55	0.33	0.87	0.64	0.53	0.29	0.53	0.40	0.47	0.10	0.62	0.70	0.49	0.21
Time behavior	0.63	0.71	0.47	0.21	0.63	0.53	0.63	0.21	0.79	0.67	0.63	0.34	0.44	1.00	0.47	0.21	0.62	0.63	0.51	0.20
Suitability	0.74	0.91	0.47	0.32	0.78	0.85	0.75	0.50	0.78	0.78	0.89	0.54	0.69	0.33	1.00	0.23	0.60	0.71	0.41	0.18
Understandability	0.56	1.00	0.74	0.41	1.00	0.89	0.93	0.82	0.78	0.89	0.78	0.54	0.79	0.81	0.44	0.28	0.90	0.65	0.49	0.29
Learnability	0.56	0.33	1.00	0.19	1.00	0.71	0.50	0.36	0.43	0.56	0.81	0.19	0.33	0.33	1.00	0.11	0.48	0.33	1.00	0.16
Operability	1.00	0.80	0.51	0.41	1.00	0.89	0.93	0.82	1.00	0.67	0.38	0.26	1.00	0.56	1.00	0.56	0.56	0.56	1.00	0.31
Attractiveness	0.42	1.00	0.82	0.35	0.82	0.93	0.41	0.31	1.00	0.41	0.47	0.19	0.72	0.57	0.44	0.18	0.55	0.52	0.67	0.19
Overall GRA- RPN	2.13				3.36				2.35				1.67				1.53

shows the GRA-FMEA average results for three experts.

Table 25. MLP ranking based on conventional GRA-FMEA.

MLPs	Overall RPN	Ranking
SoloLearn	1.53	1
Pluralsight	1.67	2
EdX	2.13	3
Duolingo	2.35	4
Khan Academy	2.36	5

indicates the final ranking according to GRA-FMEA.

The visualization for the comparative analysis of the three methods is given in Figure 15, Figure 16, Figure 17 and Figure 18.

FMEA provides a baseline evaluation of the platforms based on fault tolerance, time behavior, and other criteria, but it treats all risk factors equally without accounting for subjective differences in the importance of each factor. This leads to outliers like Pluralsight’s high fault tolerance score. GRA-FMEA refines the evaluation by considering the relational closeness to an ideal platform, moderating extreme scores like Pluralsight’s, and offering a more balanced comparison across platforms. Platforms like Khan Academy and Duolingo improve in ranking based on user-friendly criteria such as learnability and understandability. GRA-FMEA-FAHP enhances the evaluation further by introducing fuzzy logic to handle uncertainty in expert judgments. This results in a more realistic and balanced ranking of the platforms, with EdX and SoloLearn emerging as more well-rounded platforms across all criteria and Khan Academy standing out in learnability and user accessibility. The combination of FMEA, GRA, and FAHP shows the evolution of the evaluation process. FMEA provides the base risk analysis. GRA helps in making better relational comparisons. FAHP adds flexibility and refinement to adjust for expert input, making it the most robust and balanced evaluation method among the three. GRA-FMEA-FAHP is the most reliable method because it balances the quantitative risk analysis of FMEA with relational comparisons from GRA and adjusts for subjective importance and uncertainty through FAHP. This produces more robust, well-rounded results that reflect both objective and subjective factors relevant to the decision-making process. Traditional FMEA tends to produce extreme values—either high (SoloLearn) or low (Khan Academy, EdX), which may miss some nuances. The GRA-RPN method provides a relational comparison to an ideal solution, which results in more moderate scores. It highlights the relative performance differences more than absolute risk values. The FMEA-GRA-FAHP method shows the most balanced results, reflecting how risk perception changes when expert opinions and uncertainty are considered. It is a more nuanced and realistic method for comparing the platforms.

7. Discussion

The study aims to propose an efficient way to rank and choose the most suitable mobile learning platform by integrating risk analysis and multi-criteria decision-making methods to remedy the difficulty of selecting an adequate mobile learning platform among many that are available.

The proposed method uses unique, integrated risk analysis and multi-criteria decision-making approaches to overcome the shortcomings of the traditional FMEA approach. An integrated FMEA-GRA-FAHP method which is a combination of risk analysis and multi-criteria decision-making method for the suitability of mobile learning platforms. The initial implementation of grey relational analysis involved assuming that risk factors carried equal weights. In the subsequent application, distinct weights were assigned to risk factors through the utilization of Fuzzy AHP, as determined by evaluations from experts. This represents a significant advancement in the realm of failure mode and effect analysis (FMEA), as the introduction of varied weights through Fuzzy AHP introduces a novel approach to the FMEA methodology. This study aims to employ a multi-criteria method such as FMEA-GRA-FAHP to evaluate and rank the five selected MLPs. Furthermore, the study adopts the technical usability requirements concerning the ISO/IEC 9126 model to generate a suitable framework for the effective evaluation of the selected MLPs using such techniques. It was found that EdX, with a grey RPN of 19.0023, is the most reliable based on the quality evaluation process carried out in this research. Khan Academy is the 2nd in place with a grey RPN of 19.4994. For example, studies employing VIKOR, Analytical Network Process (ANP) have also found that platforms like EdX and Khan Academy consistently rank highly in terms of user satisfaction, hence usability and reliability, which is in alignment with the study findings [51,52,53]. Duolingo was third in place with an RPN value of 22.91, followed by SoloLearn with an RPN value of 25.10. Plural sight is ranked the lowest, considering the grey RPN value. According to the results, mobile learning platforms with higher grey RPN values are found to be less reliable and demonstrate highly significant risks or lower performance in certain criteria. In this case, Pluralsight, with the highest RPN, ranks the lowest, suggesting more potential risks exist as compared to the other platforms. Our study introduces the concept of risk analysis through FMEA, which has led to a few notable differences in ranking compared to other studies that emphasize user satisfaction alone. Meanwhile, Pluralsight, highly rated in terms of content quality employing the FAHP-TOPSIS approach [54], ranked lower in our analysis due to higher risks in fault tolerance and time behavior. This finding underlines the significance of integrating a risk assessment with multi-criteria decision-making as a framework for evaluating learning platforms, particularly for long-term usage in educational environments.

These findings show that the proposed methodology is robust and could be used effectively to locate a suitable mobile learning platform. The integrated FMEA-FAHP-GRA methodology provides a more nuanced analysis by prioritizing risk factors such as fault tolerance and time behavior, which previous studies using standalone methods may not have considered with the same rigor. The proposed methodology incorporates the risk assessment well with multi-criteria decision-making while compensating for the shortcomings of the conventional FMEA, where risk factors have equal weights. Another prevalent feature of this approach is the consideration of each factor using expert decision-making. GRA also has shortcomings in that risk factors have equal weights; however, in the proposed approach, different weights could be assigned by using the FAHP method, which improves the accuracy of results. Additionally, adopting a framework using ISO/IEC 9126 quality in-use international standards for software quality strengthens the evaluation framework.

The FMEA-GRA-FAHP approach has shown to be a reliable and effective strategy for assessing and classifying mobile learning platforms. The study’s findings are consistent with the idea that adding expert knowledge via Fuzzy AHP to risk analysis offers a more complex and trustworthy framework for making decisions, particularly when several platforms need to be assessed according to various criteria.

Table 13, Table 14, Table 15, Table 16 and Table 17 showed FMEA-FAHP grey RPN results obtained for each MLP. The conventional RPN is evaluated based on Equation (12). The reason was that for every FMEA assessment process carried out, the risk priority number was evaluated. The grey RPN is achieved using Equation (11).

The impact of the assigned weights is obvious after the calculation of grey RPN. It is seen that, even when the value of each risk factor is the same for two different criteria under an MLP, the grey RPN seems to be different, while the conventional RPN remains the same because the weight has a significant influence on the results. Instances are observed under EdX and Sololearn MLPs. In Table 4, for EdX MLP, under the criteria Understandability and Learnability, the RPN remain the same because of the same value of risk factors. However, the grey RPN is different. This is repeated under Sololearn MLP for learnability and operability.

The conclusive ranking of the MLPs depends on the summation of the grey RPN, which is obtained from the values of each criterion [10]. From Table 18, the MLP ranking using the FMEA-FAHP technique indicates Pluralsight MLP is ranked as the most vulnerable and has a high potential of failure, followed by SoloLearn. However, EdX was found to be the most reliable based on quality evaluation using the FMEA-FAHP technique. EdX and Khan Academy are ranked to have the least potential for failure. The final ranking of the mobile learning platforms is shown in Table 18.

The adopted framework in this study emerged from the ISO/IEC 9126 standards, and decisions made to access them using the FMEA were based on the judgment of the three experts. The FMEA was used to assess the potential risks or failures based on the adopted framework, and the FAHP was used to prioritize the weights of the risk factors of the FMEA; hence, a ranking was created for the mobile learning platforms with the integration of both FMEA and FAHP.

In this study, alternatives are ranked based on seven criteria with the use of the FMEA-FAHP technique. In previous studies where AHP was used, it was observed that AHP was unable to account for ranking inconsistencies, and hence, FAHP is preferred and proven to be more efficient than the regular AHP approach [55].

Researchers in [39] evaluated mathematical learning applications with a combination of the FAHP and TOPSIS methods. In their work, a model was proposed that considered technical and non-technical aspects to provide a framework for evaluation. The technical aspect was coined from the ISO/IEC 9126 and was based on quality evaluation, while the non-technical aspects of the framework focused on user satisfaction. In their study, the FAHP was used to assign weights to each criterion in the framework, and TOPSIS was used in ranking the mathematical applications, but in the case of this study, the FAHP is used to assign weight to the risk factors associated with FMEA and these risk factors are used to access potential risk or failures associated with each criterion in the working framework.

Other studies have employed the use of MCDM in comparing approaches to evaluation and combined them for a software evaluation, but this study combines FMEA and recognizes FMEA as an MCDM situation in prioritizing the risk factors, and hence, the combination of FMEA-FAHP is unique in terms of MLPs evaluation. This is the first study to combine both approaches and effectively rank mobile learning platforms with the use of Fuzzy AHP from the failure mode effect analysis perspective.

While this study focused on five mobile learning platforms (MLPs) in a specific educational context, the methodology—an integrated approach combining Failure Mode and Effects Analysis (FMEA), Fuzzy AHP (FAHP), and Grey relational analysis (GRA)—is designed to be scalable and adaptable to various settings. The strength of this method lies in its ability to handle multi-criteria decision-making (MCDM) problems by weighing qualitative and quantitative data. This makes it applicable to a wide range of platforms and educational environments.

The proposed method can easily handle a larger number of platforms by scaling the FMEA and FAHP processes. The ranking procedure, based on GRA –FAHP and the calculated risk priority numbers (RPNs), is not limited to a specific number of platforms. Increasing the number of platforms would primarily require more expert input for the FAHP process, but the methodology itself remains robust and can handle the added complexity.

The methodology is not only limited to the evaluation of mobile learning platforms. The evaluation framework can be easily applied to assess a variety of educational contexts using software. Risk factors, such as time behavior, fault tolerance, and operability, can be tailored to match the critical features of different applications as well. The FAHP process provides the adjustment of criteria and their relative importance depending on the specific needs of the educational context.

FMEA assessment ensures that platform-specific risks are identified and assessed based on the context. The proposed methodology provides flexible and adaptable features beyond the scope of mobile learning platforms such as e-learning systems, training platforms, or hybrid learning solutions.

8. Practical Implications

The relevant literature heavily focused on user satisfaction and content-system quality; however, educators should consider potential technical risks, and usability features are advised to since such issues may cause risks and could provide limited functionality, which can easily disrupt the learning process. Therefore, educators should consider the proposed aspects of the study in the careful selection of mobile learning platforms for their students. The findings of the study could be used as a guideline for educators who are aiming for reliable, mobile learning environments.

Developers of mobile learning platforms could benefit from the findings of the study by improving their products over problematic areas. Developers can significantly minimize the risks associated with fault tolerance and optimize time behavior to make platforms more attractive and usable. Maintaining a technically robust platform also reduces costs and platform failures.

The results of the study amplify a comprehensive and holistic risk-based approach for the integration of mobile learning tools into the curriculum, which aligns with policies supporting equitable and sustainable education for policymakers. Mobile learning platforms with high technical performance should also be available and accessible to all learners, encouraging lifelong learning.

9. Conclusions

The combination of FMEA-GRA-FAHP methods proved to be an effective technique in evaluating the quality of a mobile learning platform as well as ranking mobile learning platforms in order of their reliability.

Multi-criteria decision-making techniques play a vital role in the quality evaluation and testing of mobile learning platforms. The last COVID-19 pandemic situation has triggered rapid growth in the use of mobile learning platforms, especially in the educational sector. Evaluating such platforms by employing robust and reliable multi-criteria decision-making techniques plays a vital role in selecting proper mobile learning platforms and improving the vulnerabilities, hence providing better management of online learning.

This study utilizes the FMEA-GRA-FAHP technique in prioritizing risk factors. It is one of the most efficient weight-assigning MCDM techniques. FMEA is used to evaluate the potential risk associated with each platform based on the adopted framework which is coined from the ISO/IEC 9126 standard. The results of the FMEA are prioritized using the FAHP weights to assign risk priority numbers and, in turn, rank mobile learning platforms in their order of reliability from a quality perspective.

Other multi-criteria decision-making techniques, such as AHP, ANP, PROMETHEE, etc., can be integrated to further the selection and assessment of MLPs. Also, additional MLP samples can be added, and increasing the number of experts could be recommended for future work.

FMEA-GRA-FAHP is an efficient tool for evaluating the potential risk of a system considering the fact that the FAHP is able to compensate for the drawbacks of the traditional FMEA technique. This makes it recommendable in various industries, including the aerospace industries, electronic industries, and safety management. These are a few of the areas where the risk of failure is highly prioritized to maintain safety. Educators can leverage the use of the FMEA-GRA-FAHP approach for MLP ranking to ease the learning process and use the most efficient MLPs from a pool of available options. Institutions can also leverage the use of this technique in the teaching process to enable easier and more efficient means of learning for students and a more efficient means of teaching for lecturers.

Some potential limitations of the study could be listed as follows:

This study is delimited to enhancing FMEA methodology through the use of GRA-FAHP methods by assigning different weights to risk factors and ranking mobile learning platforms. The traditional FMEA, GRA-FMEA and the proposed methodology are also compared and contrasted in terms of robustness. However, in the future, different risk assessments and multi-criteria methods could be incorporated to perform fine-tuning of the results. In this study, the same importance was assigned to each expert judgment, as for future work, different weights could also be assigned to expert judgments. The evaluation framework emphasizes technical aspects of mobile learning platforms; however, researchers acknowledge the importance of non-technical aspects such as user engagement, accessibility, and equity for the pedagogical effectiveness of mobile learning platforms to enhance user experiences and outcomes in the educational milieu. Therefore, the crucial attributes should be considered for further evaluation. Another limitation of the study is that the number of experts is limited to three. For future work, the inclusion of more experts will yield a more representative sample of judgments. The same issue is also valid with the number of alternatives, including more mobile learning platforms for evaluation, which will improve the representability of the sample.