Next Article in Journal
Study on Driving Factors and Spatiotemporal Differentiation of Eco-Environmental Quality in Jianghuai River Basin of China
Previous Article in Journal
Practices to Improve the Sustainability of Australian Cold Storage Facilities
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Integration of Sustainability in Risk Management and Operational Excellence through the VIKOR Method Considering Comparisons between Multi-Criteria Decision-Making Methods

by
Eliana Judith Yazo-Cabuya
1,*,
Asier Ibeas
2 and
Jorge Aurelio Herrera-Cuartas
1
1
Facultad de Ciencias Naturales e Ingeniería, Universidad de Bogotá Jorge Tadeo Lozano, Carrera 4 #22-61, Bogotá 110311, Colombia
2
Departamento de Telecomunicaciones e Ingeniería de Sistemas, Universitat Autònoma de Barcelona, 08193 Barcelona, Spain
*
Author to whom correspondence should be addressed.
Sustainability 2024, 16(11), 4585; https://doi.org/10.3390/su16114585
Submission received: 10 April 2024 / Revised: 17 May 2024 / Accepted: 21 May 2024 / Published: 28 May 2024

Abstract

:
In the current context, organizations face an important challenge in managing risks related to environmental, social and governance (ESG) issues. This research presents a general method for prioritizing organizational risks with a focus on sustainability based on the characterization of five typologies of organizational risks and their respective sub-risks, based on an analysis of global reports. Subsequently, paired surveys are administered to a group of experts from various sectors, who assign importance to the organizational sub-risks. Their responses serve as the basis for the prioritization of these risks, using the VIšekriterijumsko KOmpromisno Rangiranje (VIKOR) method, which highlights the following most relevant organizational sub-risks for each type of risk: (1) Lack of ethics in the conduct of business (geopolitical risk); (2) Deficit in economic growth (economic risk); (3) Chemical safety (social risk); (4) Massive data fraud or theft incidents (technological risk); and (5) Water depletion (environmental risk). Additionally, a sensitivity analysis is performed to determine the robustness of the results of the VIKOR method and then compare the correlation coefficients with respect to the results obtained in previous studies for the AHP and ANP methods. Finally, we propose the implementation of a model to manage organizational risks, which are addressed proactively through an integral vision, allowing for continuous improvement and alignment with corporate strategy by means of an operational excellence management system (OEMS).

1. Introduction

Today’s global landscape is immersed in a complex web of challenges and pressures from multiple sources, ranging from environmental crises to social demands. In this context, global risks and stakeholder expectations are significantly shaping the environmental, social and governance (ESG) landscape [1]. These risks could impact the profitability, success and even the continuity of organizations, so it is crucial that each organization establishes its own characterization of organizational risks with a focus on sustainability. This definition must be adapted to its business model, internal and external environment, and its mission, vision and values [2]. In response to growing social awareness and the impact of global risks, regulators and governments have undertaken substantial measures to address these challenges [3].
To ensure the long-term sustainability of organizations, the best available practices should be adopted, including the integration of ESG factors into their strategy, process organization and assurance reporting. This not only involves minimizing losses, but also identifying opportunities linked to sustainability, which can generate value for both an organization and its stakeholders [4]. In addition, they must understand the risks and opportunities associated with sustainability and establish both internal and external controls. Their role is fundamental to achieving the Sustainable Development Goals (SDGs), which implies not only recognizing risks, but also managing, leading and evaluating performance effectively through a results-oriented strategy [5]. Integrity in the management of risk management associated with ESG issues must be addressed with special attention given depending on the type of organization, so an adequate identification, evaluation, prioritization and control process must be carried out to respond correctly to each risk [2]. It is recommended that risk management professionals work together with other experts to establish more precise approaches and take rapid action based on established needs and scopes. The diversity of SDG-related risks requires careful assessment and prioritization that are tailored to each organizational context. This need for prioritization is supported by international standards and recognized reports, such as the “Global Sustainable Development Report”, the “Enterprise Risk Management-Integrating with Strategy and Performance” and the “Global Risk Report” [1,2,6].
It is necessary to implement various strategies to address the challenges related to risk assessment and prioritization at the organizational level. A well-known tool in risk management is multi-criteria decision-making (MCDM) methods. These methods allow conflicts of interest to be resolved by integrating qualitative and quantitative data, facilitating the selection of appropriate actions. In addition to their versatility, they consider a wide range of stakeholders throughout the decision-making process, promoting transparency and participation at all stages of the process [7].
Additionally, in the review by Stojčić et al. (2019) [8], the importance of MCDM methods in the field of sustainable engineering is highlighted, exploring effective approaches to address problematics in this discipline, which is characterized as having various forms of uncertainty. The authors demonstrate how MCDM methods emerge as suitable tools to facilitate decision-making processes in the field of sustainable engineering.
Several studies have applied MCDM methods for risk assessment and/or prioritization in different settings. Among them is the research of Zheng et al. (2022) [9], who implemented a method combining the analytical hierarchy process (AHP) with the decision-making trial and evaluation laboratory (DEMATEL) method to assess the risks associated with floods. Additionally, Gökler and Boran (2023) [10] examine the selection of resilient and sustainable suppliers using an MCDM approach for a company in the automotive sector. In their analysis, they integrate the use of the D-number to find an efficient and feasible solution to uncertain information, together with the AHP and DEMATEL methods to determine the weights of criteria and sub-criteria. The authors indicate that the proposed model is robust within subjective assessments. The study by Balsara et al. (2019) [11] addresses the need to reduce greenhouse gas emissions in the cement industry, for which they identify climate change mitigation strategies, and priorities are established based on the use of methods such as the AHP and DEMATEL. The study by Daimi and Rebai (2023) [12] proposes a transportation sustainability governance index based on 29 indicators, applied to eight regional transit operators in Tunisia, through AHP and analytical network process (ANP) approaches in order to assess service sustainability, indicating the advantages of the ANP method over the AHP by incorporating criteria interdependencies. For their part, Liu et al. (2021) [13] propose the sustainable implementation of a product-service system, taking into account the value interactions between multiple stakeholders. Multidimensional value elements are identified and prioritized using a hybrid model incorporating DEMATEL, the ANP and Gray’s theory. Their results provide theoretical guidance for the successful implementation of a sustainable product-service system, highlighting the importance of value coordination among stakeholders.
The ideas of the aforementioned authors highlight the applicability and effectiveness of MCDM methods in risk assessment and prioritization in various contexts. In the cases highlighted, MCDM methods such as the AHP, DEMATEL and ANP have been used, leading to robust and relevant results. These findings highlight the importance of using integrated and multidisciplinary approaches to address risk and sustainability management challenges in different sectors. In this work, we propose using the VIšekriterijumsko KOmpromisno Rangiranje (VIKOR) method, whose English translation is Compromise Multicriteria Ranking, as it emphasizes dominance, which is important in this research in the framework of multicriteria optimization [14]. It is a commitment-ranking method that seeks a solution that maximizes group utility and minimizes individual regret [15]. This would complement the studies of Yazo-Cabuya et al. (2024a) [16] and Yazo-Cabuya et al. (2024b) [17] from the perspective of dealing effectively with uncertainty and functioning as a method in complex environments. Additionally, the VIKOR method offers a “compromise solution” which consists of a ranking composed of one alternative or a set of alternatives, depending on whether they meet the “lead rate” requirements.
It is important to highlight the relationship between the VIKOR, ANP and AHP methods in the context of this study. Although they are distinct approaches, each brings valuable insights to address the risk prioritization problem. While the AHP is used to structure preferences and rank criteria, the resulting weights are used in VIKOR as parameters of the algorithm to identify an optimal solution and evaluate the distance between alternatives and ideal and negative benchmarks.
By using the AHP weights from the research of Yazo-Cabuya et al. (2024a) [16] on VIKOR, information provided by experts is incorporated, and the AHP methodology is leveraged to assign weights to the criteria. This increases the credibility and robustness of the decision-making process in VIKOR, while allowing for an objective evaluation of alternatives. In addition, the integration of the AHP and VIKOR allows us to leverage the strengths of both methods, combining the structured criteria ranking of the AHP with VIKOR’s ability to find optimal and satisfactory solutions. On the other hand, the ANP and AHP are used to structure and analyze the relationships between criteria and elements, allowing for a deeper assessment of the relative importance and interdependencies between them. These methodologies complement each other, offering a comprehensive approach to prioritize risks. By evaluating the robustness and stability of the models through a sensitivity analysis and correlation coefficient, our aim is to select the model that demonstrates a greater level of consistency and reliability in its results, ensuring effective risk management.
The complexity of today’s problems requires multiple perspectives in decision making. In many cases, group decisions are necessary due to the lack of individual knowledge to solve them. Common approaches are the voting model and market mechanisms [18,19]. Decision making does not always imply total consensus, nor does it require the participation of all members in every aspect. It is crucial that each member of the group processes the data and contributes their expertise. If necessary, they can make relevant recommendations; by considering diverse points of view, collective decision making enriches each individual’s perspective [20]. The VIKOR method offers the ranking of alternatives according to their priority, with a compromised response closer to the ideal response [21]. This method is based on the concept of commitment programming to establish preference ranking, considering both individual and group regrets in relation to the outcomes [22].
Table 1 presents an analysis of the knowledge gaps in the area of organizational risks with a focus on sustainability, highlighting the critical areas in which we lack understanding or research on how to address these risks in organizational management. Through our review of different sources, gaps that require attention to improve the ability of organizations to manage risks effectively and sustainably are identified.
Throughout different studies, the VIKOR method has been employed in multi-criteria decision making and optimization problem solving, as is the case of the research of Jianxing et al. (2021) [23], in which they propose an improved risk assessment method for subsea pipelines in the oil and gas industry. They use cloud model theory and VIKOR to address linguistic uncertainties and improve assessment accuracy. A case study is included to validate the effectiveness of the proposed approach, demonstrating its superiority over existing methods. Furthermore, in the research by Bakioglu and Atahan (2021) [24], the authors highlight the importance of prioritizing the risks associated with autonomous vehicles, which are considered fundamental for a sustainable transportation system. A hybrid MCDM approach combining the AHP, the technique for order preference by similarity to an ideal solution (TOPSIS) and VIKOR in a Pythagorean fuzzy setting is proposed. The study validates the proposed model through a sensitivity analysis and shows that it produces reliable and useful results for planners and policy makers. The research by Cheng et al. (2021) [25], proposes a comprehensive framework for sustainability enterprise risk management that considers social, environmental, technological and economic aspects. A fuzzy decision-making approach, VIKOR-q-ROFSs, is introduced to identify and evaluate criteria for sustainability enterprise risk management. The VIKOR method is used to rank and evaluate manufacturing SMEs as alternatives. The results show that technological suitability is the most important risk factor, followed by technological advancement and other criteria related to safety and technological feasibility. The study demonstrates the efficiency and effectiveness of the proposed method in SME risk assessment. Our review of these highlighted studies serves as a solid basis for the development of the present methodology. Despite its use in various contexts, the VIKOR method has not been applied specifically to prioritize organizational risks with a focus on sustainability. This study addresses this gap by proposing its application in this area.
As highlighted by Pamučar et al. (2016) [26], solutions are often not analyzed with multiple MCDM methods, and consequently, sensitivity analyses, which serve to determine the quality of the results, are not performed. That is, a sensitivity analysis defines their stability or behavior against small modifications in the selected preferences, allowing one to validate the best MCDM method for a case study [27]. In addition, in the uncertainty evaluation of mathematical methods such as the AHP, according to the research of Yazo-Cabuya et al. (2024a) [16], the ANP in the study of Yazo-Cabuya et al. (2024b) [17] and VIKOR in this research, the development of a sensitivity analysis plays a fundamental role because it provides relevant information on how methods behave, their structure and their response to variations in inputs. This paper will perform a sensitivity analysis for these methods and compare their results. By assessing the consistency and coherence of the results among the three methods named above, their ability to effectively address risk prioritization can be better understood. We can determine whether VIKOR provides reliable results relative to those of the ANP and AHP, allowing for informed and evidence-based decision making in the risk management process.
Once the risks and sub-risks have been prioritized, the most robust MCDM method is used to adopt adequate organizational risk management with a focus on sustainability. For this, operational excellence management systems (OEMSs) are used as a part of a proactive approach that includes early identification, the implementation of monitoring controls and alignment with corporate strategy. It is therefore important to point out that an OEMS is presented as an integrated and structured approach that seeks to protect people, the environment and an organization’s assets, driving continuous improvement and ensuring adaptability to changing environments. The integration of risk and sub-risk prioritization results obtained through MCDM methods strengthens the OEMS’s ability to proactively manage organizational risks and ensure operational excellence.
The prioritization of organizational risks is a fundamental element for their strategic management. One of the purposes of this study is to prioritize organizational sub-risks with a focus on sustainability, contributing to the management of these risks. In this sense, this research focuses on exploring and addressing this challenge, as previously, analytical methods such as the AHP and ANP were used for this purpose [16,17]. Therefore, it is hypothesized that the VIKOR method, despite not having been greatly explored in the sustainability and risk management literature, is a powerful tool to assess organizational risk with focus on sustainability and is able to provide comparable results in the prioritization of organizational sub-risks in relation to the AHP and ANP methods. A detailed comparison between these methods is proposed to determine which is more suitable and stable in a specific context [14,15,21,22]. By offering a methodological approach to risk prioritization, the aim is to improve the quality of the decisions made and minimize possible biases in qualitative assessment. This approach addresses a fundamental aspect of organizational risk management that has so far received little attention in existing studies. Therefore, the objective of this research is to compare the results of the proposed method (VIKOR) with the AHP and ANP in order to determine which one provides the most consistent and reliable results in the specific context of the case study. Furthermore, this analysis aims not only to identify the most appropriate method for risk prioritization but also to understand how each approach responds to the complexities and dynamics of organizational risk in today’s business environment. Finally, this study seeks to enrich theoretical knowledge in risk management and provide organizational leaders with a solid and well-founded basis for strategic decision making in identifying and mitigating the risks they face.
This research is developed in five phases: (i) the methodology that will serve as a guide for the entire research process is defined; (ii) subsequently, the application of a paired survey designed using the 1AK tool (a one-click survey) to a group of experts made up of executives from various industries is carried out; (iii) using the obtained data, it is proposed to perform a prioritization of sub-risks through the VIKOR method; (iv) then, a sensitivity analysis and a comparative analysis of correlation coefficients are conducted for the results obtained with the AHP and ANP, according to the studies of Yazo et al. (2024a) [16], Yazo et al. (2024b) [17], and the results obtained with the VIKOR method in this research; and (v) finally, the development of an OEMS is proposed, for the achievement of risk-based objectives according to organizational strategy.

2. Methodology

The study of organizational risks with a focus on sustainability has been of significant importance at the business and academic levels, as they have gained special importance in terms of corporate opinions and influences. With this in mind, the implementation of decision support tools has become relevant to support companies in integrating ESG risks into their processes. Sustainability risk assessment has several sources of uncertainty, long lead times, large capital investments and diverse stakeholders. Therefore, it is relevant to use multi-criteria methodologies in order to reflect the complexity of assessments for decision makers, holistically covering organizational risks with a focus on sustainability (economic, environmental, geopolitical, social and technological risks) [16,28].
Figure 1 shows the methodological process used to address organizational risks with a focus on sustainability. The diagram details key steps, from the selection of alternatives and criteria to expert evaluations and the application of decision-making methods. The purpose of this research is to ensure efficient risk management, foresee possible challenges ahead and promote sustainability in the operation of organizations.

2.1. Selection of Alternatives and Criteria

The procedure to identify organizational risks with a focus on sustainability through global ESG reports begins with the review of recognized reports in the field of sustainability, such as the Global Sustainable Development Report of the WBCSD and COSO (2018) [2], the Enterprise Risk Management—Integrating with Strategy and Performance of the Independent Panel of Scientists appointed by the Secretary General (2019) [6] and the 2020 Global Risk Report of the World Economic Forum (WEF) [29]. Subsequently, an analysis and ranking of the identified risks and sub-risks are carried out to determine which ones will be studied. During this process, each risk is thoroughly examined to understand its relevance to organizational dynamics, leading to the characterization of five risk categories: geopolitical, economic, social, technological and environmental, along with their respective sub-risks. At the end of this phase, the characterized risks and their corresponding sub-risks are documented in Figure 2. This analysis provides a complete understanding of each type of risk, ensuring that the identification process is transparent and providing stakeholders with the necessary information to adequately address organizational risks with a focus on sustainability. In addition, this process allows us to examine emerging trends and patterns to anticipate future challenges, assessing the relevance of each risk in the specific context of the present case study [30].

2.2. Group of Experts

This research is enriched by the contribution of a diverse and experienced team of professionals from various industrial sectors. The selected panel of experts coincides with that reported in the studies of Yazo-Cabuya et al. (2024a) [16] and Yazo-Cabuya et al. (2024b) [17]. The detailed composition of this expert panel is presented in Table 2. The leaders of this team occupy key roles in their respective organizations and play a key role in decision making related to risk management.
Once the group of experts is formed, their opinions are collected through a survey using the 1AK (one-click survey) tool. The experts evaluate the influence and relevance of organizational risks with a focus on sustainability. The assessment scale provided in Table 3 facilitates a detailed comparison of the previously defined sub-risks. The diversity of disciplines among the experts guarantees an unbiased assessment of the case study, eliminating any bias towards a specific sector.
In this study, the VIKOR method was used to determine the preference of the criteria (sub-risks). After collecting the experts’ opinions through the survey, and in order to effectively integrate the information obtained from the survey, the individual comparison matrices were formed as follows: for each pair of factors (i and j), where i and j represent the sub-risks belonging to the same typology, the matrix ( C i j p ) is constructed for each pair of comparisons provided by each of the p = 79 experts:
C i j p = 0 c i j p 0 1
The determination of each cij element was performed by applying the geometric mean method (GMM), considering the evaluations of the 79 experts as follows [31]:
c i j = p = 1 79 c i j p 1 79
where p represents each expert. An iterative process was carried out following Equation (2), evaluating the different categorized sub-risks.
Research on group decision making has focused on combining preferences for continuous response distributions and processing information collectively. The challenges related to group decision making are varied and share certain fundamental characteristics, such as the presence of multiple criteria, objectives and attributes, as well as the presence of conflicts between these factors and group interests. The participation of all decision makers is sought, emergent rules are considered and individual interests within the group are encouraged [32]. This process involves evaluating criteria and alternatives, as criteria vary in importance and alternatives vary in preference. MCDMs are widely used in various fields, such as management, economics, engineering and social sciences, and are noted for their effectiveness in addressing practical decision problems. In MCDM methods, decision making is based on multiple, often conflicting, criteria that address a variety of problems. They fall into two main categories: multi-objective decision making (MODM), which focuses on continuous decision spaces, and multi-attribute decision making (MADM), which focuses on discrete decision spaces. MCDMs allow for the management of preferences over alternatives with multiple attributes, which often conflict with each other [33,34,35,36,37]. Thus, once the survey responses are collected, the analysis of results for the hierarchization of sub-risks is performed through the VIKOR method. This method was created to optimize complex systems using multiple criteria. It provides a compromise ranking list, a compromise solution and the weight stability intervals for this solution. Its approach focuses on ranking and selecting among alternatives in the presence of conflicting criteria by introducing a multi-criteria ranking index based on “closeness” to the “ideal” solution [15]. Additionally, it is essential to perform a sensitivity analysis to evaluate the stability of priorities during the decision-making process. Since the weights depend on subjective judgments, this tool allows for the exploration of variations in the criteria, providing valuable information on the consistency and stability of the prioritization [38,39].
Considering the hierarchization of organizational sub-risks with a focus on sustainability derived from the VIKOR method, as well as the subsequent sensitivity analysis and the comparative analysis of correlation coefficients in the AHP [16], ANP [17] and VIKOR methods, the need arises to address risk management from a macro-perspective based on a system denominated as OEMS. This concept is on the rise, and several organizations have been exploring it, but for many, it has been elusive. With this management system, organizations proactively address their risks through early identification of risks, implementation of monitoring controls, continuous improvement and alignment with corporate strategy. If this system is properly implemented, it helps organizations to maintain results, ensuring the protection of their stakeholders; at the same time, it guarantees sustainability and quality in their operations [40].

3. Results

The following are the results of our research along with an analysis, which are derived from the application of the methodologies and tools previously discussed.

3.1. Application of the VIKOR Method

The VIKOR method is an MCDM technique that selects the best option among alternatives by considering multiple conflicting criteria. It uses a ranking index based on the proximity to the ideal solution and provides weight stability intervals. This index was calculated using Google Colab, an online platform that provides free access to computational resources. This simplified data processing and the execution of complex algorithms. Appendix A.1, Appendix A.2, Appendix A.3, Appendix A.4 and Appendix A.5 can be consulted for additional details on the methodology and obtained results. The step-by-step development of this method is presented below [41].
Step 1: The standardization of the criteria. Normalization in the VIKOR methodology is performed to ensure that all criteria are on the same scale and can be compared equally. The codes set out in Appendix A.1, Appendix A.2, Appendix A.3, Appendix A.4 and Appendix A.5 use min–max normalization to carry out this process. The formula for min–max normalization is as follows:
c i j = max ( c ij ) i                   c i j m a x ( c i j ) i                   m i n ( c i j ) i                  
where c i j is the normalized value of criterion j for alternative i, c i j is the value of criterion j for alternative i, m i n ( c i j ) i is the minimum value of all the alternatives in criterion j and m a x ( c i j ) i is the maximum value of all the alternatives in criterion j.
The results of the normalization are shown in Table 4, Table 5, Table 6, Table 7 and Table 8.
Step 2: The determination of the criteria weights w j . The weights of the criteria ( w j ) can be determined using different methods; in this case, the AHP is used. These weights represent the relative importance of each criterion in decision making; for the case study, the vectors resulting from the research of Yazo-Cabuya et al. (2024a) are used [16].
Step 3: The calculation of the distance to the ideal solution ( S i ). The distance of each alternative to the ideal solution is calculated. This is performed by considering the difference between the minimum value of the geometric mean of the experts and the normalized value of the alternative, weighting this difference by the weight of the criterion and summing the results of all criteria, as shown below:
S j = i = 1 n   w i m a x ( c i j ) i                   c i j m a x ( c i j ) i                   m i n ( c i j ) i                    
where w i is the weight of criterion j, c i j is the value of criterion j for alternative i, m i n ( c i j ) i   is the minimum value of all alternatives in criterion j and m a x ( c i j ) i   is the maximum value of all alternatives in criterion j.
Step 4: The calculation of the distance to the negative solution ( R i ). The negative solution represents the least favourable possible outcome that an alternative can achieve in terms of the criteria considered. For each alternative, we calculate the weighted sum of the distances between the normalized value of the geometric mean of the experts and the minimum value of the criteria and then take the maximum of these sums across all alternatives, as shown below:
R j = m a x i   [ w i m a x ( c i j ) i                   c i j m a x ( c i j ) i                   m i n ( c i j ) i                   ]
where w i is the weight of criterion j, c i j is the value of criterion j for alternative i, m i n ( c i j ) i   is the minimum value of all alternatives in criterion j and m a x ( c i j ) i   is the maximum value of all alternatives in criterion j.
Step 5: The calculation of the VIKOR score ( Q i ). The VIKOR score is calculated for each alternative by combining the distances into the ideal solution and the negative solution. This is performed using a decision parameter λ, which takes the value of λ = 0.5, corresponding to a “consensus” situation, as shown below:
Q i = λ × S j m i n ( S i ) i             m a x ( S i ) i                   m i n ( S i ) i                   + 1 λ × R j m i n ( R i ) i                   m a x ( R i ) i                   m a x ( R i ) i                  
where R j is the distance to the negative solution for alternative i, S j is the distance to the ideal solution for alternative i, m i n ( R i ) i   and m a x ( R i ) i   are the minimum and maximum value of the distances to the negative solution, respectively, and m i n ( R i ) i   and m a x ( R i ) i   are the minimum and maximum value of the distances to the ideal solution, respectively.
Step 6: The prioritization hierarchy. The alternatives are ordered according to their VIKOR scores, from highest to lowest. This provides the prioritization hierarchy of the alternatives, as shown in Table 9, with the scores ordered from lowest to highest.
This allows for a systematic and detailed evaluation of the sub-risks within each risk typology. In the following section, a discussion of the findings obtained along with a sensitivity analysis for the VIKOR method will be carried out with the AHP results of the study by Yazo-Cabuya et al. (2024a) [16] and the ANP results of the study by Yazo-Cabuya et al. (2024b) [17].

3.2. Sensitivity Analysis in MCDM Methods

This section explores the sensitivity analyses for the VIKOR and ANP methods from the study by de Yazo-Cabuya et al. (2024b) [17] and the AHP method from the study by Yazo-Cabuya et al. (2024a) [16]. In addition, a comparison of the correlation coefficients by risk typology for the developed MCDM methods is presented, providing a comparative view of their effectiveness. This analysis will provide an understanding of how each methodology addresses and assesses organizational risks. It will also provide a preview of the development of the OEMS, pointing out its importance for efficient risk management. This system will be key to optimizing internal organizational processes, ensuring an effective response to prioritized risks. Prior to the development of this section, it is important to mention that a range of 0.5 to 1.0 will be used for the sensitivity analysis in the VIKOR method, specifically for changes in the λ parameter. In the case of the ANP and AHP methods, it will be applied in the criteria comparison matrices.

3.2.1. Sensitivity Analysis in VIKOR

A sensitivity analysis is performed to assess how the VIKOR results change with changes in the decision parameter λ. This range is chosen in order to assess how varying the parameter λ affects the tradeoff between the best and worst value for each criterion. By starting at 0.5 and gradually increasing to 1.0, it explores how changes in λ affect the relative ranking of the alternatives, allowing for a more comprehensive assessment of the sensitivity of the VIKOR method to different settings of λ. By allowing for an effective exploration of the sensitivity of the model to changes in this parameter, we can ensure consistency and relevance in the analysis. To this end, the VIKOR calculations are repeated for different values of λ, and it is observed how the prioritization hierarchies of the alternatives vary [15], as evidenced by the codes in Appendix A.1, Appendix A.2, Appendix A.3, Appendix A.4 and Appendix A.5. The VIKOR scores were calculated using Equation (6), and the results shown in Table 10, Table 11, Table 12, Table 13 and Table 14 were obtained.

3.2.2. Sensitivity Analysis in ANP

In the ANP sensitivity analysis, the criteria comparison matrices are adjusted by multiplying the original values by a variation factor δ; in this case, the range of 0.5 to 1.0 is maintained. This iterative process allows us to assess how the priorities of the criteria change when the relative preferences between them are altered. It is performed on the criteria comparison matrices because these represent the importance relationships between the criteria, and altering them reflects changes in decision makers’ perceptions of the relative importance of the criteria. By adjusting these values, it is possible to explore how variations in preferences affect the final priorities of the criteria, providing a more complete understanding of the sensitivity of the decision system to different scenarios and conditions. The priorities of the elements in the decision network are then recalculated using the ANP method. This involves updating the priority matrices and reapplying the normalization and aggregation process to obtain new, final priorities [17]. The development of this process is carried out through the codes in Appendix A.11, Appendix A.12, Appendix A.13, Appendix A.14, Appendix A.15, and the results shown in Table 15, Table 16, Table 17, Table 18 and Table 19 are obtained.

3.2.3. Sensitivity Analysis in AHP

In the AHP sensitivity analysis, the criteria comparison matrices are adjusted by multiplying the original values by a variation factor δ; in this case, the range of 0.5 to 1.0 is maintained. The priorities of the elements in the decision network are then recalculated using the AHP method of the study by Yazo-Cabuya et al. (2024a) [16]. The development of this process is carried out through the codes in Appendix A.6, Appendix A.7, Appendix A.8, Appendix A.9 and Appendix A.10, and the results shown in Table 20, Table 21, Table 22, Table 23 and Table 24 are obtained.

3.2.4. Correlation Coefficient

Once the results of the variations are obtained, the correlation coefficient between each resulting vector and the original priority vector is calculated (see Table 25). This coefficient shows the strength and direction of the relationship between the different models and the priority that was initially established, as shown below:
r = i = 1 n ( x i x ¯ ) ( y i y ¯ ) i = 1 n ( x i x ¯ ) 2 i = 1 n ( y i y ¯ ) 2
where r is the correlation coefficient, x i are the values of the variable x in a sample, x ¯ is the mean of the values of the variable x, y i are the values of the variable y in a sample and y ¯ is the mean of the values of the variable y.

4. Discussion

The results obtained demonstrate the importance of risk prioritization in strategic management, particularly highlighting the need to address sub-risks from a sustainability perspective. In the context of risk prioritization, it is important to note that the AHP, ANP and VIKOR methods play an important role in risk analysis and assessment, each method having its own specific relevance and benefits. Our detailed comparison of these methods has allowed us to identify relevant characteristics that contribute to a better understanding of how to address organizational risks with a focus on sustainability more effectively. The ANP shows a correlation closer to one in the comparative analysis of correlation coefficients, suggesting its robustness. The importance of the AHP should not be underestimated, as it facilitated the selection of the best alternatives and the prioritization of sub-risks and proved effective in balancing expert opinions and avoiding biases. Further, the sensitivity analysis enriched decision making, with a realistic view of the results. In the case of VIKOR, it is highlighted that it is a method that emphasizes dominance in a multi-criteria optimization context, its compromise is effective in evaluating multiple alternatives and it stands out for its ability to strike a balance between conflicting criteria. Overall, these methods offer an interesting perspective on risk assessment and prioritization and provide sound approaches to address risk management challenges efficiently.

Organizational Risk Management with a Focus on Sustainability through an OEMS

Interest in risk management has grown markedly; organizations face significant pressure to strengthen their risk management systems as stakeholder expectations are increasingly challenging [42]. In a dynamic and competitive context, improvements depend largely on the ability of organizations to manage their risks effectively. Sustainability also comes into play, as organizations must balance risk management with meeting stakeholder expectations and promoting responsible organizational practices [43,44].
To address this situation effectively, the design and implementation of an OEMS is proposed to complement the development of the hierarchical ranking methodology proposed in this study. The novelty of this system lies in a general vision of the organization, in which, starting from the articulation, three fundamental pillars are proposed (see Figure 3): (i) strategic planning; (ii) operation (which includes both processes and projects); and (iii) management systems. This macro-system provides organizations with a mechanism to proactively address their risks, allowing for early detection and the implementation of monitoring controls that contribute to continuous improvement and align with corporate strategy. According to Lutchman et al. (2019) [40], many authors define an OEMS as an integrated, organized and structured approach that seeks to protect people, the environment and the assets of organizations in order to improve. This integrated approach should help companies manage risks and opportunities more effectively, thereby improving their ability to create and preserve value for all stakeholders.
In addition to providing an overall orientation, strategic planning also provides direction and key objectives, while operation deals with the practical execution of processes and projects. Complementarily, management systems establish frameworks and standards to ensure effectiveness and compliance with requirements from a legal perspective. To support effectiveness and adaptability to changing environments, these pillars depend critically on monitoring control and assurance. Monitoring control allows for continuous supervision, identifying deviations and necessary adjustments, while assurance asserts that processes and practices adhere to standards defined during planning, materializing through periodic audits, in which auditors conduct ongoing assessments of risks, sub-risks and controls [45]. Additionally, the implementation of a performance measurement system, such as the Balanced Scorecard (BSC), helps organizations to control various activities. It fulfils the objective of providing a strategic perspective and monitors the company’s operations by aligning them with its vision and mission, encompassing both financial and non-financial metrics [46]. These elements combined constitute a holistic and dynamic approach to managing risks and ensuring continuous improvement in all areas of an organization.
An OEMS is a risk-based system and a systematic approach to identify, assess, prioritize and control risks, starting with the detailed characterization of risks that may impact the organization’s strategic objectives. Through an organizational risk ranking analysis with a focus on sustainability using MCDM methods, risks and sub-risks are assessed and prioritized, ensuring data-driven decision making. According to the studies by Yazo-Cabuya et al. (2024a) [16] and Yazo-Cabuya et al. (2024b) [17], the usefulness of the AHP and ANP methods for organizational risk prioritization with a focus on sustainability is supported, highlighting that the high similarity of these methods reinforces their usefulness in organizations.
Risk management is monitored with the BSC, in which related strategic objectives are identified, methods aligned with these are developed and resources are allocated to implement them effectively. Control and assurance are carried out through continuous monitoring and evaluation mechanisms, supported by integrated management systems and the measurement of key performance indicators (KPIs). Finally, continuous improvement is emphasized through feedback and process review activities to adapt to changes in the business environment and strengthen its long-term risk management capability. The information obtained is analyzed to monitor performance and continuous improvement and used to generate periodic reports. These are prepared following the guidelines of the Global Reporting Initiative (GRI) standards, which provide a comprehensive view of organizational performance in relation to strategic objectives and identified risks. In addition, GRI reports facilitate transparent communication and accountability by presenting detailed trend analyses, risk assessments and recommendations for continuous improvement. By disclosing results through GRI methodologies, the organization demonstrates its commitment to transparency and sustainability, promoting trust among stakeholders and strengthening its reputation in the market [47,48].
The step-by-step process for the articulation of the OEMS management model is detailed below:
  • The identification and characterization of risks
This process starts with the identification and characterization of risks that may affect the strategic objectives of the organization. In this stage, a thorough analysis is carried out to identify all potential risks that could hinder the achievement of its strategic objectives. In addition to identification, a detailed characterization of each risk is carried out, including its description, origin, potential impact and any other relevant aspect that allows for further understanding of its nature. This step is essential to establish a solid basis for effective risk assessment and management later in the process.
2.
Risk assessment and prioritization
In this stage, the identified risks are assessed and prioritized. This method, widely used in multi-criteria decision making, allows for the prioritization of risks by considering both the expected value of the risks and the associated uncertainty. By using an MCDM method to assess and prioritize risks, more informed and data-driven decision-making is ensured, allowing the organization to focus its efforts on the most significant risks and mitigate their impact on the achievement of strategic objectives.
3.
Risk management
During this step, the organizational strategy and strategic cores are defined along with their respective objectives in order to develop plans that are aligned with the organizational context. In addition, resources are allocated to address risks, following a detailed and coordinated approach as follows:
  • Definition of organizational strategy: A company’s organizational strategy should be aligned with its mission, vision and corporate values. The organizational strategy should provide clear and coherent guidance for decision making, enabling the organization to identify and address risks that may impact the achievement of its goals in a timely manner.
  • Identification of related strategic objectives: Strategic objectives that are directly related to the priority risks identified during the assessment are identified. This involves setting clear objectives that reflect the strategic direction of the organization and the KPIs that will be used to measure the company’s progress towards achieving these objectives;
  • Development of management strategies: For each prioritized risk, specific management strategies are developed that are aligned with the identified strategic objectives. These strategies may include actions to avoid, mitigate, transfer or accept the risks as appropriate for the organization’s situation and objectives;
  • Resource Assignment: The necessary resources are allocated to implement mitigation strategies, ensuring that they are prioritized and executed effectively to reduce the likelihood and impact of the identified risks.
Managing prioritized risks through strategic BSC objectives enables the organization to improve its ability to anticipate and respond to challenges in the business environment, thus ensuring the achievement of its goals and long-term success.
4.
Control and assurance
Ongoing monitoring and evaluation mechanisms are established to monitor the effectiveness of mitigation strategies and their impact on the achievement of strategic objectives. This enables adjustments and modifications to be made as necessary to ensure that risks are managed efficiently and that the desired results are achieved.
  • Integrated management systems (IMSs) play a crucial role in collecting relevant data from various sources within the organization and facilitating their analysis for performance monitoring and continuous improvement. In addition to data collection and analysis, IMSs also support the generation of regular reports that provide a comprehensive view of organizational performance in relation to its strategic objectives and identified risks. These reports include detailed trend analyses, risk assessments and recommendations for continuous improvement. They also facilitate internal and external audits to assess the effectiveness of the organization’s processes and ensure compliance with legal and regulatory requirements and strategic objectives. In this way, IMS audits complement the review process by providing an independent and objective assessment of management system performance, identifying areas for improvement and promoting transparency and accountability in the organization;
  • The measurement of KPIs, which provide a clear view of organizational performance in relation to strategic objectives and identified risks, facilitating assurance and the identification of areas for continuous improvement;
  • The implementation of tools such as the BSC, which allow for the measurement and monitoring of various activities to be managed, fulfilling the objective of providing a strategic perspective, with both financial and non-financial measurements, and obtaining control and management of the organization’s performance;
  • The disclosure of results through GRI methodologies to promote transparency and accountability.
5.
Continuous improvement
Feedback, performance analyses and process review activities are carried out to identify areas of opportunity and develop corrective and/or preventive actions. Continuous improvement allows the model to adapt to changes in the business environment, thereby strengthening the organization’s ability to effectively manage risks and achieve its long-term strategic objectives.

5. Conclusions

The development of the VIKOR model has allowed for a thorough and balanced assessment of sub-risks, providing a holistic view of the critical areas requiring priority. This assessment highlights the need to develop proactive strategies to address the risks identified in each of the five typologies characterized. By prioritizing the sub-risks within each risk typology, a solid basis is established for the effective implementation of preventive and corrective measures that contribute to comprehensive risk management in the organization. The three highest prioritized sub-risks for each risk typology were determined from the results, ordered from lowest to highest according to the results of the VIKOR method, obtaining the following:
  • Geopolitical: 1.1 Lack of ethics in the conduct of business; 1.5 Corruption and instability; and 1.3 Lack of transparency in taxation;
  • Economic: 2.6 Deficit in economic growth; 2.5 Water scarcity and sanitation; and 2.8 Partnerships to achieve objectives;
  • Social: 3.6 Chemical safety; 3.7 Demographic and health risks; and 3.8 Lack of well-being and health;
  • Technological: 4.5 Massive data fraud or theft incidents; 4.4 Large-scale cyber-attacks; and 4.6 Connectivity failures;
  • Ambiental: 5.4 Water depletion; 5.1 Carbon emissions; and 5.3 Vulnerability to climate change.
Subsequently, the results of the comparative analysis between three multi-criteria decision-making models, the AHP, ANP and VIKOR models, revealed correlation coefficients highly close to one for all models. This indicates a strong and consistent correlation between the priorities obtained under variations in the input data and the original priority vector. It is important to note that the ANP and AHP demonstrated high levels of consistency and stability in their results, as indicated by the high correlation coefficients obtained (0.9985 and 0.9965, respectively). However, although VIKOR shows a slightly lower correlation (0.9813), it has significant advantages in risk management. The VIKOR method has proven to be useful in situations in which uncertainty and ambiguity are prominent, as it enables balanced and satisfactory solutions to be found in complex and dynamic environments. This suggests that, while the ANP and AHP can provide a consistent assessment of risks, VIKOR adds a dimension with a practical and robust approach to address risk management in challenging scenarios.
Addressing issues related to organizational risk assessment and prioritization through MCDM methods allows for the integration of qualitative and quantitative data, facilitating the selection of appropriate actions and promoting stakeholder transparency and participation. In addition, our case study developed with the input of experts from various industrial fields highlights the suitability of MCDM methods to address the complexities of organizational risk assessment and prioritization with a focus on sustainability. The results obtained through the proposed methodology not only highlight the effectiveness of its application, but also reveal a high degree of consensus among the experts. This helps to consolidate confidence in the methodology’s ability to offer practical and applicable solutions in a variety of business contexts. Our analysis of the results of this study corroborates our hypothesis, demonstrating the effectiveness and relevance of MCDM methods in addressing organizational risks with a focus on sustainability, especially concerning the use of the VIKOR methodology. Furthermore, by comparing these results with alternative approaches, the theoretical benefits of using MCDM methods in risk management are highlighted, providing a solid basis for strategic decision making in their respective contexts and for future research. This study represents a significant step towards integrated risk management by providing a comprehensive framework covering risk identification, assessment, prioritization and control. By carefully structuring this methodology, a precise adaptation to the specific needs of each business sector is ensured. This enables organizations to implement customized controls and effectively address identified priority risks, helping to improve their ability to meet the challenges of today’s business environment. In addition, it improves our understanding of organizational risks, enriching our co-understanding of risk management and providing a solid basis for strategic decision making. Finally, the proposed implementation of the OEMS offers a complete view of the organization, so this is suggested to be considered further in future research. It will strengthen organizations’ resilience, facilitate adaptations to environmental changes, maximize opportunities for their sustainable growth and contribute to their long-term success.

Author Contributions

Conceptualization, E.J.Y.-C.; methodology, E.J.Y.-C., J.A.H.-C. and A.I.; software, E.J.Y.-C.; validation, E.J.Y.-C., J.A.H.-C. and A.I.; formal analysis, E.J.Y.-C.; investigation, E.J.Y.-C.; data curation, E.J.Y.-C.; writing—original draft preparation, E.J.Y.-C.; supervision, J.A.H.-C. and A.I. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

All data regarding the results of this research are available upon request from the authors.

Acknowledgments

A special thanks to Jorge Eliécer Moreno for considering the relevance of this research and actively contributing to our consultations with the technical experts along with the analyzed case studies.

Conflicts of Interest

The authors declare no conflicts of interest.

Appendix A

Appendix A.1

Appendix A.2

Appendix A.3

Appendix A.4

Appendix A.5

Appendix A.6

Appendix A.7

Appendix A.8

Appendix A.9

Appendix A.10

Appendix A.11

Appendix A.12

Appendix A.13

Appendix A.14

Appendix A.15

References

  1. WEF. World Economic Forum, The Global Risks Report; WEF: Cologny, Switzerland, 2024; Available online: https://www3.weforum.org/docs/WEF_The_Global_Risks_Report_2024.pdf (accessed on 25 January 2024).
  2. WBCSD; COSO. Enterprise Risk Management Applying Enterprise Risk Management to Environmental, Social and Governance-Related Risks. 2018. Available online: https://docs.wbcsd.org/2018/10/COSO_WBCSD_ESGERM_Guidance.pdf (accessed on 16 November 2023).
  3. Dyczkowska, J. Sustainable Performance in Business Organisations and Institutions: Measurement, Reporting and Management; Wrocławiu, W.U., Ed.; Publishing House of Wroclaw University of Economics and Business: Wrocław, Poland, 2023. [Google Scholar]
  4. Yılmaz, A.; Flouris, T. Managing corporate sustainability: Risk management process-based perspective. Afr. J. Bus. Manag. 2010, 4, 162–171. [Google Scholar]
  5. Boffo, R.; Patalano, R. ESG Investing: Practices, Progress and Challenges; OECD: Paris, France, 2020; Available online: https://oecd.org/finance/ESG-Investing-Practices-Progress-and-Challenges.pdf (accessed on 12 February 2024).
  6. Independent Group of Scientists Appointed by the Secretary-General. Global Sustainable Development Report 2019: The Future Is Now—Science for Achieving Sustainable Development; United Nations: Geneva, Switzerland, 2019. [Google Scholar]
  7. Mehak, A.; Mu, Y.; Mohsin, M.; Zhang, X.-C. MCDM-Based Ranking and Prioritization of Fisheries’ Risks: A Case Study of Sindh, Pakistan. Sustainability 2023, 15, 8519. [Google Scholar] [CrossRef]
  8. Stojčić, M.; Kazimieras Zavadskas, E.; Pamučar, D.; Stević, Ž.; Mardani, A. Application of MCDM Methods in Sustainability Engineering: A Literature Review 2008–2018. Symmetry 2019, 11, 350. [Google Scholar] [CrossRef]
  9. Zheng, Q.; Shen, S.-L.; Zhou, A.; Lyu, H.-M. Inundation risk assessment based on G-DEMATEL-AHP and its application to Zhengzhou flooding disaster. Sustain. Cities Soc. 2022, 86, 104138. [Google Scholar] [CrossRef]
  10. Gökler, S.; Boran, S. A novel resilient and sustainable supplier selection model based on D-AHP and DEMATEL methods. J. Eng. Res. 2023, in press. [Google Scholar] [CrossRef]
  11. Balsara, S.; Kumar Jain, P.; Ramesh, A. An integrated approach using AHP and DEMATEL for evaluating climate change mitigation strategies of the Indian cement manufacturing industry. Environ. Pollut. 2019, 252, 863–878. [Google Scholar] [CrossRef]
  12. Daimi, S.; Rebai, S. Sustainability performance assessment of Tunisian public transport companies: AHP and ANP approaches. Socio-Econ. Plan. Sci. 2023, 89, 101680. [Google Scholar] [CrossRef]
  13. Xiahui, L.; Deng, Q.; Gong, G.; Zhao, X.; Li, K. Evaluating the interactions of multi-dimensional value for sustainable product-service system with grey DEMATEL-ANP approach. J. Manuf. Syst. 2021, 60, 449–458. [Google Scholar] [CrossRef]
  14. Ferreira, A.; Bhaya, A. Extended Vikor (Evikor): A New Proposal for Ranking Based on Non-Dominance and Rankability. Comput. Ind. Eng. 2023. [Google Scholar] [CrossRef]
  15. Opricovic, S.; Tzeng, G.-H. Compromise solution by MCDM methods: A comparative analysis of VIKOR and TOPSIS. Eur. J. Oper. Res. 2004, 156, 445–455. [Google Scholar] [CrossRef]
  16. Yazo-Cabuya, E.; Herrera-Cuartas, J.; Ibeas, A. Organizational Risk Prioritization Using DEMATEL and AHP towards Sustainability. Sustainability 2024, 16, 1080. [Google Scholar] [CrossRef]
  17. Yazo-Cabuya, E.; Ibeas, A.; Herrera-Cuartas, J. Integrating Sustainability into Risk Management through Analytical Network Process. Sustainability 2024, 16, 2384. [Google Scholar] [CrossRef]
  18. Lu, J.; Zhang, G.; Ruan, D.; Wu, F. Multi-Objective Group Decision Making Methods, Software and Applications with Fuzzy Set Techniques; Imperial College Press: London, UK, 2007. [Google Scholar]
  19. Dong, Y.; Xu, J. Consensus Building in Group Decision Making; Springer: Singapore, 2015. [Google Scholar]
  20. Saaty, T.; Peniwati, K. Group Decision Making: Drawing Out and Reconciling Differences; RWS Publications: Pittsburgh, PA, USA, 2013. [Google Scholar]
  21. Amin, F.; Dong, Q.-L.; Grzybowska, K.; Ahmed, Z.; Yan, B.-R. A Novel Fuzzy-Based VIKOR–CRITIC Soft Computing Method for Evaluation of Sustainable Supply Chain Risk Management. Sustainability 2022, 14, 2827. [Google Scholar] [CrossRef]
  22. Huang, J.-J.; Tzeng, G.-H.; Liu, H.-H. A Revised VIKOR Model for Multiple Criteria Decision Making—The Perspective of Regret Theory. Commun. Comput. Inf. Sci. 2009, 35, 761–768. [Google Scholar] [CrossRef] [PubMed]
  23. Jianxing, Y.; Shibo, W.; Haicheng, C.; Yang, Y.; Haizhao, F.; Jiahao, L. Risk assessment of submarine pipelines using modified FMEA approach based on cloud model and extended VIKOR method. Process Saf. Environ. Prot. 2021, 155, 555–574. [Google Scholar] [CrossRef]
  24. Bakioglu, G.; Atahan, A. AHP integrated TOPSIS and VIKOR methods with Pythagorean fuzzy sets to prioritize risks in self-driving vehicles. Appl. Soft Comput. 2021, 99, 106948. [Google Scholar] [CrossRef]
  25. Cheng, S.; Jianfu, S.; Alrasheedi, M.; Saeidi, P.; Mishra, A.; Rani, P. A New Extended VIKOR Approach Using q-Rung Orthopair Fuzzy Sets for Sustainable Enterprise Risk Management Assessment in Manufacturing Small and Medium-Sized Enterprises. Int. J. Fuzzy Syst. 2021, 23, 1347–1369. [Google Scholar] [CrossRef]
  26. Pamučar, D.; Božanić, D.; Ranđelović, A. Multi-criteria decision making: An example of sensitivity analysis. Serbian J. Manag. 2016, 12, 1–27. [Google Scholar] [CrossRef]
  27. Muñoz, B.; Romana, M.; Ordóñez, J. Análisis de Sensibilidad de una Metodología de Decisión Multicriterio desarrollada para la Selección de Tipologías de Estructuras de Contención en una Autovía Urbana. In Proceedings of the Congresos de la Universitat Politècnica de València, CIT2016. Congreso de Ingeniería del Transporte, Valencia, Spain, 7–9 June 2016. [Google Scholar] [CrossRef]
  28. Martín-Gamboa, M.; Iribarren, D.; García-Gusano, D.; Dufour, J. A review of life-cycle approaches coupled with data envelopment analysis within multi-criteria decision analysis for sustainability assessment of energy systems. J. Clean. Prod. 2017, 150, 164–174. [Google Scholar] [CrossRef]
  29. WEF. World Economic Forum, The Global Risks Report; WEF: Cologny, Switzerland, 2020; Available online: https://www3.weforum.org/docs/WEF_Global_Risk_Report_2020.pdf (accessed on 15 February 2024).
  30. WEF. World Economic Forum, The Global Risks Report; WEF: Cologny, Switzerland, 2023; Available online: https://www3.weforum.org/docs/WEF_Global_Risks_Report_2023.pdf (accessed on 5 January 2024).
  31. Kułakowski, K. On the Geometric Mean Method for Incomplete Pairwise Comparisons. Mathematics 2020, 8, 1873. [Google Scholar] [CrossRef]
  32. Hwang, C.; Lin, M. Group Decision Making under Multiple Criteria: Methods and Applications; Springer: Berlin/Heidelberg, Germany, 2012. [Google Scholar] [CrossRef]
  33. Kerr, N.; Tindale, R. Group performance and decision making. Annu. Rev. Psychol. 2004, 55, 623–655. [Google Scholar] [CrossRef] [PubMed]
  34. Jassbi, J.; Ribeiro, R.; Dargam, F. Dynamic MCDM for Multi Group Decision Making. In Group Decision and Negotiation. A Process-Oriented View; Springer: Cham, Switzerland; Toulouse, France, 2014; pp. 90–99. [Google Scholar] [CrossRef]
  35. Ahmad, Q.; Khan, M.; Ahmad, N. A Group Decision-Making Approach in MCDM: An Application of the Multichoice Best–Worst Method. Appl. Sci. 2023, 13, 6882. [Google Scholar] [CrossRef]
  36. Hosseinzadeh, M.; Kamaran Hama, H.; Yassin Ghafour, M.; Masdari, M.; Hassan Ahmed, O.; Khezri, H. Service Selection Using Multi-criteria Decision Making: A Comprehensive Overview. J. Netw. Syst. Manag. 2020, 28, 1639–1693. [Google Scholar] [CrossRef]
  37. Nijssen, D. Improving Spatiality in Decision Making for River Basin Management. 2013. Available online: https://d-nb.info/1258516853/34 (accessed on 15 February 2024).
  38. Borgonovo, E.; Plischke, E. Sensitivity analysis: A review of recent advances. Eur. J. Oper. Res. 2016, 248, 869–887. [Google Scholar] [CrossRef]
  39. Borgonovo, E. Sensitivity Analysis an Introduction for the Management Scientist, 1st ed.; Springer: Cham, Switzerland, 2017. [Google Scholar] [CrossRef]
  40. Lutchman, C.; Lutchman, K.; Akula, R.; Lyons, C.; Hashmi, W. Operations Excellence Management System (OEMS): Getting It Right the First Time; Routledge: London, UK, 2019. [Google Scholar]
  41. Muñoz, B.; Romana, M. Aplicación de métodos de decisión multicriterio discretos al análisis de alternativas en estudios informativos de infraestructuras de transporte. Pensam. Matemático 2016, 6, 27–46. [Google Scholar]
  42. Paape, L.; Speklé, R. The Adoption and Design of Enterprise Risk Management Practices: An Empirical Study. Eur. Account. Rev. 2012, 21, 533–564. [Google Scholar] [CrossRef]
  43. Saeidi, P.; Parisa Saeidi, S.; Sofian, S.; Parastoo Saeidi, S.; Nilashi, M.; Mardani, A. The impact of enterprise risk management on competitive advantage by moderating role of information technology. Comput. Stand. Interfaces 2019, 63, 67–82. [Google Scholar] [CrossRef]
  44. Kashif Shad, M.; Lai, F.-W.; Lai Fatt, C.; Jaromír Klemeš, J.; Bokhari, A. Integrating sustainability reporting into enterprise risk management and its relationship with business performance: A conceptual framework. J. Clean. Prod. 2019, 208, 415–425. [Google Scholar] [CrossRef]
  45. Blanco, S. La Auditoría Interna y la Convergencia Hacia in Modelo de Aseguramiento, Monitoreo y Evaluación Continuo. 2019. Available online: https://economicas.bdigital.uncu.edu.ar/objetos_digitales/14863/blanco-fce.pdf (accessed on 15 February 2024).
  46. Kusrini, E.; Novia Sahraen, A. Risk mitigation using integration enterprise risk management and Balanced Scorecard model (A Case Study in a Consulting Services Company in Indonesia). Spektrum Ind. 2021, 19, 73–85. [Google Scholar] [CrossRef]
  47. Pulido Pineda, D.; Medina Correa, E. Tendencias en el Uso y Presentación de los Reportes de Sostenibilidad en Compañías Farmacéuticas a Nivel Mundial: “Divulgación y Transparencia Informativa para los Diferentes Stakeholders; Universidad Cooperativa de Colombia, Facultad de Ciencias Económicas, Administrativas y Contables, Contaduría Pública, Bogotá: Bogotá, Colombia, 2021; Available online: https://repository.ucc.edu.co/bitstreams/c4a0564c-3924-4f9d-bb4c-f8329d6dc6d5/download (accessed on 16 February 2024).
  48. Castillo-Muñoz, C.; Ripoll-Felieu, V.; Urquirdi, A. Divulgación de Responsabilidad Social Empresarial en América Latina: Evidencias en el uso de la Directriz GRI. Rev. Lasallista Investig. 2021, 18, 134–157. [Google Scholar] [CrossRef]
Figure 1. Research methodology flowchart.
Figure 1. Research methodology flowchart.
Sustainability 16 04585 g001
Figure 2. Characterization of organizational risks and sub-risks with a focus on sustainability. Reproduced with permission from Yazo-Cabuya et al., Sustainability, 2024a [16].
Figure 2. Characterization of organizational risks and sub-risks with a focus on sustainability. Reproduced with permission from Yazo-Cabuya et al., Sustainability, 2024a [16].
Sustainability 16 04585 g002
Figure 3. Risk management model from an OEMS approach.
Figure 3. Risk management model from an OEMS approach.
Sustainability 16 04585 g003
Table 1. Knowledge gaps on organizational risks with a focus on sustainability.
Table 1. Knowledge gaps on organizational risks with a focus on sustainability.
Author(s)TitleYearKnowledge Gap
Dong, Y.; Xu, J.Consensus Building in Group Decision Making.2015Need for group decision-making methods (requiring focus in the context of sustainability) [19].
Committee of Sponsoring Organizations of the Treadway Commission (COSO)Enterprise Risk Management: Applying enterprise risk management to environmental, social and governance-related risks.2018
  • Lack of ESG risk disclosure: approximately 35% of 170 organizations surveyed do not disclose their ESG risks [2].
  • More than 70% of professionals believe that current risk management practices with a focus on sustainability are not sufficient [2].
Independent Group of Scientists Appointed by the Secretary-General.Global Sustainable Development Report 2019: The Future is Now—Science for Achieving Sustainable Development.2019Lack of collaboration between risk management professionals and other experts to define more precise approaches to risk assessment [6].
Stojčić, M.; Kazimieras Zavadskas, E.; Pamučar, D.; Stević, Ž.; Mardani, A.Application of MCDM Methods in Sustainability Engineering: A Literature Review 2008–2018.2019There is a lack of research on validation of results in the context of sustainability, using multiple MCDM methods [8].
Amin, F.; Dong, Q.-L.; Grzybowska, K.; Ahmed, Z.; Yan, B.-RA Novel Fuzzy-Based VIKOR–CRITIC Soft Computing Method for Evaluation of Sustainable Supply Chain Risk Management2022Lack of application of advanced methods in sustainable risk management (requiring focus in the context of organizational sustainability) [21].
Ferreira, A.; Bhaya, A.Extended Vikor (Evikor): A New Proposal for Ranking Based on Non-Dominance and Rankability2023Lack of studies investigating the robustness and sensitivity of the VIKOR method in organizational risk prioritization with a focus on sustainability compared to other perspectives of multi-criteria decision making [14].
World Economic ForumGlobal Risk Report.2024Lack of understanding of the interaction between global risks and stakeholder expectations in the context of sustainability [1].
Table 2. Group of experts. Reproduced with permission from Yazo-Cabuya et al., Sustainability, 2024a [16].
Table 2. Group of experts. Reproduced with permission from Yazo-Cabuya et al., Sustainability, 2024a [16].
Type of PositionNumber of PeopleAverage Years of Experience
Accountants with specialization and/or Master’s degree in auditing, digital transformation and/or sustainability2220
Economists with specialization in risk management1815
Industrial engineers with specialization in occupational health and safety, sustainability and/or risk management1215
Systems engineering professionals with specialization in cybersecurity725
Psychologists with specialization in human resources1025
Environmental professionals with specialization in risk management1015
Table 3. VIKOR method comparison scale. Adapted from Opricovic et al. (2004) [15].
Table 3. VIKOR method comparison scale. Adapted from Opricovic et al. (2004) [15].
Linguistic ScaleValue
No influence1
Very low influence3
Low influence5
High influence7
Very high influence9
Table 4. Normalized geopolitical risk matrix.
Table 4. Normalized geopolitical risk matrix.
1.11.21.31.41.51.61.71.81.91.10
1.10.00000.00000.00000.00000.00000.01850.00000.00000.02050.0280
1.20.10360.06020.08500.10110.06930.04220.01940.02960.02390.0144
1.30.05250.02360.04000.06400.01090.04690.02800.01380.02150.0052
1.40.08130.01980.03630.06100.04400.00180.02240.01720.01490.0136
1.50.05050.04140.06310.05800.04030.00000.00190.00340.00000.0000
1.60.09810.07920.06780.11880.11490.06220.02210.01730.01940.0125
1.70.16110.10400.10730.12390.14020.09560.04950.03590.04220.0483
1.80.14810.09080.11640.12160.13070.09420.05660.04860.03750.0349
1.90.14230.10410.12290.13680.14910.10000.05350.05820.04570.0460
1.100.11400.10730.13290.13260.14240.10050.04980.05970.04740.0485
Table 5. Normalized economic risk matrix.
Table 5. Normalized economic risk matrix.
2.12.22.32.42.52.62.72.8
2.10.07330.13150.10020.05370.03700.15630.15530.0960
2.20.01320.08350.08560.00000.02510.07030.15770.0507
2.30.02930.07520.08240.09760.10420.16830.11620.1289
2.40.08230.12320.06890.07510.13150.06850.11190.1310
2.50.06980.07990.06950.03200.01070.04910.06340.0616
2.60.00000.07180.00000.06470.01070.04910.10210.0000
2.70.03180.00000.06770.04470.00000.00000.05750.1185
2.80.06660.08690.06970.04470.01070.08230.00000.0616
Table 6. Normalized social risk matrix.
Table 6. Normalized social risk matrix.
3.13.23.33.43.53.63.73.83.93.10
3.10.05390.08110.10460.07350.08100.14400.12160.11730.09850.1077
3.20.04660.06240.09420.05810.07280.11650.09830.06440.05820.0802
3.30.02830.01110.06170.05070.05290.12470.07010.08960.02460.0508
3.40.03890.05880.07080.05630.08510.07320.09300.10640.09800.0965
3.50.04450.05460.08430.03190.06900.14820.11790.08470.07530.0253
3.60.02830.00570.00000.04050.02380.00000.00000.00000.00000.0000
3.70.00630.00000.04780.02910.00000.10810.05620.06590.04990.0735
3.80.00000.05650.01430.00000.04920.12360.04750.05910.04830.0820
3.90.02590.06460.09040.00940.05710.13470.06830.07080.06010.0626
3.100.00200.03480.07810.01120.08770.15410.04110.02860.06290.0651
Table 7. Normalized technological risk matrix.
Table 7. Normalized technological risk matrix.
4.14.24.34.44.54.6
4.10.11350.06570.02620.15910.30730.0914
4.20.12950.08200.08990.24540.31720.1106
4.30.13650.07730.08290.25600.27170.1044
4.40.08580.03990.00000.00000.00000.0000
4.50.00000.00000.00000.01910.01980.0218
4.60.11720.07110.07360.26380.25590.0961
Table 8. Normalized environmental risk matrix.
Table 8. Normalized environmental risk matrix.
5.15.25.35.45.55.65.75.8
5.10.07290.02970.02980.16450.03300.10890.15750.0832
5.20.00000.00000.00000.00000.00000.00000.00000.0000
5.30.15270.04770.06980.05590.02850.02810.06290.1056
5.40.05670.05150.13860.13730.09290.09650.08410.0571
5.50.09460.05330.09810.04180.04140.04480.01000.0308
5.60.01350.02820.09510.05960.04140.04480.02800.0420
5.70.01290.04990.08130.11990.11890.10360.07380.0807
5.80.02020.03490.01730.09780.05100.04300.03270.0404
Table 9. Prioritization matrix by risk typology.
Table 9. Prioritization matrix by risk typology.
Item1. GeopoliticalItem2. EconomicItem3. SocialItem4. Technological Item5. Environmental
1.10.00002.10.93243.10.95564.10.86755.10.1180
1.20.55392.20.62603.20.70384.21.00005.21.0000
1.30.26962.30.99863.30.63414.30.89815.30.2060
1.40.33762.40.78083.40.67334.40.14395.40.0000
1.50.23952.50.13633.50.83414.50.00005.50.5866
1.60.64702.60.12523.60.00004.60.85665.60.5688
1.70.97162.70.23963.70.4887 5.70.3717
1.80.90672.80.16233.80.5815 5.80.5306
1.90.9551 3.90.7229
1.100.9168 3.100.7640
Table 10. Geopolitical sensitivity analysis with VIKOR method.
Table 10. Geopolitical sensitivity analysis with VIKOR method.
WeightVIKOR (Qi) Geopolitical Risk Scores
1.11.21.31.41.51.61.71.81.91.10
λ = 0.50.00000.55390.26960.33760.23950.64700.97160.90670.95510.9168
λ = 0.60.00000.55120.26930.32510.23450.63990.96590.90760.96410.9282
λ = 0.70.00000.54840.26910.31260.22960.63280.96020.90850.97300.9396
λ = 0.80.00000.54570.26890.30010.22470.62570.95450.90940.98200.9510
λ = 0.90.00000.54290.26870.28760.21970.61860.94880.91030.99100.9624
λ = 1.00.00000.54020.26850.27510.21480.61150.94320.91121.00000.9738
Table 11. Economic sensitivity analysis with VIKOR method.
Table 11. Economic sensitivity analysis with VIKOR method.
WeightVIKOR (Qi) Economic Risk Scores
2.12.22.32.42.52.62.72.8
e = 0.50.93240.62600.99860.78080.13630.12520.23960.1623
λ = 0.60.94590.57520.99840.82030.16350.10010.20040.1790
λ = 0.70.95950.52450.99810.85980.19080.07510.16110.1957
λ = 0.80.97300.47370.99780.89920.21800.05010.12180.2125
λ = 0.90.98650.42290.99760.93870.24530.02500.08250.2292
λ = 1.01.00000.37210.99730.97810.27250.00000.04320.2459
Table 12. Social sensitivity analysis with VIKOR method.
Table 12. Social sensitivity analysis with VIKOR method.
WeightVIKOR (Qi) Social Risk Scores
3.13.23.33.43.53.63.73.83.93.10
λ = 0.50.95560.70380.63410.67330.83410.00000.48870.58150.72290.7640
λ = 0.60.96450.71070.61270.69210.81130.00000.46750.55160.70170.7168
λ = 0.70.97340.71760.59120.71080.78850.00000.44630.52160.68040.6696
λ = 0.80.98230.72450.56980.72950.76580.00000.42500.49170.65910.6224
λ = 0.90.99110.73140.54830.74820.74300.00000.40380.46170.63780.5752
λ = 1.01.00000.73830.52690.76690.72020.00000.38250.43180.61660.5280
Table 13. Technological sensitivity analysis with VIKOR method.
Table 13. Technological sensitivity analysis with VIKOR method.
WeightVIKOR (Qi) Technological Risk Scores
4.14.24.34.44.54.6
λ = 0.50.86751.00000.89810.14390.00000.8566
λ = 0.60.84771.00000.90840.12940.00000.8640
λ = 0.70.82791.00000.91880.11480.00000.8715
λ = 0.80.80821.00000.92920.10020.00000.8790
λ = 0.90.78841.00000.93960.08570.00000.8864
λ = 1.00.76861.00000.95000.07110.00000.8939
Table 14. Environmental sensitivity analysis with VIKOR method.
Table 14. Environmental sensitivity analysis with VIKOR method.
WeightVIKOR (Qi) Environmental Risk Scores
5.15.25.35.45.55.65.75.8
λ = 0.50.11801.00000.20600.00000.58660.56880.37170.5306
λ = 0.60.10431.00000.21060.00000.55320.55640.31800.5301
λ = 0.70.09051.00000.21510.00000.51990.54400.26430.5297
λ = 0.80.07681.00000.21960.00000.48650.53160.21060.5292
λ = 0.90.06311.00000.22410.00000.45320.51910.15700.5287
λ = 1.00.04931.00000.22870.00000.41980.50670.10330.5282
Table 15. Geopolitical sensitivity analysis with ANP method.
Table 15. Geopolitical sensitivity analysis with ANP method.
WeightGeopolitical ANP Scores
1.11.21.31.41.51.61.71.81.91.10
δ = 0.50.39510.28250.33800.34700.37430.26730.15710.16520.13240.1349
δ = 0.60.00090.00070.00080.00080.00090.00060.00040.00040.00030.0003
δ = 0.70.00430.00310.00370.00380.00410.00290.00170.00180.00140.0015
δ = 0.80.01640.01170.01400.01440.01550.01110.00650.00680.00550.0056
δ = 0.90.05310.03800.04540.04670.05030.03590.02110.02220.01780.0181
δ = 1.00.15230.10890.13030.13380.14430.10300.06060.06370.05100.0520
Table 16. Economic sensitivity analysis with ANP method.
Table 16. Economic sensitivity analysis with ANP method.
WeightEconomic ANP Scores
2.12.22.32.42.52.62.72.8
δ = 0.50.22570.34200.27000.26210.34060.41880.39540.3393
δ = 0.60.00050.00080.00060.00060.00080.00100.00090.0008
δ = 0.70.00250.00370.00290.00290.00370.00460.00430.0037
δ = 0.80.00930.01420.01120.01080.01410.01730.01640.0140
δ = 0.90.03030.04600.03630.03520.04580.05630.05320.0456
δ = 1.00.08700.13180.10410.10100.13130.16150.15240.1308
Table 17. Social sensitivity analysis with ANP method.
Table 17. Social sensitivity analysis with ANP method.
WeightSocial ANP Scores
3.13.23.33.43.53.63.73.83.93.10
δ = 0.50.14790.21610.27250.19720.23230.37710.31120.30160.25850.2794
δ = 0.60.00030.00050.00060.00050.00050.00090.00070.00070.00060.0007
δ = 0.70.00160.00240.00300.00210.00250.00410.00340.00330.00280.0030
δ = 0.80.00610.00890.01130.00820.00960.01560.01290.01250.01070.0116
δ = 0.90.01990.02910.03660.02650.03120.05070.04180.04050.03470.0376
δ = 1.00.05700.08330.10510.07600.08960.14540.12000.11630.09970.1077
Table 18. Technological sensitivity analysis with ANP method.
Table 18. Technological sensitivity analysis with ANP method.
WeightTechnological ANP Scores
4.14.24.34.44.54.6
δ = 0.50.39380.24900.26900.63900.71590.3270
δ = 0.60.00090.00060.00060.00150.00170.0008
δ = 0.70.00430.00270.00290.00700.00780.0036
δ = 0.80.01630.01030.01110.02650.02960.0135
δ = 0.90.05290.03350.03620.08590.09620.0440
δ = 1.00.15180.09600.10370.24640.27600.1261
Table 19. Environmental sensitivity analysis with ANP method.
Table 19. Environmental sensitivity analysis with ANP method.
WeightEnvironmental ANP Scores
5.15.25.35.45.55.65.75.8
δ = 0.50.38490.15120.35670.41240.31430.29170.39860.2841
δ = 0.60.00090.00040.00080.00100.00070.00070.00090.0007
δ = 0.70.00420.00160.00390.00450.00340.00320.00430.0031
δ = 0.80.01590.00630.01480.01710.01300.01210.01650.0118
δ = 0.90.05170.02030.04790.05540.04220.03920.05360.0382
δ = 1.00.14840.05830.13750.15900.12120.11240.15370.1095
Table 20. Geopolitical sensitivity analysis using AHP method.
Table 20. Geopolitical sensitivity analysis using AHP method.
WeightGeopolitical AHP Scores
1.11.21.31.41.51.61.71.81.91.10
δ = 0.50.12980.10630.11830.11990.12520.10260.07700.07920.07060.0712
δ = 0.60.13600.10690.12160.12350.13010.10250.07270.07510.06540.0661
δ = 0.70.14230.10730.12470.12710.13500.10220.06840.07110.06050.0613
δ = 0.80.14850.10750.12760.13050.13980.10180.06430.06720.05590.0568
δ = 0.90.15480.10750.13030.13370.14450.10120.06040.06340.05150.0525
δ = 1.00.16110.10730.13290.13680.14910.10050.05660.05970.04740.0485
Table 21. Economic sensitivity analysis with AHP method.
Table 21. Economic sensitivity analysis with AHP method.
WeightEconomic AHP Scores
2.12.22.32.42.52.62.72.8
δ = 0.50.10180.12900.11220.11120.12990.14510.14130.1295
δ = 0.60.09760.12960.10970.10840.13050.14940.14460.1301
δ = 0.70.09360.13020.10730.10570.13100.15390.14780.1305
δ = 0.80.08970.13070.10490.10290.13130.15850.15110.1308
δ = 0.90.08600.13110.10250.10020.13150.16330.15440.1310
δ = 1.00.08230.13150.10020.09760.13150.16830.15770.1310
Table 22. Social sensitivity analysis with AHP method.
Table 22. Social sensitivity analysis with AHP method.
WeightSocial AHP Scores
3.13.23.33.43.53.63.73.83.93.10
δ = 0.50.07420.09120.10350.08640.09400.12520.11130.10940.10040.1043
δ = 0.60.06970.08930.10390.08380.09280.13070.11350.11110.10020.1051
δ = 0.70.06540.08730.10420.08120.09150.13630.11560.11280.09990.1058
δ = 0.80.06140.08520.10450.07860.09020.14210.11760.11440.09950.1065
δ = 0.90.05750.08320.10460.07600.08900.14800.11960.11590.09910.1071
δ = 1.00.05390.08110.10460.07350.08770.15410.12160.11730.09850.1077
Table 23. Technological sensitivity analysis with AHP method.
Table 23. Technological sensitivity analysis with AHP method.
WeightTechnological AHP Scores
4.14.24.34.44.54.6
δ = 0.50.15580.12150.12690.21740.23730.1411
δ = 0.60.15250.11300.11910.22730.25270.1352
δ = 0.70.14900.10480.11150.23700.26850.1292
δ = 0.80.14510.09690.10410.24630.28450.1231
δ = 0.90.14090.08920.09690.25530.30080.1169
δ = 1.00.13650.08200.08990.26380.31720.1106
Table 24. Environmental sensitivity analysis with AHP method.
Table 24. Environmental sensitivity analysis with AHP method.
WeightEnvironmental AHP Scores
5.15.25.35.45.55.65.75.8
δ = 0.50.13890.08300.13240.14560.12290.11850.14190.1168
δ = 0.60.14170.07620.13370.14960.12220.11680.14510.1147
δ = 0.70.14440.06980.13500.15340.12150.11500.14830.1126
δ = 0.80.14720.06390.13620.15720.12070.11300.15140.1103
δ = 0.90.15000.05840.13740.16090.11980.11100.15450.1080
δ = 1.00.15270.05330.13860.16450.11890.10890.15750.1056
Table 25. Comparison of correlation coefficients by risk typology and MCDM method.
Table 25. Comparison of correlation coefficients by risk typology and MCDM method.
RiskAHPANPVIKOR
Geopolitical0.99890.99890.9983
Economic0.99840.99910.9667
Social0.98820.99750.9664
Technological0.99900.99970.9964
Environmental0.99820.99730.9787
Average0.99650.99850.9813
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Yazo-Cabuya, E.J.; Ibeas, A.; Herrera-Cuartas, J.A. Integration of Sustainability in Risk Management and Operational Excellence through the VIKOR Method Considering Comparisons between Multi-Criteria Decision-Making Methods. Sustainability 2024, 16, 4585. https://doi.org/10.3390/su16114585

AMA Style

Yazo-Cabuya EJ, Ibeas A, Herrera-Cuartas JA. Integration of Sustainability in Risk Management and Operational Excellence through the VIKOR Method Considering Comparisons between Multi-Criteria Decision-Making Methods. Sustainability. 2024; 16(11):4585. https://doi.org/10.3390/su16114585

Chicago/Turabian Style

Yazo-Cabuya, Eliana Judith, Asier Ibeas, and Jorge Aurelio Herrera-Cuartas. 2024. "Integration of Sustainability in Risk Management and Operational Excellence through the VIKOR Method Considering Comparisons between Multi-Criteria Decision-Making Methods" Sustainability 16, no. 11: 4585. https://doi.org/10.3390/su16114585

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop