1. Introduction
With the continuous development of autonomous underwater vehicles (AUVs) in recent years, the confrontation ability of AUVs is improving gradually. In a complex underwater environment, target threat assessment and attack optimization are essential to determine countermeasures. They are also hot issues in intelligent decision making.
Threat assessment [
1,
2] aims to analyze the capability and intention of the target and provide a basis for the subsequent decision by quantifying the features contained in the target through a specific method. The core of threat quantification is also the process of data fusion in the situation of confrontation. This concept was first proposed in the 1980s, and threat assessment belongs to three-level data fusion [
3]. Due to the uncertainty of the confrontation environment and the complexity of the objects involved in the confrontation [
4], there are inevitably many problems that need to be solved in threat assessment. The threat assessment problem of multi-AUVs can be regarded as group objective analysis [
5]. In this kind of research, two types of information are generally used; one is the objective attributes of the environment and target [
6], and the other is subjective cognition based on previous expert experience [
7]. In the context of confrontation, multi-target threat assessment can be regarded as a kind of analytical decision problem [
8], and multi-target analysis can be regarded as the calculation of the comprehensive value of the target. Previous research on threat assessment algorithms has focused on information preprocessing and description of the objective world, such as interval fuzzy set [
9] and intuitionistic fuzzy set [
10] developed according to the fuzzy set principle [
11] but has seldom considered the principle of the target analysis method and the relationship with the analyzed target.
The problem addressed in this paper is multi-objective threat assessment under underwater adversarial conditions. Considering the existing research background, the following challenges are identified:
How to establish an efficient and reasonable threat assessment system and prove the effectiveness of the system;
Calculation and determination of multi-attribute weights, as the computation and determination of weights for multiple attributes pose a challenge in this research;
Insufficient analysis accuracy due to data dimensionality reduction caused by data fusion during the threat assessment process.
Threat assessment in the counter environment is subject to strong coupling, which is not only the connection among multiple targets but also the interaction among the inherent attributes contained by the target itself. The purpose of this paper is to work on a method based on target comprehensive value assessment through analysis of the inherent attributes of many incoming targets, as well as analysis of the relationship between targets and attributes, to assess the threat of incoming targets. In this method, the weight of the target attribute is unknown, and the method used to calculate the weight is crucial and should not only include the accuracy of the objective weight but also integrate the subjective cognition of the prior information. Compared with previous studies, the research presented in this paper has the following three main characteristics:
First, by integrating subjective and objective weightings in a comprehensive attribute evaluation algorithm, the proposed method enhances the applicability of the threat assessment system by facilitating the interactive analysis of threat assessment indicators;
Secondly, this paper introduces an ANP-entropy-based algorithm for calculating and combining subjective and objective weights. This method effectively addresses the issue of decreased accuracy caused by dimensionality reduction resulting from data fusion;
Finally, experimental analysis demonstrates that the proposed algorithm improves the accuracy of threat assessment, even when faced with reduced data dimensionality. This improvement is achieved by enhancing data complexity. The impact of system integrity on threat assessment is analyzed.
The organizational structure of this paper is as follows. First, we summarize previous research in
Section 2, providing a solid foundation for the research presented in this paper. In
Section 3, the multi-AUV threat assessment system is studied, establishing a more applicable multi-AUV threat assessment system for underwater adversarial scenarios based on a comprehensive attribute evaluation algorithm. On the basis of establishing the threat assessment system,
Section 4 puts forward the target comprehensive value assessment algorithm. In
Section 5, the calculation of the index weight in target comprehensive value evaluation is explained in detail.
Section 4 and
Section 5 mainly describe the secondary fusion of subjective and objective weights in the comprehensive attribute evaluation algorithm. This algorithm provides an approach for analysis of the weights of indicators in threat assessment. Experiments are designed in
Section 6 and
Section 7 to prove the effectiveness and superiority of the proposed algorithm.
Section 6 and
Section 7 provide experimental evidence for the effectiveness of the proposed algorithm. The advantages of the algorithm are comprehensively analyzed in terms of effectiveness, superiority, and accuracy. Finally, in
Section 8 and
Section 9, the research work of this paper is discussed, and the limitations of this paper are analyzed to suggest directions for future work.
2. Literature Review and Related Works
2.1. Research on Threat Assessment Systems
Threat assessment is a technology used to grade or rank assessed targets based on data analysis and data fusion [
12]. The main process of multi-objective threat assessment is shown in
Figure 1. As shown in the figure, the threat assessment process generally consists of the following steps: (1) analysis of target attributes, (2) identification of threat assessment indicators, (3) design of threat assessment algorithms, and (4) computation of threat assessment results. In essence, the process of building a threat assessment system is a complete analysis process of threat assessment [
13]. First, determine the evaluation object and analyze the properties of the evaluated object. The process can be based on sensor detection and a priori intelligence information collection, and the threat assessment index is determined based on the analysis of the target. Second, the data related to the index are preprocessed, especially in the evaluation process, where both accurate information and fuzzy information are preprocessed so that the index information can be uniformly analyzed in the evaluation model. Finally, a suitable threat assessment algorithm, which includes a learning and clustering algorithm [
14], a prediction algorithm [
15,
16,
17], multi-attribute decision making (MADM) [
18], etc., is designed to calculate the result of multi-target threat assessment by combining target attributes and index characteristics.
In recent years, research on the design of battlefield threat assessment systems has become the focus of intelligent decision making. The design of a threat assessment system is generally based on two aspects: an assessment object and an assessment algorithm. The evaluation index is determined according to the evaluation object, and the evaluation algorithm is designed to determine the elements to be included in the system. Based on the analysis of the environment [
19], a two-tier threat assessment system for multiple agents is established based on a transferable belief model, with one layer used to collect threat information and processing other information. Chen et al. [
20] divided threat targets into dynamic targets, static targets, and tactical targets to design a threat assessment grading system based on different threat sources. Yao et al. [
15,
16] comprehensively considered underwater threat assessment factors and established an underwater threat assessment system based on a dynamic Bayesian network, which provides a research basis for the analysis and formulation of underwater threat assessment criteria. Gao et al. [
18] started from the battlefield environment, comprehensively considered the characteristics of threat targets and environmental factors, and established a three-way decision-making threat assessment system. Carling [
21] analyzed the situation and threat assessment process used by the Navyand adopted an expert knowledge system to assess naval battlefield threat. Zhou et al. [
17] used a probability-based threat assessment system for accident prediction and collision avoidance in intelligent vehicle systems [
22]. Xu et al. [
23] designed a threat assessment system based on the Markov decision process and state feedback. Based on the above research on threat assessment system, it can be seen that there the following challenges remain in research on underwater multi-target threat assessment systems:
From the perspective of research objects, there are few studies on underwater threat assessment. Although the threat assessment algorithm has a certain degree of portability, the design of an assessment system for different confrontation scenarios often needs to consider environmental factors and target characteristics;
From the point of view of the problems associated with threat assessment, although there are some studies on AUV threat assessment systems, they are generally aimed at traditional problems such as path planning, which cannot be better applied to underwater countermeasures.
2.2. Threat Index Analysis and Weighting Algorithm
The problem of threat assessment in counter tasks is mainly associated with the measurement of the threat index and the design of a threat algorithm. The measurement of threat indicators includes the design, calculation, and determination of indicator weights. According to the existing research, there is no unified standard for the determination of indicators, and specific analysis should be conducted according to the characteristics of threat targets and the background of problems, while the calculation of indicators and their weights requires a systematic algorithm for reasonable evaluation. For the index weight determination method, the algorithm can be divided into two categories: subjective weight and objective weight. Typical objective weight methods include the information entropy method [
24,
25], the CRITIC weight method [
25], and the DS evidence theory method [
26]. The subjective weighting method is also widely used in solving battlefield threat assessment, such as analytic hierarchy process (AHP) [
27], fuzzy analytic hierarchy process (FAHP) [
28,
29], the expert system method [
30], etc.
In previous studies [
31], threat indicators in confrontation problems were comprehensively considered, and the AHP method was used to empower threat assessment indicators to solve threat assessment problems in naval battlefields. Zhao et al. [
32] carried out triangulation fuzzy processing on threat indicators and used the CRITIC weight method to objectively weight indicators to solve the problem of dynamic target threat assessment. With the deepening of the research, when solving the problem of multi-target threat assessment, the single weight calculation method cannot achieve comprehensive analysis of the target. The objective weighting algorithm relying on information collection and analysis has high requirements on the accuracy and differentiation of the data, while the subjective weighting method relying on prior information of the commanding decision maker cannot adapt to the rapidly changing objective world. The index weighting algorithm gradually develops towards the direction of a subject–object fusion algorithm. In [
24], a threat assessment algorithm based on the combination of AHP and information entropy was introduced to solve the problem of multi-target threat assessment in air combat, and subjective and objective weights were integrated to analyze air combat targets. Bao et al. [
33] combined AHP and the gray correlation method to solve the problem of index weight in multi-objective analysis. Subjective and objective fusion weighting offers advantages in the calculation of the weight of threat indicators in order to more objectively and profoundly show the actual situation. However, the AHP algorithm, which is often used in the calculation of subjective weight, requires the strict independence of threat assessment indicators when calculating the weight of indicators, while indicators in practical problems are often intricate and mutually influencing. The scientific nature and effectiveness of weight fusion are also urgent problems to be solved.
2.3. Research on Threat Assessment Algorithm
Multi-objective threat assessment algorithms are generally divided into two categories. The first category of algorithms selects assessment indicators based on the characteristics of the analysis target and environment and analyzes the target threat through index quantization and index weight calculation. This category belongs to the multi-attribute decision-making problem. The other category of algorithms is based on the method of machine learning, target analysis, and clustering to solve the target threat assessment ranking problem. TOPSIS [
34,
35,
36] is one of the methods often used to solve multi-target threat assessment. GAO et al. [
37] studied the multi-target threat problem in air combat, determined the index weight based on the cloud model, and calculated the threat degree of the incoming target using the TOPSIS algorithm. On this basis, Xiang et al. [
38] improved the index set of the multi-target threat ranking problem and sorted and graded air combat target threats by considering survivability, weapons, electronic warfare systems, and many other factors. In [
39], an intuitionistic fuzzy set was used not only to consider a more comprehensive index setbut also to carry out data processing and analysis on the indicators in TOPSIS, improving the existing methods of solving problems in data processing and using the interval SD-G1 method combined with subjective and objective calculation of interval number weights. Although the TOPSIS method achieves good performance in solving the multi-objective ranking problem, it is difficult to determine index weight in practical application, and it is difficult to distinguish multi-objective ranking when the gap between indices is not obvious. Therefore, the VIKOR decision algorithm [
40,
41] is often used in multi-attribute analysis and decision making. Some scholars [
25] used the VIKOR algorithm to analyze the attack ability, speed, distance, and angle of a target and finally complete the threat degree ranking of the incoming target. On the basis of the above research, Khorram [
42] comprehensively considered subjective and objective weights, conducted a comprehensive analysis of index weights, and further improved the weight calculation algorithm in the VIKOR algorithm.
In addition to the multi-target threat assessment algorithm based on multi-attribute decision making, there are also some studies that consider solving the threat assessment problem using a machine learning algorithm from the perspective of target clustering. Lee et al. [
8] recognized the threat intention of each fighter by predicting the movement of cells in the region through a Markov chain and modeled the cell transition using a neural network. Although machine learning algorithms have achieved good results in many aspects, for target analysis in a hostile environment, especially in a complex underwater environment, machine learning methods rely too much on the training of large amounts of data. If the authenticity and quantity of data cannot be guaranteed, the model cannot achieve generalization. With respect to the multi-objective threat ranking problem, existing studies focus on data processing and weight determination, while ignoring the mutual influence relationship between targets and the influence of indicators on targets. The PROMETHEE method [
43,
44] makes up for this problem. On the basis of solving the weight problem between indicators, more attention should be paid to the information transmitted between multiple objectives and the influence of this information on problem analysis.
3. Multi-AUV Threat Assessment System and Analysis of Threat Indicators
The basis of multi-AUV threat assessment is to master the information of the threat targets and conduct fusion and analysis based on this information. In this paper, a threat assessment system is designed as shown in
Figure 2. It first analyzes the elements of the underwater confrontation process and quantifies the information value of each element of a multi-AUV. On this basis, the indicators are classified and summarized corresponding to the threat assessment criteria. The threat assessment criteria are used to determine the direct dominance degree of different indicators to compare different indicators and obtain the matrix of direct dominance comparison. In order to reflect the interaction between different indicators, other criteria with indirect effects are taken as indirect dominance criteria. The indirect dominance comparison matrix can be obtained by considering the influence of other indicators on the indirect dominance criteria. Together, the two matrices form the threat index matrix.
After evaluating and determining each criterion using indicators, the subjective weight can be obtained by the convergence of the threat indicator matrix. According to the entropy value of information transmitted to the target, the objective weight based on data can be obtained. Through the fusion calculation of weights, the subjective and objective combined weights can be obtained to address the threat assessment problem. Finally, the PROMETHEE-II algorithm is used to analyze the actual problem and obtain the final multi-AUV countermeasures. It can be seen from the above analysis that the design of a multi-AUV threat assessment system has three steps: the establishment of threat criteria, the analysis and modeling of threat indicators, and the design of the weighting algorithm.
3.1. Loading and Testing of a Comprehensive Criterion Evaluation System for Targets
In underwater countermeasures, many factors should be considered in the analysis of incoming AUVs. In this paper, the basic factors are integrated and analyzed to form a comprehensive index. Based on the analysis of AUVs’ motion and detection characteristics, the cooperative value of an incoming AUV, the ability value of the incoming AUV, the cost of damaging the target AUV (i.e., the survival value), and the consistency between the incoming AUV intent and the task (i.e., the intention value) are considered.
The cooperative value of target (
A) is the effect on an incoming coalition of AUVs if the target is attacked or disabled. In this study, the boundary contribution rate in a cooperative game is introduced to measure the influence of certain goals on other goals. The Shapley value [
45] is used to quantify the boundary contribution in the alliance. The boundary contribution value (
) of the target is described as:
where
U is the set composed of all players in the game:
N is any finite carrier of
v,
,
.
is the ability value of the AUV, which refers to the economic value of the target in general. In underwater confrontation, the value of the incoming AUV is often measured by its size, number of weapons, and maneuvering ability.
is the survival value of the target AUV, which is affected by the distance between the defense unit and the incoming AUV, the detection capability of both sides, the weapon strike capability, the difference in velocity, etc.
is the intention value of the target AUV, which reflects the degree of influence of the target on the main antagonistic task. The more consistent the intent of the goal is with the overall task of the confrontation, the greater the importance of the goal.
The following deterministic criteria are included in the criterion evaluation system:
, and fuzzy criterion
. In order to achieve unified measurement, the classification threshold is designed to evaluate
, and the comprehensive value of the definite and fuzzy criteria of the incoming targets. The classification and quantification methods of each criterion are shown in
Table 1. It can be seen from the table that the values of the conformity indices
calculated by the basic indices are very different, which leads to certain data having a great influence on the calculation result due to the difference of dimensions in the comprehensive analysis so the data are preprocessed in one step. According to the four evaluation criteria, each incoming target (
) corresponds to a criterion vector at the current moment
.
3.2. Analysis of Threat Indicators
In order to describe the indicators in threat assessment in a more detailed manner, the above criteria are further broken down into assessment indicators. Indicators are described and calculated as follows.
3.2.1. Heading Advantage Indicators
The heading advantage between the incoming AUV and the defense unit is described as the relation between the relative chord angles of both sides, as shown in
Figure 3. It is assumed that the red AUV is one of the incoming AUVs. When analyzing its heading advantage, two heading conditions are comprehensively considered. The heading indicator is defined as Equation (
3), where
and
are the entry angles of our AUV and the incoming AUV, respectively. The heading advantage is obtained by evaluating the geometric relationship between the advantage of offensive and defensive tasks.
3.2.2. Velocity Advantage Indicators
When analyzing the velocity advantage indicators, similarly, the velocity difference between the two sides is analyzed. According to [
24], the AUV velocity advantage function is obtained by comparative analysis of the underwater environment after parameter test adjustment as follows:
3.2.3. Confrontation Position Advantage Indicators
In underwater confrontation, the confrontation position advantage indicator is derived from the characteristics of the encounter and the assessment of the incoming threat. In addition to assessing the consistency of the goal with the task, this measure is closely related to the criterion defined as “the cost of damaging the target”. The azimuth boundary limits of the incoming AUV weapon are shown in
Figure 4, where
P is the hit point of the weapon launched by the incoming AUV at point
W based on the premise that the heading and velocity are not changed.
When the attacking AUV position, defense AUV position, and the sailing speed and heading of both sides satisfy Equation (
5), the attacking intention of the incoming AUV is most evident. Based on reference research on counter-position advantage presented in [
46], the method of calculating attack position advantage is obtained by geometric simplification of attack behavior.
where
m is the ratio of the velocity of the attacked AUV to the velocity of the weapon. The incoming AUV has strong attack intention in the circle, with the weapon hit point as the center of the circle and the maximum range of the weapon as the radius. The advantage in this case is shown in Equation (
6), where
is the distance between the incoming AUV and the defense AUV, and
is the ultimate range of the weapon.
If not, the advantage of the incoming AUV is as follows:
3.2.4. Bearing Advantage Indicators
In contrast to heading advantage, the relationship between the chord angles of both sides should be considered to analyze the superiority of the incoming AUV. The azimuth indicator is one of the damage effect indicators of the response to the incoming AUV, which is mainly used to reflect the position relationship between the target and the defense unit. The advantages of azimuth indicators should be considered in conjunction with the prediction of the incoming AUVs’ intentions. The determination methods used under different intentions are shown in
Table 2.
3.2.5. Heading Change Rate, Acceleration, Capability of Attack, and Distance Indicators
Heading change rate, acceleration, capability of attack, and distance indicators are simple to obtain in multi-target analysis by comparing the sizes of incoming AUVs in pairs.
4. PROMETHEE Algorithm for Incoming AUV Comprehensive Threat Assessment
PROMETHEE [
44], a sorting method based on an outranking preference model, is used to evaluate the intent of multi-AUVs. The comprehensive value assessment problem for each objective is determined by building an outranking relation diagram. The algorithm works as follows:
Establish a priority function among the criteria;
Construct an outranking relation diagram between the targets based on the indicator priority function;
The net flow of information of indicators is calculated using the outranking relation diagram, and the target threat degree is sorted based on the net flow.
4.1. Priority Function of Multi-AUVs
A priority function is used to describe the priority of different incoming AUVs under the same criteria, that is, to judge the importance of incoming targets according to certain criteria. Therefore, the PROMETHEE method designs a function for each criterion to evaluate the importance of the target under the same index. The range of the function is defined between 0 and 1. The smaller the function value is, the less the difference between the two objectives under the unified evaluation index. The closer the function value is to 1, the greater the difference in importance between the two objectives. When the function is set to 1, goal is strictly more important than goal .
According to the above analysis, the criteria are divided into five levels. The priority function is defined as follows: for the
kth criterion, the evaluation levels of target
and
are set as
and
, respectively. There is a priority function (
) as shown in Equation (
8). For two targets under the same criterion, the difference between the two levels is not obvious, and the comparison function values are 0.2. When the difference between two targets is more than two levels, the difference becomes obvious, and the values are defined as 0.7 and 1. If not, the advantage of the incoming AUV is as follows:
4.2. Outranking Relation Diagram
The above priority function compares the importance of different objectives under the same criteria. The comparison between multiple objects and attributes is described by the outranking relation diagram. Such a diagram is constructed by calculating the priority indicators.
For any two targets (
and
), assume that the weight of each criterion is
,
, where
n is the number of indicators. The priority indicator is defined as:
The outranking relation diagram is comprised of the above relationship between nodes, as shown in
Figure 5. As shown in
Figure 5, the priority index is calculated by pairwise comparison of target priorities and indicator weights under the same criterion, which enables attainment of the necessary level of fuzzy preference programming higher than the graph in the PROMETHEE algorithm. The nodes in the figure refer to the incoming multi-AUV, and the lines between the nodes are the priority indices calculated by the targets according to different indicators. If
is higher than
,
, but
is not necessarily equal to 1.
4.3. Determine the Comprehensive Value of Incoming AUVs
Based on the outranking relation diagram, the information inflow and outflow are defined for each incoming target to measure the comprehensive value of the target. The threat assessment results of multi-incoming AUVs are obtained through comprehensive analysis of the information flow among AUVs.
AUV information inflow is defined as:
AUV information outflow is defined as:
Based on the defined information inflow and outflow of the incoming AUVs, there are usually two algorithms used to analyze the importance of the target to other targets: one involves defining two full orders, that is PROMETHEE-I, and the other involves the use of the net flow method, that is PROMETHEE-II. This paper adopts the net flow method to measure the comprehensive value of the incoming AUVs and defines the net flow of
as Equation (
12). The net flow of all incoming AUV nodes can be obtained, and the threat assessment results (
) of multiple incoming AUV nodes are determined according to the net flow size.
5. Weight Fusion Algorithm of Multi-AUV Threat Assessment Indicators
There are two main research focuses for threat assessment of incoming AUVs: the assessment model and weight calculation. The weights determine the effectiveness and the scientific validity of the assessment results. Neither the subjective expert system method, the AHP method, nor the objective weighting method that relies on data statistics can adequately describe the multi-objective analysis indicators in a complex confrontation environment. Therefore, this study proposes a weight fusion algorithm that integrates subjective and objective weighting.
5.1. Entropy Weighting Algorithm
In solving multi-AUV threat assessment, once the priority function (
) between the targets is determined, the weight of the indicators should be calculated according to the principle of information theory. Using the entropy method, the entropy value of
kth target is calculated as:
where
represents the value of the
kth criterion of the
ith target after processing. Finally, the weight of the
kth criterion can be obtained as follows:
5.2. ANP Weighting Algorithm
To compensate for the shortcomings of the pure objective algorithm in solving real problems, this study uses ANP to assign weights to indicators in the threat assessment of incoming AUVs.
For the characteristics of the underwater confrontation problem, analysis, and comparison of various factors, the correlation of influencing factors is analyzed. According to the detection characteristics of AUVs and the above multi-objective comprehensive indicator ranking system, a hierarchical multi-AUV threat assessment weighting system is designed, as shown in
Figure 6. The indicator weight analysis system consists of two layers: the control layer and the network layer. Based on the structure of the ANP algorithm, the system can analyze the relationship between the indicator layer and the criterion layer and analyze the mutual influence of the indicator.
ANP is improved from AHP with the central idea of decomposing the multi-attribute decision problem into multiple factors that affect the decision. The specific process is shown in
Figure 7. Unlike the requirement of independence among factors in AHP, the ANP algorithm emphasizes the interdependencies among factors. This ensures that the ANP algorithm can better represent the actual threat assessment problems involving the coupling of indicators. ANP divides all factors into two layers. The first layer is the control layer, which includes the target layer and the criterion layer. In this layer, all criteria are independent of each other and only governed by the target layer. The control layer is a hierarchical structure. The second layer is the network layer, which includes the criterion layer and the factor layer. The factors of the factor layer are governed by the criterion layer. The factors in the same criterion layer are independent of each other, while those in different layers are interdependent. The control layer and network layer constitute the basic structure of ANP. The steps of the ANP decision-making method for underwater multi-target threat assessment are as follows:
Establish the indicator factor set of underwater multi-target tracking;
Construct the corresponding ANP network model;
Construct the limit super-matrix;
Calculate the final decision result.
In contract to the paired contrast matrix of indicators and targets in the AHP model, the super-matrix needs to be constructed using the ANP model. First, the concept of dominance needs to be introduced, which can be divided into direct dominance and indirect dominance. Direct dominance is the degree of comparative advantage between two factors under a given criterion. The degree of indirect dominance is the degree of advantage obtained by comparing two indicators against the same criterion while applying a third indicator to another criterion.
A criterion (
) in the criterion set of the ANP model is selected, and factor
in
is used as the subcriterion to compare the degree of indirect advantages of factor
in
to
to obtain the judgment matrix of
to
. On the basis of the premise that the matrix meets the consistency requirement, the weight vector
about
to
is obtained by the eigenvalue method. The weight vectors are computed to form the matrix (
).
If factor
in
is not affected by factor
in
,
is 0. The super-matrix of the ANP model is formed using the above weight vectors, as shown in Equation (
17).
For the factors that need to calculate direct dominance, the weight coefficient between the factors should be calculated. For example, to compare the direct dominance of factor
and
in criterion
, the dominance comparison matrix determined by expert experience should be introduced, the structure of which is as follows:
The weight vector () of the dominance contrast matrix is calculated by the eigenvalue method. The comparison matrix () of a different incoming target () is established, and the sorting vector () under factors is calculated using the super-matrix to obtain the final super-matrix (). After column normalization and convergence, the weight () of each criterion is obtained.
5.3. Subjective and Objective Weight Fusion Algorithm
It can be seen that the data dimension is reduced due to the fusion and grading of indicators in the analysis process, which may lead to a decline in the accuracy of the analysis. Therefore, we design a fusion algorithm to refine the performance of weights to improve this problem. As for the fusion of weights, this paper endeavors to reflect the importance of indicators in the context of multi-AUV threats. When calculating weights, the entropy weight based on objective data and the weight based on subjective ANP are integrated to consider both subjective experience value and accurate analysis of objective data. In this paper, the quadratic fusion algorithm is adopted, and the final weight is obtained by geometric average double fusion after weighted summation and multiplication, respectively.
5.3.1. Weighted Sum
The objective weight (
) and subjective weight
are obtained. According to the characteristic that the weighted average [
47] can comprehensively consider the two weights without losing generality, the weights are fused first. Taking into account the importance of both weights, the fusion weight of the
jth indicator can be obtained through fusion according to Equation (
19), where
are the proportion of the objective weight and subjective weight, respectively, and
,
.
5.3.2. Multiplication Synthesis
Compared with the weighted summation method, the multiplicative synthesis method considers both the objective and subjective experience data while focusing more on enlarging the difference between threat levels among various indicators [
48]. The fusion method of multiplication synthesis can be expressed as follows:
5.3.3. Double Fusion
Double fusion algorithms, weighted summation, and multiplicative synthesis are used to fuse the subjective and objective weights to obtain the fusion weights (
and
). For the characteristics of the comprehensive fusion algorithm, the geometric average idea is adopted, as shown in Equation (
21) for the second fusion of the fusion weights. Finally,
is obtained by being substituted into Equation (
9) to obtain the priority sequence of multiple incoming AUVs.
6. Examples of Multi-AUV Threat Assessment
Simulation experiments for five incoming AUVs are designed. The entropy and ANP weights are calculated to assign weights to indicators. The weights of indicators are merged twice to address the limitations of subjective target analysis and the deficiencies of objective data analysis. The data for the simulation experiments were obtained using our developed AUV simulation platform, which runs on Ubuntu 18.04.5 with ROS 1.14.11 and Gazebo 9.19.0. The algorithm simulations reported in this paper were conducted in a Windows 10 environment with VS2013.
Figure 8 shows the AUV in the simulation platform, along with its experimental parameters. The specific parameters are listed in
Table 3. The incoming AUVs’ initial states are given in
Table 3.
Table 3 shows that the indicators required by the threat sorting algorithm can be calculated according to the data in the table, and the effectiveness of the algorithm can be analyzed accordingly.
6.1. Examples of Entropy Method Weighting
According to the above initial state, five incoming AUVs can be obtained, and the four criteria of the PROMETHEE method can be evaluated as shown in
Table 4. After data preprocessing and preliminary data fusion calculation, the sensor data obtained from the simulation platform are processed and integrated according to the data required by the algorithm in this paper, and the indicator values shown
Table 4 are obtained, including cooperative value, ability value, survival value, and intention value. Criterion value classification can be determined to obtain the priority function values of different AUVs. According to the entropy in information theory, the weight of the criterion is
.
6.2. Examples of ANP Weighting
Using the ANP model, the underwater multi-AUV threat sequencing problem is solved. The paired comparison matrix for different criteria is calculated, and the weight vector in the ANP model is calculated according to the primary and secondary criteria. The initial super-matrix (
W) is obtained as shown in
Table 5. The limit super-matrix (
) is obtained by normalization and convergence according to the ultimate super-matrix results, and the weights of the indicators of the ANP can be obtained as follows:
.
6.3. Fusion Subjective and Objective Weights and Incoming AUV Threat Assessment
After the subjective and objective weights are obtained, they are integrated according to the quadratic fusion algorithm.
Experiment 1: A weighted fusion coefficient is designed as
. The weighted fusion weight is obtained as follows:
Then, multiplication fusion is performed using the multiplication fusion algorithm, and the fusion weight is obtained as follows:
Finally, the weight of the final indicator is obtained by secondary fusion of the two groups of fusion weights according to the geometric average idea:
According to the priority indicator and assignment level obtained by the PROMETHEE-II algorithm, the information inflow, outflow, and net flow of incoming AUVs can be obtained as shown in
Table 6. The order of threat degree of incoming AUVs is then determined. Through the calculation of the target net flow, the final sequencing result of the incoming AUVs can be written as
.
Experiment 2: A weighted fusion coefficient is designed as
. The weighted fusion weight is obtained as follows:
Multiplicative fusion is independent of the weighting coefficient and remains unchanged. Double fusion weight is obtained as follows:
The information inflow, outflow, and net flow of incoming AUVs can be obtained as shown in
Table 7. The effectiveness of the algorithm is analyzed through the above two experiments. It can be seen that objective
, where all analysis indicators are dominant and balanced, has the highest threat degree. Although target
has the highest contribution rate among multiple attack targets, it has the lowest threat level because it is not superior in other aspects. From the point of view of the characteristic of the target index, the algorithm proposed in this paper can distinguish the target analysis to a certain degree.
6.4. Parameter Sensitivity Experiment
In the previous process, the effectiveness of the threat assessment algorithm proposed in this paper was proven by experiments. This section focuses on the parameter index sensitivity analysis of the algorithm. The sensitivity of the threat assessment system to the number of indicators is analyzed to demonstrate the importance of system integrity to the assessment. It can also be seen which kind of parameters in the threat assessment have a more obvious impact on the assessment results.
As can be seen from
Figure 9, the experiment compared the results of the complete indicator threat assessment and the missing indicator threat assessment. Under the same initial conditions, the lack of capability value, survival value, and intention value in the threat assessment system has an impact on the result, which makes the differentiation of the assessment result decline. The absence of two criteria at random was a fatal blow to the evaluation results, which showed that two objectives were not differentiated.
7. Analysis of Algorithm Superiority
On the basis of the effectiveness analysis of the above algorithm, the superiority of the algorithm is further analyzed. In this section, the superiority of the proposed algorithm is analyzed from three perspectives: the result of single-weight threat assessment, the influence of single-weight fusion on the result of threat assessment, and the superiority of the quadratic fusion algorithm.
According to further analysis and processing of the data, the calculation results of index weights shown in
Figure 10a can be obtained. It can be intuitively seen from the figure that, regardless of whether the formula uses the entropy method alone and purely relies on objective data to weigh indicators or only relies on expert experience to analyze and process indicators, there will be certain problems, such as inevident differentiation of indicators and a large gap between indicators.
Figure 10c shows whether the ANP subjective weighting method is used alone or the entropy objective weighting algorithm is used to inadequately differentiate target analysis. ANP is indistinguishable from objectives
and
, and entropy is indistinguishable from objectives
and
.
The results shown in
Figure 10d were obtained through the analysis of the experimental results presented in the previous section. It can be seen that the average difference of the results of the ANP algorithm is small, so that if the target indicators are similar in the analysis, there may be an indistinguishable situation. Entropy weighting is overly dependent on the data information in the process of confrontation. As shown in
Figure 10d, entropy generates a maximum difference in threat assessment results that is much higher than other algorithms, resulting in a lack of stability.
A weighted summation method can solve this problem, but the influence of the weighted fusion coefficient on the fusion result is too obvious. It can be seen from
Figure 10a,b that the single-weight fusion algorithm leads to the inconspicuous analytical differentiation of some indicators. The results of experiments 1 and 2 are shown in
Figure 10b. In the weighted fusion of subjective and objective weights, the different selection of coefficients may lead to significant changes in index weights. The quadratic fusion calculation method proposed in this paper can reduce the influence of the weighted coefficients on weight fusion so that the evaluation results do not change significantly with changes in the coefficients, making the algorithm more robust.
The superiority of the algorithm often cannot be analyzed through a single experiment. In this paper, 100 experiments are designed for the threat assessment problem of five incoming targets to verify the superiority of the algorithm. Different incoming target situations are randomly generated, and experimental situations in which the target itself is not distinguishable are eliminated. The experimental results are shown in
Table 8. The worst outcome is that in which two of the five targets are indistinguishable because the experiment was designed to exclude cases in which the difference is too small. The results show that the subject-dependent ANP weighting algorithm has difficulty in distinguishing the target threat assessment when the subjective concern indicators are not different. As can be seen from
Table 8, the algorithm proposed in this paper improves the accuracy by 10–13% compared with other algorithms in multiple experiments conducted to assess the threat degree of the target. As for the entropy algorithm, weighting is overly dependent on data information, and as the number of experiments grows and the analysis scenarios become more similar, it becomes more difficult to distinguish between objectives. Comparatively speaking, the algorithm proposed in this paper solves the existing problems from two perspective. One involves improving the accuracy of target analysis by considering the subjective and objective weights comprehensively. On the other hand, as shown in the table, the weighting coefficient in the secondary fusion changes, but the result changes little, which also indicates that the algorithm reduces the dependence on the weighting coefficient.
The product synthesis method makes it difficult to show the advantages of data with obvious magnitude differences in the fusion process. The second fusion not only takes the characteristics of subjective and objective data into account but also further reduces the dependence on the coefficient in the process of weighted summation.
Figure 10b and Equations (
22) and (
25) above show that the simple use of weighted fusion results in significant changes in weight, along with changes in the weighting coefficient. As a result, the preference of decision commanders has too much influence on the results. However,
Figure 10 and Equations (
24) and (
26) show that the algorithm proposed in this paper effectively reduces the dependence on the fusion coefficient.
8. Discussion
This study investigates the problem of threat assessment in multi-AUV adversarial missions. We accomplished the following tasks:
Reviewed the existing research on threat assessment under underwater adversarial conditions;
Proposed a comprehensive attribute evaluation algorithm to establish a more applicable threat assessment system. In this part of the work, we emphasized the importance of evaluating the integrity of the system through experimental analysis of the identified metrics in threat assessment. Through rigorous experimentation, we demonstrated the significance of a comprehensive set of metrics in accurately assessing threats. The experimental analysis provided valuable insights into the effectiveness and reliability of the threat assessment system. By evaluating the system’s completeness through empirical validation, we highlighted its critical role in ensuring accurate and reliable threat assessment results. These findings underscore the importance of considering a wide range of metrics and their interdependencies to achieve robust and effective threat assessment in multi-AUV scenarios;
Developed an ANP-entropy-based method for calculating and determining the weights of multiple attributes; Addressed the issue of data dimension reduction caused by data fusion during threat assessment, ensuring analysis accuracy; We conducted a comprehensive analysis focusing on the necessity of weight fusion algorithms in data processing; We demonstrated the significance of the quadratic fusion algorithm in both theoretical and practical aspects, highlighting its necessity and its role in compensating for the inaccuracies resulting from data dimensionality reduction;
Through a combination of theoretical discussions and practical experiments, we provided evidence supporting the importance of weight fusion algorithms. Specifically, we showcased how the quadratic fusion algorithm addresses the issue of reduced accuracy caused by data dimensionality reduction. Theoretical and empirical findings emphasized the crucial role of weight fusion algorithms in achieving more accurate and reliable results in the context of multi-AUV threat assessment. By evaluating the system’s completeness through empirical validation, we highlighted its critical role in ensuring accurate and reliable threat assessment results;
Conducted extensive experiments to validate the effectiveness, superiority, and accuracy of the proposed algorithm. Analyzed and compared the performance of the algorithm against existing approaches. Presented the results and findings, highlighting the advantages and contributions of the proposed algorithm in threat assessment for multi-AUV adversarial missions.
Our research contributes to advancement in the field of threat assessment under underwater adversarial conditions and provides valuable insights for enhancing mission planning and decision making in multi-AUV scenarios. However, it is also important to acknowledge the limitations of the proposed approach. For example, in the experiments on the impact of attribute reduction on the experimental results, we observed that an improved threat assessment system is a double-edged sword. While it enhances accuracy, it also introduces certain challenges. In adversarial missions, the loss of information due to sensor failure or other reasons directly affects the discriminability of threat assessment. In the next steps, it is crucial to address and research this issue and design more sophisticated threat assessment algorithms to overcome this challenge, providing a solid foundation for AUV game strategy selection and control algorithms in adversarial missions.
9. Conclusions and Future Work
A multi-AUV threat assessment analysis indicator system is proposed in this paper, providing a research basis for multi-target threat analysis in underwater combat. The proposed fused multi-AUV threat ranking algorithm also provides a foundation for information processing and decision making in underwater countermeasures. A multi-AUV threat indicator processing problem for underwater multi-AUV countermeasures was systematically studied in this paper using the multi-attribute indicator threat quantification method. The quantitative processing method of the detection index is provided, and the deterministic and fuzzy indicators were analyzed and quantified in a unified manner, which not only retains the fuzzy information but also facilitates practical use. The network evaluation method is put forward for the relationship between different indicators, and the relationships between indicators, indicators and criteria, and criteria and criteria are described in detail. Based on the comprehensive analysis of subjective and objective weighting methods, a fused algorithm for multi-objective analysis is proposed. The effectiveness of the proposed algorithm is verified through simulation experiments.
In the future, as analysis and studies of underwater AUVs are further deepened, an increasing number of analysis indicators can be introduced to enrich the threat assessment system. Threat assessment in underwater confrontation is not a task that can be completed overnight, so we should consider dynamic threat assessment from the perspective of continuous time series. However, it is still necessary to explore the threat assessment algorithm in uncertain situations, such as under the influence of information loss, so as to provide accurate and robust algorithm support for threat assessment in underwater countermeasure missions.
Author Contributions
D.Y., H.W. and B.L. designed the study, performed the research, analyzed data, and wrote the paper. Z.W., J.R. and X.W., contributed to refining the ideas, carrying out additional analyses, and finalizing this paper. All authors have read and agreed to the published version of the manuscript.
Funding
This research work is supported by the National Science and Technology Innovation Special Zone Project (21-163-05-ZT-002-005-03), the National Key Laboratory of Underwater Robot Technology Fund (No. JCKYS2022SXJQR-09), and a special program to guide high-level scientific research (No. 3072022QBZ0403).
Institutional Review Board Statement
Not applicable.
Informed Consent Statement
Not applicable.
Data Availability Statement
Not applicable.
Conflicts of Interest
The authors declare no conflict of interest.
References
- Xia, J.; Chen, M.; Fang, W. Normal Wiggly Probabilistic Hesitant Fuzzy Set and Its Application in Battlefield Threat Assessment. Int. J. Fuzzy Syst. 2023, 25, 145–167. [Google Scholar] [CrossRef]
- Engler, E.; Baldauf, M.; Banyś, P.; Heymann, F.; Gucma, M.; Sill Torres, F. Situation Assessment—An Essential Functionality for Resilient Navigation Systems. J. Mar. Sci. Eng. 2020, 8, 17. [Google Scholar] [CrossRef] [Green Version]
- Azimirad, E.; Haddadnia, J. Target threat assessment using fuzzy sets theory. Int. J. Adv. Intell. Inform. 2015, 1, 57. [Google Scholar] [CrossRef] [Green Version]
- Wilson, A.G.; McNamara, L.A.; Wilson, G.D. Information integration for complex systems. Reliab. Eng. Syst. Saf. 2007, 92, 121–130. [Google Scholar] [CrossRef]
- Kong, D.; Chang, T.; Wang, Q.; Sun, H.; Dai, W. A threat assessment method of group targets based on interval-valued intuitionistic fuzzy multi-attribute group decision-making. Appl. Soft Comput. 2018, 67, 350–369. [Google Scholar] [CrossRef]
- Deng, Y. A threat assessment model under uncertain environment. Math. Probl. Eng. 2015, 2015, 878024. [Google Scholar] [CrossRef] [Green Version]
- Alali, M.; Almogren, A.; Hassan, M.M.; Rassan, I.A.; Bhuiyan, M.Z.A. Improving risk assessment model of cyber security using fuzzy logic inference system. Comput. Secur. 2018, 74, 323–339. [Google Scholar] [CrossRef]
- Lee, H.; Choi, B.J.; Kim, C.O.; Kim, J.S.; Kim, J.E. Threat evaluation of enemy air fighters via neural network-based Markov chain modeling. Knowl.-Based Syst. 2017, 116, 49–57. [Google Scholar] [CrossRef]
- Torra, V. Hesitant fuzzy sets. Int. J. Intell. Syst. 2010, 25, 529–539. [Google Scholar] [CrossRef]
- Pang, Q.; Wang, H.; Xu, Z. Probabilistic linguistic term sets in multi-attribute group decision making. Inf. Sci. 2016, 369, 128–143. [Google Scholar] [CrossRef]
- Zadeh, L.A. Fuzzy sets. Inf. Control 1965, 8, 338–353. [Google Scholar] [CrossRef] [Green Version]
- Carling, R.L. A Knowledge-Base System for the Threat Evaluation and Weapon Assignment Process. Nav. Eng. J. 1993, 105, 31–41. [Google Scholar] [CrossRef]
- Tuma, K.; Widman, M. Seven Pain Points of Threat Analysis and Risk Assessment in the Automotive Domain. IEEE Secur. Priv. 2021, 19, 78–82. [Google Scholar] [CrossRef]
- Cao, Y.; Kou, Y.X.; Xu, A.; Xi, Z.F. Target Threat Assessment in Air Combat Based on Improved Glowworm Swarm Optimization and ELM Neural Network. Int. J. Aerosp. Eng. 2021, 2021, 4687167. [Google Scholar] [CrossRef]
- Yao, H.; Wang, H.; Li, Y.; Wang, Y.; Han, C. Research on unmanned underwater vehicle threat assessment. IEEE Access 2019, 7, 11387–11396. [Google Scholar] [CrossRef]
- Yao, H.; Wang, H.; Wang, Y. UUV autonomous decision-making method based on dynamic influence diagram. Complexity 2020, 2020, 8565106. [Google Scholar] [CrossRef]
- Zhou, H.; Zhong, Z. Evasive behavior-based method for threat assessment in different scenarios: A novel framework for intelligent vehicle. Accid. Anal. Prev. 2020, 148, 105798. [Google Scholar] [CrossRef]
- Gao, Y.; Li, D.-S.; Zhong, H. A novel target threat assessment method based on three-way decisions under intuitionistic fuzzy multi-attribute decision making environment. Eng. Appl. Artif. Intell. 2020, 87, 103276. [Google Scholar] [CrossRef]
- Rogova, G.L.; Ilin, R. Multi-Agent System for Threat Assessment and Action Selection under Uncertainty and Ambiguity. In Proceedings of the 2019 22th International Conference on Information Fusion (FUSION), Ottawa, ON, Canada, 2–5 July 2019; IEEE: Piscataway, NJ, USA, 2019; pp. 1–8. [Google Scholar]
- Chen, H.; Cao, X.; Hao, Y.; Fan, J.; Chen, D.; Chen, D. Multi-target threat assessment in unmanned cluster confrontation scenarios. In Proceedings of the 2020 3rd International Conference on Unmanned Systems (ICUS), Harbin, China, 27–28 November 2020; IEEE: Piscataway, NJ, USA, 2020; pp. 785–790. [Google Scholar]
- Carling, R.L. Naval situation assessment using a real-time knowledge-based system. Nav. Eng. J. 1999, 111, 173–187. [Google Scholar] [CrossRef]
- Shin, D.; Kim, B.; Yi, K.; Carvalho, A.; Borrelli, F. Human-centered risk assessment of an automated vehicle using vehicular wireless communication. IEEE Trans. Intell. Transp. Syst. 2018, 20, 667–681. [Google Scholar] [CrossRef]
- Xu, C.; Zhao, W.; Wang, C. An integrated threat assessment algorithm for decision-making of autonomous driving vehicles. IEEE Trans. Intell. Transp. Syst. 2019, 21, 2510–2521. [Google Scholar] [CrossRef]
- Luo, R.; Huang, S.; Zhao, Y.; Song, Y. Threat Assessment Method of Low Altitude Slow Small (LSS) Targets Based on Information Entropy and AHP. Entropy 2021, 23, 1292. [Google Scholar] [CrossRef] [PubMed]
- Fan, C.; Fu, Q.; Song, Y.; Lu, Y.; Li, W.; Zhu, X. A New Model of Interval-Valued Intuitionistic Fuzzy Weighted Operators and Their Application in Dynamic Fusion Target Threat Assessment. Entropy 2022, 24, 1825. [Google Scholar] [CrossRef]
- Lu, Y.; Lei, X.; Zhou, Z.; Song, Y. Approximate reasoning based on IFRS and DS theory with its application in threat assessment. IEEE Access 2020, 8, 160558–160568. [Google Scholar] [CrossRef]
- Zhang, Q.; Hu, J.; Feng, J.; Liu, A. Air multi-target threat assessment method based on improved GGIFSS. J. Intell. Fuzzy Syst. 2019, 36, 4127–4139. [Google Scholar] [CrossRef]
- Liang, S.; Yang, X.; Bi, F.; Ye, C. V essel traffic scheduling method for the controlled waterways in the upper Yangtze River. Ocean. Eng. 2019, 172, 96–104. [Google Scholar] [CrossRef]
- Kaewfak, K.; Huynh, V.N.; Ammarapala, V.; Ratisoontorn, N. A risk analysis based on a two-stage model of fuzzy AHP-DEA for multimodal freight transportation systems. IEEE Access 2020, 8, 153756–153773. [Google Scholar] [CrossRef]
- Rak, M.; Salzillo, G.; Granata, D. ESSecA: An automated expert system for threat modelling and penetration testing for IoT ecosystems. Comput. Electr. Eng. 2022, 99, 107721. [Google Scholar] [CrossRef]
- Yu, D.; Wang, H.; Yuan, J. Method of Unknown Target Risk Analysis and Threat Assessment for UUVs. In Proceedings of the 2020 39th Chinese Control Conference (CCC), Shenyang, China, 27–29 July 2020; IEEE: Piscataway, NJ, USA, 2020; pp. 2888–2893. [Google Scholar]
- Zhao, R.; Yang, F.; Ji, L. An extended fuzzy CPT-TODIM model based on possibility theory and its application to air target dynamic threat assessment. IEEE Access 2022, 10, 21655–21669. [Google Scholar] [CrossRef]
- Bao, L.; Wang, W.; Liu, K.; Yu, L.; Niu, H. Multi-objective risk analysis and documents assessment of bridge construction based on AHP-GRA Y. Adv. Sci. Lett. 2011, 4, 2543–2546. [Google Scholar] [CrossRef]
- Li, D.F. TOPSIS-based nonlinear-programming methodology for multiattribute decision making with interval-valued intuitionistic fuzzy sets. IEEE Trans. Fuzzy Syst. 2010, 18, 299–311. [Google Scholar] [CrossRef]
- Wang, T.C.; Chang, T.H. Application of TOPSIS in evaluating initial training aircraft under a fuzzy environment. Expert Syst. Appl. 2007, 33, 870–880. [Google Scholar] [CrossRef]
- Ma, S.; Zhang, H.; Yang, G. Target threat level assessment based on cloud model under fuzzy and uncertain conditions in air combat simulation. Aerosp. Sci. Technol. 2017, 67, 49–53. [Google Scholar] [CrossRef]
- Gao, Y.; Yu, M.; Wang, Z. A new method of multi-target threat assessment for air combat. In Proceedings of the 2017 IEEE International Conference on Cybernetics and Intelligent Systems (CIS) and IEEE Conference on Robotics, Automation and Mechatronics (RAM), Ningbo, China, 19–21 November 2017; IEEE: Piscataway, NJ, USA, 2017; pp. 779–784. [Google Scholar]
- Wang, J.; Fan, K.; Su, Y.; Liang, S.; Wang, W. Air combat effectiveness assessment of military aircraft using a fuzzy AHP and TOPSIS methodology. In Proceedings of the 2008 Asia Simulation Conference—7th International Conference on System Simulation and Scientific Computing, Beijing, China, 10–12 October 2008; IEEE: Piscataway, NJ, USA, 2008; pp. 655–662. [Google Scholar]
- Xiang, C.; Xing, W.; Hubiao, Z.; Yuheng, X.; You, C.; Xiaotian, W. Interval TOPSIS with a novel interval number comprehensive weight for threat evaluation on uncertain information. J. Intell. Fuzzy Syst. 2022, 42, 4241–4257. [Google Scholar] [CrossRef]
- Bazzazi, A.A.; Osanloo, M.; Karimi, B. Deriving preference order of open pit mines equipment through MADM methods: Application of modified VIKOR method. Expert Syst. Appl. 2011, 38, 2550–2556. [Google Scholar] [CrossRef]
- Zeng, S.; Chen, S.M.; Kuo, L.W. Multiattribute decision making based on novel score function of intuitionistic fuzzy values and modified VIKOR method. Inf. Sci. 2019, 488, 76–92. [Google Scholar] [CrossRef]
- Khorram, S. A novel approach for ports’ container terminals’ risk management based on formal safety assessment: FAHP-entropy measure—VIKOR model. Nat. Hazards 2020, 103, 1671–1707. [Google Scholar] [CrossRef]
- Greco, S.; Ishizaka, A.; Tasiou, M.; Torrisi, G. The ordinal input for cardinal output approach of non-compensatory composite indicators: The PROMETHEE scoring method. Eur. J. Oper. Res. 2021, 288, 225–246. [Google Scholar] [CrossRef]
- Abdel-Basset, M.; Gamal, A.; Moustafa, N.; Abdel-Monem, A.; El-Saber, N. A security-by-design decision-making model for risk management in autonomous vehicles. IEEE Access 2021, 9, 107657–107679. [Google Scholar] [CrossRef]
- Bahmani, R.; Karimi, H.; Jadid, S. Cooperative energy management of multi-energy hub systems considering demand response programs and ice storage. Int. J. Electr. Power Energy Syst. 2021, 130, 106904. [Google Scholar] [CrossRef]
- Fengduan, G.; Jin, A.; Yichuan, W. Impact analysis of platform positioning and orientation error on warship weapon target strike. Evol. Intel. 2021, 1–6. [Google Scholar] [CrossRef]
- Peng, J.-J.; Wang, J.-Q.; Zhang, H.-Y.; Sun, T.; Chen, X.-H. OWA aggregation over a continuous fuzzy argument with applications in fuzzy multi-criteria decision-making. J. Intell. Fuzzy Syst. 2014, 27, 1407–1417. [Google Scholar] [CrossRef]
- Bhuvaneswari, T.; Prasad, V.C.; Singh, A.K.; Prasad, P.W. Weights Binary Decision Diagram (WBDD) and its application to Matrix Multiplication. In Proceedings of the 2009 Innovative Technologies in Intelligent Systems and Industrial Applications, Kuala Lumpur, Malaysia, 25–26 July 2009. [Google Scholar] [CrossRef]
| Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).