Formation Cooperative Intelligent Tactical Decision Making Based on Bayesian Network Model

Abstract

This paper proposes a method based on a Bayesian network model to study the intelligent tactical decision making of formation coordination. For the problem of formation coordinated attack target allocation, a coordinated attack target allocation model based on the dominance matrix is constructed, and a threat degree assessment model is constructed by calculating the minimum interception time. For the problem of real-time updating of the battlefield situation in the formation confrontation simulation, real-time communication between the UAV formation on the battlefield is realized, improving the efficiency of communication and target allocation between formations on the battlefield. For the problem of UAV autonomous air combat decision making, on the basis of the analysis of the advantage function calculation of the air combat decision-making model and a Bayesian network model analysis, the network model’s nodes and states are determined, and the air combat decision-making model is constructed based on the Bayesian network. Our formation adopts the Bayesian algorithm strategy to fight against the blue side’s UAVs, and the formation defeats the blue UAVs through coordinated attack, which proves the reasonableness of coordinated target allocation. An evaluation function is established, and the comprehensive scores of our formation are compared with those of other algorithms, which proves the accuracy and intelligibility of the decision making of the Bayesian network.

1. Introduction

Unmanned combat platforms (UCPs) have unique advantages in the field of aircraft confrontation due to their low cost and simple operation. With the increasing complexity of battlefield situations and the change in emerging technologies, unmanned aerial vehicles (UAVs) play an increasingly important role in modern aircraft confrontation [1,2]. With the continuous development of UAV-related technology, UAV cluster cooperative confrontation has emerged, and its combat mode is accelerating the change in aircraft confrontation patterns and system combat modes [3].

Due to the limited level of intelligence of the current UAV development, it is difficult to realize a fully autonomous cooperative combat mode among UAV clusters [4,5]. In terms of UAV formation tactical decision making, all countries hope to develop UAVs in the form of lead UAVs that can make accurate decisions, as well as unmanned systems in the form of wingmen that can realize intelligence and cooperate with manned combat UAVs in coordinated operations, so as to form formation-coordinated intelligent tactics in air combat, i.e., lead UAV–wingman coordinated operations [6,7]. The United States has adopted the Loyal Wingman, Skyborg, Long Shot, and SoSITE programs under a number of operational concepts, and has conducted research and test validation based on these programs. Research and test validation have been carried out on the basis of programs such as Skyborg, Long Shot, and SoSITE [8]. France conducted flight tests to further test the tactical coordination of AWACS, fighters, and UAVs in air combat [9]. Russia has conducted cooperative flight tests, and the test UAVs include Su-57 fighter jets and “Hunter” drones [10]. The United Kingdom has developed and manufactured the Mosquito unmanned verification UAV, which will be used as a “loyal wingman” to carry out coordinated operations with manned fighters [11].

In addition, in the formation cooperative combat mode, scholars from various countries have proposed a large number of decision-making methods for air combat [12,13], such as decision-making methods based on genetic learning, differential countermeasures, expert knowledge, neural network, and matrix countermeasures [14,15,16,17,18]. Liu, H. et al. [19] proposed an Evolution Strategies (ESs) optimization method and applied it to large-scale UAV swarms to address the issues of high dimensionality and high dynamics in the adversarial game process, where traditional optimal control algorithms fail to meet timeliness requirements. Ou Y. et al. [20] proposed a multi-UAV cooperative air combat maneuvering decision-making method based on LSTM and competitive graph convolutional deep reinforcement learning by introducing an LSTM network for processing air combat information with strong temporal correlations and building a graph convolutional network for inter-UAV communication. Ma W. et al. [21] proposed an algorithm that combines game theory with deep reinforcement learning, using a Deep Q-Network (DQN) to handle the aircraft’s continuous infinite state space and employing linear programming to solve the optimal value function for stage games while training the network to approximate this value function. Li Y.F. et al. [22] proposed a deep reinforcement learning algorithm-based autonomous maneuver decision-making model for air combat, which enables unmanned aerial vehicles to autonomously select tactical actions in air combat, greatly improving the combat efficiency of unmanned aerial vehicles.

Compared to countermeasure theory-based and heuristic learning maneuver-based decision-making methods, Bayesian network decision-making methods can introduce expert experience to build decision-making models based on datasets when solving uncertainty problems, as well as the ability to train the models based on sample data [23,24,25]. Bayesian networks have been widely used in complex decision-making systems, and the networks can better fit the environment and make reasonable decisions based on domain expert knowledge and objective information [26]. The air warfare battlefield environment is complex, and Bayesian networks can solve the uncertainty problem based on formulating the problem description as conditional probability relationships [27,28]. The Bayesian network method used in this paper can effectively make decisions about the next moment of formation maneuvers based on the battlefield situation. The results of simulation experiments show that the construction of a maneuver decision-making model based on a Bayesian network improves the battlefield adaptability of the UAV system and ensures the autonomy and effectiveness of the decision-making process. Formation coordination tactical decision making is the core and key of UAV coordinated combat utilization [29]; how to choose better decision-making tactics based on Bayesian networks in formation coordination to achieve the combat effectiveness of “1 + 1 > 2” has important research value and military significance [30,31,32].

The research objective of this paper is to carry out research on formation cooperative intelligent tactical decision making based on Bayesian networks, compare different decision-making methods in the simulation and confrontation platform, and verify the effectiveness of the decision-making method adopted in this paper by analyzing the posture of the enemy and us. In this paper, under the framework of the air combat decision-making process, real-time communication between formations is realized through OpenDDS, and a Bayesian network modeling algorithm is adopted to enable UAV formations to autonomously and cooperatively attack targets. By analyzing the process and results of formation combat target allocation, the rationality of cooperative target allocation and the accuracy of Bayesian network decision making are proved. In the research process, for the target allocation problem, a coordinated attack target allocation model is constructed, and a threat degree assessment model is constructed by calculating the minimum interception time; for the autonomous air combat decision-making problem, on the basis of the air combat decision-making model calculated by the dominance function and the Bayesian network model, the nodes and states of the network model are determined, and the air combat decision-making model based on the Bayesian network is constructed.

The following sections give a specific introduction to the main contents of this paper. Among them, Section 2 introduces the formation coordination target allocation model established in this paper; Section 3 introduces the Bayesian network-based air combat decision-making model established in this paper; Section 4 carries out simulation validation for communication, target allocation, and decision-making evaluation; and Section 5 summarizes the whole paper.

2. Formation Collaboration Target Allocation Model

The air combat decision-making process involves the red formation conducting action planning under the current mission and posture environment. This includes target allocation between the lead UAV and wingman after calculating the threat assessment to obtain the interception time. The formation then selects tactics to realize tactical decision making according to the Bayesian network, and executes the mission under the control of the lead UAV, with the cooperation of the wingman, and collaboratively through communication. Based on the above ideas, this section first discusses threat assessment theory, and then, studies the target allocation process based on coordinated attack to clarify the threat assessment and target allocation process of the red formation as well as the tactical decision-making method.

2.1. Threat Assessment Model

The threat assessment task is to judge the extent of the threat of the current battlefield events occurring and the severity of the threat from the blue side to the red side’s formation. Through the model of threat assessment by interception time, the threat to the red formation from the blue side is transformed into judging whether the blue side poses a threat to the red side and the size of the threat posed by the distance and the length of the attack time.

Two criteria, threat distance threshold and interception time threshold, are used as the criteria for judging the threat level. The threat distance threshold is used to judge whether the target poses a threat to the red formation; when the blue target is within the threat distance threshold, it is judged that the target poses a threat to the red formation. Interception time is used to estimate the time required by the red formation to intercept the blue target; when the estimated value of the interception time is less than the interception time threshold, it is judged that the target poses a threat to the red formation, and the shorter the interception time is, the bigger the threat is [33].

In the following, the threat assessment model is established by calculating the interception time. In the current decision cycle, the set target maintains a constant speed and linear motion, while our interceptor UAV maintains the current speed and moves in a straight line, so as to calculate the threat index of our formation to the enemy’s formation of UAVs at this moment. The interceptor UAV can turn towards the interception point to fly in a straight line.

The equation for the interception time $t$ can be derived as follows:

$$\left\{\begin{array}{l}{‖AP‖}^2={‖AT‖}^2+{‖TP‖}^2-2‖AT‖·‖TP‖·\cos \left(q\right)\\ ‖AP‖=V_a·t\\ ‖TP‖=V_t·t\end{array}\right.$$

(1)

The symbols in Formula (1) are described as shown in

Table 1. Formula symbol table.

Symbol	Description
$T$	The current position of the target
$A$	The current position of the interceptor
$P$	The interception point
$‖AT‖$	The distance between the interceptor and the target
$‖AP‖$	The distance between the interceptor and the target
$‖TP‖$	The distance between the interception point and the target
$q$	The target aspect angle
$V_t$	The target’s velocity
$V_a$	The interceptor’s velocity

.

By simplifying, we obtain

$$(V_a^2-V_t^2)·t^2+2‖AT‖·V_t·\cos (q)·t-{‖AT‖}^2=0$$

(2)

Letting $s=‖{\displaystyle \frac{AT}{t}}‖$, the equation can be further simplified to

$$s^2-2V_t·\cos (q)·s+V_t^2-V_a^2=0$$

(3)

Solving that we obtain

$$s=V_t·\cos (q)\pm \sqrt{V_a^2-V_t^2\sin {(q)}^2}$$

(4)

When $s=V_t·\cos (q)+\sqrt{V_a^2-V_t^2\sin {(q)}^2}$, the minimum interception time can be obtained. If $s<0$ or $s$ is undefined, meaning $V_a<V_t·\sin (q)$, the red interceptor UAV cannot intercept the target.

When $s$ is meaningful and $s>0$, the threat assessment model can be used to evaluate the minimum time for the red UAV to intercept the blue UAV, thus assessing the threat level of the blue UAV to the red UAV under the current situation.

2.2. Target Allocation Model Based on Coordinated Attack

The target allocation process utilizes the command-and-control system of the command UAV in order to give full play to the comprehensive advantages of the attacking UAV. It optimally allocates combat resources according to the target posture, the threat level, the combat effectiveness of the red side’s weapons, and the destruction requirements to achieve the effect of improving the combat effectiveness and reducing the cost of the attack [34,35,36]. After evaluating the threat level, target allocation is carried out, and the lead UAV and wingman cooperate to attack the blue UAV.

The exact steps of target allocation are shown in the Algorithm 1 below.

Algorithm 1 Target Allocation Model

Require: Red UAV $R$, Blue UAV $B$, Combat matrix $C$ Ensure: Attack assignments 1: $T\leftarrow$calculate_threat_indices$(B,R,C)$ 2: assignments$\leftarrow \left\{ \right\}$           $⊳$Initialize assignments dictionary 3: while $T$ is not empty do 4:   $b_{\mathrm{target}}\leftarrow T[0]$      $⊳$Find the blue UAV with highest threat index 5:   $r_{\mathrm{UAV}}\leftarrow$find_min_threat_red_UAV$(b_{\mathrm{target}},R,C)$  $⊳$Find red UAV with lowest threat index 6:   assign_ attack$(r_{\mathrm{UAV}},b_{\mathrm{target}},\mathrm{assignments})$   $⊳$Assign red UAV to attack blue UAV 7:   remove_ threat$(b_{\mathrm{target}},T,C)$   $⊳$Remove assigned blue UAV and threat indices 8:   if is_attack_capacity_exceeded$(r_{\mathrm{UAV}},\mathrm{assignments})$ then 9:    remove_red_UAV$(r_{\mathrm{UAV}},C)$ 10:   end if 11: end while 12: return assignments

In step 1, the specific strategy of calculate_threat_indices is as Algorithm 2.

Algorithm 2 calculate_threat_indices

Require: Red UAV $R$, Blue UAV $B$, Combat matrix $C$ Ensure: Threat vector $T$ 1: $T\leftarrow \varnothing$ 2: for each $b$ in $B$ do 3:  for each $r$ in $R$ do 4:    $s\leftarrow {\displaystyle \frac{‖AT‖}{t}}$ 5:    $q\leftarrow \arccos \left({\displaystyle \frac{AT·TP}{‖AT‖‖TP‖}}\right)$ 6:    threat_index $\leftarrow$ $V_i\cos (q)\pm \sqrt{V_i^2-V_a^2{\sin }^2(q)}$      $⊳$Using Equation (4) 7:    $T.\mathrm{append}((b,\mathrm{threat}\_\mathrm{index}))$ 8:   end for 9: end for 10: $T\leftarrow \mathrm{sort}(T,\mathrm{descending}=\mathrm{True})$ 11: return $T$

In addition, after each target allocation is completed the lead UAV will not issue another command to change the attack target based on the existing assignment scheme. However, in the simulation, it is necessary to select a suitable time interval cycle to perform the threat judgment again and complete a new round of target allocation.

2.3. Air Combat Tactical Action

According to the action and posture characteristics of UAVs in the battlefield, the attack tactical actions are as follows:

• Head-on reception and enemy-first attack tactics.

Head-on attack tactics are used for attack within the target entry angle. When the radar detection distance of the red UAV and the maximum attack distance of its missiles are better than the radar detection distance of the blue target UAV and the maximum attack distance of its missiles, head-on attack tactics are used.

2.

Single side to receive the enemy or double side of the sandwich tactics.

The red UAV, under guidance, approaches the blue UAV from the route side or both sides of the blue UAV close to the blue UAV; at a distance from the blue UAV of 100 km ~ 120 km, to avoid the radar it turns to the side of the target, after the interception of the target UAV, for attack. The flanking strike and attack avoids the best position for radar detection by the blue UAV, and it cuts in from outside of the blue UAV’s radar detection area. A schematic of the flanking detour of the occupation attack is shown in Figure 1; where $R_1$ and $R_2$ represent the red formation, while $B_1$ represents the blue UAV. This tactic is used when the radar detection distance and missile range of the target UAV are significantly better than that of the red UAV.

3.

Covert approach and sudden attack tactics.

The red UAV receives the blue UAV at a low position with negative altitude difference, and when approaching the missile attack distance, it jumps up to the target UAV’s altitude to attack. When the red UAV’s radar’s upward vision is superior to the blue UAV’s downward vision, we choose the tactic of concealed reception.

4.

Head-on around-the-side attack tactics.

The red UAV quickly turns to the outside so that the target azimuth angle reaches 90°, and the radial relative velocity of the UAV is extremely small, so it is difficult for the blue UAV’s radar to stably intercept the red UAV. When it reaches the edge of the target’s radar detection, the red UAV quickly turns to the side of the target so that the target azimuth angle is 0 or relatively small to carry out the attack. When the radar and missile performance of the red UAV is at an obvious disadvantage, and the initial posture of the red UAV and the blue UAV is head-on, we choose head-on around-the-side attack tactics.

5.

Evasive maneuver tactics.

The evasive maneuver tactic means that the red UAV adopts 90° side-turning and tail-placing maneuvers to reduce the detection distance of radar when the blue UAV is in the side-turning and tail-placing state. When the red UAV is captured by the blue UAV’s radar but does not reach the blue UAV’s missile launching conditions, we choose the evasive attack tactics.

2.4. Air Combat Tactical Decision Control

The motion model of the UAV is as follows:

$$\left\{\begin{array}{c}\dot{x}=v\cos \gamma \sin \psi \\ \dot{y}=v\cos \gamma \cos \psi \\ \dot{z}=y\sin \gamma \end{array}\right.$$

(5)

$x$, $y$, $z$ are the position coordinates in the coordinate system. $v$ is the velocity of the UAV. $\gamma$ is the heading angle of the UAV trajectory, which represents the angle of the velocity vector and the horizontal plane x-O-y. $\psi$ represents the heading angle of the UAV trajectory, indicating the angle between the projection v’ of the velocity vector on the x-O-y plane and the y-axis.

The relative postures of the blue and red UAVs are shown in Figure 2.

In the above equations, $V_1$ is the velocity vector of the red UAV, $V_2$ is the velocity vector of the blue UAV, ${\phi }_A$ is the target azimuth angle, representing the angle between the red UAV’s heading and the target line. ${\phi }_T$ is the target entry angle, representing the angle between the blue UAV’s heading and the target line. $S_{MA}$ is the maximum range of the red UAV’s missile, and $S_{MT}$ is the maximum range of the blue UAV’s missile. $D$ is the relative distance between the blue UAV and the red UAV.

By analyzing the impact of various parameters on tactical decisions, the threat function is controlled to the same magnitude for subsequent modeling, and the control variables are defined as follows:

• Angular advantage.

$$S_T=({\phi }_T-{\phi }_A)/180$$

(6)

where $S_T\in [-1,1]$, ${\phi }_T$ is the target entry angle, and ${\phi }_A$ is the target azimuth angle. When $S_T=1$, the angular advantage is at its maximum; when $S_T=0$, the angular advantage is at its minimum.

2.

Missile range advantage

$$S_M=(S_{MA}-S_{MT})/10$$

(7)

where $S_M\in [-5,5]$, $S_{MA}$ is the maximum range of the red UAV’s missile, and $S_{MT}$ is the maximum range of the blue UAV’s missile.

3.

Relative distance advantage.

$$S_D=D/100$$

(8)

where $S_D\in (0,1)$, and $D$ is the relative distance between the red UAV and the blue UAV.

4.

Radar line-of-sight capability advantage.

$$S_R=(D_{SS}-D_{XS})/50$$

(9)

where $S_R\in (-1,1)$, $D_{SS}$ is red’s radar line-of-sight capability, and $D_{XS}$ is blue’s radar line-of-sight capability.

5.

Enemy missile launch status

The enemy missile launch status is denoted by $F$. The state is represented as a logistic variable taking the value Y/N.

Assume $S_T$, $S_M$, and $S_R$ can be fuzzified as P, Z, N; $S_D$ can be fuzzified as L, S; and $F$ can be fuzzified as Y, N. The tactics chosen for each cycle are determined by the results of fuzzy reasoning, and their values are defined as “1”, “2”, “3”, “4”, or “5”, respectively, representing head-on reception and enemy-first attack tactics, single side to receive the enemy or double side of the sandwich tactics, covert approach and sudden attack tactics, head-on around-the-side attack tactics, and evasive maneuver tactics; see

Table 2. Action strategy numbering.

No.	Action Strategy
1	Head-on reception and enemy-first attack tactics
2	Single side to receive the enemy or double side of the sandwich tactics
3	Covert approach and sudden attack tactics
4	Head-on around-the-side attack tactics
5	Evasive maneuver tactics

for the corresponding number of these action strategies.

2.5. Reasoning Rules for Tactical Decision Making

Set $S_T$, $S_M$, $S_D$, $S_R$, and $F$ as nodes, and set 7 fixed reasoning rules. As shown in

Table 3. Reasoning rules.

	${\mathit{S}}_{\mathit{T}}$	${\mathit{S}}_{\mathit{M}}$	${\mathit{S}}_{\mathit{D}}$	${\mathit{S}}_{\mathit{R}}$	$\mathit{F}$	Action Strategy
$R_1$	$P/Z$	$P/Z$			$N$	1
$R_2$	$N$	$P/Z$	$L$		$N$	1
$R_3$	$N$	$P/Z$	$S$		$N$	3
$R_4$		$P/Z$	$L$	$P$	$N$	2
$R_5$		$N$		$Z/N$	$N$	3
$R_6$			$L$		$Y$	4
$R_7$			$S$		$Y$	5

, it is stipulated that each reasoning rule can derive corresponding action strategies when the node is in the state shown in the table. Among them, $P/Z$ represents that the node is in state $P$ or state $Z$, $Z/N$ represents that the node is in state $Z$ or state $N$, the statements from 1 to 5 in “action strategy” are consistent with those in Table 2.

3. Air Combat Decision-Making Model Based on Bayesian Networks

This section introduces an air combat decision model based on Bayesian network decision making. After the target allocation is completed, the formation cooperative combat evolves into a one vs. one confrontation combat mode between the red UAVs and the assigned target. For each of the red UAVs, the air combat decision-making model can represent the current enemy–us situation based on five control variables, i.e., the dominance function, which then determine five tactical decision-making actions. Meanwhile, for the Bayesian network-based air combat decision-making model, this section introduces its updating method and decision-making method, as well as the nodes and node relationships in the model. According to the input and output values of the decision model, the current optimal decision method can be selected.

3.1. Bayesian Network Modeling

A Bayesian network is an uncertainty handling model that simulates causal relationships in human reasoning and can be represented as a probabilistic graphical model through a directed acyclic graph [37]. The directed acyclic graph represents a definite direction in the information flow process, and the connection relationship between nodes represents the conditional independent semantics of Bayesian networks [38,39]. A Bayesian network consists of nodes and a table of conditional probabilities between nodes, with a series of probability values that allow the calculation of any given joint probability, i.e., the inference process of a Bayesian network.

The Bayesian network analysis method is shown in Figure 3.

3.2. Bayesian Network Update Method

For a Bayesian network, suppose some node $X$ has $m$ child nodes ($Y_1,Y_2,\dots ,Y_m$) and $n$ parent nodes ($Z_1,Z_2,\dots ,Z_n$). $\lambda$ represents the information of the node, and $\pi$ represents the causal relationship between the parent and child nodes. The probability of event x occurring in the child node set $X$ given the state z in the parent node set $Z$ is

$$M_{X|Z}=P(X=x|Z=z)$$

(10)

Since $X$ has multiple discrete values, $\lambda (x)$ and $\pi (x)$ are vector factors. Their elements are related to each discrete value of $X$:

$$\lambda (x)=[\lambda (X=x_1),\lambda (X=x_2),\dots ,\lambda (X=x_l)]$$

(11)

$$\pi (x)=[\pi (X=x_1),\pi (X=x_2),\dots ,\pi (X=x_l)]$$

(12)

The inference algorithm of the Bayesian network centers on each node, obtaining $\lambda$ values from the child nodes and $\pi$ values from the parent nodes. The posterior probability of the nodes is then calculated and updates are made to the neighboring nodes. The update process is as follows.

The posterior probability of updating a node is as follows:

$$Bel(x)=\alpha \lambda (x)\pi (x)$$

(13)

Here, $\alpha$ is the normalization factor, satisfying the condition $\underset{x}{\Sigma }Bel(x)=1$, $\lambda (x)=\underset{j}{\mathrm{\Pi }}{\lambda }_{Y_j}(x)$, $\pi (x)=\underset{i}{\mathrm{\Pi }}{\pi }_{z_i}{M}_{X|Z}$.

It is updated from bottom to top as follows:

$${\lambda }_X(z)=\lambda (x)M_{X|Z}$$

(14)

It is updated from top to bottom as follows:

$${\pi }_{Y_j}(x)=\alpha \pi (x)\underset{k\ne j}{\mathrm{\Pi }}{\lambda }_{Y_k}(x)$$

(15)

3.3. Bayesian Network Modeling for Tactical Decision Making

A Bayesian network model is the basis of Bayesian network decision making. Establishing a Bayesian network model must follow certain principles and rules, and the process includes confirming the network nodes, confirming the network state, and confirming the relationship between nodes.

From the analysis of tactical decision making, it can be seen that the control variables in the decision-making process include angular advantage $S_T$, missile range advantage $S_M$, relative distance advantage $S_D$, the red UAV’s radar line-of-sight capability advantage $S_R$, and the blue UAV’s missile launch status $F$. Therefore, the Bayesian network model nodes consist of $S_M$, $S_T$, $S_D$, $S_R$, and $F$. For node $S_D$, there are two states: L and S. The blue UAV’s missile launch status $F$ has two states: Y and N. The decision node D has five states: T1, T2, T3, T4, and T5, which correspond to head-on reception and enemy-first attack tactics, single side to receive the enemy or double side of the sandwich tactics, covert approach and sudden attack tactics, head-on around-the-side attack tactics, and evasive maneuver tactics, respectively. The final selected tactical action is determined by $S_M$, $S_T$, $S_D$, $S_R$, and $F$, constructing the Bayesian network model as shown in Figure 4.

In the Bayesian network model, ${\phi }_A$ and ${\phi }_T$ are parameters determining the angular advantage, $S_{MA}$ and $S_{MT}$ are parameters determining the missile range advantage, $A\mathrm{xis}A$ and $A\mathrm{xis}T$ are parameters determining the relative distance advantage, $D_{SS}$ and $D_{XS}$ are parameters determining the red UAV’s radar line-of-sight capability and blue’s radar line-of-sight capability advantages, and $StaticM$ is the parameter determining the enemy missile launch status.

3.4. Definition of Inputs and Outputs in Decision Modeling

For the above Bayesian network decision model, the input and output variables in the combat mission are shown in

Table 4. Input variables.

Input Quantity	Range of Values
Angular advantage, $S_T$	[−1, 1]
Missile range advantage, $S_M$	[−5, 5]
Relative distance advantage, $S_D$	(0, 1)
Radar line-of-sight capability advantage, $S_R$	(−1, 1)
Enemy missile launch status, $F$	$Y/N$

and

Table 5. Output variables.

Output Quantity	Range of Values
Attack UAV tactics	$T_1,T_2,T_3,T_4,T_5$

, respectively. Based on the input and output variables, the tactical decision chosen at each moment can be judged.

3.5. Air Combat Decision Evaluation Model

The purpose of close air combat is to maneuver the UAV as close as possible to the tail of the target UAV, and at the same time avoid the target UAV from approaching our UAV’s tail. Based on the above air combat decision-making model, an evaluation model is established to evaluate the decision-making process of the confrontation between us and the enemy. According to the spatial position and relative motion state of the UAV and the target, the following four functions are designed, the time advantage function, the speed advantage function, the altitude advantage function, and the weapon advantage function, which together constitute the evaluation function of the air combat decision evaluation function.

• Time advantage function.

In the decision-making process of UAV confrontation, the combat time is used as an index for judging the decision-making efficiency of UAVs, and the shorter the combat time is, the higher the decision-making efficiency of UAVs is within the range of allowed combat time.

$$Q_1=1-{\displaystyle \frac{1}{1+e^{—\frac{T_o-T_s}{T_{up}-T_{down}}}}}$$

(16)

where $T_o$ and $T_s$ are the attack start and attack end times, respectively. $T_{up}$ and $T_{down}$ are the maximum and minimum time allowed for UAV operations, respectively. The greater the interval between the end time of the attack and the start time of the attack, the greater the temporal advantage that the current decision-making method possesses, within the operationally permissible time frame.

2.

Speed advantage function.

In the UAV confrontation decision-making process, speed and altitude together characterize the UAV’s energy advantage, which is an important indicator of the UAV’s situational superiority. Within the range of permissible flight speeds, a UAV with a higher speed relative to an enemy aircraft has a more favorable posture.

$$Q_2=1-{\displaystyle \frac{1}{1+e^{\frac{v_U-E_v}{v_{up}-v_{down}}}}}$$

(17)

where $v_U$ is the flight speed of the blue UAV during the attack, and $E_v$ is the average speed of the UAV in each round. $v_{up}$ and $v_{down}$ are the maximum and minimum speeds at which the UAV is allowed to fly, respectively. The greater the flight speed of our UAV during the attack compared to its average speed during the round, the greater the speed advantage of the current decision is considered to be.

3.

Altitude advantage function.

In the decision-making process of UAV confrontation, in the range of permitted flight altitudes, the red UAV has a higher altitude relative to the blue UAV, and the posture is more advantageous.

$$Q_3=1-{\displaystyle \frac{1}{1+e^{\frac{z_U-E_z}{Z_{\max }-Z_{\min }}}}}$$

(18)

where $z_U$ is the altitude of the red UAV at the time of the attack, and $E_z$ is the average altitude of the UAV in each round. $Z_{\max }$ and $Z_{\min }$ are the maximum and minimum altitudes at which the UAV is allowed to fly, respectively. The greater the altitude of our drone’s flight at the time of the attack compared to its average altitude during the round, the greater the altitude advantage of the current decision.

4.

Weapon advantage function.

In the decision-making process of UAV confrontation, the remaining weapons as well as the confrontation results are used as evaluation indexes, setting the number of remaining weapons on the red side as $F_1$ and the number of remaining weapons on the blue side as $F_2$. The result of the match is $R$, which is recorded as $R=1$ in the case of the red side’s victory, and $R=0$ in the case of the blue side’s victory.

$$Q_4=\left\{\begin{array}{l}1,R=1,\hspace{0.33em}and\hspace{0.33em}{F}_1>F_2\\ 0,\hspace{0.33em}else\\ -1,R=0,\hspace{0.33em}and\hspace{0.33em}{F}_1<F_2\end{array}\right.$$

(19)

When the red side wins and has fewer weapons left, $Q_4=1$. Conversely, when the blue side wins and has fewer weapons left, $Q_4=-1$.

5.

Air combat decision evaluation function.

On the basis of designing the time advantage function, speed advantage function, altitude advantage function, and weapon advantage function, the three advantage functions are restricted by setting adjustment parameters through a hierarchical analysis method; in order to be able to directly represent the weight of each advantage function, the different coefficients of the advantage function are normalized, and the air combat decision-making evaluation function is obtained as follows:

$$Q={\alpha }_1\times Q_1+{\alpha }_2\times Q_2+{\alpha }_3\times Q_3+{\alpha }_4\times Q_4$$

(20)

where ${\alpha }_1$, ${\alpha }_2$, ${\alpha }_3$, and ${\alpha }_4$ are the weight coefficients; the specific weights can be calculated using the hierarchical analysis method through the evaluation function as a measure of the advantages and disadvantages of air combat decision making [40].

4. Simulation and Validation

In this section, the air combat decision-making model is validated based on the Bayesian network-based air combat decision-making method. In addition, the communication method of sending battlefield information via OpenDDS between the lead UAV and the wingman is introduced. After that, two groups of simulation confrontation are carried out on the simulation platform with two vs. two as the simulation scenario. Finally, the rationality of the target allocation method and the effectiveness of the air combat model updated and decided by the Bayesian network are proved by the red side winning.

4.1. Formation Cooperative Combat Target Allocation Model

In the simulation training, the default numbers of the lead UAV and wingman are 1 and 2, respectively, and the numbers of the two blue UAVs are 3 and 4, respectively, and the communication events of the lead UAV–wingman are shown in

Table 6. Lead UAV–wingman communication events.

Communication Modes	Function	Communication Form and Explanation
Lead UAV --> Wingman	Lead UAV writes for DDS publication	a–b–c a: Total number of UAVs on the battlefield b: Identifier for the attacking side c: Identifier for the defending side
	DDS subscription write for wingman read
Wingman --> Lead UAV	Wingman writes for DDS publication	d–e d: Number of blue UAVs detected on the battlefield e: Number of blue UAVs shot down
	DDS subscription write for lead UAV read

.

The lead UAV sends the wingman the current attack mission, i.e., the attacking and attacked parties, and this message indicates the number in the attacking party and the number in the attacked party based on the number of UAVs in the current formation. The wingman sends to the lead UAV the detected status of the blue side on the current battlefield, which indicates the number of detectable blue UAVs and the number of blue hits.

This constant communication between the leader and wingman determines the target of the attack each cycle.

4.2. Experimental Environment and Parameter Setting

The simulation platform is based on the air combat confrontation simulation software, which is mainly oriented to formation air combat, based on simulation, human–computer interaction, and other technical principles, to recreate the key elements of formation air confrontation with high precision. The platform supports algorithm implementation using the C++ language, and adopts two vs. two red and blue confrontation, with the red side as the attacking side, and the red and blue sides each controlling two UAVs to carry out air combat games within the limited air combat task area.

The air combat area is a closed rectangular three-dimensional space, with minimum flight height and maximum flight height restrictions. The red side’s UAVs are initially located in the red side’s battle area, and the blue side’s UAVs are initially located in the blue side’s battle area. The combat scenario is set up to simulate the confrontation according to the currently set action strategy. The initial parameters are shown in

Table 7. Initial parameters of operational scenarios.

Regional Platform	Parameter	Initial Value	Parameter	Initial Value	Unit
Setup of the combat simulation platform	Longitude of the red UAV	102.4	Longitude of the blue UAV	103.7	deg
	Latitude of the red UAV	29.1	Latitude of the blue UAV	28.8	deg
	Altitude of the red UAV	6000	Altitude of the blue UAV	6000	m
	Mach number of the red UAV	0.8	Mach number of the blue UAV	0.8	Ma
	Heading of the red UAV	90	Heading of the blue UAV	270	deg

.

4.3. Training Results

4.3.1. Experiment One

During the simulation, both our side and the enemy have two UAVs. Our side is the red side, with the lead and wingman corresponding to no. 1 and no. 2, respectively. The enemy is the blue side, corresponding to no. 3 and no. 4. Both our aircraft and the enemy aircraft are set to fly at an initial altitude of 6000 m. Our side flies according to the set strategy, while the enemy flies according to the waypoint. Both sides are equipped with the same number of missiles.

The simulation scenario of the enemy UAVs flying along waypoints in a combat environment is shown in Figure 5a. The enemy UAVs flew along the waypoint and the combat results are shown in Figure 5b. The red sector is our formation radar detection and missile guidance area, and the blue sector is the enemy formation radar detection and missile guidance area.

The operation of the red formation’s attack strategy selection for each cycle is shown in Figure 6. Figure 6 describes the strategy selection results of the lead UAV and wingman in each cycle during the confrontation, updated and decided through the Bayesian net-work.

The evaluation function is set according to the hierarchical analysis method, setting the speed advantage level as 1, the height advantage level as 2, and the weapon advantage level as 1/2, and the evaluation function is calculated as follows:

$$Q=0.31190476\times Q_2+0.49047619\times Q_3+0.19761905\times Q_4$$

(21)

Thirty sets of simulation confrontations were conducted, and a comparison between the scores of the red and blue sides is shown in Figure 7.

In order to make a more visual comparison, the data in Figure 7 were counted and calculations performed, and Figure 8 was drawn based on the processed data.

In Figure 7, the red color is the combined score of the red UAVs and the blue color is the combined score of the blue UAVs. Overall, the red UAVs’ scores are higher than the blue UAVs’ scores in the same round.

It can be seen from the statistics that the fluctuation in the decision-making score of the red UAVs is 0.02, which is lower than the decision-making score of the blue side of 0.04, while the maximum and minimum values of the red decision-making score are higher than the score of the blue side, and the fluctuation interval of the score of the red side is 0.13, which is lower than that of the blue side of 0.22. It can be seen from Figure 8 that the decision-making evaluation system can play a reasonable role in evaluating the decision-making level of the UAVs in each round.

4.3.2. Experiment Two

In the second group of confrontations, we set the red UAVs to use the Bayesian algorithm strategy and the blue UAVs to use the attack-avoidance strategy to start the confrontation. Both sides carry the same number of missiles.

The attack-avoidance strategy is as follows: Blue’s no. 3 and no. 4 UAVs have the same strategy, and both can calculate their respective air combat capability index. When a red UAV enters the safety zone set by blue, the blue UAV will fly in the opposite direction to avoid the red UAV. When the red UAV is detected entering the attack zone, the blue UAV launches missiles.

The trajectory of one round of aerial combat is recorded; 1001 represents no. 1 UAV, 1002 represents no. 2, 1003 and 1004 represent no. 3 and no. 4, and 6024 represents the launch missile marker. The confrontation trajectories of the red and blue UAVs are plotted, and the flight trajectory diagram is shown in Figure 9. During the process of launching the missile to hitting the target, 10 moments are selected according to the evaluation model of experiment one, and the scores of the four UAVs are plotted, as shown in Figure 10.

In Figure 9, the x and y axes represent latitude and longitude, respectively, and the z-axis represents altitude. The real flight trajectory of the UAV is described by this 3D scene.

The time from the launch of the missile until the missile is about to hit the target is taken, and ten of these moments are taken to calculate the score of each of the four UAVs, as shown in Figure 10.

As can be seen in Figure 9, the lead UAV assigns the wingman to attack target no. 4, and when the blue UAV enters the attack area, the wingman fires a missile-guided hit on the blue UAV, and after the hit the target red formation continues to attack target no. 3. As can be seen in Figure 10, 1002 and 1004 are in attack and evasion states, respectively, and their scores are lowest when the blue UAV is hit.

4.3.3. Experiment Three

In the third set of confrontation experiments, the blue UAVs and the red UAVs are still flying at an initial altitude of 6000 m. On the basis of the target allocation and communication method set in this paper, the red side adopts the Bayesian algorithm and the particle swarm algorithm strategy to carry out the comparison experiment. The blue side starts the confrontation by flying according to the same waypoints, and both the blue side and red side mounted the same number of missiles.

The parameters related to the particle swarm algorithm experiment are shown in

Table 8. Particle swarm algorithm parameters.

Parameter	Representation	Initial Value
Particle count	$N$	60
Particle dimension	$D$	10
Inertia weight	$\omega$	0.8
Learning factor	$c_1$	0.1
Learning factor	$c_2$	0.1
Maximum number of iterations	$T_{\max }$	100

.

According to the simulation environment of experiment two, 30 groups of simulation confrontation are launched and the scores of red’s strategy are compared. According to the hierarchical analysis method, the evaluation function was set: the time advantage level was set as 1, the speed advantage level as 3, the height advantage level as 4, the weapon advantage level as 1/2. The evaluation function calculation method is as follows:

$$Q=0.12056036\times Q_1+0.2501542\times Q_2+0.52824038\times Q_3+0.10104506\times Q_4$$

(22)

A graph comparing the scores of the methods used in this paper and the particle swarm algorithm is shown in Figure 11.

In Figure 11, the red color is the comprehensive score of our UAVs using the Bayesian algorithm, and the blue color is the comprehensive score of our UAVs using the particle swarm algorithm. In general, the combined score of the Bayesian algorithm is higher than the combined score of the particle swarm algorithm.

For the two decision-making methods, the number of battle losses of UAVs in the enemy and our formations in 30 rounds is compared, and the comparison graph is shown in Figure 12. Figure 12a shows the number of battle losses in the Bayesian decision-making method, and Figure 12b shows the number of battle losses in the particle swarm decision-making method.

During the 30-group confrontation, the number of battle losses of our UAVs under the Bayesian decision-making method is lower than that of the particle swarm decision-making method, and the annihilation of the enemy aircraft is more efficient. Figure 12 shows that the decision-making method used in this paper is better than the particle swarm decision-making method, and the decision-making method in this paper is effective.

4.3.4. Simulation Results and Analysis

In the first set of experiments, it can be seen in Figure 5 that after both sides’ radars detected each other and entered the attack range, the red formation collaborated to destroy two blue UAVs and won the victory. In Figure 6, it can be seen that the red formation constantly chooses the most appropriate attack strategy for the current situation and effectively avoids the attack of the blue UAVs. Comparing the scores of the blue side and us through Figure 7 and Figure 8, the comprehensive scores of the red UAVs are higher than the comprehensive scores of the blue UAVs and the fluctuation is small, which reflects the rationality of the evaluation function established in this paper.

In the second group of experiments, the flight trajectories of the red formation and the blue formation are plotted in Figure 9, and the scores of the four UAVs in the confrontation process are calculated in Figure 10. In order to protect the lead UAV in the target allocation process, the wingman is preferentially assigned to perform the attack task. It can be seen by plotting the curve graph that the wingman and the blue UAV are in the attack and avoidance states, respectively.

In the third set of experiments, the red UAVs flew and attacked the blue UAVs using the Bayesian algorithm and particle swarm algorithm, respectively. Figure 11 compares the comprehensive scores of the red UAVs under the two strategies, and the Bayesian algorithm’s comprehensive score is generally higher than that of the particle swarm algorithm; Figure 12 compares the number of aircraft losses of the red and blue sides under the two algorithms, and the average number of aircraft losses of the Bayesian algorithm is generally lower, which verifies that the decision-making strategy of the blue side’s UAVs in attacking and evading is advantageous.

4.3.5. Experimental Conclusion

According to the lead UAV–wingman attack strategy, it can be seen that with the lead UAV in the main decision-making role, the wingman, in the main detection as well as an attack role, constantly strikes the blue UAV; according to the principle of allocation, the lead UAV and the wingman coordinate the attack. By comparing the combined scores of the two sides and comparing with other algorithms, the decision-making framework established in this paper is shown to be effective, proving the rationality of assigning targets according to the threat degree. Through the DDS communication, the efficiency of the coordination between the lead UAV and the wingman is improved, which verifies the reliability of the Bayesian network decision making.

5. Conclusions

This paper addresses the problem of formation cooperative intelligent tactical decision making, and proposes a method based on a Bayesian network model to study formation cooperative intelligent tactical decision making.

The main work of this paper is as follows.

For the formation coordinated attack problem of allocating targets, a coordinated attack target allocation model based on a dominance matrix is constructed, which improves the efficiency of communication and target allocation between formations on the battlefield. The effectiveness of this communication method is proved through simulation and confrontation.

For the problem of autonomous air combat decision making of UAVs, the air combat decision-making model based on a Bayesian network is constructed. An evaluation function is established, and by comparing it with other decision-making methods, the accuracy and intelligence of the Bayesian network decision making are confirmed. The effectiveness of the algorithm model is verified through simulation.

For subsequent research directions we will further optimize the Bayesian decision network, increase the number of simulated confrontations, and start the confrontation in a more complicated combat environment.