A Hierarchical Mission Planning Method for Simultaneous Arrival of Multi-UAV Coalition

Featured Application

Cooperative task allocation and path planning problems of multi-UAV are becoming more and more important. The proposed mission planning method can be easily applied to some civilian scenarios, such as multi-UAV cooperative forest fire fighting and some military scenarios.

Abstract

A hierarchical mission planning method was proposed to solve a simultaneous attack mission planning problem for multi-unmanned aerial vehicles (UAVs). The method consisted of three phases aiming to decouple and solve the mission planning problem. In the first phase, the Pythagorean hodograph (PH) curve was used in the path estimation process for each UAV, which also served as the input for the task allocation process. In the second phase, a task allocation algorithm based on a negotiation mechanism was proposed to assign the targets. Considering the resource requirement, time-dependent value of targets and resource consumption of UAVs, the proposed task allocation algorithm can generate a feasible allocation strategy and get the maximum system utility. In the last phase, a path planning method was proposed to generate a simultaneous arrival PH path for each UAV considering UAV’s kinematic constraint and collision avoidance with obstacles. The comparison simulations showed that the path estimation process using the PH curve and the proposed task allocation algorithm improved the system utility, and the hierarchical mission planning method has potential in a real mission.

1. Introduction

Over the last decades, unmanned aerial vehicles (UAVs) have been widely used in a large range of applications, such as environmental monitoring [1], searching and attacking [2], forest fire fighting [3] and so on. Due to the capability limitation of a single UAV, multiple UAVs working together in coordination are expected to perform more complex missions under various types of attack [4]. For example, multiple UAVs can accomplish an attack mission much better than a single UAV when those UAVs arrive simultaneously to the target.

Two main aspects in multi-UAV attack mission are task allocation and path planning. The complexity of these problems increases with the number of UAVs and tasks. In addition, in some specific scenarios, the imposed constraints, such as resource requirement, collision avoidance and simultaneous arrival constraints further increase the complexity of those problems.

Many researchers treated the task allocation and path planning as two sub-problems separately in the in multi-UAV attack mission, they use a centralized or distributed approach to obtain a feasible task allocation strategy and then use path planning algorithms to generate feasible paths for UAVs. Sharma et al. [5] proposed a cooperative ad-hoc network framework for multi-UAV using a neural network, cooperative task allocation and path planning algorithms can be used in the framework in different applications. Yao et al. [6] developed an iterative mission planning framework for the task allocation and path planning problems, an improved auction algorithm with an iterative strategy was proposed in the task allocation process. When multi-UAV is used to perform multi-task, the task allocation and path planning process are coupled [7], as the performance of task allocation depends on an accurate path length of UAVs, and the path planning depends on the UAV’s task allocation results.

To solve the coupled task allocation and path planning problems, there are two main ways. In the first class of methods, Euclidean distance between UAVs and targets is seen as the input in the task allocation process [8,9]. After obtaining the feasible allocation results, the path planning process is used to generate flyable and safe paths/trajectories for UAVs considering a variety of constraints. However, the length of the generated paths/trajectories are quite different from the Euclidean distance, especially when there are obstacles in the environment. This deviation will result in poor quality of solutions obtained in the task allocation process. The second method considers the task allocation and path planning process as an integrated optimization problem, and encoding the solution of the coupled problem, genetic-based [10,11] and decision tree [12] algorithms are usually used. However, simultaneously solving task allocation and path planning needs a large amount of computation.

The presented task allocation studies above assume each target can be performed by single UAV and the resources of UAVs do not deplete. However, in a real situation, some targets may need a variety of resources and should be performed by multi-UAV simultaneously, such as for a simultaneous attacking mission, multiple UAVs are required to reach the target simultaneously with a different entry angle to maximize the mission performance. This problem can be called the single-task, multi-robot (ST-MR) problem according to the taxonomy criteria proposed by Gerkey et al. [13]. The ST-MR problem is also known as the coalition formation problem [14], a coalition means a team of UAVs performing the same target.

To solve the coalition formation problem for multiple UAVs performing multi-target, two distributed sub-optimal coalition formation algorithms were proposed to form coalition with minimum time and size to destroy targets simultaneously [15,16]. In a search and destroy mission for multiple UAVs, a welfare-based coalition formation algorithm for heterogeneous UAVs was presented by Kim et al. [17], the UAVs use resources in a balanced manner to improve the system’s robustness in the dynamic and uncertain environment. However, the works cited above assume that only the UAV performing a search task can send a bid to form a coalition, they do not consider the overlapping coalitions, which will decrease the mission performance. For the cooperative task allocation and rendezvous problems, Sharma et al. [18] proposed a global optimization algorithm inspired by the biological features of the hill Myna and desert Sparrow, the algorithm can be applied to ad-hoc networks in various applications. For the overlapping coalitions problem, Oh et al. [19] proposed two distributed coalition formation algorithms for multi-UAV based on the contract net protocol, the sequence of UAVs and pop-up tasks are considered in the dynamic environment with limited communication range. The studies cited above do not take into account the obstacles and no-fly-zones in the environment, which significantly limits the application of the coupled task allocation and path planning algorithm.

To maximize the system performance, the paths for the UAVs in the same coalition should satisfy simultaneous arrival as well as some other constraints, such as the kinematic constraint of a UAV and collision avoidance with obstacles. For the simultaneous arrival problem (also called rendezvous problem), the velocity-based [20,21] method has been used. The simultaneous arrival problem of multiple UAVs is addressed by a velocity control approach [20], the UAVs communicate with each other to make a consensus on the estimated time of arrival. A distributed optimization algorithm based on game theory [21] was proposed for a UAV rendezvous problem, each UAV adjusts its velocity according to the states of its neighbors in a predefined time interval. However, this method is easy to cause velocity saturation [22] and a large amount of communication is required. Some other studies [7,23] assume all the UAVs fly at the same and constant speed, so the simultaneous arrival for multi-UAV can be achieved by planning paths of equal length. In our previous work, a cooperative particle swarm optimization (CPSO) algorithm is proposed to generate Pythagorean hodograph (PH) paths of equal length for each UAV. However, the task sequence of UAVs is not considered.

In this paper, the focus is on the coupled task allocation and path planning problems for multi-UAV performing simultaneous arrival tasks. Considering the target’s resource requirement, simultaneous arrival, time-dependent values of targets, resource consumption of UAVs and collision avoidance with obstacles. A hierarchical mission planning method is proposed to solve the coupled task allocation and path planning problem for a multi-UAV performing a multi-task, which is also the contribution of this paper. The method consists of three phases. In the first phase, the PH curve is calculated as the estimated path for each UAV, which can provide a more accurate input for the task allocation process. Then, a task allocation algorithm is proposed to form a coalition for each target considering the task sequence of UAVs and the resource requirement of targets. Finally, the path planning process generates the flyable and safe paths for the fixed-wing UAVs in the same coalition to realize the simultaneous arrival.

The rest of this paper is organized as follows: In Section 2, a description of the multi-UAV cooperative attacking problem is given. Section 3 describe the task allocation method for multi-UAV. In Section 4, path planning method to achieve simultaneous arrival is presented. Experimental results are shown in Section 5. In Section 6, the conclusions are given.

2. Description of the Multi-UAV Cooperative Attack Problem

The problem considered in this study consists of assigning a fleet of heterogeneous UAVs to a set of targets. At the initial time, there are some targets in the environment, it is assumed the information of these targets are known at the initial time. The UAVs are required to attack the targets.

In this paper, the UAVs and targets are modeled based on their resources. Each target may need a coalition of UAVs. According to the information of the targets and UAVs, the appropriate UAVs are allocated to each target. In order to maximize the system performance, the UAVs performing the same target are required to arrive to the target simultaneously. Then, the path planning process is used to generate simultaneous arrival paths for the UAVs in the same coalition. The resource requirement and simultaneous arrival of targets, the UAV’s kinematic constraints and collision avoidance with obstacles are taken into account.

In this study, some assumptions were made to simplify the problem:

• For simplicity, the take-off and landing process of UAVs were not taken into account, the collision avoidance among UAVs were achieved by altitude layering.
• In the task allocation process, the communication topology remained unchanged.
• The aerodynamic and attitude of UAVs were not considered.
• The obstacles and targets considered in this paper were static.

2.1. Targets and UAVs

Let $T=\{T_1,T_2,\dots ,T_M\}$ be the set of $M$ stationary targets, it is assumed that the information of targets is known at the initial time. Each target requires m-different types of resources, the resource requirement of target $T_j$ can be expressed as Equation (1), for example, $R^{T_j}=(3,3,2)$ denotes that there are three types of resources required to destroy target $T_j$ and the required amount of each resource is 3, 3 and 2, respectively.

$$R^{T_j}=(R_1^{T_j},R_2^{T_j},\dots ,R_m^{T_j}),j=1,\dots ,M$$

(1)

Let $Value=(Valu{e}_1,\dots ,Valu{e}_M)$ be the set of values of the targets at the initial time, the value of targets decreases with time as follows:

$$rewar{d}_j=Valu{e}_j\cdot e^{-\beta t_j}$$

(2)

where $\beta >0$ is a user-defined coefficient, which denotes the descent rate of the target’s value. $t_j$ is the time of arrival of the UAV coalition assigned to attack the target $T_j$.

Let $U=(U_1,U_2,\dots ,U_N)$ be the set of $N$ heterogeneous UAVs. The heterogeneity of UAVs is reflected in their different dynamic characteristics and the amount of resources as shown below:

$$R^{U_i}=(R_1^{U_i},R_2^{U_i},\dots ,R_m^{U_i}),i=1,\dots ,N$$

(3)

It is assumed that all UAVs fly in a two-dimensional plane. The motion of UAV $U_i$ is governed by the following equation:

$$\left[\begin{array}{c}{\dot{x}}_i\\ {\dot{y}}_i\\ {\dot{\phi }}_i\end{array}\right]=\left[\begin{array}{cc}\cos {\phi }_i& 0\\ \sin {\phi }_i& 0\\ 0& 1\end{array}\right]\left[\begin{array}{c}{V}_i\\ {\omega }_i\end{array}\right]$$

(4)

where $(x_i,y_i)$ and ${\phi }_i$ represents the Cartesian coordinates and the heading angle of UAV $U_i$, respectively. $V_i$ and ${\omega }_i$ denote the velocity and turn rate, respectively. It is assumed that the UAVs fly at a constant and same velocity.

The control input for each UAV can be described as: $u_i(t)={\omega }_i(t)$, which is subject to the constraints in Equation (5). $R_{\min }^i$ is the minimum turning radius of UAV $U_i$.

$${\omega }_i\le \left|V_i/R_{\min }^i\right|$$

(5)

In this paper, the UAVs were required to perform the attacking mission to the targets. The objective of the mission planning problem is to obtain the assignment strategy for each UAV to maximize the objective function and provide flyable and safe paths for these UAVs.

2.2. Objective Function and Constraints

The objective of the task allocation problem is to assign the targets to the UAV team to obtain the maximum system utility. To satisfy the resource requirement of targets, there may be some UAVs forming a coalition to attack one target. The reward to destroy targets and the cost of the UAV coalitions were considered in the objective function, so, two criteria were selected to quantify the objective function of this mission planning problem: The reward to destroy targets and the cumulative path length of all the UAV coalitions. Thus, the objective function to be maximized is as follows:

$$J={\displaystyle \sum _{j=1}^M(}{w}_1\times Valu{e}_j\times e^{-\beta t_j}-w_2\times {\displaystyle \sum _{i=1}^{\left|C_j\right|}{L}_{ij})}$$

(6)

where $w_1,w_2$ respectively denotes the weights of the reward and cost. $C_j$ represents the UAV coalition assigned to attack the target $T_j$, $\left|C_j\right|$ denotes the number of UAVs in the coalition $C_j$, $L_{ij}$ denotes the path length from the initial location of UAV $U_i$ to the target $T_j$. Assuming $A^{(i)}$ is the task sequence of UAV $U_i$, tasks before $T_j$ should be considered in calculating $L_{ij}$. $t_j$ is the estimated time of arrival (ETA) of the latest UAV in the coalition assigned to the target $T_j$:

$$t_j=\underset{i\in C_j}{\max }(ETA(i,j))$$

(7)

The constraints considered in this paper can be categorized into UAV-related constraints, target-related constraints and environment-related constraints.

(1) UAV-related constraints

First of all, UAVs have limited fuel, so a maximum flying distance constraint should be satisfied:

$$L(A^{(i)})\le L_{\max }^i,i=1,\dots ,N$$

(8)

where $L(A^{(i)})$ is the total path length of UAV $U_i$, $L_{\max }^i$ denotes the maximum flight distance of UAV $U_i$.

Each UAV can carry limited resources that deplete with use, so each UAV cannot consume more resources than it carries.

$$R(A^{(i)})\le R^{U_i}$$

(9)

where $R(A^{(i)})$ represents the total resource consumption for performing the targets assigned to $U_i$.

A flyable curve [24] is required to satisfy the UAV’s kinematic constraint. In a two-dimensional plane, a curve is determined by its curvature. Let ${\kappa }_i(q)$ be the curvature of UAV $U_i$ at point $q$, the curvature of the path should be continuous and bounded as follows:

$${\kappa }_i(q)\le 1/R_{\min }^i$$

(10)

(2) Target-related constraints

Each target is required to be performed by a UAV coalition simultaneously, and the resource requirement should be satisfied:

$$({\displaystyle \sum _{U\in C_j}{R}_p^U)\ge R_p^{T_j},p=1,\dots ,m;j=1,\dots M}$$

(11)

(3) Environment-related constraints

Let $\mathrm{\Omega }$ be the two-dimensional space in which UAVs fly. Let $\mathrm{\Theta }$ denote the set of obstacles in the environment. The collision avoidance between UAVs and obstacles must be satisfied:

$$(x(t),y(t))\cap \mathrm{int}(\mathrm{\Theta })=\varnothing$$

(12)

(4) Simultaneous arrival constraints

According to the resource model, each target may need a coalition of UAVs. To maximize the system performance, the UAVs in the same coalition are required to arrive to the target simultaneously as follows:

$$t_{i,j}^{path}=t_{l,j}^{path},i,l\in \{1,\dots ,\left|C_j\right|\};i\ne l$$

(13)

where $U_i$ and $U_l$ are two different UAVs in coalition $C_j$, $T_j$ is the target assigned to the coalition $C_j$.

Thus, the mission planning problem was provided as the objective function (Equation (6)) with constraints (Equations (8)–(13))

3. Task Allocation

3.1. Framework of the Mission Planning Method

Inspired by the hierarchical control framework [25,26,27] of a multi-UAV system. The mission planning method was divided into three layers in this paper, as shown in Figure 1. The first layer was the path estimation process, a Particle Swarm Optimization-based method (shown in Section 4.2) was employed to generate a PH path for each UAV as an estimated path. The length of the estimated path was seen as the input for the task allocation process, the estimated path was used to decouple the task allocation and path planning problems. In the second layer, the task allocation process assigned targets to UAVs that optimized the system performance. A task allocation algorithm was proposed to assign the targets. In the task allocation process, the resource requirement of targets, resource consumption and task sequence of UAVs were taken into account. In the last layer, according to the task allocation strategy, a path planning method was proposed to generate flyable and safe paths for the UAVs considering the task sequence of each UAV. In this layer, the simultaneous arrival, UAV’s kinematic and collision avoidance constraints were considered (detailed explanation can be found in Section 4.3). Finally, a path following process was used to generate the control signals for the UAV team. As the path following was not the focus of this paper, a path following method based on the nonlinear model predictive control [28] (NMPC) was used to track the generated PH paths.

3.2. Task Allocation Algorithm

The task allocation process defined in Section 2.2 is a Non-deterministic Polynomial hard problem. With the increasing size of the problem, the computational complexity will be increased sharply. It will be time-consuming and even impossible to obtain the optimal solution. In order to obtain a feasible solution within a reasonable time, in this paper, the task allocation algorithm assigns the most profitable target in each allocation round. In each allocation round, each UAV selects the most beneficial target to itself. Then, if there is a conflict among different UAVs, a consensus mechanism is used. The resource requirement of targets and task sequence of UAVs were considered. As mentioned above, the task allocation and path planning were coupled. In this paper, the path estimation process using the PH curve was used to decouple these two problems. The proposed task allocation algorithm based on a negotiation mechanism is presented as Algorithm 1.

Algorithm 1. Task allocation algorithm

Input:$T$: The set of targets and initial states; $R^{T_j},j=1,\dots ,M$: Resource requirement of targets; $U$: The set of UAVs and initial states; $R^{U_i},i=1,\dots ,N$: Resources of UAVs; Output:$A$: The allocation matrix of the UAV team 1: for$k=1:M$do 2:  for $U_i,i=1,\dots ,N$ do 3:   for $j=1$ to the number of unallocated targets 4:    calculate estimated reward for $U_i$ performing target $T_j$ 5:   end for 6:   select the pre-allocated target for $U_i$ 7:  end for 8:  consensus on pre-allocated target in this allocation round and select the manger UAV 9:  $C_j\leftarrow$ coalition_formation $(U_j^P,V{a}_j,R^{T_j})$    ---Algorithm 2 10:  the UAVs in the coalition $C_j$ add the target $T_j$ into their task sequence 11:  calculate resource consumption for UAVs in the coalition $C_j$ 12:  cooperative path planning for simultaneous arrival to target $T_j$    ---Section 4 13: end for

There were $M$ targets at the initial time, each allocation round would assign one target (Line 1, Algorithm 1). Then, the estimated reward to perform all unallocated targets was calculated for each UAV using Equation (14; Line 4, Algorithm 1). Considering the task sequence of a UAV, the ETA of the UAV $U_i$ to the target $T_j$ can be represented as Equation (15).

$$Reward(i,j)=Valu{e}_j\times e^{-\beta \cdot ETA(i,j)}$$

(14)

$$ETA(i,j)=\{\begin{array}{ll}{L}_{ij}^e/V_i,& if{A}^{(i)}=\varnothing \\ L_{im}^{before}/V_i+L_{mj}^e/V_i,& otherwise\end{array}$$

(15)

where $A^{(i)}$ is the task sequence of the UAV $U_i$, if $A^{(i)}=\varnothing$, which denotes there is no target assigned to the UAV $U_i$, then $L_{ij}^e$ is the estimated path length between the current location of the UAV $U_i$ to target $T_j$, which can be obtained in the path estimation process (Section 4.2). Otherwise, there has been some targets assigned to the UAV $U_i$, these assigned targets should be considered in calculating the $ETA(i,j)$. Assuming the last target in the task sequence of UAV $U_i$ is the target $T_m$, the ETA from UAV $U_i$ to the target $T_j$ can be divided into two parts. The first part is the accurate path length of the assigned targets $L_{im}^{before}$, which can be obtained by the path planning process (Section 4.3) in the previous allocation round; the second part $L_{ij}^e$ is the estimated path length from the target $T_m$ to $T_j$, which can be obtained in the path estimation process.

Then, the UAV $U_i$ selects the most appropriate target $j^{*}$ as its pre-allocated target (Line 6 Algorithm 1):

$$j_i{}^{*}=\underset{j\in M^{\prime }}{\arg \max (Reward(i,j))}$$

(16)

where $M^{\prime }$ represents the targets, which have not been assigned.

After all the UAVs have computed their pre-allocated targets, there may be conflicts among UAVs. For example, more than one UAV selects the same target as their pre-allocated target. Here, a consensus algorithm [19] is employed for conflict resolution (Line 8, Algorithm 1). Each UAV achieves a consensus on the pre-allocated target by negotiating with others. The UAV with the highest estimated reward is seen as the manager UAV $i_{manager}$ using Equation (17), the corresponding pre-allocated target of the manager UAV is regarded as the target to be allocated in this allocation round.

$$i_{manager}=\underset{i\in N}{\arg \max (Reward(i,j_{{}^i}^{*}))}$$

(17)

After the consensus process, every UAV knows the manger and the target to be allocated in this allocation round. Other UAVs, who have the required resource for the target are seen as the potential coalition members and will send applications to the manager UAV, the applications include their remaining resources and ETA. Then, the manager UAV uses algorithm 2 to determine the UAV coalition $C_j$ to perform the target to be assigned in this allocation round (for simplicity, expressed as $T_j$ in the following algorithm). Then, the UAVs in the coalition $C_j$ will augment the target into their task sequence (Line 10, Algorithm 1).

Algorithm 2. Select the coalition $C_j$ to perform target $T_j$

Input: Potential coalition members $U_j^P$, number of potential coalition member $n$, resource requirement $R^{T_j}$ of the target $T_j$, initial value of target $Valu{e}_j$. Output: The coalition $C_j$ to perform the target $T_j$ 1: for $k=1:n$ do 2:  calculate all the potential coalitions of size $k$ 3:  for $m=1$ to the number of potential coalitions do 4:   $R(m)=$ total resources of the potential coalition $m$ 5:   if $R(m)\ge R^{T_j}$ 6:    $C_{feasible}\leftarrow$ append the potential coalition 7:   end if 8:  end for 9:  if isempty $(C_{feasible})$ 10:   continue 11:  else 12:   select the coalition $C_j$ which have the highest utility to perform target $T_j$ 13:  end if 14: end for

The manager UAV and the applicants were regarded as the potential coalition members for the target $T_j$, they would form a list $U_j^P$. The coalition formation algorithm begins with finding all the feasible coalitions with sufficient resources and the minimum size (line 1–8, Algorithm 2). Then, the manager UAV selects the most appropriate coalition $C_j$ among all the feasible coalitions using Equation (18; Line 12, Algorithm 2). The utility for each coalition $c\in C_{feasible}$ is calculated using Equation (6).

$$C_j=\underset{c\in C_{feasible}}{\arg \max (utility(c))}$$

(18)

Once the coalition $C_j$ is determined to perform the target $T_j$, the coalition members will calculate the resource consumption that satisfying the target’s resource requirement (Line 11, Algorithm 1). A greedy resource management method was used in this paper. Assuming the type-p resource requirement of target $T_j$ is $R_p^{T_j}$. The type-p resource consumption $R_p^{U_k}(\mathrm{cost})$ of UAV $U_k$ in the coalition $C_j$ can be represented as:

$$R_p^{U_k}(\mathrm{cost})=\{\begin{array}{ll}0,& if{\displaystyle \sum _{i\in U_{b\_k}}{R}_p^{U_i}}\ge R_p^{T_j}\\ \min (R_p^{T_j}-{\displaystyle \sum _{i\in U_{b\_k}}{R}_p^{U_i}}),& otherwise\end{array}$$

(19)

where $U_{b\_k}$ represents the UAVs before $U_k$, when elements in $C_j$ are arranged in descending order of reward.

After the calculation of resource consumption, the UAVs in the coalition $C_j$ update their remaining resources. Then, the simultaneous arrival paths for the UAVs in the coalition $C_j$ are generated by the path planning process (Section 4.3).

3.3. Complexity Analysis of the Task Allocation Algorithm

Given the information of the operating environment and configurations, the proposed task allocation algorithm can provide solutions at low complexity. Assuming there are $N$ UAVs and $M$ targets in the environment. In Algorithm 1, each allocation round will assign one target, it requires $M$ allocation rounds to allocate all the targets. In each allocation round, step 2 in Algorithm 1 requires $N$ iterations. For step 3–5 in Algorithm 1, they undergo $M$ iterations for the worst case. The complexity for the consensus process [19] in Algorithm 1, step 8 is ${(N-1)}^2$ for the worst case. In Algorithm 2, at most $N$ elements are sorted, the complexity of the Algorithm 2 is $N\log (N)$. Summing up the numbers analyzed above in each allocation round, the complexity of the proposed task allocation algorithm in the worst case can be computed as $NM+{(N-1)}^2+N\log (N)$. For $M$ targets (line 1 in Algorithm 1), the complexity of the proposed task allocation algorithm in the worst case can be expressed as $M^{2}N+M{(N-1)}^2+MN\log (N)$.

4. Path Estimation and Simultaneous Arrival Path Planning

4.1. Pythagorean Hodograph Curve

In this paper, the PH curve was used in the path estimation and path planning processes, so a brief overview of PH curve is stated here.

In the two-dimensional plane, the Pythagorean hodograph (PH) curve $r(q)=[x(q),y(q)]$, $q\in [0,1]$ is a parameterized polynomial curve whose hodographs $x^{\prime }(q),y^{\prime }(q)$ satisfy a Pythagorean condition:

$${x^{\prime }}^2(q)+{y^{\prime }}^2(q)={\sigma }^2(q)$$

(20)

The PH curve can also be expressed in the Bézier form of degree $n$:

$$r(q)={\displaystyle \sum _{k=0}^n{C}_n^k{(1-q)}^{n-k}{q}^k{P}_k},q\in [0,1]$$

(21)

where $P_k=(x_k,y_k),k=1,\dots ,n$ are control points. The length of a PH path parameterized with parameter $q$ is calculated by:

$$L(q)={\displaystyle {\int }_{q_1}^{q_2}\Vert \dot{r}(q)\Vert }dq$$

(22)

In order to obtain the PH curve with continuous curvature, the quantic PH curve was used as the path for each UAV in this paper. The theory and calculation method of the PH curve were not repeated here, relevant contents can be found in [29,30]. It should be noted that the kinematics constraint of a UAV is satisfied by bounding the curvature of the PH curve in a two-dimensional plane, so the maximum curvature on a flyable PH curve should satisfy:

$$\max (\kappa (q))=\frac{\Vert r^{\prime }(q)\times r^{″}(q)\Vert }{{\Vert r(q)\Vert }^3}\le {\kappa }_{\max },q\in [0,1]$$

(23)

where ${\kappa }_{\max }$ represents the maximum curvature of the UAV.

When the position and attitude of the starting point and ending point are given, the shape of the PH curve is determined by the modulus of the direction vectors of the starting and ending points. Increasing the modulus value can reduce the curvature of the PH curve, but it will increase the length of the curve. Therefore, it is necessary to select the appropriate modulus values to make the generated PH curve satisfy the kinematics constraints of the UAV, obstacle avoidance and some other coordination constraint.

4.2. Path Estimation

In the task allocation process, according to Equation (15), each UAV needs to calculate the ETA from its current location to the target. For the UAV $U_i$, if $A^{(i)}=\varnothing$, $L_{ij}^e$ needs to be estimated; otherwise, the ETA from the UAV $U_i$ to the target $T_j$ consists of two parts, the first part is the accurate path length of the assigned targets $L_{im}^{before}$ (can be obtained in Section 4.3), the second part $L_{mj}^e$ requires to be estimated, where $T_m$ denotes the last target that has been assigned to the UAV $U_i$. For both $L_{ij}^e$ and $L_{mj}^e$, they can be obtained directly by calculating the length of the PH curve from one point to another.

Assuming the starting and ending points are $P_s$ and $P_f$, the flyable and safe path from $P_s$ to $P_f$ can be obtained by a constraint optimization problem as follows:

$$\begin{array}{l}\min J(x)\\ s.t.Path\notin obstacles\\ \kappa (Path)\le {\kappa }_{\max }\end{array}$$

(24)

where $x$ denotes the decision variable, the collision avoidance and curvature constraints are shown in the first and second constraint, respectively. $J(x)$ means the cost function to be minimized. In this paper, two criteria are selected to quantify the cost function, which are the length and the smoothness of the path as follows:

$$J=w\times J_{length}+(1-w)\times J_{smooth}$$

(25)

where $w\in [0,1]$ means the weight coefficient; $J_{length}$ represents the cost function of the path length. The length of the path is very important in the path planning process, the shorter path will save time and fuel. The length of the PH curve was used in this paper. $J_{smooth}$ denotes the cost function of the smoothness of the generated path, which can be expressed by the elastic energy of the generated PH curve [29].

To the constraint optimization problem, the penalty function for violating the constraints can be represented as:

$$J^{\prime }(x)=P_J\times J$$

(26)

where $P_J$ is the penalty parameter for violating the collision avoidance or curvature constraint, in this paper, $P_j=100$.

As mentioned above, a PH curve can be determined by the modulus values of the direction vectors of the starting and ending points, which are represented as $h_0$ and $h_1$, respectively. In this paper, A PSO-based algorithm was designed to solve the constraint optimization problem.

First, the initial position and velocity of each particle is initialized as $z_i=(h_{i,0},h_{i,1})$ and $v_i=(v_{i,0},v_{i,1})$, respectively. $i=1,\dots ,M_p$, where $M_p$ denotes the population size of the PSO algorithm. If the particle satisfies the collision avoidance and curvature constraint, its fitness can be calculated by Equation (25); otherwise, Equation (26) is used to calculate the particle’s fitness. The best position of each particle is represented as $P_{best}=(p_{i,0},p_{i,1})$, the global best position of all particles is represented as $g_{best}=(g_0,g_1)$. Then, each particle updates its position and velocity as follows:

$$\begin{array}{l}{\mathit{v}}_{i,d}^{k+1}=\varpi {\mathit{v}}_{i,d}^k+c_1{r}_1({\mathit{p}}_{i,d}^k-{\mathit{h}}_{i,d}^k)+c_2{r}_2({\mathit{g}}_d^k-{\mathit{h}}_{i,d}^k)\\ {\mathit{h}}_{i,d}^{k+1}={\mathit{h}}_{i,d}^k+{\mathit{v}}_{i,d}^{k+1}\end{array}$$

(27)

where $i=1,\dots ,M_p$; $d=0,1$; $k=1,\dots ,T$; $T$ is the iteration number. $c_1$ and $c_2$ are positive constants, $r_1$ and $r_2$ are random numbers between $[0,1]$.

After $T$ number of iterations, the global extremum is obtained. The estimated path length between the starting and ending point can be calculated by Equation (22).

4.3. Path Planning for Simultaneous Arrival

When the coalition $C_j,T_j\in T$ is determined to attack the target $T_j$, the UAVs in the coalition $C_j$ should arrive at the target simultaneously to maximize the system performance. The simultaneous arrival problem for the UAVs in the coalition $C_j$ can be regarded as a constrained optimization problem:

$$\begin{array}{l}\min J(x_i)i=1,\dots ,\left|C_j\right|\\ s.t.Path\_i\notin \mathrm{obstacles}\\ \kappa (Path\_i)\le {\kappa }_{\max ,i}\\ t_{i,j}^{path}=t_{l,j}^{path},i,l\in \{1,\dots ,\left|C_j\right|\};i\ne l;j=1,\dots ,M\end{array}$$

(28)

The cost function and the first two constraints are the same as Equation (24). For the simultaneous arrival problem, the third constraint requires the UAVs in the same coalition to realize the simultaneous arrival. It should be noted that, in this paper, the UAVs and targets are modeled by their resources. Each target may require a UAV coalition, and each UAV can attack a set of targets. So, the task sequence of UAVs should be considered in the third constraint, which means that the target assigned before $T_j$ should be considered in the simultaneous arrival paths. The following two cases illustrate how to obtain the simultaneous arrival paths for the UAVs in the coalition $C_j$.

(1) If the target $T_j$ is the first assigned target.

If the target $T_j$ is the first assigned target, which means the target $T_j$ is the first target in the task sequence for the UAVs in the coalition $C_j$. A cooperative particle swarm optimization (CPSO) algorithm in our previous study [31] can be used to solve the simultaneous arrival problem. Here is a brief explanation: A sub-swarm is generated for each UAV, the positions and velocities of the particles in each sub-swarm are initialized. Each particle’s fitness was calculated using Equation (25) or Equation (26; if the particle violates the first two constraints). Then, each sub-swarm selects its best global position as its representative particle. The particles in each sub-swarm modifies their fitness by communicating with other sub-swarms’ representative particles. After several rounds of iteration, the UAVs in the coalition $C_j$ can obtain the paths of equal length.

(2) If the target $T_j$ is not the first assigned target.

If the target $T_j$ is not the first assigned target, the UAVs in the coalition $C_j$ may have tasks before the target $T_j$. For the simultaneous arrival problem, the UAVs in the coalition $C_j$ should consider the targets performed before the target $T_j$, a modified CPSO algorithm is presented here.

In the modified CPSO algorithm, a sub-swarm is initialized for each UAV in the coalition $C_j$. The positions and velocities of the particles in sub-swarm $i$ can be represented as $z_{ik}=(h_{ik,0},h_{ik,1})$ and $v_{ik}=(v_{ik,0},v_{ik,1})$, respectively. Where $i=1,\dots ,\left|C_j\right|$, $k=1,\dots ,M_p$, $M_p$ denotes the population size of each sub-swarm. It should be noted that each particle in sub-swarm $i$ determines a PH path from the location of target $T_m$ to $T_j$, where $T_m$ denotes the target performed by the UAV $U_i$ just before the target $T_j$. The fitness of the particles in each sub-swarm were calculated using Equation (25) or Equation (26). However, as the target $T_j$ was not the first assigned target, the target assigned before $T_j$ should be considered in the cost function. For example, for the particles in sub-swarm $i$, the cost function for path length is modified as:

$$J_{length}=L_{ij}=L_{im}^{before}+L_{mj}$$

(29)

where $L_{im}^{before}$ denotes the accurate path length from the current location of UAV $U_i$ to the target $T_m$, which can be obtained in the previous allocation round. $L_{mj}$ denotes the path length from the location of the target $T_m$ to $T_j$.

Then, a representative particle was selected in each sub-swarm, the representative particle with the maximum path length was selected as the reference particle $p_{ref}$. The particles in each sub-swarm modifies their fitness according to the reference particle. For example, the particle $k$ in sub-swarm $i$ modifies its fitness as follows:

$$J_{i,k}^{\prime }=J_{i,k}+J_{penalty,k}$$

(30)

where $J_{penalty,k}$ is the penalty function for the particle $k$ in sub-swarm $i$:

$$J_{penalty,k}=\{\begin{array}{ll}100\times {\left|L_{ij}-L_{ref}\right|}^2,& if{p}_{ref}\notin sub-swarmi\\ 0& otherwise\end{array}$$

(31)

It should be noted that, when calculating $L_{ij}$ and $L_{ref}$, the target before $T_j$ should be considered using Equation (29).

5. Simulation Results

In this section, numerical simulations were conducted to demonstrate the performance of the mission planning method for multiple UAVs. All the algorithms were compiled in MATLAB 2016b and ran on a personal computer with an Intel Core i5-4590 @ 3.3 GHz with 8GB of RAM. In the first experiment, a general scenario was used to demonstrate the effectiveness of the mission planning method. In the second experiment, a sample run and the Monte Carlo simulations were presented to show the effect of path estimation to task allocation. The difference of a straight line and PH curve used in path estimation process were compared. Finally, the performance of the proposed task allocation algorithm was compared with the other two algorithms by the Monte Carlo simulation.

5.1. General Scenario

A general scenario is presented to show the whole process of the multi-UAV mission planning, where six UAVs needed to attack two targets. A square region with the size of $3200\mathrm{m}\times 3200\mathrm{m}$ was used. At the initial time, the information of the targets is known as shown in

Table 1. The initial information of the targets.

Targets	Initial Location (m)	Resource Requirement	Initial Value
Target 1	$(1500,1600)$	$(4,2)$	100
Target 2	$(2000,1000)$	$(5,4)$	90

. The UAVs were required to form two coalitions that satisfy the resource requirement of the targets. The UAVs in the same coalition were required to arrive to their target simultaneously. The initial information of the six UAVs are shown in

Table 2. The initial information of the unmanned aerial vehicles (UAVs).

UAVs	Initial Location (m)	Heading Angle (rad)	Initial Resource
UAV 1	$(0,0)$	$pi/6$	$(2,1)$
UAV 2	$(1500,0)$	$pi/6$	$(1,0)$
UAV 3	$(3000,0)$	$3\cdot 4/pi$	$(3,2)$
UAV 4	$(500,2200)$	$pi/6$	$(2,2)$
UAV 5	$(2000,2000)$	$pi/6$	$(1,1)$
UAV 6	$(3000,3000)$	$-3\cdot 4/pi$	$(2,1)$

. The parameters used in the mission planning method are shown in

Table 3. Control parameters in the proposed mission planning method.

Parameters	Notation	Value
Descent rate of target’s reward	$\beta$	0.05
The minimum turning radius of the UAV	$R_{\min }$	50 $\mathrm{m}$
Weight value of the target reward in Equation (6)	$w_1$	0.8
Weight value of the cost in Equation (6)	$w_2$	0.2
Weight value in Equation (25)	$w$	0.8
Velocity of the UAVs	$V$	$50\mathrm{m}/\mathrm{s}$
Population size of the PSO algorithm	$M_p$	50
Acceleration coefficient of the PSO algorithm	$c_1$	2
Acceleration coefficient of the PSO algorithm	$c_2$	2

.

At the initial time, the task allocation algorithm was used to form coalitions for the two targets. In the first allocation round, the UAV $U_5$ was selected as the manager UAV, the target $T_1$ was first to be assigned. The coalition $C_1=[U_5,U_4,U_2]$ was formed to attack the target $T_1$ simultaneously. As the target $T_1$ was the first assigned target, the CPSO algorithm was used to generate simultaneous arrival paths for these UAVs as shown in Figure 2a. The path lengths of the three UAVs in the coalition $C_1$ from their initial location to the target $T_1$ were $1862.5\mathrm{m}$, $1863.8\mathrm{m}$ and $1862.7\mathrm{m}$, respectively. Then, the UAVs in the coalition $C_1$ updated their resources according to Equation (19). The ETAs for the three UAVs are shown in Figure 3a. It can be seen that, the three UAVs could arrive simultaneously to the target $T_1$.

In the second allocation round, the UAV $U_3$ was selected as the manager UAV to assign the target $T_2$. The coalition $C_2=[U_3,U_4,U_1]$ was formed to attack the target $T_2$. As the target $T_2$ was not the first assigned target. The modified CPSO algorithm was used to generate simultaneous arrival paths for the UAVs in the coalition $C_2$. The generated paths for these UAVs are shown in Figure 2a. The path lengths of the UAVs from their initial locations to the target $T_2$ were $3008.03\mathrm{m}$, $3008.7\mathrm{m}$ and $3008.31\mathrm{m}$, respectively. It can be seen that the UAV $U_4$ had to perform the target $T_1$ before $T_2$, this was considered in the modified CPSO algorithm. The ETAs for the UAVs in the coalition $C_2$ are shown in Figure 3b. These UAVs could arrive simultaneously to the target $T_2$. Using the proposed algorithms, the formed coalitions could obtain the maximum system utility in Equation (6). The UAV $U_4$ was selected to be the part of the two coalitions $C_1$ and $C_2$. The UAV $U_6$ had not been assigned any target, which denotes that the UAV $U_6$ was always at its base.

It can be seen from Figure 2a that the paths for these UAVs could avoid collision with the obstacles. The curvature of each UAV’s path is shown in Figure 2b, which demonstrated that these paths had continuous curvature and satisfied the maximum curvature constraint.

5.2. Effects of Different Path Estimation Methods on Task Allocation

Consider a scenario that involves four UAVs and two targets. The performance of the path estimation process using the PH curve was compared to that using a straight line. Four obstacles are in the region size $3200\mathrm{m}\times 3200\mathrm{m}$. The initial information of UAVs and targets are shown in

Table 4. Initial information of UAVs.

UAVs	Initial Location (m)	Heading Angle (rad)	Initial Resource
UAV 1	$(700,900)$	$pi/6$	$(2,2)$
UAV 2	$(700,600)$	0	$(2,2)$
UAV 3	$(2100,2100)$	$pi/2$	$(2,2)$
UAV 4	$(2300,2200)$	$pi/2$	$(2,2)$

and

Table 5. Targets location and resource requirement.

Targets	Initial Location (m)	Resource Requirement	Initial Value
Target 1	$(700,2100)$	$(4,4)$	100
Target 2	$(2300,900)$	$(4,4)$	90

. Other control parameters are the same with Section 5.1.

The UAVs used the proposed mission planning method to obtain the allocation strategy and feasible paths. When the straight line and PH curve were used in the path estimation process, the mission planning (task allocation and path planning) results are shown in Figure 4a and Figure 5a, respectively. The curvature of each UAV’s path of these two path estimation methods are shown in Figure 4b and Figure 5b, respectively, the curvature of each UAV’s path satisfied the maximum curvature constraint.

It can be seen from Figure 4 and Figure 5 that when the straight line and PH curve were used in the path estimation process, respectively, the task allocation methods generated different allocation strategies. When the straight line was used in the path estimation process, the generated coalitions were $C_1=[U_1,U_3]$ and $C_2=[U_2,U_4]$ for the target $T_1$ and $T_2$, respectively. That was because the UAV $U_1$ was closer to target $T_1$ than target $T_2$ when the straight line was used in the path estimation. However, when there are obstacles in the environment, the path planning process should generate paths avoiding collision with obstacles, which will have a large deviation from the path estimation process using straight line. When path estimation using the PH curve was used in the mission planning method, it produced a more reasonable allocation strategy, the generated coalitions were $C_1=[U_3,U_4]$ and $C_2=[U_1,U_2]$ for the target $T_1$ and $T_2$, respectively.

Table 6. Performance of different path estimation methods.

Figure #	Path Estimation	Mission Time (s)	System Utility
4a	Straight line	51.7	1.3
5a	PH curve	35.6	26.5

summarizes the results of these two path estimation methods for this study case. In Table 6, the mission time denotes the total time to perform all the targets, the system utility was calculated by Equation (6). It can be seen that, the mission time using the straight line in the path estimation was longer than that using the PH curve, which also led to a lower system utility.

In order for a deeper comparison of performance differences of the two path estimation methods, the performance of the two path estimation methods with increasement in the number of targets was compared by Monte Carlo simulation. In the set of simulation, there were six UAVs in the team, and the number of targets varied from 10 to 30. For each simulation, obstacles were taken into consideration. The location of UAVs and targets were randomly generated in a bounded region. Each target needed three different types of resources, each required resource was a random integer between three and five. Each UAV also carried three types of resources, and each type of resource was randomly generated between 20 and 30. Using two different path estimation methods, the comparison results of average mission completion time and system utility are shown in Figure 6.

Figure 6a shows the average mission time using the two path estimation methods when the number of targets increased. The vertical axis denotes the average mission completion time of 100 runs. The horizontal axis indicates the number of targets. When the PH curve was used in the path estimation process, it could obtain a smaller mission completion time compared to the path estimation using the straight line. That was because the deviation between the estimated path and the actual path was smaller, when the PH curve was used in the path estimation process. When the number of targets increased, the difference between the mission completion time of path estimation using the PH curve and straight-line increased.

Figure 6b shows the average system utility when using different path estimation methods. While the number of targets increasing, the system utility of the two path estimation methods increased. However, when the straight line was used in the path estimation process, the system utility increased slightly when the number of targets increased. It can be seen in Figure 6, the path estimation process using the PH curve had a much better performance than that using the straight line, especially when the number of targets increased.

5.3. Comparison of Three Task Allocation Algorithms

The performance of the proposed task allocation algorithm was demonstrated in comparison with a PSO task allocation algorithm and a modified resource welfare-based [18] task allocation algorithm via Monte Carlo simulation. The system utility and mission computation time using different task allocation algorithms were compared.

For a comparative study, the resource welfare-based task allocation algorithm was modified to adapt to the problem in this paper. For a target to be assigned, the UAVs consumed resources in a balanced way, the coalition with the highest resource welfare was selected to attack the target simultaneously. Algorithm 1 was also used in the modified resource welfare-based task allocation algorithm, the Line 9 and 11 of algorithm 1 were replaced by the resource welfare method.

For the PSO-based task allocation algorithm, assuming there were $N$ UAVs and $M$ targets in the environment. The $l^{th}$ particle at the $k^{th}$ iteration in PSO task allocation algorithm is represented as:

$$X_l^k=(x_{11}^{l,k},\dots ,x_{1M}^{l,k},x_{21}^{l,k},\dots ,x_{2M}^{l,k},\dots ,x_{N1}^{l,k},x_{N2}^{l,k},\dots ,x_{NM}^{l,k})$$

(32)

where $x_{ij}^{l,k}\in \{0,1,\dots ,M\}$ denotes that the UAV $U_i$ will attack target $x_{ij}^{l,k}$ in $j^{th}$ sequence. The dimension of each particle is $N\times M$. Each particle represents a solution to the task allocation problem. The fitness of each particle can be calculated using Equation (6). If the task allocation solution represented by a particle violates the task allocation constraints in Equations (8)–(13), the fitness of this particle is set to $-100$.

To compare the performance of the three task allocation algorithms, a set of simulation with different number of targets were designed. The parameter setting of these simulations is the same with the Monte Carlo simulation shown in Section 5.1. The PH curve was used in the path estimation process for all these task allocation algorithms. All these simulations used the proposed path planning method. The comparison results of the average mission completion time and system utility are shown in Figure 7. It should be noted that, as the dimension of the PSO-based algorithm was too large, we only recorded the feasible solutions obtained within 24 h of the PSO-based task allocation algorithm.

Figure 7a shows the comparison of the average mission completion time of these task allocation (TA) algorithms. Obviously, with the increase in the number of targets, the average mission completion time increased. When the number of targets was 10, the PSO-based task allocation algorithm could obtain the shortest mission completion time. However, the mission completion time using the PSO-based TA algorithm significantly increased with the number of targets. That was because when using the PSO-based TA algorithm, the dimension of particle was too high and increased with the number of targets, it was difficult for the PSO algorithm to obtain the optimal solution in a short time. From Figure 7a, the proposed TA algorithm performed better than the PSO-based and resource welfare-based TA algorithms.

Figure 7b presents the average system utility obtained by these TA algorithms. When the number of targets was 10, the PSO-based TA algorithm could obtain a higher system utility than the other two TA algorithms. However, with the increasing number of targets, the proposed algorithm outperformed the PSO-based TA algorithm as well as the resource welfare-based TA algorithm. That was because the PSO-based TA algorithm could not obtain the optimal solution when the dimension of the particle was high. This also led to a slight decline in the system utility when the number of targets increased further (the number of targets was 30) using the PSO-based TA algorithm.

6. Conclusions

In this paper, a hierarchical mission planning was proposed to decouple and solve the coupled task allocation and path planning problem for a multi-UAV attacking a multi-target. Three phases were involved in the method. In the path estimation process, each UAV utilized the PSO-based method to estimate the path length to targets. Then, a task allocation algorithm based on a negotiation mechanism was proposed to assign the targets. Finally, a path planning method was proposed to generate flyable and safe paths for the UAVs. Through the corresponding simulation, some key points were illustrated.

• Considering the resource requirement of targets, UAV’s resource consumption and task sequence, the proposed task allocation algorithm could obtain feasible solutions for the UAVs.
• The proposed path planning method could generate flyable and safe paths for the UAVs considering several constraints, such as simultaneous arrival, UAV’s kinematic and collision avoidance with obstacles.
• The Monte Carlo results showed that path estimation using the PH curve performed better than that using the straight line. The proposed task allocation algorithm could obtain a lower mission completion time and higher system utility compared with the PSO algorithm and the modified resource welfare-based task allocation algorithm.

Although this paper focused on the problem of cooperative attacking using multiple UAVs, the proposed mission planning method can be applied to some civilian scenarios, such as multi-UAV cooperative fire fighting, where each UAV can carry different fire fighting resources. There are also some limitations in the proposed mission planning approach. The proposed task allocation algorithm assumes that all the UAVs can communicate with each other. Some practical issues that may appear in real environment, such as communication delay or even interruption will be considered in future work.