Coordinated Multi-UAV Reconnaissance Scheme for Multiple Targets

Abstract

This study addresses dynamic task allocation challenges in coordinated surveillance involving multiple unmanned aerial vehicles (UAVs). A significant concern is the increased UAV flight distance resulting from the assignment of new missions, leading to decreased reconnaissance efficiency. To tackle this issue, we introduce a collaborative multi-target and multi-UAV reconnaissance scheme. Initially, the multitasking constrained multi-objective optimization framework (MTCOM) is employed to optimize task allocation and reconnaissance time in static scenarios. Subsequently, in case of emergency, we iteratively refine the outcomes of static task allocation through an enhanced auction-based distributed algorithm, effectively reducing UAV flight costs in response to new missions, UAV withdrawal, or damage. Simulation results demonstrate the efficacy of our proposed multi-UAV and multi-target cooperative reconnaissance scheme in resolving dynamic task allocation issues. Additionally, our approach achieves a 5.4% reduction in UAV flight distance compared to traditional allocation methods. The main contribution of this paper is to consider a dynamic scenario model involving UAV damage and the emergence of new reconnaissance areas. Then we propose an innovative collaborative multi-target and multi-UAV reconnaissance scheme to address this issue and, finally, conduct experimental simulations to verify the effectiveness of the algorithm.

1. Introduction

The utilization of intelligence is experiencing a notable surge, and with the increasing automation of robot design [1], UAVs have emerged as essential tools in the information battlefield [2]. Nonetheless, due to the intricacy and variability of reconnaissance tasks and the ever-increasing number of targets requiring reconnaissance, a single UAV is insufficient to meet these task demands [3]. In light of this, the integration of multiple UAVs, exhibiting mutual cooperation and complementary advantages, has gained significant traction. This paradigm of multi-UAV cooperation facilitates diverse reconnaissance missions, encompassing the reconnaissance of multiple target areas, thereby substantially enhancing the overall reconnaissance efficiency [4]. Within the realm of multi-UAV collaboration, the technology facilitating multi-UAV and multi-target cooperative reconnaissance assumes paramount importance as it leverages the cooperative advantages of employing multiple UAVs together [5]. This technology encompasses two vital components, namely task allocation and path planning. Task allocation strives to effectively orchestrate the entire mission by intelligently assigning each UAV with specific targets and tasks. On the other hand, path planning pertains to determining the optimal trajectory for each UAV, guiding them from their initial positions to the designated target locations. The primary objective of path planning is to discover an optimal route that enables more efficient executions of all assigned tasks [6].

The task allocation problem in the context of multi-UAV and multi-target cooperative reconnaissance represents a complex and NP-hard challenge. Presently, the prevailing models utilized to address this problem encompass the Traveling Salesman Problem (TSP) model [7] and its extended variations [8,9]. These models require the UAVs to cover all designated target points during reconnaissance missions, rendering them well-suited for scenarios characterized by extended flight times. Notably, they offer the advantages of low computational complexity and robust scalability. However, the application of these models require the establishment of pertinent constraints aligned with the specific problem context. In tackling the multi-UAV cooperative reconnaissance task allocation problem, two primary approaches are commonly employed: centralized methods and distributed methods [10].

In the centralized method, each individual UAV transmits its state information to a central processing unit, which then consolidates the global information and formulates an optimal allocation scheme [11]. Noteworthy centralized methods include optimization techniques [12,13], and metaheuristic methods [14,15,16]. These approaches demonstrate favorable attributes in terms of globality and solution quality, as they are capable of producing high-quality solutions. However, the centralized method does suffer from certain limitations, including low timeliness and robustness, primarily stemming from the requirement to integrate global information. As a consequence, it is most suitable for static problems, such as the pre-allocation of reconnaissance tasks. In practical reconnaissance operations, real-time challenges often arise, such as the emergence of new tasks or UAV damage, rendering centralized methods inadequate in meeting these real-time requirements [17].

Addressing the dynamic and real-time demands of practical reconnaissance operations has led to significant research focus on the distributed reconnaissance task allocation method. Unlike the centralized approach, the distributed method substantially reduces reliance on a central processing unit, fostering information sharing, negotiation, and decision making among the UAVs to formulate a reconnaissance task allocation scheme. Prominent distributed methods encompass the contract net protocol algorithm [18,19,20], and the distributed auction algorithm [21,22,23], both of which utilize a market auction mechanism as their core principle. The distributed method boasts advantages in terms of timeliness, flexibility, and robustness owing to its decentralized nature. Nonetheless, certain factors, such as the limited computing power of UAVs and the inherent limitations of distributed algorithms in achieving global optimization, may result in suboptimal allocation schemes [24].

This study presents an innovative cooperative reconnaissance scheme involving multiple UAVs and targets, aimed at addressing the challenge of attaining a solution with smaller flight distance and higher reconnaissance benefits during dynamic reconnaissance operations. Different from other schemes, the proposed method comprises two pivotal components. Firstly, a centralized heuristic algorithm is employed to obtain a static allocation solution because the centralized algorithm can use the global information, and can obtain a better solution than the distributed algorithm. Subsequently, an enhanced distributed algorithm, based on the auction mechanism, is utilized for dynamic adaptation of the allocation scheme in real time, compensating for the shortcomings of centralized algorithms in handling dynamic scenarios.

The pre-allocation of solutions based on global information can be represented as a constrained multi-objective problem (CMOP). A variety of constrained multi-objective algorithms (CMOEAs) have been developed to address CMOPs [25]. An essential aspect of handling CMOPs lies in effectively balancing conflicts between constraints and objectives during the evolutionary process. CMOEAs that progressively adjust the weights of constraints and objectives [26,27] may encounter challenges in escaping large unfeasible regions. To overcome these issues, several CMOEAs [28,29] employing two-stage optimization have been proposed to approach the unconstrained Pareto frontier (UPF) and mitigate the impact of large unfeasible regions. However, these approaches may not perform optimally when dealing with CMOPs characterized by small and discrete feasible regions. Moreover, CMOEAs [30,31] based on dual population optimization have been developed to tackle these problems, but they may lack effectiveness in solving CMOPs with optimal feasible regions situated far from the UPF, as the auxiliary population may not adequately maintain and utilize promising infeasible solutions in the later evolutionary stages. To address these challenges, a novel multitasking constrained multi-objective optimization framework (MTCMO) has been proposed in the literature [32]. This framework generates dynamic auxiliary tasks through knowledge transfer, aiding in the resolution of complex CMOPs (primary task), while allowing promising infeasible solutions to persistently contribute to the later evolutionary stages. The MTCMO framework has demonstrated robust performance in solving CMOPs. Hence, in this study, we adopt this framework to obtain pre-allocation solutions in the first stage.

The primary contribution of this work is to introduce and investigate a dynamic scenario model that considers UAV damage and the emergence of new reconnaissance areas. Additionally, an innovative collaborative multi-target and multi-UAV reconnaissance scheme is proposed, which utilizes heuristic algorithms and distributed computing as two phases of our algorithm to address the problem. Finally, we conduct simulations to validate the effectiveness of the algorithm.

The rest of this paper is as follows. Section 2 introduces the related work on multi-UAV multi-target cooperative reconnaissance task allocation. In Section 3, the research problem is described and modeled. Section 4 describes the proposed multi-UAV multi-target cooperative reconnaissance scheme in detail. In Section 5, the effectiveness of the algorithm is verified by simulation experiments. Finally, conclusions are given in Section 6.

2. Related Work

The application of heuristic algorithms to address the challenges of multi-UAV cooperative reconnaissance has gained widespread prominence. Gao et al. [33] proposed a novel mathematical model for reconnaissance, which classified the reconnaissance targets into point targets, line targets, and area targets, and introduced a new group ant colony optimization algorithm for this model. Ye et al. [34] considered the collaborative task assignment problem of heterogeneous UAVs in air defense missions, and designed a task assignment model according to the different capabilities and maneuverability of UAVs as well as task coupling and precedence constraints. Then, they proposed an improved genetic algorithm using a multi-genotype chromosome coding method to solve this problem. Seeking to expedite convergence in multi-UAV task assignment, Lu et al. [35] proposed a discrete wolf pack algorithm integrating principles from Particle Swarm Optimization (PSO) and Genetic Algorithms (GA). Experimental outcomes showcased its efficacy in high-dimensional spaces. Further contributions within this domain include the work of Lin et al. [36], who combined the improved Artificial Potential Field (APF) and the bat algorithm to advance the optimal success rate strategy. This strategy accelerated the convergence rate of bat position updates while refining the adaptive inertia weight of the bat algorithm. Additionally, they incorporated a chaotic strategy to avert local optimization pitfalls. In pursuit of rapidly obtaining optimal paths, Li et al. [37] integrated the strengths of P-RRT* (potential functions based rapidly-exploring random trees) and Quick-RRT* to proposed a novel algorithm, PQ-RRT*, which guarantees a fast convergence to an optimal solution. The aforementioned studies primarily prioritize the enhancement of algorithm fitness within biomimetic algorithms and the avoidance of local optimization challenges. Furthermore, in the method of task assignment that guarantees the execution of high levels of expectations and specifications, Alexandros et al. [38] presented a scalable procedure for the time-constrained planning of a class of uncertain nonlinear multi-robot systems and designed decentralized and robust control laws, so that each robot meets an individual high-level specification expressed as metric interval temporal logic. It provides an algorithmic framework for practical robot applications. Alexandros et al. [39] proposed a decentralized control protocol that ensures collision avoidance among agents, as well as with obstacles and workspace boundaries, by introducing certain distance constraints. In the field of multi-robot space exploration, Gul et al. [40] have proposed a hybrid framework called the coordinated multi-robot exploration aquila optimizer, which ensures the acquisition of an optimal collision-free path in a barrier-filled environment by generating a finite map. Forestiero et al. [41] examined an approach based on ant systems to replicate and map Grid services information on Grid hosts according to the semantic classification of such services. This algorithm can drive query messages towards a cluster of peers having information about resources belonging to the requested class.

In response to the advancement of neural networks, an emerging trend is the integration of these models into solutions for multi-UAV cooperative reconnaissance task allocation challenges. Wang et al. [42] introduced a path planning approach grounded in the attention mechanism, employing an attention neural network to formulate collaborative reconnaissance strategies for UAVs. This technique surpasses traditional heuristic algorithms in terms of solution efficiency and robustness. Zhao et al. [43] presented a rapid task allocation algorithm founded on Q-learning, effectively shifting online computations to an offline learning process. Expanding upon the Q-learning paradigm, Zhu et al. [44] introduced an enhanced half-random Q-learning algorithm that adeptly circumvents the pitfalls of local optimization. However, this algorithm falls short in addressing task allocation within uncertain contexts. Bayerlein et al. [45] addressed the path planning predicament by translating it into a discrete partially observable Markov decision process, subsequently employing deep reinforcement learning to derive optimal UAV control strategies. Hu et al. [46] synergistically combined the double screening sampling method with the depth deterministic strategy gradient algorithm, culminating in the REL-DDPG algorithm, notable for its heightened convergence rate. The evolution of neural networks and reinforcement learning has notably shaped strategies to overcome these challenges. Nevertheless, these neural network-based methodologies often demand substantial quantities of training data and time, potentially impeding their suitability for dynamic scenarios necessitating rapid decision making.

Distributed algorithms have found extensive application in task scenarios characterized by emergent situations. In the context of optimization problems within dynamic scenarios, Xu et al. [47] proposed a cooperative co-evolutionary algorithm that dynamically groups decision variables to effectively address MOPs with changing decision variables. This approach excels in terms of diversity, convergence, and the spread of solutions on most benchmark optimization problems. Ramirez-Atencia et al. [48] introduced a multi-objective genetic algorithm (MOGAMR) specifically tailored to address real-time task reassignment predicaments. This approach generates a set of feasible solutions within stipulated timeframes by leveraging previous solution information when encountering new tasks. However, this process is time-intensive. Yang et al. [49] devised a distributed method aimed at handling dynamic events arising during the original scheduling process. This method efficiently delivers conflict-free solutions while minimizing data exchange and runtime. In the realm of task assignment, which entails the generation of novel tasks, Buckman et al. [50] extended the consensus-based bundle algorithm (CBBA). By reconfiguring the final task allocation for each agent, this algorithm mitigates the magnitude of reprogramming required. Despite the rapid execution and adaptability of distributed algorithms in dynamic task assignment scenarios, the solutions derived from these approaches frequently fall short of optimality.

The integration of online dynamic allocation capabilities into a multi-UAV system offers substantial benefits, notably bolstering the fault tolerance and overall reliability of the UAV infrastructure. Pioneering this direction, Gao et al. [51] introduced the application of evolutionary algorithms to task assignment and harnessed the contract network protocol method for task reassignment in emergency contexts. Their work thoughtfully combines the advantages of high-quality solutions from centralized algorithms with the rapid responsiveness inherent to distributed algorithms. Building upon this foundation, this paper introduces a novel multi-UAV multi-target cooperative reconnaissance scheme to tackle the challenge of obtaining high-quality solutions in dynamic reconnaissance operations. The proposed approach consists of two key components. Firstly, a heuristic algorithm is utilized, leveraging global information to perform pre-allocation of reconnaissance tasks, thereby obtaining superior pre-allocation solutions. Subsequently, an enhanced distributed algorithm, grounded on the auction mechanism, is employed to dynamically adjust the allocation scheme in response to real-time environmental changes during task execution. This improved distributed algorithm aims to minimize the total flight distance of UAVs while yielding higher-quality solutions as compared to conventional distributed algorithms.

3. Problem Model

The problem to be solved in this paper is defined as follows.

Suppose that $U=\{U_1,U_2,...,U_{N_U}\}$ and $A=\{A_1,A_2,...,A_{N_T}\}$ are the sets of UAVs and reconnaissance task areas, respectively, where $N_U$ and $N_T$ refer to the size of the corresponding set. Given the location information of the elements in A and U, each reconnaissance task area in A must be initially allocated to a UAV in U. Then the reconnaissance sequence, q, for each $U_i\in U$ and the reconnaissance time, t, for each $A_i\in A$ must be planned so as to maximize the reconnaissance benefits $f_1$ and minimize the flight distance $f_2$.

3.1. Problem Description

In a two-dimensional environment, the location information of the task area and the UAV-related information are known in advance. After the task assignment set, reconnaissance sequence and reconnaissance time are planned, the process of UAV reconnaissance to the mission area is shown in Figure 1. To satisfy various constraints such as endurance time, reconnaissance capability, and flight speed, the task area in the multi-UAV cooperative reconnaissance area is shown in Figure 1.

If a sudden situation occurs unexpectedly during the task, as shown in Figure 2, the task area is dynamically redistributed according to the sudden situation to ensure the successful completion of the task. These emergencies may include:

• Some UAVs fail temporarily and cannot continue to perform tasks. It becomes necessary to allocate the remaining unfinished targets to other UAVs;
• Some new task areas to be reconnoitered appear unexpectedly and need to be assigned to UAVs for reconnaissance;
• Some existing task areas are cancelled temporarily, and the original task allocation of UAVs requires adjustments.

Therefore, it is necessary to design a dynamic task area allocation mechanism. This mechanism aims to ensure that each task area is detected while simultaneously maximizing total reconnaissance revenue and minimizing total flight distance. When an emergency occurs, the task should be allocated as reasonably as possible.

3.2. Model Establishment

Based on the analysis of the dynamic task allocation problem in multi-UAV cooperative reconnaissance, this paper makes the following reasonable assumptions to simplify the model:

• Each UAV must achieve a minimum reconnaissance revenue before leaving the task area and moving on to the next target location;
• Each task area can only be reconnoitered once by a single UAV;
• The flight path of a UAV is barrier-free, and there are no collisions between UAVs; that is, when UAVs travel from the current position to the next location, they just fly along a straight line between two points, and the Euclidean distance between two points is the path length of this flight;
• When a new task emerges, its location, area, value coefficient, and other relevant information can be obtained.

In general, the reconnaissance of task areas by UAVs is constrained by their cruising time and the reconnaissance resources they carry. It is often challenging to ensure complete information reconnaissance for each task area. Therefore, the information certainty metric is considered to measure the reconnaissance revenue of a UAV in the task area in a specific time [52]. The formula is listed as follows:

$$I\left(t\right)=I_0+I_1(1-e^{-\beta t}),$$

(1)

where I is the information certainty, $I\in [0,1]$; $I_0$ is the known information of the UAV in the task area before the start of reconnaissance. In this paper, we assume that $I_0=0$; $I_1$ refers the information uncertainty part of the UAV in the task area, which satisfies $I_0+I_1=1$ [52]; $\beta$ denotes the ability index of the reconnaissance load carried by the UAV for reconnaissance of the task area.

In this paper, the performance of the UAV reconnaissance payload is known. The reconnaissance capability of the UAV for each task area can be expressed by the known flight speed and scanning width. Therefore, the reconnaissance capability index ${\beta }_{ij}$ of the UAV${}_i$ in the task area j is defined as follows:

$${\beta }_{ij}=\frac{w_i·v_i}{S_j},$$

(2)

where $S_j$ is the size of the task area; $w_i$ is the effective scanning width of the UAV for the task area; and $v_i$ is the flight speed of the UAV.

It can be seen from assumption 3 that the Euclidean distance between a and b is the path length of the UAV flying from a to b. Assuming that the coordinates of a are $(x_a,y_a)$ and the coordinates of b are $(x_b,y_b)$, the distance between the two places is calculated as follows:

$$d(a,b)=\sqrt{{(x_a-x_b)}^2+{(y_a-y_b)}^2}.$$

(3)

The reconnaissance sequence $q_i$ stores the task areas that need to be reconnoitered by UAV${}_i$ in a specific reconnaissance sequence. For a given reconnaissance sequence $q_i$, the distance of UAV${}_i$ consists of three parts: the path length from the base to the first reconnaissance target, the cumulative path length from the first reconnaissance target to the last reconnaissance target following the reconnaissance sequence, and the path length from the last reconnaissance target back to the base. The formula for calculating the total distance of a single UAV is as follows:

$$D_i=d(b_i,a_{q_{i,1}})+d(a_{q_{i,|q_i|}},b_i)+\sum _{k=1}^{|q_i|-1}d(a_{q_{i,k}},a_{q_{i,k+1}}),$$

(4)

where $a_j$ represents task area j; $b_i$ represents the base of UAV${}_i$; $d\left(\right)$ is the function used to calculate the distance between two places; $q_{i,k}$ represents the k-th reconnaissance task area of UAV${}_i$; and $|q_i|$ is the length of the reconnaissance sequence $q_i$, which denotes the number of task areas to be reconnoitered by UAV${}_i$.

Based on the analysis above, we can establish the mathematical model for the multi-UAV multi-target coordinated reconnaissance problem as follows:

$$max{f}_1=\sum _{j=1}^{N_t}{c}_j·(1-e^{-{\beta }_{(U^j,j)}·t_j}),$$

(5)

$$min{f}_2=\sum _{i=1}^{N_u}{D}_i,$$

(6)

The constraints are

$$G_{min}\le c_j·(1-e^{-{\beta }_{(U^j,j)}·t_j}),j=1,2,\dots ,N_t,$$

(7)

$$\frac{D_i}{v_i}+\sum _{k=1}^{|q_i|}{t}_{q_{(i,k)}}\le T_i,i=1,2,\dots ,N_u.$$

(8)

In Formula (5), $f_1$ is the total revenue of the multi-UAV cooperative reconnaissance. $N_t$ represents the total number of task areas. $c_j$ refers to the reconnaissance value coefficient of task area j, $U^j$ stands for the UAV assigned to the j-th task area, ${\beta }_{(U^j,j)}$ is the reconnaissance ability of the UAV $U^j$, to task area j. $t_j$ means the reconnaissance time assigned to task area j by the UAV $U^j$. In Formula (6), $f_2$ represents the total flight distance of all UAVs, and $N_u$ is the total number of UAVs. Formula (7) indicates that the reconnaissance revenue of each UAV in a task area must be greater than the minimum reconnaissance revenue $G_{min}$ of the corresponding task area. This ensures that the information obtained by the UAV after reconnaissance of the task area is not too little or nothing and prevents the reconnaissance of the task area from losing its significance. Formula (8) is a constraint which means the sum of the time spent on the road and the reconnaissance of each UAV cannot exceed the endurance time. It ensures that the UAV can return to the base smoothly after performing the task. In Formula (8), $v_i$ refers the flight speed of UAV_i, $t_{q_{(i,k)}}$ represents the reconnaissance time allocated by the i-th UAV to the k-th task area, and $T_i$ is the endurance time of UAV${}_i$.

4. Proposed Method

4.1. Algorithm Framework

The proposed approach in this paper can be divided into two phases. In the first phase, the MTCMO algorithm is employed to solve the static problem based on global information (described in Section 4.1). Subsequently, upon encountering dynamic scenarios, the second phase of the approach is initiated. This phase utilizes the Improved Task Planning Algorithm to dynamically adjust the allocation scheme in response to real-time environmental changes during task execution (described in Section 4.2). The flow chart of the algorithm is shown in Figure 3.

4.2. Division of Decision Variables

A solution should include the reconnaissance task set p, reconnaissance sequence set q, as well as reconnaissance time set t. Each $solution=p\cup q\cup t$, and each decision vector is explained as follows:

• The task assignment set of the UAV.
  Different UAVs are assigned to different tasks, and the task set $p_i$ defined as $p_i=\{p_{i1},p_{i2},\dots ,p_{i|p_i|}\}$ stores the tasks assigned to the i-th UAV. Here, $|p_i|$ represents the number of tasks in the task set.
• The reconnaissance sequence of the UAV on the task set.
  Each UAV should execute the tasks in the corresponding task set $p_i$ with the sequence of its own reconnaissance sequence $q_i=\{q_{i1},q_{i2},\dots ,q_{i|p_i|}\}$, where $q_i$ means the reconnaissance sequence of the i-th UAV, and $q_{i|p_i|}$ refers to the $|p_i|$-th task to be scouted by the i-th UAV. The reconnaissance sequence length is the same as the task set length, i.e., $|q_i|=|p_i|$.
• The reconnaissance time of the UAV on the task area.
  The reconnaissance time of a task in the reconnaissance sequence is denoted by $t_i=\{t_{i1},t_{i2}\dots ,t_{i|q_i|}\}$, where i represents the UAV’s serial number. $t_{ij}$ represents the reconnaissance time for the task area of the j-th reconnaissance conducted by the i-th UAV.

4.3. Static Reconnaissance Allocation Scheme

Static reconnaissance refers to a situation where the positions, value coefficients $c_j$, and minimum reconnaissance revenue $G_{min}$ of all the task areas are known, as well as the positions, endurance time $T_i$, and reconnaissance ability $\beta$ of the UAVs. The task of static reconnaissance allocation is to plan the task assignment set p, reconnaissance sequence q, and reconnaissance time t under the constraints defined in Formulas (7) and (8), with the objective of achieving the goals outlined in Formulas (5) and (6).

In this paper, the MTCMO algorithm is used to optimize the static reconnaissance problem. The MTCMO algorithm [32] is a relatively advanced constrained multi-objective evolutionary algorithm, characterized by its use of decreasing constraint boundaries, so that auxiliary tasks and master tasks maintain a high correlation during the evolution process. In addition, an improved $ϵ$ method is designed for auxiliary tasks, which can retain infeasible solutions with excellent objective values to help the main population cross the infeasible region and provide supplementary directions for infeasible solutions with lower constraint values to help the main population enter the feasible region from the infeasible region.

• Calculate the Degree of Constraint Violation.
  In general, multi-objective optimization problems will have various constraints, which are of the form

  $$\left\{\begin{array}{cc}{g}_i\left(x\right)\le 0,\hfill & i=1,\dots ,k\hfill \\ h_i\left(x\right)=0,\hfill & i=k+1,\dots ,n,\hfill \end{array}\right.$$

  (9)

  where $g\left(x\right)$ and $h\left(x\right)$ are inequality and equality constraints, and the k and $n-k$ are the number of inequality and equality constraints, respectively. For convenience, equality constraints are often transformed into inequality constraints by using positive boundary correction parameters:

  $$|h_i\left(x\right)|-\sigma \le 0,i=k+1,\dots ,n.$$

  (10)
  According to Formulas (9) and (10), the constraint violation degree of each constraint for the individual is calculated using the following formula:

  $$c{v}_i\left(x\right)=\left\{\begin{array}{cc}max(0,g_i\left(x\right)),\hfill & 1\le i\le k\hfill \\ max(0,|h_i\left(x\right)|-\sigma ),\hfill & k+1\le i\le n.\hfill \end{array}\right.$$

  (11)
  Finally, the constraint violation degree of an individual is calculated as follows:

  $$CV\left(x\right)=\sum _{i=1}^{n}c{v}_i\left(x\right).$$

  (12)
  If the constraint violation degree of an individual is not greater than 0, it is called a feasible solution; otherwise, it is an infeasible solution. According to the Constraint Dominance Criterion (CDP) [53], feasible solutions are considered better than infeasible solutions, and the comparison between infeasible solutions can be carried out using the constraint violation degree. Individuals with a small constraint violation degree are considered better than those with a large constraint violation degree.
• $ϵ$ Method.
  Set a constraint bound ${ϵ}_T$ that shrinks with the number of iterations and satisfies the following formula:

  $${ϵ}_T={ϵ}_0·{(1-\frac{T}{MaxT})}^{\left(\frac{-log\left({ϵ}_0\right)-\sigma }{log(1-pp)}\right)},$$

  (13)
  $MaxT$ represents the maximum number of iterations, $pp$ is a parameter that controls the rate of decrease of ${ϵ}_T$, and ${ϵ}_0$ denotes the initial constraint violation, which is equal to the maximum CV among the initial primary and auxiliary populations.
• MTCMO Algorithm Flow.
  The main process of the MTCMO algorithm is illustrated in Figure 4. Firstly, two populations with NP individuals, denoted as $P_1$, and $P_2$, are randomly initialized in the search space. Each individual represents a set of decision variables, that is, an allocation scheme. Next, the two parent populations $P_1$ and $P_2$ use the Multi-objective Evolutionary Algorithm (MOEA) operator to generate offspring populations $O_1$ and $O_2$ with NP/2 individuals, respectively. Subsequently, for $P_1$, after mixing $O_1$, $O_2$ and $P_1$, using the algorithm calculating $CV$ according to Formula (12), the top NP individuals are selected as the new $P_1$ based on CDP. As for $P_2$, after mixing $O_1$, $O_2$ and $P_2$, using the algorithm setting ${ϵ}_T$ according to Formula (13), the first NP individuals are selected as the new $P_2$ using the improved $ϵ$ method. The above process represents one generation in the evolution, so the above process is iterated until the preset maximum evolution generation is reached, thereby obtaining a series of optimal decision variables and ultimately attaining the optimal static task pre-allocation scheme.

After obtaining the solution, we apply the 2-optimization (2-opt) [54] operator to locally optimize the reconnaissance sequence q, thereby preventing the intersection of flight paths.

4.4. Improved Dynamic Task Planning Algorithm Based on Market Auction Mechanism

After the pre-assignment of the static the reconnaissance target, the UAV executes the task based on the task assignment. During the execution, some unexpected events may occur, such as the generation of new tasks to be reconnoitered, the cancellation of tasks, or the destruction of the UAV, etc. In such cases, it becomes necessary to dynamically adjust the task allocation to ensure the successful completion of reconnaissance.

The flow of the improved dynamic task planning algorithm is shown in Figure 5. The reallocation of tasks among multiple UAVs should focus on the real-time capability, feasibility, and aim of Formula (6) to obtain a new task assignment set p and reconnaissance sequence q. Then the reconnaissance time t is reallocated with Formula (5) as the target.

The details of the algorithm are outlined in Section 4.4.1, Section 4.4.2 and Section 4.4.3.

4.4.1. Insertion Mechanism for Task Areas

Task area insertion is a common case in dynamic mission planning. When new task areas emerge on the map, they need to be inserted into the existing UAV reconnaissance sequence. When a UAV is damaged during the mission, the unfinished task area should be treated as a new task area and inserted into the reconnaissance sequence of the other UAVs. The task area insertion mechanism proposed in this paper considers the additional travel cost required by UAVs to add new task areas to the original reconnaissance sequence. It dynamically allocates the inserted task areas by means of bidding, and multiple UAVs compete for the newly inserted reconnaissance targets, as shown in Figure 6 where the extra distance (extra cost) needed for UAV${}_i$ travel to the new task area is inserted between $w_1^i$ and $w_2^i$ is $cos{t}_{i,1}=d_3+d_2-d_1$.

Let $q_{new}$ denote the set of newly added task areas, and calculate the priority of the corresponding task targets based on the value coefficient of the task targets and the reconnaissance area. The priority coefficient of the new task area is defined as shown in Formula (14):

$$R_i=\frac{J_i}{{\sum }_{i=1}^{q_{new}}{J}_i}\alpha +(1-\frac{s_i}{{\sum }_{i=1}^{q_{new}}{s}_i})\gamma ,$$

(14)

where $R_i$ is the priority coefficient corresponding to the newly added task in $q_{new}$; targets with higher priority coefficient have higher priority and are added first. $J_i$ denotes the value coefficient of the i-th task target in $q_{new}$, and $s_i$ denotes the area of the i-th task in $q_{new}$. $\alpha$ and $\gamma$ are the weights of the value coefficient and area, respectively. This formula can make the task area with a high numerical coefficient and small area be prioritized by UAV competition.

In order to ensure that the UAV reconnaissance sequence still satisfies the time constraint in Formula (8) after adding a new task area, this paper refers to the degree of constraint violation and proposes a feasibility index. This index is the endurance time (T) of the UAV minus the minimum reconnaissance time of the UAV unexecuted task area in the sequence and the time spent on the journey, then minus the minimum reconnaissance time of the new task. The feasibility $FD\left(i\right)$ model of the i-th UAV is as follows:

$$FD\left(i\right)=T_i-\frac{D_i+cos{t}_i}{v_i}-\sum _{k=1}^{|q_i|-|w^i|}{t}_{q_i,k}-\sum _{k=1}^{|w^i|}min{t}_{q_i,k}-t_n.$$

(15)

where $t_{q_{i,k}}$ represents the reconnaissance time allocated by the UAV to the task area before the new task appears, and $min{t}_{q_{i,k}}$ is the reconnaissance time needed to meet the minimum reconnaissance revenue of the task area, and $t_n$ is the minimum reconnaissance time of the new task area. The UAV is eligible to compete for the task area only if $FD\ge 0$. Otherwise, UAVs with $FD<0$ abandon competition for the new mission areas. If the $FD$ of each UAV is less than 0, the problem is considered unsolvable.

When a new task area emerges or a UAV is damaged, the task execution status of all the UAVs at that time is recorded. The task sequence that needs to be scouted of UAV${}_i$ is denoted as $w^i$, $w_1^i$ is the next task area for reconnaissance by the UAV, and the number of unexecuted task areas is denoted as $nu{m}_i$.

The emergence of new task areas can sometimes involve the appearance of multiple task areas simultaneously. In this paper, the priority coefficient $R_i$ of each new task area in $q_{new}$ is calculated using Formula (14), then each new task area is inserted into the $w^i$ one by one, following the order of $R_i$.

After the insertion order has been determined, the new task areas will be inserted into the unexecuted reconnaissance sequence one by one, and the new task area currently inserted is denoted as $a_{new}$. The detailed steps of the new task area insertion process are presented as follows:

Step 1: For all UAVs, check whether the number of unexecuted areas of each UAV is less than 1 or not. If the number of unexecuted areas is less than 1, $a_{new}$ is directly taken as the next task area of the UAV, and the increased distance of the UAV is recorded as $cos{t}_i$ and then Step 4 is executed. Otherwise, the following steps 2–4 are executed;

Step 2: Insert $a_{new}$ between unexecuted areas $w_1$ and $w_2$. Calculate the additional distance (additional cost) that the UAV${}_i$ needs to travel in this insertion as $cos{t}_{i,1}$, then calculate $cos{t}_{i,2},cos{t}_{i,3}\dots cos{t}_{i,nu{m}_i}$ in turn;

Step 3: Record the minimum cost of UAV${}_i$ competition $a_{new}$ as $cos{t}_i$ and consider this insertion point as the best one for UAV${}_i$, denoted as $po{s}_i$;

Step 4: Calculate the feasibility FD of the UAV${}_i$ using Formula (15), and the UAVs with $FD\ge 0$ join the bidding for $a_{new}$; The remaining UAVs with $FD<0$ will give up competing for the task;

Step 5: For all UAVs participating in the bidding, the UAV with the lowest cost and $FD\ge 0$ is selected as the winner of $a_{new}$, and $a_{new}$ is inserted into the position of $pos$ for the UAV reconnaissance sequence.

Figure 7 shows the insertion process for a new task area.

After all new task areas $q_{new}$ have been inserted, the reconnaissance time reallocation method described in Section 4.4.3 is employed to determine the optimal reconnaissance time t corresponding for the new reconnaissance sequence q.

4.4.2. Task Area Removal Mechanism

When reconnaissance missions in certain task areas are abruptly canceled, they need to be removed from the reconnaissance sequence $w_i$ of the UAV. This article uses a simple and effective removal mechanism to address this issue.

If the task area to be removed has already been scouted, it has no impact on the remaining tasks and does not require any further processing. However, if the task area to be removed has not been scouted, it can be directly removed from the corresponding reconnaissance sequence, and the UAV will skip over the task area and proceed to the next one for reconnaissance. After adjusting the reconnaissance sequence, the UAV reconnaissance time t is redistributed by the method described in Section 4.4.3 to optimize the UAV’s reconnaissance revenue $f_1$. The rest of the unaffected UAVs do not change the task sequence and reconnaissance time.

4.4.3. Reconnaissance Time Redistribution

In the reconnaissance process, to maximize reconnaissance benefits, it is necessary to reallocate the reconnaissance time t for the UAVs that have changed their reconnaissance sequences. The reallocation of reconnaissance time can be regarded as a constrained single-target optimization problem. This paper comprehensively considers the area and task value coefficient of the reconnaissance target, as well as the endurance time constraint, taking Formula (5) as the objective function, under the Formula (7) condition of minimum reconnaissance benefit constraint and Formula (8) endurance time constraint. The GA (Genetic Algorithm) [55] is used to optimize the reconnaissance time in the remaining task areas of UAVs, and the reallocated reconnaissance time is obtained in a short time to obtain the maximum reconnaissance benefit.

5. Simulation Experiment

Taking the actual situation of a multi-UAV reconnaissance of multiple targets task as an example, the static task pre-allocation and the improved dynamic task planning strategy are simulated and verified. These experiments are conducted using the framework of PLATEMO in MATLAB. The hardware configuration used for the algorithm simulation experiments in this chapter is Intel(R) Core(TM)i7-10700 CPU @ 2.90 GHz with 16.0 GB RAM. The software environment is MATLAB 2020a. First, a non-dominated solution from the population obtained by solving the static task pre-assignment problem is selected as the initial task assignment plan. This assignment plan is used as the initial solution for dynamic reassignment simulation experiments, which are processed using the task reassignment mechanism proposed in this paper.

5.1. Static Task Assignment Simulation

To validate the effectiveness of the first stage to the proposed algorithm, which involves the uniformity and rationality of task assignment (ensuring that each UAV is assigned with an appropriate number of tasks, neither excessive nor insufficient, and that the assigned task areas are not far from the location of the UAV), and to assess the reasonableness of the reconnaissance sequence (minimizing the UAV’s flight distance), the following scenarios are set for simulation experiments for the given UAV and task area parameters.

We assume that there are UAVs performing reconnaissance in multiple task areas in a 300 km × 300 km 2D operational environment, and the location of each UAV and task area is fixed. Altogether, three UAVs take off from different bases for reconnaissance tasks, and the initial status information of each UAV is shown in

Table 1. Parameters of UAV performances.

UAV No.	Speed (km/h)	Endurance Time (h)	Endurance Time (h)	Take-Off Coordinate (km)	Landing Coordinate (km)
1	200	10	0.3	(224, 289)	(224, 289)
2	200	10	0.3	(265, 23)	(265, 23)
3	200	10	0.3	(71, 295)	(71, 295)

.

We assume that there are 12 task areas waiting to be reconnoitered, and the reconnaissance values of different task areas are different. The value coefficients of the task areas are denoted by a random variable that follows a uniform distribution between [0,1], and the initial state information of the task areas is shown in

Table 2. Parameters of reconnaissance task area.

No.	Task Area Value Factor	Coordinate (km)	Reconnaissance Areas (km${}^2$)	Minimun Reconnaissance Revenue
1	0.4	(140, 116)	50	0.3
2	0.35	(191, 169)	15	0.2
3	0.45	(298, 197)	32	0.2
4	0.47	(167, 296)	38	0.3
5	0.31	(209, 218)	15	0.2
6	0.4	(245, 243)	18	0.2
7	0.7	(30, 62)	49	0.3
8	0.52	(171, 54)	41	0.3
9	0.57	(201, 120)	49	0.2
10	0.37	(143, 257)	57	0.3
11	0.4	(62, 254)	59	0.2
12	0.39	(104, 177)	30	0.2

.

By using the MTCMO algorithm to solve the initial task assignment problem, with a population size of 200 and a maximum evaluation count of 500,000, the results of the initial task assignment for reconnaissance by 3 UAVs for 12 task areas are presented in

Table 3. UAV initial task assignment results.

UAV No.	New Sequence of Reconnaissance Task Areas	Reconnaissance Time (h)
1	{4, 5, 2, 3, 6}	{2.159, 1.024, 1.196, 2.100, 1.373}
2	{8, 7, 1, 9}	{1.558, 2.157, 1.689, 1.895}
3	{11, 12, 10}	{3.238, 2.058, 3.202}

, and the reconnaissance roadmap for each UAV is shown in Figure 8.

The results from the MTCMO algorithm for solving the initial task assignment are as follows: the UAV’s total reconnaissance gain is 5.048, the total flight distance is 1056.226 km, and the total reconnaissance time is 23.650 h. From Figure 8, we can observe that the task assignment is uniform, and there are no intersecting flight paths. The simulation results show that the algorithm can effectively solve the multi-objective problem of multi-UAV task assignment under the complex constraints of a static task. The time consumed by the task allocation algorithm is 130.57 s.

5.2. Dynamic Task Planning Simulation

The traditional dynamic assignment model gives more priority to the constraints than to the optimization objective, and the evaluation of new reconnaissance sequences temporarily disregards the objective of maximizing the gain. The traditional dynamic allocation mechanism for updating the reconnaissance sequence prioritizes the insertion of new reconnaissance targets after the closest task area, considering the goal of distance minimization to some extent. In contrast, the dynamic allocation mechanism proposed in this paper achieves an improvement by inserting the new reconnaissance target between the two nearest task areas and minimizing the cost of additional distance traveled.

In the simulation experiments, the multi-UAV task assignment results, which are shown in Table 3, are selected as the initial assignment plan. A random moment is generated when dynamic task planning is required, and two contingency cases of decreasing the number of UAVs and increasing the number of task areas are triggered, respectively, followed by dynamic task assignment to satisfy the dynamicity requirement. The processing results of the two assignment models for each case are as follows.

5.2.1. UAV Fails and Cannot Complete Its Mission

Randomly delete one UAV and assign its remaining reconnaissance targets to other UAVs. In this simulation, the randomly deleted UAV is selected as No. 3, and the contingency occurs at the time of 3.6 h, when the No. 3 UAV has completed the reconnaissance of task area 11. The remaining reconnaissance tasks of task areas 12 and 10 need to be reassigned.

The results of the traditional dynamic task planning strategy solution are shown in

Table 4. Task planning results of traditional assignment model with reduced UAV.

UAV No.	New Sequence of Reconnaissance Task Areas	Reconnaissance Time (h)
1	{4, 5, 10, 2, 3, 6}	{2.159, 1.024, 1.585, 1.590, 1.590, 1.590}
2	{8, 7, 1, 12, 9}	{1.558, 2.157, 1.159, 1.159, 1.159}
3	{11}	{3.238}

, and the corresponding path planning is shown in Figure 9.

The results of the dynamic task planning strategy solution proposed in this paper are shown in

Table 5. Task planning results of improved assignment model with reduced UAV.

UAV No.	New Sequence of Reconnaissance Task Areas	Reconnaissance Time (h)
1	{4, 5, 10, 2, 3, 6}	{2.159, 1.024, 1.585, 1.585, 1.585, 1.585}
2	{8, 7, 12, 1, 9}	{1.558, 2.157, 0.817, 0.817, 0.817}
3	{11}	{3.238}

, and the corresponding path planning is shown in Figure 10.

In the case of UAV reduction, the traditional dynamic assignment model solves the dynamic task planning to obtain a UAV’s total reconnaissance gain of 4.588 and a total flight distance of 1479.981 km. The dynamic assignment mechanism proposed in this paper solves the dynamic task planning to obtain a UAV’s total reconnaissance gain of 4.619 and a total flight distance of 1442.816 km. The time consumed by the algorithm is 0.079 s. We can see that the proposed method is more advantageous and can effectively improve the UAV’s reconnaissance gain and reduce the flight distance.

5.2.2. New Task Areas to Be Reconnoitered Appear

Randomly add a certain number of task areas and assign them to the UAVs as reconnaissance tasks. In this simulation, task areas 13 and 14 are added for reassignment, and the parameters of the new task areas are shown in

Table 6. Parameters of newly added task areas.

No.	Task Area Value Factor	Coordinate (km)	Reconnaissance Areas (km${}^2$)	Minimum Reconnaissance Revenue
13	0.4	(90, 130)	28	0.3
14	0.35	(170, 220)	36	0.3

.

The results of the traditional dynamic task planning strategy solution are shown in

Table 7. Planning results of traditional assignment model with added task areas.

UAV No.	New Sequence of Reconnaissance Task Areas	Reconnaissance Time (h)
1	{4, 5, 14, 2, 3, 6}	{2.159, 1.024, 1.288, 1.288, 1.288, 1.288}
2	{8, 7, 1, 13, 9}	{1.558, 2.157, 0.991, 0.991, 0.991}
3	{11, 12, 10}	{3.238, 2.058, 3.202}

, and the corresponding path planning is shown in Figure 11.

The results of the dynamic task planning strategy solution proposed in this paper are shown in

Table 8. Planning results of improved assignment model with added task areas.

UAV No.	New Sequence of Reconnaissance Task Areas	Reconnaissance Time (h)
1	{4, 5, 14, 2, 3, 6}	{2.159, 1.024, 1.312, 1.312, 1.312, 1.312}
2	{8, 7, 1, 13, 9}	{1.558, 2.157, 0.923, 0.923, 0.923}
3	{11, 12, 10}	{3.238, 2.058, 3.202}

, and the corresponding path planning is shown in Figure 12.

In the case of additional task areas, the traditional dynamic assignment model solves the dynamic task planning to obtain a UAV’s total reconnaissance gain of 5.227 and a total flight distance of 1537.092 km. The dynamic assignment mechanism proposed in this paper solves the dynamic task planning to obtain the UAV’s total reconnaissance gain of 5.245 and total flight distance of 1454.920 km, and the time consumed by the algorithm is 0.056 s. The dynamic allocation mechanism proposed in this paper is once again proved to be more advantageous.

All the simulation results indicate that the dynamic task assignment mechanism proposed in our work has an excellent capability to handle two contingency situations, which are the reduction in the number of UAVs and the increase in the number of task areas. Furthermore, the objective of minimizing the flight distance of the UAVs can be satisfied, and the total reconnaissance gain of the UAVs is improved, which is an enhancement of the traditional dynamic task assignment model. In the case of 3 UAVs conducting reconnaissance in 12 mission areas, the time required by the algorithm for dynamic planning in the second stage is within 0.1s, which is much less than the time of 130s for the algorithm in the first stage of pre-allocation. Thus, the proposed algorithm is effective in real time in the second stage.

6. Conclusions

In this work, we have proposed a multi-UAV multi-target cooperative reconnaissance scheme. Firstly, based on global information, it adopts a heuristic algorithm for obtaining solutions with greater reconnaissance profits and shorter flight distances than the distributed algorithm. Subsequently, an improved distributed algorithm based on the auction mechanism is employed to dynamically adjust the allocation scheme based on real-time environmental changes, compensating for the defect of centralized algorithms in processing dynamic scenarios. Simulation results demonstrate that the proposed multi-UAV multi-target collaborative reconnaissance scheme effectively solves the task allocation problem for multiple UAVs and targets, and effectively handles unexpected situations such as the emergence of new tasks and UAV damage. Furthermore, the improved distributed algorithm significantly reduces the flight distance cost of the UAV compared to the traditional distributed algorithm, resulting in greater reconnaissance gains.

The primary contribution of this paper is to introduce and investigate a dynamic scenario model that considers UAV damage and the emergence of new reconnaissance areas. Additionally, an innovative collaborative multi-target and multi-UAV reconnaissance scheme is proposed to address the problem, which effectively solves the task reassignment problem when the UAV is damaged or a new mission area appears. Finally, we conduct simulations to validate the effectiveness of the algorithm.

Our work addresses the allocation of UAV cooperative reconnaissance missions in dynamic situations. However, it does not take into account certain communication challenges that may arise in practical applications, nor does it consider the requirements of heterogeneous target reconnaissance in complex scenarios. Specifically, the reconnaissance of heterogeneous targets necessitates the use of various types of UAVs for target reconnaissance. The inclusion of multiple UAV types and target categories introduces a more complex scheduling optimization problem, producing a challenge to scheduling algorithms. In future work, we plan to incorporate the communication status among UAVs into our planning process and explore cooperative strategies for both heterogeneous targets and UAVs to meet the demands of even more complex reconnaissance scenarios.