1. Introduction
The utilization of intelligence is experiencing a notable surge, and with the increasing automation of robot design [
1], UAVs have emerged as essential tools in the information battlefield [
2]. Nonetheless, due to the intricacy and variability of reconnaissance tasks and the ever-increasing number of targets requiring reconnaissance, a single UAV is insufficient to meet these task demands [
3]. In light of this, the integration of multiple UAVs, exhibiting mutual cooperation and complementary advantages, has gained significant traction. This paradigm of multi-UAV cooperation facilitates diverse reconnaissance missions, encompassing the reconnaissance of multiple target areas, thereby substantially enhancing the overall reconnaissance efficiency [
4]. Within the realm of multi-UAV collaboration, the technology facilitating multi-UAV and multi-target cooperative reconnaissance assumes paramount importance as it leverages the cooperative advantages of employing multiple UAVs together [
5]. This technology encompasses two vital components, namely task allocation and path planning. Task allocation strives to effectively orchestrate the entire mission by intelligently assigning each UAV with specific targets and tasks. On the other hand, path planning pertains to determining the optimal trajectory for each UAV, guiding them from their initial positions to the designated target locations. The primary objective of path planning is to discover an optimal route that enables more efficient executions of all assigned tasks [
6].
The task allocation problem in the context of multi-UAV and multi-target cooperative reconnaissance represents a complex and NP-hard challenge. Presently, the prevailing models utilized to address this problem encompass the Traveling Salesman Problem (TSP) model [
7] and its extended variations [
8,
9]. These models require the UAVs to cover all designated target points during reconnaissance missions, rendering them well-suited for scenarios characterized by extended flight times. Notably, they offer the advantages of low computational complexity and robust scalability. However, the application of these models require the establishment of pertinent constraints aligned with the specific problem context. In tackling the multi-UAV cooperative reconnaissance task allocation problem, two primary approaches are commonly employed: centralized methods and distributed methods [
10].
In the centralized method, each individual UAV transmits its state information to a central processing unit, which then consolidates the global information and formulates an optimal allocation scheme [
11]. Noteworthy centralized methods include optimization techniques [
12,
13], and metaheuristic methods [
14,
15,
16]. These approaches demonstrate favorable attributes in terms of globality and solution quality, as they are capable of producing high-quality solutions. However, the centralized method does suffer from certain limitations, including low timeliness and robustness, primarily stemming from the requirement to integrate global information. As a consequence, it is most suitable for static problems, such as the pre-allocation of reconnaissance tasks. In practical reconnaissance operations, real-time challenges often arise, such as the emergence of new tasks or UAV damage, rendering centralized methods inadequate in meeting these real-time requirements [
17].
Addressing the dynamic and real-time demands of practical reconnaissance operations has led to significant research focus on the distributed reconnaissance task allocation method. Unlike the centralized approach, the distributed method substantially reduces reliance on a central processing unit, fostering information sharing, negotiation, and decision making among the UAVs to formulate a reconnaissance task allocation scheme. Prominent distributed methods encompass the contract net protocol algorithm [
18,
19,
20], and the distributed auction algorithm [
21,
22,
23], both of which utilize a market auction mechanism as their core principle. The distributed method boasts advantages in terms of timeliness, flexibility, and robustness owing to its decentralized nature. Nonetheless, certain factors, such as the limited computing power of UAVs and the inherent limitations of distributed algorithms in achieving global optimization, may result in suboptimal allocation schemes [
24].
This study presents an innovative cooperative reconnaissance scheme involving multiple UAVs and targets, aimed at addressing the challenge of attaining a solution with smaller flight distance and higher reconnaissance benefits during dynamic reconnaissance operations. Different from other schemes, the proposed method comprises two pivotal components. Firstly, a centralized heuristic algorithm is employed to obtain a static allocation solution because the centralized algorithm can use the global information, and can obtain a better solution than the distributed algorithm. Subsequently, an enhanced distributed algorithm, based on the auction mechanism, is utilized for dynamic adaptation of the allocation scheme in real time, compensating for the shortcomings of centralized algorithms in handling dynamic scenarios.
The pre-allocation of solutions based on global information can be represented as a constrained multi-objective problem (CMOP). A variety of constrained multi-objective algorithms (CMOEAs) have been developed to address CMOPs [
25]. An essential aspect of handling CMOPs lies in effectively balancing conflicts between constraints and objectives during the evolutionary process. CMOEAs that progressively adjust the weights of constraints and objectives [
26,
27] may encounter challenges in escaping large unfeasible regions. To overcome these issues, several CMOEAs [
28,
29] employing two-stage optimization have been proposed to approach the unconstrained Pareto frontier (UPF) and mitigate the impact of large unfeasible regions. However, these approaches may not perform optimally when dealing with CMOPs characterized by small and discrete feasible regions. Moreover, CMOEAs [
30,
31] based on dual population optimization have been developed to tackle these problems, but they may lack effectiveness in solving CMOPs with optimal feasible regions situated far from the UPF, as the auxiliary population may not adequately maintain and utilize promising infeasible solutions in the later evolutionary stages. To address these challenges, a novel multitasking constrained multi-objective optimization framework (MTCMO) has been proposed in the literature [
32]. This framework generates dynamic auxiliary tasks through knowledge transfer, aiding in the resolution of complex CMOPs (primary task), while allowing promising infeasible solutions to persistently contribute to the later evolutionary stages. The MTCMO framework has demonstrated robust performance in solving CMOPs. Hence, in this study, we adopt this framework to obtain pre-allocation solutions in the first stage.
The primary contribution of this work is to introduce and investigate a dynamic scenario model that considers UAV damage and the emergence of new reconnaissance areas. Additionally, an innovative collaborative multi-target and multi-UAV reconnaissance scheme is proposed, which utilizes heuristic algorithms and distributed computing as two phases of our algorithm to address the problem. Finally, we conduct simulations to validate the effectiveness of the algorithm.
The rest of this paper is as follows.
Section 2 introduces the related work on multi-UAV multi-target cooperative reconnaissance task allocation. In
Section 3, the research problem is described and modeled.
Section 4 describes the proposed multi-UAV multi-target cooperative reconnaissance scheme in detail. In
Section 5, the effectiveness of the algorithm is verified by simulation experiments. Finally, conclusions are given in
Section 6.
2. Related Work
The application of heuristic algorithms to address the challenges of multi-UAV cooperative reconnaissance has gained widespread prominence. Gao et al. [
33] proposed a novel mathematical model for reconnaissance, which classified the reconnaissance targets into point targets, line targets, and area targets, and introduced a new group ant colony optimization algorithm for this model. Ye et al. [
34] considered the collaborative task assignment problem of heterogeneous UAVs in air defense missions, and designed a task assignment model according to the different capabilities and maneuverability of UAVs as well as task coupling and precedence constraints. Then, they proposed an improved genetic algorithm using a multi-genotype chromosome coding method to solve this problem. Seeking to expedite convergence in multi-UAV task assignment, Lu et al. [
35] proposed a discrete wolf pack algorithm integrating principles from Particle Swarm Optimization (PSO) and Genetic Algorithms (GA). Experimental outcomes showcased its efficacy in high-dimensional spaces. Further contributions within this domain include the work of Lin et al. [
36], who combined the improved Artificial Potential Field (APF) and the bat algorithm to advance the optimal success rate strategy. This strategy accelerated the convergence rate of bat position updates while refining the adaptive inertia weight of the bat algorithm. Additionally, they incorporated a chaotic strategy to avert local optimization pitfalls. In pursuit of rapidly obtaining optimal paths, Li et al. [
37] integrated the strengths of P-RRT* (potential functions based rapidly-exploring random trees) and Quick-RRT* to proposed a novel algorithm, PQ-RRT*, which guarantees a fast convergence to an optimal solution. The aforementioned studies primarily prioritize the enhancement of algorithm fitness within biomimetic algorithms and the avoidance of local optimization challenges. Furthermore, in the method of task assignment that guarantees the execution of high levels of expectations and specifications, Alexandros et al. [
38] presented a scalable procedure for the time-constrained planning of a class of uncertain nonlinear multi-robot systems and designed decentralized and robust control laws, so that each robot meets an individual high-level specification expressed as metric interval temporal logic. It provides an algorithmic framework for practical robot applications. Alexandros et al. [
39] proposed a decentralized control protocol that ensures collision avoidance among agents, as well as with obstacles and workspace boundaries, by introducing certain distance constraints. In the field of multi-robot space exploration, Gul et al. [
40] have proposed a hybrid framework called the coordinated multi-robot exploration aquila optimizer, which ensures the acquisition of an optimal collision-free path in a barrier-filled environment by generating a finite map. Forestiero et al. [
41] examined an approach based on ant systems to replicate and map Grid services information on Grid hosts according to the semantic classification of such services. This algorithm can drive query messages towards a cluster of peers having information about resources belonging to the requested class.
In response to the advancement of neural networks, an emerging trend is the integration of these models into solutions for multi-UAV cooperative reconnaissance task allocation challenges. Wang et al. [
42] introduced a path planning approach grounded in the attention mechanism, employing an attention neural network to formulate collaborative reconnaissance strategies for UAVs. This technique surpasses traditional heuristic algorithms in terms of solution efficiency and robustness. Zhao et al. [
43] presented a rapid task allocation algorithm founded on Q-learning, effectively shifting online computations to an offline learning process. Expanding upon the Q-learning paradigm, Zhu et al. [
44] introduced an enhanced half-random Q-learning algorithm that adeptly circumvents the pitfalls of local optimization. However, this algorithm falls short in addressing task allocation within uncertain contexts. Bayerlein et al. [
45] addressed the path planning predicament by translating it into a discrete partially observable Markov decision process, subsequently employing deep reinforcement learning to derive optimal UAV control strategies. Hu et al. [
46] synergistically combined the double screening sampling method with the depth deterministic strategy gradient algorithm, culminating in the REL-DDPG algorithm, notable for its heightened convergence rate. The evolution of neural networks and reinforcement learning has notably shaped strategies to overcome these challenges. Nevertheless, these neural network-based methodologies often demand substantial quantities of training data and time, potentially impeding their suitability for dynamic scenarios necessitating rapid decision making.
Distributed algorithms have found extensive application in task scenarios characterized by emergent situations. In the context of optimization problems within dynamic scenarios, Xu et al. [
47] proposed a cooperative co-evolutionary algorithm that dynamically groups decision variables to effectively address MOPs with changing decision variables. This approach excels in terms of diversity, convergence, and the spread of solutions on most benchmark optimization problems. Ramirez-Atencia et al. [
48] introduced a multi-objective genetic algorithm (MOGAMR) specifically tailored to address real-time task reassignment predicaments. This approach generates a set of feasible solutions within stipulated timeframes by leveraging previous solution information when encountering new tasks. However, this process is time-intensive. Yang et al. [
49] devised a distributed method aimed at handling dynamic events arising during the original scheduling process. This method efficiently delivers conflict-free solutions while minimizing data exchange and runtime. In the realm of task assignment, which entails the generation of novel tasks, Buckman et al. [
50] extended the consensus-based bundle algorithm (CBBA). By reconfiguring the final task allocation for each agent, this algorithm mitigates the magnitude of reprogramming required. Despite the rapid execution and adaptability of distributed algorithms in dynamic task assignment scenarios, the solutions derived from these approaches frequently fall short of optimality.
The integration of online dynamic allocation capabilities into a multi-UAV system offers substantial benefits, notably bolstering the fault tolerance and overall reliability of the UAV infrastructure. Pioneering this direction, Gao et al. [
51] introduced the application of evolutionary algorithms to task assignment and harnessed the contract network protocol method for task reassignment in emergency contexts. Their work thoughtfully combines the advantages of high-quality solutions from centralized algorithms with the rapid responsiveness inherent to distributed algorithms. Building upon this foundation, this paper introduces a novel multi-UAV multi-target cooperative reconnaissance scheme to tackle the challenge of obtaining high-quality solutions in dynamic reconnaissance operations. The proposed approach consists of two key components. Firstly, a heuristic algorithm is utilized, leveraging global information to perform pre-allocation of reconnaissance tasks, thereby obtaining superior pre-allocation solutions. Subsequently, an enhanced distributed algorithm, grounded on the auction mechanism, is employed to dynamically adjust the allocation scheme in response to real-time environmental changes during task execution. This improved distributed algorithm aims to minimize the total flight distance of UAVs while yielding higher-quality solutions as compared to conventional distributed algorithms.
3. Problem Model
The problem to be solved in this paper is defined as follows.
Suppose that and are the sets of UAVs and reconnaissance task areas, respectively, where and refer to the size of the corresponding set. Given the location information of the elements in A and U, each reconnaissance task area in A must be initially allocated to a UAV in U. Then the reconnaissance sequence, q, for each and the reconnaissance time, t, for each must be planned so as to maximize the reconnaissance benefits and minimize the flight distance .
3.1. Problem Description
In a two-dimensional environment, the location information of the task area and the UAV-related information are known in advance. After the task assignment set, reconnaissance sequence and reconnaissance time are planned, the process of UAV reconnaissance to the mission area is shown in
Figure 1. To satisfy various constraints such as endurance time, reconnaissance capability, and flight speed, the task area in the multi-UAV cooperative reconnaissance area is shown in
Figure 1.
If a sudden situation occurs unexpectedly during the task, as shown in
Figure 2, the task area is dynamically redistributed according to the sudden situation to ensure the successful completion of the task. These emergencies may include:
Some UAVs fail temporarily and cannot continue to perform tasks. It becomes necessary to allocate the remaining unfinished targets to other UAVs;
Some new task areas to be reconnoitered appear unexpectedly and need to be assigned to UAVs for reconnaissance;
Some existing task areas are cancelled temporarily, and the original task allocation of UAVs requires adjustments.
Therefore, it is necessary to design a dynamic task area allocation mechanism. This mechanism aims to ensure that each task area is detected while simultaneously maximizing total reconnaissance revenue and minimizing total flight distance. When an emergency occurs, the task should be allocated as reasonably as possible.
3.2. Model Establishment
Based on the analysis of the dynamic task allocation problem in multi-UAV cooperative reconnaissance, this paper makes the following reasonable assumptions to simplify the model:
Each UAV must achieve a minimum reconnaissance revenue before leaving the task area and moving on to the next target location;
Each task area can only be reconnoitered once by a single UAV;
The flight path of a UAV is barrier-free, and there are no collisions between UAVs; that is, when UAVs travel from the current position to the next location, they just fly along a straight line between two points, and the Euclidean distance between two points is the path length of this flight;
When a new task emerges, its location, area, value coefficient, and other relevant information can be obtained.
In general, the reconnaissance of task areas by UAVs is constrained by their cruising time and the reconnaissance resources they carry. It is often challenging to ensure complete information reconnaissance for each task area. Therefore, the information certainty metric is considered to measure the reconnaissance revenue of a UAV in the task area in a specific time [
52]. The formula is listed as follows:
where
I is the information certainty,
;
is the known information of the UAV in the task area before the start of reconnaissance. In this paper, we assume that
;
refers the information uncertainty part of the UAV in the task area, which satisfies
[
52];
denotes the ability index of the reconnaissance load carried by the UAV for reconnaissance of the task area.
In this paper, the performance of the UAV reconnaissance payload is known. The reconnaissance capability of the UAV for each task area can be expressed by the known flight speed and scanning width. Therefore, the reconnaissance capability index
of the UAV
in the task area
j is defined as follows:
where
is the size of the task area;
is the effective scanning width of the UAV for the task area; and
is the flight speed of the UAV.
It can be seen from assumption 3 that the Euclidean distance between
a and
b is the path length of the UAV flying from a to b. Assuming that the coordinates of a are
and the coordinates of b are
, the distance between the two places is calculated as follows:
The reconnaissance sequence
stores the task areas that need to be reconnoitered by UAV
in a specific reconnaissance sequence. For a given reconnaissance sequence
, the distance of UAV
consists of three parts: the path length from the base to the first reconnaissance target, the cumulative path length from the first reconnaissance target to the last reconnaissance target following the reconnaissance sequence, and the path length from the last reconnaissance target back to the base. The formula for calculating the total distance of a single UAV is as follows:
where
represents task area
j;
represents the base of UAV
;
is the function used to calculate the distance between two places;
represents the
k-th reconnaissance task area of UAV
; and
is the length of the reconnaissance sequence
, which denotes the number of task areas to be reconnoitered by UAV
.
Based on the analysis above, we can establish the mathematical model for the multi-UAV multi-target coordinated reconnaissance problem as follows:
In Formula (
5),
is the total revenue of the multi-UAV cooperative reconnaissance.
represents the total number of task areas.
refers to the reconnaissance value coefficient of task area
j,
stands for the UAV assigned to the
j-th task area,
is the reconnaissance ability of the UAV
, to task area
j.
means the reconnaissance time assigned to task area
j by the UAV
. In Formula (
6),
represents the total flight distance of all UAVs, and
is the total number of UAVs. Formula (
7) indicates that the reconnaissance revenue of each UAV in a task area must be greater than the minimum reconnaissance revenue
of the corresponding task area. This ensures that the information obtained by the UAV after reconnaissance of the task area is not too little or nothing and prevents the reconnaissance of the task area from losing its significance. Formula (
8) is a constraint which means the sum of the time spent on the road and the reconnaissance of each UAV cannot exceed the endurance time. It ensures that the UAV can return to the base smoothly after performing the task. In Formula (
8),
refers the flight speed of UAV
i,
represents the reconnaissance time allocated by the
i-th UAV to the
k-th task area, and
is the endurance time of UAV
.
6. Conclusions
In this work, we have proposed a multi-UAV multi-target cooperative reconnaissance scheme. Firstly, based on global information, it adopts a heuristic algorithm for obtaining solutions with greater reconnaissance profits and shorter flight distances than the distributed algorithm. Subsequently, an improved distributed algorithm based on the auction mechanism is employed to dynamically adjust the allocation scheme based on real-time environmental changes, compensating for the defect of centralized algorithms in processing dynamic scenarios. Simulation results demonstrate that the proposed multi-UAV multi-target collaborative reconnaissance scheme effectively solves the task allocation problem for multiple UAVs and targets, and effectively handles unexpected situations such as the emergence of new tasks and UAV damage. Furthermore, the improved distributed algorithm significantly reduces the flight distance cost of the UAV compared to the traditional distributed algorithm, resulting in greater reconnaissance gains.
The primary contribution of this paper is to introduce and investigate a dynamic scenario model that considers UAV damage and the emergence of new reconnaissance areas. Additionally, an innovative collaborative multi-target and multi-UAV reconnaissance scheme is proposed to address the problem, which effectively solves the task reassignment problem when the UAV is damaged or a new mission area appears. Finally, we conduct simulations to validate the effectiveness of the algorithm.
Our work addresses the allocation of UAV cooperative reconnaissance missions in dynamic situations. However, it does not take into account certain communication challenges that may arise in practical applications, nor does it consider the requirements of heterogeneous target reconnaissance in complex scenarios. Specifically, the reconnaissance of heterogeneous targets necessitates the use of various types of UAVs for target reconnaissance. The inclusion of multiple UAV types and target categories introduces a more complex scheduling optimization problem, producing a challenge to scheduling algorithms. In future work, we plan to incorporate the communication status among UAVs into our planning process and explore cooperative strategies for both heterogeneous targets and UAVs to meet the demands of even more complex reconnaissance scenarios.