Decentralized UAV Swarm Control: A Multi-Layered Architecture for Integrated Flight Mode Management and Dynamic Target Interception

Abstract

Uncrewed Aerial Vehicles (UAVs) are increasingly deployed across various domains due to their versatility in navigating three-dimensional spaces. The utilization of UAV swarms further enhances the efficiency of mission execution through collaborative operation and shared intelligence. This paper introduces a novel decentralized swarm control strategy for multi-UAV systems engaged in intercepting multiple dynamic targets. The proposed control framework leverages the advantages of both learning-based intelligent algorithms and rule-based control methods, facilitating complex task control in unknown environments while enabling adaptive and resilient coordination among UAV swarms. Moreover, dual flight modes are introduced to enhance mission robustness and fault tolerance, allowing UAVs to autonomously return to base in case of emergencies or upon task completion. Comprehensive simulation scenarios are designed to validate the effectiveness and scalability of the proposed control system under various conditions. Additionally, a feasibility analysis is conducted to guarantee real-world UAV implementation. The results demonstrate significant improvements in tracking performance, scheduling efficiency, and overall success rates compared to traditional methods. This research contributes to the advancement of autonomous UAV swarm coordination and specific applications in complex environments.

1. Introduction

Uncrewed Aerial Vehicles (UAVs), renowned for their flexibility, swift deployment, and advanced surveying capabilities, significantly augment the Internet of Things (IoT) ecosystem [1,2]. Various examples can be found in agricultural monitoring [3], unidentified aircraft engagement [4,5], and post-disaster search and rescue operations [6,7]. As aerial mobile nodes, UAVs bridge the gap between physical devices and digital systems by facilitating efficient data collection, transmission, and processing [8,9]. However, given a single UAV’s limited coverage for large-scale and complex missions, the coordination of multiple UAVs–or a UAV swarm–can significantly amplify their collective strengths [10]. Compared to single-unit setups, swarms provide greater coverage, sensing, computing, and processing power while also improving overall efficiency through information sharing [11] and dynamic task allocation [12]. Beyond that, it enhances the system’s robustness and fault tolerance, ensuring mission continuity and completion even when a few individual units fail [13]. A swarm architecture not only simplifies the complexity of tasks assigned to each UAV, thereby boosting energy efficiency, but also lowers the environmental adaptation requirements [14], enabling new possibilities for a broader spectrum of UAV applications.

With population growth and technological progression, the scale and complexity of UAV-involved tasks are continuously increasing. Breakthroughs in swarm control technology are necessary. Beni and Wang [15] developed a behavior-based UAV swarm control architecture that defines a set of basic actions (e.g., obstacle avoidance, aggregation, dispersion) to achieve coordination under simple conditions. Although straightforward and intuitive, this approach lacks flexibility and adaptability. To overcome computational bottlenecks and communication overhead of centralized control methods, Kada et al. [16] introduced a distributed control framework utilizing consensus algorithms that enable each UAV to update control inputs independently based on local state and neighbor data while achieving global coordination and simultaneously enhancing robustness and scalability. Similarly, drawing inspiration from natural foraging behaviors, the authors of [17] proposed an ant colony optimization (ACO)-based strategy for UAV swarms in which the UAVs release and detect pheromones to coordinate their search in unknown environments, facilitating efficient and distributed cooperative exploration.

Despite progress in traditional (or code-based) methods, significant challenges are still encountered in dynamic environments with uncertainties. Predefined control rules struggle to cover all possible scenarios and cannot adapt to changes. Moreover, these methods often require extensive manual tuning of parameters, complicating the achievement of optimal swarm performance [18,19]. To further enhance the autonomy and intelligence of UAV swarms, it is crucial to integrate advanced artificial intelligence (AI) techniques, such as machine learning [20] and reinforcement learning, enabling UAV swarms to learn and optimize from experience [21].

In [22], Weng et al. borrowed concepts from artificial immune systems to design an adaptive UAV swarm control strategy mimicking the recognition, learning, and memory functions of immune cells to dynamically respond to environmental changes and quickly generate countermeasures. Among supervised learning approaches, Tang et al. [23] employed convolutional neural networks (CNNs) [24] to enable UAV swarm navigation and obstacle avoidance in complex environments. By collecting a vast dataset of aerial images and corresponding control commands, the CNN model learns a mapping function from visual inputs to navigation decisions, eliminating the need for handcrafted features and control rules.

On the other hand, researchers have explored multi-agent reinforcement learning (MARL) for swarm control. Chen et al. [25] developed a centralized-critic MARL framework for surveillance and monitoring, which employs centralized value function estimation alongside decentralized policy optimization, enabling efficient coverage of large areas with minimal overlap by a swarm of UAVs. Building upon this, Qie et al. [26] introduced an actor–critic-based multi-agent deep deterministic policy gradient (MADDPG) algorithm in a fully cooperative scenario. This algorithm enhances the stability and convergence of MARL, making it capable of handling nonstationary environments and learning robust swarm coordination policies.

However, MARL systems face inherent challenges. As the complexity of the environment grows, standard policy gradient methods suffer from high variance estimates, leading to suboptimal or unstable learning [27]. Furthermore, policy changes in one agent can alter the environment’s dynamics perceived by others, further destabilizing the learning process. Additionally, the curse of dimensionality [28] presents significant obstacles in MARL; as the number of agents increases, the state–action space grows exponentially, demanding substantial computational resources to learn effective policies.

To address these challenges, researchers have developed various approaches. Lowe et al. [29] introduced a "centralized training and decentralized execution" paradigm to tackle the nonstationarity issue in MARL. During the centralized training phase, all agents’ observation and action information is collected in a central critic network that learns to estimate the value function of each agent’s decisions. In the decentralized execution phase, each agent uses only its own observational data, which maintains autonomy while leveraging additional information gathered during training to stabilize the learning process. Besides, Li et al. [30] enhanced the MADDPG algorithm with transfer learning techniques to improve generalization capabilities across different tasks. Specifically, they first trained the MADDPG agents on a set of source tasks, covering various environmental settings and objectives. Then, meta-learning techniques were used to embed this acquired knowledge into a meta-policy network, allowing quick adaptation to new tasks with minimal tuning, significantly speeding up the learning process and boosting adaptability in new environments.

Based on the above discussion, although significant advancements in UAV swarm control methods have been made, limitations persist when facing increasingly complex mission requirements and unpredictable environmental conditions. To clarify, traditional code-based methods lack flexibility and adaptability, struggling to cope with dynamic and uncertain environments [31]. Supervised learning methods, although possessing learning capabilities, have limited effectiveness in handling dynamic constraints and with insufficient pre-provided training data [32,33]. While reinforcement learning methods demonstrate powerful real-time decision-making intelligence through interaction with the environment, they still encounter issues in multi-objective and large-scale swarm cases, including safety concerns, energy consumption, and practical deployment [34]. This paper develops a multilayered UAV swarm control approach that addresses these problems for multiple dynamic intruder aircraft interception. It utilizes a modular control system design that combines reinforcement learning capabilities to handle uncertainties with the determinacy brought by code-based algorithms. From low-level operational controls to high-level strategic decisions, the system enables the generation of real-time states for each ownship, supports various flight modes and their autonomous transitions, and implements a coordination mechanism for information sharing within multi-ownship systems.

The remainder of this paper is organized as follows: Section 2 formulates the problem, reviews related previous works, and highlights the main contributions. Section 3 elaborates the proposed distributed swarm control framework and associated algorithms involved in the system. Section 4 presents numerical and simulation results showcasing the effectiveness and robustness of the proposed method. An analysis of the feasibility of applying the control system to real-world UAV swarms is provided as well. Finally, Section 5 summarizes this work and outlines future research directions.

2. Related Work and Problem Definition

In order to develop an efficient swarm control strategy suitable for complex tasks, we took cues from our earlier work. In [35], we proposed a hybrid control framework that combines the Soft Actor–Critic method with a Fuzzy Inference System (SAC-FIS) for single UAV dynamic multi-target interception. This framework boosts training efficiency and ensures robust generalization by integrating universal expert experience through the FIS, effectively tackling prevalent reinforcement learning challenges in dynamic and multi-target environments. The framework’s modular design enhances its scalability to a variety of problems, allowing for straightforward modifications or upgrades to specific modules. Additionally, we developed a UAV autonomous navigation and collision avoidance system using two deep reinforcement learning (DRL) methods in [36,37], which was evaluated in multi-agent scenarios including two independent ownships and three cooperative UAVs (one leader and two followers). In [38], we introduced an integrated UAV emergency landing system designed for GPS-denied environments, ensuring safe mission aborts and landings under emergency conditions.

This paper extends the SAC-FIS controller to multi-UAV systems. The ownships employ a decentralized target selection algorithm to identify unauthorized UAVs as targets. Upon successful interception of a current target, an ownship may either proceed to another intruder or transition to the safe returning mode, landing in a designated area. The environment situations between each ownship and its target are entirely unknown and unpredictable. For any given ownship, other ownships and unselected intruder aircraft are treated as mobile obstacles. This setup demands dynamic real-time response to evolving threats, ensuring mission safety and success.

In summary, this study synthesizes previous works and introduces significant innovations with the aim of developing a hybrid approach that combines code-based and neural network-based methods. This integration seeks to enhance scheduling efficiency within the swarm and improve the resilience of various objectives. By dynamically and flexibly distributing tasks within the swarm, the proposed approach not only simplifies the assignments for each UAV but also ensures that the mission can proceed without disruption even if one or more ownships fail. However, managing the inherent complexity of this swarm problem involves addressing multiple facets: ensuring safety by preventing unintended collisions, enhancing mission efficiency by guaranteeing that ownships complete all interception tasks asynchronously, and managing contingencies to ensure the overall mission’s success despite potential individual failures.

Contributions

Table 1. Comparative overview of key differences between previous work and the current study.

Features	Previous Work	The Current Study
Control Framework	1. SAC-FIS controller for single-ownship systems [35]. 2. A single outer loop controller for one specific operation, such as target tracking or collision avoidance [35,36,37]. 3. None.	1. Extended the well-trained SAC-FIS agents for multi-ownship systems 2. Integrated multiple learning-based and rule-based control algorithms. 3. Established a practical cooperative protocol.
Algorithm Implementation	1. Target selection algorithm for single-ownship systems [35]. 2. Pre-planned waypoint navigation and landing algorithm [38]. 3. None.	1. Extended to a decentralized multi-ownship coordination and dynamic target selection algorithm. 2. Development of a seamless flight mode transitioning strategy with randomly generated safe landing sites. 3. Optimized control signals for safe physical drone operations.

displays the improvements and innovations made in this paper based on our previous work. The main contributions of this paper are also outlined below.

• A new hybrid layered control architecture for intercepting swarms: A three-layer control system is developed to address advanced swarm control problems. This architecture establishes a highly flexible and scalable framework for swarm activities across different scales, ensuring robust control in dynamic and unpredictable environments.
• A new decentralized target allocation algorithm: A dynamic target selection algorithm is designed to enable distributed target allocation for each ownship with capabilities for prompt responses in emergency situations.
• An integrated method for enhancing safe flight operations: A strategy for random waypoint generation is created for adaptable return mode processes, and a control signal optimizer is employed to enhance the security of physical UAV applications.

3. Methodology

As depicted in Figure 1, the use cases involve a swarm of n ownships (blue quadrotors) and m intruder aircraft (in other colors). A rectangular drone hub acts as both a home base and a safe landing site. The ownships are initially stationed at the hub and will intercept all intruder aircraft if they enter a specified airspace. Successfully intercepted intruders will hover at their capture points. After successfully intercepting all intruders or in emergencies, ownships automatically revert to safe returning mode, landing and disarming at the drone hub. Additionally, each ownship is equipped with a 3D LiDAR sensor, which has azimuth (${\theta }_{az}$) and elevation (${\theta }_{el}$) limits defined by ${\theta }_{az}\in [-{179}^{\circ },{179}^{\circ }]$ and ${\theta }_{el}\in [-{15}^{\circ },{15}^{\circ }]$, respectively, and a maximum range of 20 m.

Figure 2 details the architecture of the proposed hybrid swarm control system. This system integrates SAC-FIS and PID controllers to manage distinct flight modes, processing environment feedback at each time step; concurrently, intruder UAVs follow particular and predefined trajectories, providing continuous position and velocity data.

To address the challenges posed by the defined swarm control problem, the framework includes several strategies and algorithms. These encompass an inner loop controller that converts underactuated inputs (roll, pitch, yaw, thrust) into UAV states, outer loop controllers that generate desired real-time model inputs, and a decentralized swarm control protocol. This protocol features a host of capabilities: a dynamic target selection algorithm, methods for assessing target capture success, emergency response strategies, flight mode transition criteria, and test termination conditions. The rest of this section provides a comprehensive breakdown of each component within this system.

3.1. Inner Loop Control

In this study, we employed the North–East–Downward (NED) coordinate system for both the earth coordinate system ($S_e-O_e{x}_e{y}_e{z}_e$), which is also regarded as the inertial reference frame, and the body coordinate system ($S_b-O_b{x}_b{y}_b{z}_b$) of the UAV.

We derived the equations of motion for the ‘+’ type quadrotor using the Newton–Euler formulation. Although the fourth-order Runge–Kutta method (RK4) [40,41] is more accurate, it requires more complex computations and is less intuitive. It is important to note that the Newton–Euler equations of motion are usually derived in the inertial reference frame, whereas describing the motion of the object is more convenient in the body coordinate system. For example, the UAV’s position ${\mathbf{X}}_e={[x_e,y_e,z_e]}^T$ and attitude ${\mathbf{\Theta }}_e={[\varphi ,\theta ,\psi ]}^T$ are variables in the earth coordinate system, whereas the translational velocity ${\mathbf{V}}_b={[u,v,w]}^T$ and rotational velocity ${\mathit{\omega }}_b={[p,q,r]}^T$ are defined in the body frame. In addition, the data collected by onboard sensors is also relative to the body-fixed frame. Therefore, a transform matrix from the body-fixed frame to the inertial reference frame is necessary.

The rotation matrix, defined as ${\mathbf{R}}_b^e$, follows the ZYX rotation sequence (yaw $\psi$–pitch $\theta$–roll $\varphi$), and is provided by

$$\begin{array}{cc}\hfill {\mathbf{R}}_b^e& ={\mathbf{R}}_{\psi }{\mathbf{R}}_{\theta }{\mathbf{R}}_{\varphi }\hfill \\ \hfill & =\left[\begin{array}{ccc}cos\psi & -sin\psi & 0\\ sin\psi & cos\psi & 0\\ 0& 0& 1\end{array}\right]\left[\begin{array}{ccc}cos\theta & 0& sin\theta \\ 0& 1& 0\\ -sin\theta & 0& cos\theta \end{array}\right]\left[\begin{array}{ccc}1& 0& 0\\ 0& cos\varphi & -sin\varphi \\ 0& sin\varphi & cos\varphi \end{array}\right]\hfill \\ \hfill & =\left[\begin{array}{ccc}cos\theta cos\psi & sin\varphi sin\theta cos\psi -cos\varphi sin\psi & cos\varphi sin\theta cos\psi +sin\varphi sin\psi \\ cos\theta sin\psi & sin\varphi sin\theta sin\psi +cos\varphi cos\psi & cos\varphi sin\theta sin\psi -sin\varphi cos\psi \\ -sin\theta & sin\varphi cos\theta & cos\varphi cos\theta \end{array}\right].\hfill \end{array}$$

(1)

The angular velocity transformation matrix ${\mathbf{\Omega }}_b^e$ can be derived according to Equation (2) as follows:

$$\left[\begin{array}{c}p\\ q\\ r\end{array}\right]=\left[\begin{array}{c}\dot{\varphi }\\ 0\\ 0\end{array}\right]+{\mathbf{R}}_{\varphi }\left[\begin{array}{c}0\\ \dot{\theta }\\ 0\end{array}\right]+{\mathbf{R}}_{\varphi }{\mathbf{R}}_{\theta }\left[\begin{array}{c}0\\ 0\\ \dot{\psi }\end{array}\right]={\left({\mathbf{\Omega }}_b^e\right)}^{-1}\left[\begin{array}{c}\dot{\varphi }\\ \dot{\theta }\\ \dot{\psi }\end{array}\right].$$

(2)

Therefore, the transformation matrix is

$${\mathbf{\Omega }}_b^e=\left[\begin{array}{ccc}1& sin\varphi tan\theta & cos\varphi tan\theta \\ 0& cos\varphi & -sin\varphi \\ 0& sin\varphi sec\theta & cos\varphi sec\theta \end{array}\right].$$

(3)

During the inner loop control process, the inputs are Roll ($\varphi$), Pitch ($\theta$), Yaw ($\psi$), and Total Thrust (T), representing the desired values. The outputs are the states, including the position, attitude, and velocities. Initially, as presented in Equation (4), the required torques ${\tau }_{\varphi }$, ${\tau }_{\theta }$, ${\tau }_{\psi }$ are calculated using PID controllers based on the difference between the current and desired attitude angles:

$$\left\{\begin{array}{c}{\tau }_{\varphi }=PI{D}_{\varphi }({\varphi }_{\mathrm{desired}}-{\varphi }_{\mathrm{actual}})\hfill \\ {\tau }_{\theta }=PI{D}_{\theta }({\theta }_{\mathrm{desired}}-{\theta }_{\mathrm{actual}})\hfill \\ {\tau }_{\psi }=PI{D}_{\psi }({\psi }_{\mathrm{desired}}-{\psi }_{\mathrm{actual}})\hfill \end{array}\right.$$

(4)

where

$$PI{D}_i=K_p·e_i+K_i·\int e_{i}dt+K_d·{\displaystyle \frac{d{e}_i}{dt}}.$$

For the tuning process (gain parameters $K_p$, $K_i$, $K_d$) of all PID controllers designed in this paper, the pidTuner tool from MATLAB was utilized. The primary objectives are to ensure the stability of the closed-loop system at all predetermined operating points, minimize the response time, reduce overshoot, and maintain smooth system output. For specific PID controllers (e.g., Section 3.2.2), appropriate integral action was employed to reduce or eliminate steady-state errors.

Simultaneously, the torques from Equation (4) can also be expressed using thrusts, as shown below.

$$\begin{array}{cc}\hfill \mathrm{Total}\mathrm{thrust}T:& T=T1+T2+T3+T4\hfill \\ \hfill \mathrm{Pitch}\mathrm{torque}{\tau }_{\theta }:& {\tau }_{\theta }=L_d(T1-T3)\hfill \\ \hfill \mathrm{Roll}\mathrm{torque}{\tau }_{\varphi }:& {\tau }_{\varphi }=L_d(T4-T2)\hfill \\ \hfill \mathrm{Yaw}\mathrm{torque}{\tau }_{\psi }:& {\tau }_{\psi }=\kappa (T1-T2+T3-T4)\hfill \end{array}$$

(5)

Because all of the thrust–torque relationships can be directly represented as the product of a unit matrix, the form of the thrust distribution matrix (Equation (6)) becomes more concise when assuming that both the arm length $L_d$ and torque coefficient $\kappa$ are equal to 1:

$$\left[\begin{array}{c}T1\\ T2\\ T3\\ T4\end{array}\right]={\displaystyle \frac{1}{4}}\left[\begin{array}{cccc}1& -1& 0& -1\\ 1& 1& 0& -1\\ 1& 0& -1& 1\\ 1& 0& 1& 1\end{array}\right]\left[\begin{array}{c}T\\ {\tau }_{\theta }\\ {\tau }_{\varphi }\\ {\tau }_{\psi }\end{array}\right].$$

(6)

At this point, the linear velocity, position, angular velocity, and attitude can be updated over time accordingly.

3.1.1. Linear Velocity and Position Update

Assuming that the mass of the UAV is m, the acceleration is $\mathbf{a}$, and the total force is $\mathbf{F}={[F_x,F_y,F_z]}^T$, according to Newton’s second law we have:

$$m\mathbf{a}=\mathbf{F}.$$

(7)

For the quadcopter, the total force $\mathbf{F}$ can be decomposed into thrust and gravity:

$$\mathbf{F}={\mathbf{R}}_b^e\left[\begin{array}{c}0\\ 0\\ -T\end{array}\right]+\left[\begin{array}{c}0\\ 0\\ mg\end{array}\right].$$

(8)

Therefore,

$$m\left[\begin{array}{c}\ddot{x}\\ \ddot{y}\\ \ddot{z}\end{array}\right]={\mathbf{R}}_b^e\left[\begin{array}{c}0\\ 0\\ -T\end{array}\right]+\left[\begin{array}{c}0\\ 0\\ mg\end{array}\right]$$

(9)

and

$$\left\{\begin{array}{c}\ddot{x}={\displaystyle \frac{T}{m}}(cos\varphi sin\theta cos\psi +sin\varphi sin\psi )\hfill \\ \ddot{y}={\displaystyle \frac{T}{m}}(cos\varphi sin\theta sin\psi -sin\varphi cos\psi )\hfill \\ \ddot{z}={\displaystyle \frac{T}{m}}(cos\theta cos\varphi )-g\hfill \end{array}\right..$$

(10)

The updated linear velocities ($\dot{x},\dot{y},\dot{z}$) can be obtained by integrating the corresponding linear accelerations ($\ddot{x},\ddot{y},\ddot{z}$). Assuming that the velocities at the previous time step are given by ${\mathbf{v}}_t={[{\dot{x}}_t,{\dot{y}}_t,{\dot{z}}_t]}^T$ with a time step of $\Delta t$, the velocities for the current time step can be computed as follows:

$${\left[\begin{array}{c}\dot{x}\\ \dot{y}\\ \dot{z}\end{array}\right]}_{t+\Delta t}={\left[\begin{array}{c}\dot{x}\\ \dot{y}\\ \dot{z}\end{array}\right]}_t+\left[\begin{array}{c}\ddot{x}\\ \ddot{y}\\ \ddot{z}\end{array}\right]\Delta t.$$

(11)

Similarly, the position is updated through numerical integration of the linear velocity:

$${\left[\begin{array}{c}x\\ y\\ z\end{array}\right]}_{t+\Delta t}={\left[\begin{array}{c}x\\ y\\ z\end{array}\right]}_t+{\left[\begin{array}{c}\dot{x}\\ \dot{y}\\ \dot{z}\end{array}\right]}_t\Delta t.$$

(12)

3.1.2. Angular Velocity and Attitude Update

The relationship between angular velocity and angular acceleration can also be described by the Newton–Euler equation:

$$\mathit{\tau }=\mathbf{I}\mathit{\alpha }+\mathit{\omega }\times \left(\mathbf{I}\mathit{\omega }\right)$$

(13)

where $\mathit{\tau }={[{\tau }_{\varphi },{\tau }_{\theta },{\tau }_{\psi }]}^T$, $\mathbf{I}$ is the inertia matrix, $\mathit{\alpha }={[{\alpha }_x,{\alpha }_y,{\alpha }_z]}^T$ is the angular acceleration, and $\mathit{\omega }={[p,q,r]}^T$ is the angular velocity.

The angular acceleration is calculated using

$$\mathit{\alpha }={\mathbf{I}}^{-1}(\mathit{\tau }-\mathit{\omega }\times \left(\mathbf{I}\mathit{\omega }\right)).$$

(14)

The updated angular velocities are obtained by integrating the angular accelerations. Assuming that the initial angular velocities from the last time step are ${\mathit{\omega }}_t={[p_t,q_t,r_t]}^T$ and the angular accelerations are provided by $\mathit{\alpha }={[{\alpha }_x,{\alpha }_y,{\alpha }_z]}^T$ with a time increment of $\Delta t$, the new angular velocities can be computed as follows:

$${\left[\begin{array}{c}p\\ q\\ r\end{array}\right]}_{t+\Delta t}={\left[\begin{array}{c}p\\ q\\ r\end{array}\right]}_t+\left[\begin{array}{c}{\alpha }_x\\ {\alpha }_y\\ {\alpha }_z\end{array}\right]\Delta t.$$

(15)

The attitude is then updated:

$${\left[\begin{array}{c}\varphi \\ \theta \\ \psi \end{array}\right]}_{t+\Delta t}={\left[\begin{array}{c}\varphi \\ \theta \\ \psi \end{array}\right]}_t+{\left[\begin{array}{c}\dot{\varphi }\\ \dot{\theta }\\ \dot{\psi }\end{array}\right]}_t\Delta t$$

(16)

where the attitude rates in the earth coordinate system $\dot{\varphi },\dot{\theta },\dot{\psi }$ are obtained through the angular velocity transformation matrix:

$$\left[\begin{array}{c}\dot{\varphi }\\ \dot{\theta }\\ \dot{\psi }\end{array}\right]={\mathbf{\Omega }}_b^e\left[\begin{array}{c}p\\ q\\ r\end{array}\right].$$

(17)

3.2. Outer Loop Control

As the inner loop control manages the transformation of desired inputs into updated UAV states, the outer loop control’s role is to determine these desired inputs at each time step.

In this study, two outer loop controllers are utilized. During the intruder chasing phase, each ownship employs a DRL-based intelligent SAC-FIS agent. Additionally, PID controllers are applied to regulate the safe returning mode.

3.2.1. SAC-FIS Controller for Mobile Intruder Interception

In our previous work [35], we detailed the design and validation of the SAC-FIS controller for single-ownship operations. Specifically, the FIS processes readings from onboard 3D LiDAR to perform object detection and real-time obstacle avoidance by controlling yaw. The SAC model, which controls roll, pitch, and total thrust, serves as the core of the algorithm. It progressively understands the UAV dynamic model and the logic provided by the FIS via its neural networks. Through training, it continuously strengthens cooperation with the FIS, allowing this approach to achieve efficient control and dynamic tracking in unpredictable environments without compromising the UAV’s maneuverability. This paper will not revisit the specific design process explained in that work, and will instead focus on new enhancements for swarm control settings.

The SAC-FIS agent relies on environmental information and feedback to determine actions, which are the inputs to the quadrotor model.

For each individual ownship, the observation space (S) is constructed as follows:

$$S=\left(d_{\mathrm{ot}};{\mathbf{\Theta }}_{ZYX};{\mathbf{v}}_{\mathrm{o}};{\mathit{\omega }}_{\mathrm{o}};\Delta {\mathbf{x}}_{\mathrm{ot}};{\mathbf{v}}_{\mathrm{t}};{\mathit{\omega }}_{\mathrm{xy},\mathrm{o}};{\mathbf{a}}_{\mathrm{input}};{\mathbf{u}}_{\mathrm{FIS}};p_{\Vert };p_{\perp };n_{\mathrm{captured}}\right)$$

(18)

which includes:

• $d_{\mathrm{ot}}$: The Euclidean distance between the ownship and selected target ship.
• ${\mathbf{\Theta }}_{ZYX}$: The ownship’s Euler angles in ZYX order, represented as $\psi ,\theta ,\varphi$.
• ${\mathbf{v}}_{\mathrm{o}}$: The ownship’s linear velocities; ${\mathbf{v}}_{\mathrm{o}}=[vx;vy;vz]$.
• ${\mathit{\omega }}_{\mathrm{o}}$: The ownship’s angular velocities, represented as p, q, r.
• $\Delta {\mathbf{x}}_{\mathrm{ot}}$: The coordinate differences between the ownship and selected target ship in the XYZ directions, represented as $x_e$, $y_e$, and $z_e$.
• ${\mathbf{v}}_{\mathrm{t}}$: The selected target ship’s linear velocities.
• ${\mathit{\omega }}_{\mathrm{xy},\mathrm{o}}$: The ownship’s angular velocities about the earth coordinate system’s X and Y axes, represented by ${\omega }_X$ and ${\omega }_Y$.
• ${\mathbf{a}}_{\mathrm{input}}$: The roll ($u_{\varphi }$), pitch ($u_{\theta }$), yaw ($u_{\psi }$), and thrust ($u_T$) actions.
• ${\mathbf{u}}_{\mathrm{FIS}}$: Inputs of the FIS, consisting of the ownship–target angle (${\theta }_{\mathrm{target}}$) and sampled readings from the 3D LiDAR, including the front distance ($d_{\mathrm{front}}$) and lateral distance error ($d_{\text{left-right}}$).
• $p_{\Vert }$: The speed vector’s projection onto the direction (vector) formed between the ownship and the selected target ship.
• $p_{\perp }$: The component of the speed vector that is perpendicular to the vector formed between the ownship and the selected target ship.
• $n_{\mathrm{captured}}$: The number of successfully intercepted intruders.

In addition, to maintain high sensitivity to slight state changes within complex and dynamic environments, exponential terms are utilized to formulate an effective reward function for each ownship in the swarm. This design ensures that each component possesses bounded outputs, typically ranging from −1 to 0 or from 0 to 1, which facilitates the fine-tuning of weights for each component according to the importance of different objectives:

$$\begin{array}{cc}\hfill r_t=& {\alpha }_1·e^{{\alpha }_2{x}_e^2+{\alpha }_3{y}_e^2+{\alpha }_4{z}_e^2}+{\alpha }_5·\left(1-e^{{\alpha }_6·p_{\Vert }}\right)+{\alpha }_7·\left(1-e^{{\alpha }_8·p_{\perp }}\right)\hfill \\ \hfill & +{\alpha }_9·\left(1-e^{{\alpha }_{10}·({\alpha }_{11}·u_{\varphi }^2+{\alpha }_{12}·u_{\theta }^2)}\right)+{\alpha }_{13}·\left(1-e^{{\alpha }_{14}·{\omega }_X^2}\right)+{\alpha }_{13}·\left(1-e^{{\alpha }_{15}·{\omega }_Y^2}\right)\hfill \\ & +{\beta }_1·\Delta n_{\mathrm{captured}}\hfill \end{array}$$

(19)

where ${\alpha }_1$ to ${\alpha }_{15}$ and ${\beta }_1$ are weight coefficients. For this study, these coefficients were tuned as follows:

$$\begin{array}{c}{\alpha }_1=0.5,{\alpha }_2={\alpha }_3={\alpha }_4=-0.01,{\alpha }_5=0.4\mathrm{if}{p}_{\Vert }\ge 0\vee {\alpha }_5=-0.4\mathrm{if}{p}_{\Vert }<0\hfill \\ {\alpha }_6=-0.3\mathrm{if}{p}_{\Vert }\ge 0\vee {\alpha }_6=0.3\mathrm{if}{p}_{\Vert }<0, {\alpha }_7=-0.25\hfill \\ {\alpha }_8=-0.3\mathrm{if}{p}_{\perp }\ge 0\vee {\alpha }_8=0.3\mathrm{if}{p}_{\perp }<0,{\alpha }_9=-0.2,{\alpha }_{10}=-8,{\alpha }_{11}=2,{\alpha }_{12}=3\hfill \\ {\alpha }_{13}=-0.2,{\alpha }_{14}={\alpha }_{15}=-0.05.\hfill \end{array}$$

and

$$\begin{array}{c}{\beta }_1=60.\hfill \end{array}$$

This reward function comprises multiple elements. The primary element employs the ownship’s distance to the designated target, encouraging the minimization of this difference, while the second and third components guide the agent to advance directly toward the chosen target by imposing penalties for deviations in alternative directions. Additionally, penalties are applied to the control inputs ($u_{\varphi }$ and $u_{\theta }$) and angular velocities (${\omega }_X$ and ${\omega }_Y$) to mitigate oscillations and excessive fluctuations, thereby enhancing safety and conserving energy. The term with $\Delta n_{\mathrm{captured}}$ is included to reward the agent for each successfully intercepted intruder UAV.

Furthermore, the reset function utilized for the agent training process, originally described in [35], has been improved to accommodate swarm systems and ensure adaptive generalization. At the start of each episode, the initial orientation (yaw angle) of each ownship is randomized, and their initial positions on the drone hub are also randomly generated. This step guarantees that each ownship maintains a minimum distance of one meter from the drone hub boundary and at least two meters from one another. Additionally, the initial positions of each intruder aircraft are randomly assigned along their predefined paths.

3.2.2. PID Control for Safe Returning

When an ownship has successfully captured all its allocated targets and no unassigned intruders remain, it automatically switches to return mode. This also occurs if it encounters an emergency, such as impending battery depletion, prompting a return to the drone hub. The specific conditions for triggering the flight modes are detailed in Section 3.3.2.

Upon returning, the system generates a random landing point on the drone hub for the ownship, defined as $[x_{\mathrm{land}},y_{\mathrm{land}},0]$. Similar to the mechanism for generating the initial positions of ownships, this point must be at least one meter from the drone hub’s edge and two meters from any previously assigned landing points of other ownships, unless it is the first ownship to return.

Given a specified three-dimensional coordinate and the current UAV position, the waypoint-following control that guides the quadrotor to the specified location can be designed using the PID controllers based on the quadrotor model described in Section 3.1. The same control principle is applicable to the predefined path following of intruder aircraft. Specifically, if the desired landing point or waypoint is $[x_{\mathrm{d}},y_{\mathrm{d}},z_{\mathrm{d}}]$ and the current position is $[x_{\mathrm{c}},y_{\mathrm{c}},z_{\mathrm{c}}]$, then the desired velocities can be calculated as follows:

$$\left\{\begin{array}{c}{\dot{x}}_{\mathrm{d}}=PI{D}_x(x_{\mathrm{d}}-x_{\mathrm{c}})\hfill \\ {\dot{y}}_{\mathrm{d}}=PI{D}_y(y_{\mathrm{d}}-y_{\mathrm{c}})\hfill \\ {\dot{z}}_{\mathrm{d}}=PI{D}_z(z_{\mathrm{d}}-z_{\mathrm{c}})\hfill \end{array}\right.$$

(20)

and

$$\left\{\begin{array}{c}{\ddot{x}}_{\mathrm{d}}=PI{D}_{\dot{x}}({\dot{x}}_{\mathrm{d}}-{\dot{x}}_{\mathrm{c}})\hfill \\ {\ddot{y}}_{\mathrm{d}}=PI{D}_{\dot{y}}({\dot{y}}_{\mathrm{d}}-{\dot{y}}_{\mathrm{c}})\hfill \\ {\ddot{z}}_{\mathrm{d}}=PI{D}_{\dot{z}}({\dot{z}}_{\mathrm{d}}-{\dot{z}}_{\mathrm{c}})\hfill \end{array}\right..$$

(21)

Given the desired accelerations, according to Equation (10), the desired inputs at each time step are as follows:

$$\mathrm{Roll}\mathrm{Control}:{\varphi }_{\mathrm{input}}=\mathrm{atan}2\left({\displaystyle \frac{(-{\ddot{x}}_{d}sin\left({\psi }_c\right)+{\ddot{y}}_{d}cos\left({\psi }_c\right))cos\left({\theta }_c\right)}{{\ddot{z}}_d+g}}\right),$$

(22)

$$\mathrm{Pitch}\mathrm{Control}:{\theta }_{\mathrm{input}}=\mathrm{atan}2\left({\displaystyle \frac{-{\ddot{x}}_{d}cos\left({\psi }_c\right)-{\ddot{y}}_{d}sin\left({\psi }_c\right)}{{\ddot{z}}_d+g}}\right),$$

(23)

$$\mathrm{Thrust}\mathrm{Control}:T_{\mathrm{input}}={\displaystyle \frac{m({\ddot{z}}_d+g)}{cos\left({\theta }_c\right)cos\left({\varphi }_c\right)}}.$$

(24)

Because the quadrotor’s yaw angle does not affect its velocity in any direction [42], the desired yaw can be simply calculated by

$$\mathrm{Yaw}\mathrm{Control}:{\psi }_{\mathrm{input}}=\mathrm{atan}2\left({\displaystyle \frac{y_{\mathrm{d}}-y_{\mathrm{c}}}{x_{\mathrm{d}}-x_{\mathrm{c}}}}\right).$$

(25)

To ensure the flight stability and control response of the quadrotor, the roll and pitch control should be bounded:

$$\begin{array}{cc}\hfill -0.135\pi & \le {\varphi }_{\mathrm{input}}\le 0.135\pi ,\hfill \\ \hfill -0.135\pi & \le {\theta }_{\mathrm{input}}\le 0.135\pi .\hfill \end{array}$$

(26)

3.3. Decentralized Swarm Control

At this stage, it is crucial to develop an efficient and decentralized swarm control strategy that can coordinate all ownships. This strategy encompasses a dynamic target selection algorithm, conditions that direct individual ownships to transition to landing mode, and criteria for concluding a test.

3.3.1. Dynamic Target Selection Algorithm

Developing a target selection algorithm for single-ownship scenarios is straightforward. However, the challenges escalate in swarm scenarios involving n ownships and m intruder aircraft. When $n>m$, the situation is similar to $n=m$ as only m ownships are needed, with each assigned an intruder based on proximity. Conversely, when $n<m$, some ownships must intercept more than one intruder UAV to complete the mission.

At the mission’s outset, each ownship can be uniformly assigned an intruder aircraft according to a sequence (Initial Allocation Phase). Yet, the completion times for the first interception will vary among ownships, necessitating a distributed dynamic target selection mechanism in which the ownship that completes its task first is promptly reassigned a new target (Dynamic Reallocation Phase). This cycle continues until an ownship completes its current target’s interception and no unassigned intruder aircraft remain in the environment.

To implement such a strategy, ownships in the swarm must share their intruder status information, indicating whether each intruder UAV has been assigned (represented by 1 or 0); for instance, if an ownship encounters an emergency, it will abandon its assigned intruder, whose status then reverts to ’unassigned’. Due to the information-sharing mechanism, one of the other ownships will intercept this intruder after completing its current task, ensuring the overall mission’s success.

A flowchart of the distributed swarm control strategy is illustrated in Figure 3, and the detailed pseudocode of the dynamic target selection algorithm is provided in Algorithm 1.

3.3.2. Conditions for Triggering Safe Return Mode

For each individual ownship, the return mode is triggered under two specific conditions:

• The ownship successfully neutralizes its current target and no unassigned intruder aircraft remain in the environment.
• The ownship encounters an emergency situation, e.g., a collision or $d_{\mathrm{trav}}\ge d_{\mathrm{threshold}}$.

In testing, we simulated low battery conditions by adjusting the ‘distance traveled’ threshold $d_{\mathrm{threshold}}$ of the corresponding ownship, thereby further validating the effectiveness of the information-sharing and reallocation mechanism. The traveled distance $d_{\mathrm{trav}}$ can be calculated by:

The distance change at each time step $\Delta d$ is

$$\Delta d=\sqrt{{(x_{t+\Delta t}-x_t)}^2+{(y_{t+\Delta t}-y_t)}^2+{(z_{t+\Delta t}-z_t)}^2},$$

(27)

$$d_{\mathrm{trav}}=\sum _{i=0}^N\Delta d_i,$$

(28)

where N is the number of time steps.

Figure 3. Flowchart of the distributed swarm control strategy.

Figure 3. Flowchart of the distributed swarm control strategy.

Algorithm 1: Dynamic Target Selection Algorithm for Multiple Ownships and Intruders

Require:  Number of ownships $\mathbf{n}$ ($Q_1,Q_2,\dots ,Q_n$); Number of intruder UAVs $\mathbf{m}$ ($P_1,P_2,\dots ,P_m$); Ownship positions ${\mathbf{Q}}_{\mathrm{pos}}=[{\mathbf{Q}}_{1,\mathrm{pos}},\dots ,{\mathbf{Q}}_{n,\mathrm{pos}}]$; Intruder aircraft positions and velocities ${\mathbf{P}}_{\mathrm{pos}}=[{\mathbf{P}}_{1,\mathrm{pos}},\dots ,{\mathbf{P}}_{m,\mathrm{pos}}]$ and ${\mathbf{P}}_{\mathrm{vel}}=[{\mathbf{P}}_{1,\mathrm{vel}},\dots ,{\mathbf{P}}_{m,\mathrm{vel}}]$; Number of captured intruders for each ownship ${\mathbf{Q}}_{\mathrm{cap}}=[Q_{1,\mathrm{cap}},\dots ,Q_{n,\mathrm{cap}}]$; Intruder aircraft status ${\mathbf{P}}_{\mathrm{stat}}=[P_{1,\mathrm{stat}},\dots ,P_{m,\mathrm{stat}}]$. Ensure:  Each ownship is assigned an intruder as a target and receives the target’s position and velocity information: ${\mathbf{Q}}_{\mathrm{tgt}}=[{\mathbf{Q}}_{1,{\mathrm{P}}_{i,\mathrm{pos}}},{\mathbf{Q}}_{1,{\mathrm{P}}_{i,\mathrm{vel}}},\dots ,{\mathbf{Q}}_{n,{\mathrm{P}}_{j,\mathrm{pos}}},{\mathbf{Q}}_{n,{\mathrm{P}}_{j,\mathrm{vel}}}]$; Number of unselected targets ${\mathbf{N}}_{\mathrm{Unsel}}$; Each intruder is assigned to an ownship ${\mathbf{P}}_{\mathrm{sel}}=[{\mathbf{P}}_{1,{\mathrm{Q}}_i},\dots ,{\mathbf{P}}_{m,{\mathrm{Q}}_j}]$. Step 1: Initialization Initialize persistent variables ${\mathbf{Q}}_{\mathrm{tgt},\mathrm{prev}}$, ${\mathbf{N}}_{\mathrm{Unsel},\mathrm{prev}}$, and ${\mathbf{P}}_{\mathrm{sel},\mathrm{prev}}$ to store previous assignment results. Step 2: Initial Target Allocation if ${\sum }_{i=1}^n{Q}_{i,\mathrm{cap}}=0$ and ${\mathbf{N}}_{\mathrm{Unsel},\mathrm{prev}}=m$ then    for $i\leftarrow 1$ to n do      Calculate distances: $d_{i,j}=\parallel {\mathbf{Q}}_{i,\mathrm{pos}}-{\mathbf{P}}_{j,\mathrm{pos}}\parallel$, $j=1,\dots ,m$ {Euclidean distance}      Find nearest intruder: $j^{\ast }=arg{min}_j{d}_{i,j}$      ${\mathbf{Q}}_{i,\mathrm{tgt},\mathrm{prev}}\leftarrow [{\mathbf{P}}_{j^{\ast },\mathrm{pos}},{\mathbf{P}}_{j^{\ast },\mathrm{vel}}]$, ${\mathbf{P}}_{j^{\ast },\mathrm{sel},\mathrm{prev}}\leftarrow {\mathbf{Q}}_i$      Remove ${\mathbf{P}}_{j^{\ast },\mathrm{pos}}$ and ${\mathbf{P}}_{j^{\ast },\mathrm{vel}}$ from ${\mathbf{P}}_{\mathrm{pos}}$ and ${\mathbf{P}}_{\mathrm{vel}}$    end for    ${\mathbf{N}}_{\mathrm{Unsel},\mathrm{prev}}\leftarrow m-n$ end if   Step 3: Dynamic Reallocation for $i\leftarrow 1$ to n do    if $Q_{i,\mathrm{cap}}>Q_{i,\mathrm{cap},\mathrm{prev}}$ and ${\mathbf{N}}_{\mathrm{Unsel},\mathrm{prev}}>0$ then      Calculate distances: $d_{i,j}=\parallel {\mathbf{Q}}_{i,\mathrm{pos}}-{\mathbf{P}}_{j,\mathrm{pos}}\parallel$ for each ${\mathbf{P}}_{j,\mathrm{pos}}$ in ${\mathbf{P}}_{\mathrm{pos}}$ {Euclidean distance to unassigned intruders}      Find nearest intruder: $j^{\ast }=arg{min}_j{d}_{i,j}$      ${\mathbf{Q}}_{i,\mathrm{tgt},\mathrm{prev}}\leftarrow [{\mathbf{P}}_{j^{\ast },\mathrm{pos}},{\mathbf{P}}_{j^{\ast },\mathrm{vel}}]$, ${\mathbf{P}}_{j^{\ast },\mathrm{sel},\mathrm{prev}}\leftarrow {\mathbf{Q}}_i$      Remove ${\mathbf{P}}_{j^{\ast },\mathrm{pos}}$ and ${\mathbf{P}}_{j^{\ast },\mathrm{vel}}$ from ${\mathbf{P}}_{\mathrm{pos}}$ and ${\mathbf{P}}_{\mathrm{vel}}$      ${\mathbf{N}}_{\mathrm{Unsel},\mathrm{prev}}\leftarrow {\mathbf{N}}_{\mathrm{Unsel},\mathrm{prev}}-1$    end if    $Q_{i,\mathrm{cap},\mathrm{prev}}\leftarrow Q_{i,\mathrm{cap}}$ {Update previous capture count for each ownship} end for   Step 4: Intruders’ Status Information Sharing Mechanism ${\mathbf{P}}_{\mathrm{stat}}$ contains m elements (0 or 1), updated in real-time and shared among the ownships in the swarm. for $i\leftarrow 1$ to m do    if $P_{i,\mathrm{stat}}=1$ then      $P_i$ has been assigned to ownship $Q_j$.      ${\mathbf{Q}}_{j,\mathrm{tgt}}\leftarrow [{\mathbf{P}}_{i,\mathrm{pos}},{\mathbf{P}}_{i,\mathrm{vel}}]$, ${\mathbf{P}}_{i,\mathrm{sel}}\leftarrow {\mathbf{Q}}_j$, ${\mathbf{N}}_{\mathrm{Unsel},\mathrm{prev}}\leftarrow {\mathbf{N}}_{\mathrm{Unsel},\mathrm{prev}}-1$    end if    if Emergency occurs for $Q_j$ then      ${\mathbf{Q}}_{j,\mathrm{tgt}}\leftarrow [\mathrm{null},\mathrm{null}], {\mathbf{P}}_{i,\mathrm{sel}}\leftarrow \mathrm{null}, {\mathbf{N}}_{\mathrm{Unsel},\mathrm{prev}}\leftarrow {\mathbf{N}}_{\mathrm{Unsel},\mathrm{prev}}+1, P_{i,\mathrm{stat}}\leftarrow 0$ {At this point, $P_i$ is reset to unassigned state and will wait to be reassigned to other ownships.}    end if end for   Step 5: Output Assignment Results ${\mathbf{Q}}_{j,\mathrm{tgt}}\leftarrow {\mathbf{Q}}_{j,\mathrm{tgt},\mathrm{prev}}$ ${\mathbf{P}}_{\mathrm{sel}}\leftarrow {\mathbf{P}}_{\mathrm{sel},\mathrm{prev}}$ ${\mathbf{N}}_{\mathrm{Unsel}}\leftarrow {\mathbf{N}}_{\mathrm{Unsel},\mathrm{prev}}$

3.3.3. Termination Conditions

Compared to single-ownship tasks, utilizing a swarm not only significantly enhances the mission’s fault tolerance but also markedly improves its time efficiency. A test concludes under one of the following conditions:

• Each ownship is positioned more than 30 m away from its assigned target ship:

  $$\mathrm{For}\mathrm{any}\mathrm{individual}\mathrm{ownship},\mathrm{if}∥{\mathbf{Q}}_{i,\mathrm{pos}}-{\mathbf{P}}_{j,\mathrm{pos}}∥>30\mathrm{m}.$$
• The duration surpasses the predefined maximum time threshold:

  $$\mathrm{If}t\ge T_{\mathrm{final}}=30\mathrm{s}.$$
• All intruders have been successfully intercepted and all ownships have been safely disarmed on the drone hub (note that if any ownship lands outside the drone hub, it is considered a failed mission):

  $$\mathrm{An}\mathrm{intruder}\mathrm{is}\mathrm{successfully}\mathrm{intercepted}\mathrm{if}∥{\mathbf{Q}}_{i,\mathrm{pos}}-{\mathbf{P}}_{j,\mathrm{pos}}∥\le 2\mathrm{m}(\mathit{is}\_\mathit{Captured}),$$

  $$\mathrm{All}\mathrm{intruder}\mathrm{aircraft}\mathrm{are}\mathrm{intercepted}\mathrm{when}:\sum _{i=1}^n{n}_{i,\mathrm{captured}}=m\mathrm{and}{\mathbf{N}}_{\mathrm{Unsel}}=0.$$
• Each individual ownship has collided with an obstacle, identified by a minimal LiDAR reading (from the 3D point cloud) dropping below 0.75 m (note: in this study, the distance from the center of an ownship to its edge should not exceed 0.4 m):

  $$\mathrm{For}\mathrm{any}\mathrm{individual}\mathrm{ownship},\mathrm{if}{d}_{i,3\mathrm{DPC},\min }\le 0.75\mathrm{m}.$$

To summarize, this section has first offered a detailed overview of the UAV model applicable to each quadrotor, including ownships and intruder aircraft. Next, we have covered the inner loop controller (Section 3.1) and the various outer loop control strategies suitable for different flight modes (Section 3.2). Finally, we have introduced a decentralized control algorithm for the swarm featuring a dynamic target selection algorithm, intruder information sharing mechanism, and methods to ensure ownship safety in emergency situations and mission completion (Section 3.3). In the subsequent Section 4, we validate this control system through simulation results.

4. Results

In this chapter, we demonstrate and analyze the effectiveness and flexibility of the proposed control system across various scenarios, employing the MATLAB UAV Toolbox (2023b) for incorporation of realistic UAV dynamics and consideration of environmental factors. As established in Section 3, scenarios with more ownships (n) than intruder aircraft (m) are treated as if $n=m$. Therefore, we do not further discuss the $n>m$ case, instead focusing on the $n<m$ and $n=m$ scenarios, including situations where ownships face emergencies during interception. Additionally, the smoothing of control signals and the feasibility of the control system for use with physical drones are analyzed.

The tuning and training process of the RL agent for intruder interception is detailed in [35], and will not be reiterated here. This paper applies well-trained SAC-FIS agents, known for their strong generalization efficacy in managing intruder interception and collision avoidance tasks across various configurations. Additionally, as this study uses the NED coordinate system, all z-axis values above ground are represented as negative. The drone hub is defined by coordinates $[21,-1,0;21,21,0;-1,21,0;-1,-1,0]$, with ownships depicted as blue quadrotors and intruder aircraft in other colors. Each intruder’s motion pattern exhibits different speeds: the purple intruder moves at approximately 0.8 m/s, the orange one at 1 m/s, and the green one at 1.2 m/s. Meanwhile, since the magnitudes of the control inputs for each ownship are bounded, the speed of each individual ownship can be up to 10 m/s.

4.1. Equal Number of Ownships and Intruder Aircraft ($n=m$)

In this scenario, as illustrated in Figure 4, three ownships (${\mathbf{Q}}_i$) are stationed at the drone hub, with three intruders breaching the airspace. The initial positions of the ownships are ${\mathbf{Q}}_{1,\mathrm{pos}\mathrm{initial}}=(10,12,0)$, ${\mathbf{Q}}_{2,\mathrm{pos}\mathrm{initial}}=(7,14,0)$, and ${\mathbf{Q}}_{3,\mathrm{pos}\mathrm{initial}}=(14,7,0)$. The intruders’ initial positions are ${\mathbf{P}}_{1,\mathrm{pos}\mathrm{initial}}=(0,20,-5)$, ${\mathbf{P}}_{2,\mathrm{pos}\mathrm{initial}}=(20,20,-8)$, and ${\mathbf{P}}_{3,\mathrm{pos}\mathrm{initial}}=(20,0,-11)$, with their flight paths delineated by the waypoints $[0,0,-5;0,20,-5;20,20,-5;20,0,-5]$, $[0,0,-8;0,20,-8;20,20,-8;20,0,-8]$, and $[0,0,-11;0,20,-11;20,20,-11;20,0,-11]$, respectively.

At the beginning of the test, based on the target selection algorithm and the ownships’ sequence, ${\mathbf{Q}}_1$ first selected the nearest intruder ${\mathbf{P}}_1$ as its target ship. Subsequently, ${\mathbf{Q}}_2$ locked on to ${\mathbf{P}}_2$, leaving ${\mathbf{P}}_3$ to be automatically assigned to ${\mathbf{Q}}_3$. Figure 5 shows the complete simulation results, including both the flight modes of the ownships and their trajectories during this task. The figure also indicates the takeoff and landing points for each ownship (e.g., $O_1$ and $L_1$ represent the off-ground and landing points of ${\mathbf{Q}}_1$). The results show that each ownship successfully completed the task and safely returned to the drone hub.

Figure 6, Figure 7 and Figure 8 detail each ownship’s performance via time-based metrics. Specifically, the test duration was 15.4 s, well below the 30-second limitation. Figure 6 depicts the distance from each ownship to its target ship. Ownship 1 (${\mathbf{Q}}_1$) intercepted its target in 6.2 s and subsequently initiated a return, disarming at 13.8 s and waiting for the others to return. Similarly, Ownship 3 took 7.1 s to intercept its assigned intruder, while Ownship 2 took 9.3 s. Upon successful completion, each intruder hovered in place. The trial concluded after ${\mathbf{Q}}_2$ returned to the hub.

Figure 7 and Figure 8 quantify the reward values from the reward function’s first and second terms. The first term calculates the coordinate errors between an ownship and its target across the X, Y, and Z axes, with each component constrained between 0 and 1. Smaller differences result in higher rewards. The second term evaluates the alignment of the speed vector’s projection with the ownship–target vector, within bounds of $-0.4$ to $0.4$, rewarding positive alignments and penalizing negative ones.

4.2. Fewer Ownships than Intruder Aircraft ($n<m$)

Next, as depicted in Figure 9, we created a scenario with fewer ownships compared to intruders.

In this case, the initial positions of the ownships are ${\mathbf{Q}}_{1,\mathrm{pos}\mathrm{initial}}=(15,15,0)$ and ${\mathbf{Q}}_{2,\mathrm{pos}\mathrm{initial}}=(5,5,0)$. To increase the complexity of the mission, the intruders’ predefined trajectories are diversified, and each displays a distinct movement pattern. The intruders’ paths are as follows: ${\mathbf{P}}_{1,\mathrm{pos}\mathrm{initial}}=(20,5,-5)$ adheres to $[0,0,-11;0,20,-11;20,20,-5;20,0,-5]$, ${\mathbf{P}}_{2,\mathrm{pos}\mathrm{initial}}=(12,20,-6.8)$ follows $[0,0,-5;0,20,-5;20,20,-8;20,0,-8]$, and ${\mathbf{P}}_{3,\mathrm{pos}\mathrm{initial}}=(8,0,-9.2)$ traces $[0,0,-8;0,20,-8;20,20,-11;20,0,-11]$.

At mission onset, Ownship 1 targeted the nearest intruder, ${\mathbf{P}}_2$, and Ownship 2 pursued ${\mathbf{P}}_3$, while ${\mathbf{P}}_1$ was initially unassigned, pending the completion of the first interception.

The resulting dynamic is detailed in Figure 10, Figure 11, Figure 12 and Figure 13, where Figure 11 records Ownship 1 rapidly engaging ${\mathbf{P}}_2$ in 5.5 s. This was faster than Ownship 2, which took 6.9 s. Consequently, according to the dynamic reallocation phase of the target selection algorithm, ${\mathbf{P}}_1$ was automatically assigned to Ownship 1, serving as its second target. Ultimately, Ownship 1 required 14.2 s to intercept two intruder UAVs, including the landing process, with the total mission duration amounting to 18.8 s.

4.3. Equal Number of Ownships and Intruder Aircraft ($n=m$ with Emergency Situations)

To further verify the effectiveness of the proposed control system, an emergency scenario simulation was conducted. This test aimed to assess whether other ownships could successfully intercept the intruder initially assigned to the failed ownship. The scenario utilized the mission layout from Section 4.1 (Figure 4).

For Ownship 2, $d_{\mathrm{threshold}}$ was set to 2 m, meaning that this ownship would enter an emergency state when its flight distance reached two meters. The originally assigned ${\mathbf{P}}_2$ would then become unallocated and be reassigned to either Ownship 1 or Ownship 3 depending on the timing of their initial interceptions. Figure 14, Figure 15, Figure 16 and Figure 17 show the detailed simulation results for each ownship.

From these figures, it is evident that Ownship 2 encountered an emergency situation in less than two seconds of flight. During landing, it significantly reduced its speed to minimize further risk. Meanwhile, thanks to the dynamic reallocation mechanism and information-sharing with regard to the intruders’ status, Ownship 1 was assigned the remaining intruder UAV after completing the first target interception. It took 12 s to successfully intercept the second intruder. Together with the landing process, the total duration of this mission was 22.1 s.

These findings confirm the resilience and efficacy of the swarm control protocol, demonstrating that the mission objectives can be achieved even with limited ownship availability or in emergency situations.

4.4. Control System Feasibility Analysis

The preceding subsections have validated the control system’s performance across various scenarios; however, determining the practical applicability of these control signals on physical UAVs remains a critical step.

In this study, the mass of each quadrotor UAV is set to $m=0.1\mathrm{kg}$ and the gravity is set to $g=9.81{\mathrm{m}/\mathrm{s}}^2$. The UAV’s weight is adjustable; however, this requires retraining the SAC-FIS agent. Taking Ownship 1 (${\mathbf{Q}}_1$) from Section 4.3 as an example, its original control signals throughout the mission are displayed in Figure 18. It was propelled by the SAC-FIS controller during the initial 17.2 s, then subsequently by the PID controller. Clearly, the control variables (roll, pitch, yaw, thrust) exhibit significant fluctuations which, while effective in simulations, could lead to mechanical wear and safety hazards when applied to physical drones [43].

Therefore, such control signals must be further smoothed to meet both the maneuverability requirements for successfully completing the mission and the safety constraints of physical UAV motors. We tested and compared various filters to identify the best method for smoothing the control signals. Traditional low-pass filters [44] can delay signals and reduce amplitude by filtering out high frequencies. The performance of the Kalman filter [45,46] would significantly decrease if the system’s dynamic model and noise statistical characteristics were inaccurate, making it unsuitable for this study’s model-free context. After testing, we finally opted for a moving average filter. A window size of 10 was used for the SAC-FIS controller (intruder interception phase), and a window size of 400 was applied to the PID controller (landing process), based on their different noise frequencies. The final results are shown in Figure 19.

The simulations presented in Section 4 applied the filtered signals, further confirming that this control signal processing method successfully achieves a balance between performance and applicability.

5. Conclusions

This research presents a novel multilayered control architecture that integrates intelligent and rule-driven methods for multi-UAV systems. This integration enables real-time path planning and execution of complex tasks, which are difficult with a single method alone. It also provides a safe and efficient solution to the swarm control problem. Specifically, the proposed framework encompasses an inner-loop controller, two outer-loop controllers, and a decentralized swarm control strategy. The main strengths of this system can be summarized as follows: first, the introduction of dual flight modes enhances performance by not only enabling autonomous decision-making in unpredictable environments but also ensuring higher reliability under controllable conditions, thereby improving overall system safety and efficiency; second, the dynamic target selection algorithm can perform distributed target allocation for each ownship through information sharing and accommodate diverse challenges, thus efficiently coordinating ownships, saving time, and enhancing system fault tolerance and mission success rates; finally, the entire system is designed with a modular architecture, facilitating the replacement and upgrading of components to adapt to different problems. However, the proposed framework still exhibits certain limitations, such as the use of PID controllers at certain points. While straightforward to deploy, these controllers introduce steady-state errors and potential for integral windup, making them suboptimal for control scenarios that demand high precision.

In the future, we will continue to explore various combinations of neural network-based and code-based algorithms for more complex problems, such as leader–follower structures and environments with dense and randomly localized obstacles. By defining and adjusting the control scope of AI methods, we aim to unlock new possibilities for UAV applications.