*Article* **Cooperative Friendly Jamming Techniques for Drone-Based Mobile Secure Zone**

**Ga-Hye Jeon <sup>1</sup> , Ji-Hyun Lee <sup>2</sup> , Yeon-Su Sung <sup>3</sup> , Hyun-Ju Park <sup>4</sup> , You-Jin Lee <sup>5</sup> , Sun-Woo Yun <sup>1</sup> and Il-Gu Lee 1,\***


**Abstract:** Threats of eavesdropping and information leakages have increased sharply owing to advancements in wireless communication technology. In particular, the Internet of Things (IoT) has become vulnerable to sniffing or jamming attacks because broadcast communication is usually conducted in open-network environments. Although improved security protocols have been proposed to overcome the limitations of wireless-communication technology and to secure safe communication channels, they are difficult to apply to mobile communication networks and IoT because complex hardware is required. Hence, a novel security model with a lighter weight and greater mobility is needed. In this paper, we propose a security model applying cooperative friendly jamming using artificial noise and drone mobility, which are autonomous moving objects, and we demonstrate the prevention of eavesdropping and improved security through simulations and field tests. The Cooperative Friendly Jamming Techniques for Drone-based Mobile Secure Zone (CFJ-DMZ) can set a secure zone in a target area to support a safe wireless mobile communication network through friendly jamming, which can effectively reduce eavesdropping threats. According to the experimental results, the average information leakage rate of the eavesdroppers in CFJ-DMZ-applied scenarios was less than or equal to 3%, an average improvement of 92% over conventional methods.

**Keywords:** IoT; RF radio communication; Wi-Fi direct; D2D; drone-based mobile secure zone; friendly jamming; mobility

## **1. Introduction**

Wireless communication network technology is evolving to meet the needs of users who want to use high-speed, high-capacity multimedia content without the limitations of location and time. However, cases of information leakage have been continually occurring owing to the fundamental limitations of wireless communication, which is vulnerable to eavesdropping [1]. To solve this problem, protocols and mechanisms have been proposed that improve the security or secure safe communication channels within the time, frequency, and space domains [2]. However, conventional wireless secure communication methods require complex hardware, which reduces the energy efficiency and data transmission performance, limiting the application to highly reliable wireless autonomous moving objects that exchange confidential information [3,4].

Among wireless network technologies, the Internet of Things (IoT) has become an essential element in all industries and everyday life, and accordingly, security vulnerabilities of wireless networks have become a larger issue [5]. Most IoT devices are light in weight

**Citation:** Jeon, G.-H.; Lee, J.-H.; Sung, Y.-S.; Park, H.-J.; Lee, Y.-J.; Yun, S.-W.; Lee, I.-G. Cooperative Friendly Jamming Techniques for Drone-Based Mobile Secure Zone. *Sensors* **2022**, *22*, 865. https:// doi.org/10.3390/s22030865

Academic Editor: Margot Deruyck

Received: 17 December 2021 Accepted: 20 January 2022 Published: 24 January 2022

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

with specifications that are insufficient to apply to regular personal computers or mobile devices. The existing communication systems are vulnerable to side-channel attacks, which are attacks that utilize physical information generated from the physical layer. Thus, the security protocol key can be leaked and encryption can be disabled. As they communicate in open network environments, they are easily exposed to security threats such as eavesdropping. This vulnerability can be exploited to easily collect confidential information [6].

To improve the security in IoT environments that are easily exposed to security threats, security architectures should be designed by considering the light weight and mobility of mobile IoT devices [7]. In this paper, we proposed Cooperative Friendly Jamming Techniques for Drone-based Mobile Secure Zone (CFJ-DMZ) to enhance the security in wireless communication environments for drones, which are IoT devices that can move autonomously while exchanging information with other surrounding devices. As eavesdropping is a passive attack that leaves no evidence of attacks, it is impossible to detect eavesdroppers in a wireless communication environment. Therefore, in this paper, a proactive prevention method was proposed to reduce the eavesdropping probability of unspecified potential eavesdroppers. The CFJ-DMZ uses the mobility and artificial interference of autonomous moving objects to form a secure zone, which guarantees safe data communication in the wireless communication environment of mobile IoT and flexibly controls the zone to be protected, effectively mitigating eavesdropping threats.

To evaluate the proposed CFJ-DMZ, we implemented a network simulation model and validated its performance. In the network simulation model, the transmission node moves to near the receiving node, and safe short-range device-to-device (D2D) communication is then conducted in the secure zone formed through a cooperative jammer drone. Here, three drones communicate with each other and create a locally secure zone based on the boundary of the artificial interference signal reaching range to protect the confidentiality of communication between nodes.

The main contributions of this paper are as follows:


The remainder of this paper is organized as follows. Section 2 introduces studies on conventional wireless communication security technologies and major studies on the wireless communication security of autonomous moving objects. Section 3 then describes the security performance criteria of friendly jamming and the friendly jamming model. Section 4 proposes a CFJ-DMZ model, and Section 5 validates the security performance of the proposed model through simulations and field experiments. Finally, Section 6 provides some concluding remarks and describes future research directions.

#### **2. Related Work**

#### *2.1. Introduction to Friendly Jamming*

Wyner defined the concept of security capacity in a study on eavesdropping channels and proved that security can be achieved according to the information theory perspective when the quality is poorer in the eavesdropper's channel than in the legitimate receiver [8]. Security channel capacity (secrecy capacity) is defined as the difference in the channel capacity between a legitimate sender and a legitimate receiver and the channel capacity between a legitimate sender and an eavesdropper. If this value is negative, the security channel capacity is zero, which means that no information can be safely sent. Here, signals and artificial interference can be generated and sent to improve the channel quality of the receiver and deteriorate the channel quality of the eavesdropper. Various studies have been conducted for this, including jamming, beamforming jamming, relay jamming, and friendly jamming techniques for improvements in wireless security. It has recently been proven that if a beam is formed using multiple antennas for jamming signals that can

reduce the eavesdropping performance of malicious eavesdroppers, the communication security and reliability between legitimate communicators can be significantly improved. However, beamforming technology that uses multiple antennas or a massive antenna is complex and requires a large power consumption, making it difficult to use in IoT or mobile devices [9,10].

#### *2.2. Friendly Jamming Security Model*

Various studies are underway on friendly jamming security techniques applying jamming signals for security purposes [11–18]. A friendly jamming technique is a method of emitting friendly artificial interference signals to prevent malicious eavesdroppers from overhearing when the sender communicates with a legitimate receiver. A jammer is used to protect the wireless communication network and send messages with confidentially, allowing legitimate senders to communicate in a secure manner. In friendly jamming security technique research, security metrics are defined to demonstrate the effectiveness and validity of jamming. Friendly jamming security techniques include a method of using anti-jamming that automatically arranges multiple friendly jammers to deliver information safely between moving objects in a mobile communication network [16–18].

#### *2.3. Friendly Jamming Security Model for Mobility Environment*

Research is underway for friendly jamming security models using multiple unmanned aerial vehicles (UAVs) in mobile communication networks. The Friendly UAV Jamming (Fri-UJ) method has shown that the eavesdropping probability decreases as the number of jammers deployed near the protection zone increases because the signal to interference and noise ratio (SINR) of the eavesdropping device drops through the transmitted signals of the jammers [19]. In addition, there is a study that proved its effectiveness by applying Fri-UJ to IoT technology in the medical field [20]. However, using the Fri-UJ, as the number of UAVs increases, the number of jammers increases, thereby increasing the loss cost, and in practice, making it difficult to install an infinite number of jammers. Therefore, research is required to compare, analyze, and propose an effective friendly jamming model in a mobile communication environment based on the size of the secure zone, the number of jammers, and the security efficiency relative to cost.

In addition, research on location optimization considering the mobility of UAVs is being conducted. The location of the drone is important in order to reduce the waiting time and delay of the user or improve the quality of the service provided to users [21]. In [22], the disturbance power intensity and power trajectory path that occur when protecting legitimate nodes located on the ground using friendly jamming signals were studied. At that time, it was calculated by estimating the location of the eavesdropper, but the applicability is low because it is difficult to determine the location of the eavesdropper in the actual situations. Accordingly, in [23], randomness was modeled without identifying the location of the eavesdropper. At that time, in order to maximize the secret ratio of several legitimate receivers, the area was determined using the signal-to-noise ratio (SNR). However, as the confidentiality of the receiver in the security area is not guaranteed equally, some devices have a high potential for eavesdropping. In addition, when the optimal location is determined, only some factors change depending on the situation, so the area is not flexible and the mobility of UAVs cannot be utilized. In the case of battery usage, it is inefficient because jamming signals must be transmitted all the time.

Table 1 shows a summary of prior studies related to friendly jamming. In the table header, "Paper Title" refers to the title of the study, and "Research Topic" refers to the keywords of the study. "Number of Jammers" refers to the number of jammers used in the model: Here, "single" indicates that the model used one jammer, and "multiple" indicates that the model used two or more jammers. In addition, "Main Idea" refers to the model proposed in each paper, and "Limitation" refers to those analysis results that are a limitation of the indicated study. The models in the prior studies are inefficient and inflexible in terms of creating a secure zone for secure wireless communication because they consume large amounts of power and have no mobility. In this study, however, we proposed a model that creates an efficient and flexible secure zone by using only three drones that transmit friendly jamming signals, and we demonstrated the effectiveness of the model through simulations and field experiments.


**Table 1.** Comparison of prior studies on friendly jamming.

#### **3. Friendly Jamming Technique**

Equation (1) shows the information leakage rate (ILR) metrics defined for use as performance evaluation metrics of friendly jamming. The relationship between the bit error rate (BER) and friendly jamming was defined using the ILR. If the BER exceeds 0.5, then the ILR is 0, indicating that it is impossible to extract information because the BIR is high [24–26]. By contrast, if the BER is less than or equal to 0.5, then information extraction is possible, and ILR has a value greater than or equal to 0. As the ILR increases, the security decreases, indicating that more information can be extracted.

$$\text{BER} = 0, \text{ if BER} > 0.5\\\text{LLR} = 1 - \frac{\text{BER}}{0.5}, \text{ if BER} \le 0.5\tag{1}$$

Figure 1 shows the network configuration for verifying the security performance of friendly jamming. Figure 1a shows a friendly jamming technique model, which uses an open channel to send data (*h* ∗ *sd*) to the sender node (Source, S) or receiver node (Destination, D) that uses a single antenna. Here, the eavesdropping device (Eve, E) also receives the data (*h* ∗ *se*). A drone (Jammer, J) located near the source node transmits friendly jamming signals (*h* ∗ *je*) to form a jamming zone. The eavesdropping quality of the eavesdropping device in the friendly jamming zone deteriorates as an effect of jamming, reducing the possibility of eavesdropping.

Figure 1 shows the network configuration for verifying the security performance of friendly jamming. Figure 1a shows a friendly jamming technique model, which uses an

<sup>∗</sup> ). A drone (Jammer, J) located near the source node transmits friendly jam-

<sup>∗</sup> ) to form a jamming zone. The eavesdropping quality of the eavesdrop-

tion, D) that uses a single antenna. Here, the eavesdropping device (Eve, E) also receives

ping device in the friendly jamming zone deteriorates as an effect of jamming, reducing

<sup>∗</sup> ) to the sender node (Source, S) or receiver node (Destina-

**Figure 1.** Network configuration for friendly jamming: (**a**) friendly jamming model and (**b**) CFJ-DMZ model. **Figure 1.** Network configuration for friendly jamming: (**a**) friendly jamming model and (**b**) CFJ-DMZ model.

#### **4. CFJ-DMZ Model 4. CFJ-DMZ Model**

open channel to send data (ℎ௦ௗ

the possibility of eavesdropping.

the data (ℎ௦

ming signals (ℎ

This section introduces the CFJ-DMZ model that forms a secure zone by using the mobility of devices and the friendly jamming method. Three drones transmit jamming signals to the outside of the secure zone. These cooperative jamming signals reduce the eavesdropping probability of eavesdroppers and facilitate secure communication in the secure zone. This section introduces the CFJ-DMZ model that forms a secure zone by using the mobility of devices and the friendly jamming method. Three drones transmit jamming signals to the outside of the secure zone. These cooperative jamming signals reduce the eavesdropping probability of eavesdroppers and facilitate secure communication in the secure zone.

Figure 1b shows the CFJ-DMZ model proposed in this paper. The CFJ-DMZ network consists of a source node (S), destination node (D), arbitrary eavesdropper (E), and three friendly jamming drones (J). The drones transmit jamming signals after forming a secure zone, as shown in Figure 1b. The source and destination nodes communicate D2D inside the secure zone. As it is difficult to use a multi-antenna communication interface for lowpower lightweight drones, we considered a communication method using a single an-Figure 1b shows the CFJ-DMZ model proposed in this paper. The CFJ-DMZ network consists of a source node (S), destination node (D), arbitrary eavesdropper (E), and three friendly jamming drones (J). The drones transmit jamming signals after forming a secure zone, as shown in Figure 1b. The source and destination nodes communicate D2D inside the secure zone. As it is difficult to use a multi-antenna communication interface for lowpower lightweight drones, we considered a communication method using a single antenna in this study.

tenna in this study. Figure 2 shows the flowchart of a scenario for the CFJ-DMZ model, which consists of three stages. Figure 2 shows the flowchart of a scenario for the CFJ-DMZ model, which consists of three stages. *Sensors* **2022**, *22*, x FOR PEER REVIEW 6 of 17

**Figure 2.** Simulation flow chart. **Figure 2.** Simulation flow chart.

In the first stage, three drones are used to form a secure zone. When D2D communication is determined, the receiver (D) delivers its location coordinate information using In the first stage, three drones are used to form a secure zone. When D2D communication is determined, the receiver (D) delivers its location coordinate information using GPS

GPS to the sender (D). In addition, S sends the determined location information to D and moves to the determined location, and D uses the arriving coordinate information of S

Moreover, D creates a virtual circle around its position. Here, given that the maximum transmittable distance according to the strength of the transmitted jamming signals of J is

2

Here, D selects three arbitrary points to make an equilateral triangle on the virtual circle created by centering on the position of D and sending the selected location to J. Each J moves to the received location. The J that arrives at each location moves to D until their respective jamming signals are no longer caught while transmitting to adjust the size of the secure zone. After the secure zone is created, J stops sending the jamming signals and

The second stage is the jamming signal transmission stage. After arriving at the final target coordinates, S sends the jamming signal transmission start time and the jamming signal transmission maintenance time to both D and J. The three units of J transmit jamming signals simultaneously based on the time information, and S and D can send secure data during the jamming signal transmission maintenance time. As such, J sends jamming signals only when communication is made according to the data transmission time. If jamming signals are continuously transmitted irrespective of the data transmission, the security inside the secure zone improves. However, continuous jamming signals interfere with the communication of other nearby transmitting and receiving objects [27]. Furthermore, it is inefficient to transmit jamming signals continuously in terms of energy. In the CFJ-DMZ, J therefore sends jamming signals only when S and D are communicating to minimize the effects of such signals on other nearby sender and receiver nodes and to use their batteries efficiently. In addition, it is assumed that the secure zone is formed only when the legitimate nodes exchange confidential information with the help of surrounding UAVs used for other purposes. Therefore, the cost of friendly jamming drones for CFJ-DMZ was not considered in this paper. On the other hand, very small control logic can be added to friendly jamming drones and the ground user's hardware to implement the pro-

posed method, but it is assumed that the added hardware cost is trivial.

<sup>3</sup> √3 (2)

Radius =

x, the radius of the circle is as shown in Equation (2):

waits until S arrives at the final target coordinates.

to the sender (D). In addition, S sends the determined location information to D and moves to the determined location, and D uses the arriving coordinate information of S and the current coordinate information of D to calculate the locations of the drones (J). Moreover, D creates a virtual circle around its position. Here, given that the maximum transmittable distance according to the strength of the transmitted jamming signals of J is x, the radius of the circle is as shown in Equation (2):

$$\text{Radius} = \frac{2}{3}\sqrt{3} \text{x} \tag{2}$$

Here, D selects three arbitrary points to make an equilateral triangle on the virtual circle created by centering on the position of D and sending the selected location to J. Each J moves to the received location. The J that arrives at each location moves to D until their respective jamming signals are no longer caught while transmitting to adjust the size of the secure zone. After the secure zone is created, J stops sending the jamming signals and waits until S arrives at the final target coordinates.

The second stage is the jamming signal transmission stage. After arriving at the final target coordinates, S sends the jamming signal transmission start time and the jamming signal transmission maintenance time to both D and J. The three units of J transmit jamming signals simultaneously based on the time information, and S and D can send secure data during the jamming signal transmission maintenance time. As such, J sends jamming signals only when communication is made according to the data transmission time. If jamming signals are continuously transmitted irrespective of the data transmission, the security inside the secure zone improves. However, continuous jamming signals interfere with the communication of other nearby transmitting and receiving objects [27]. Furthermore, it is inefficient to transmit jamming signals continuously in terms of energy. In the CFJ-DMZ, J therefore sends jamming signals only when S and D are communicating to minimize the effects of such signals on other nearby sender and receiver nodes and to use their batteries efficiently. In addition, it is assumed that the secure zone is formed only when the legitimate nodes exchange confidential information with the help of surrounding UAVs used for other purposes. Therefore, the cost of friendly jamming drones for CFJ-DMZ was not considered in this paper. On the other hand, very small control logic can be added to friendly jamming drones and the ground user's hardware to implement the proposed method, but it is assumed that the added hardware cost is trivial.

Finally, S and D communicate inside the secure zone formed by the jamming-signal transmission of J. Here, because the distance between S and D is close, the transmission signal strength of S is reduced. The three units of J transmit cooperative jamming signals. As a result, the probability of success of the eavesdroppers decreases, improving the security of D2D communication in the secure zone.

In this study, we conducted simulations and field experiments to prove the security of the CFJ-DMZ model. Octave was used for the simulations, and Raspberry Pi 3 was used in the field experiments. Section 5 discusses the security verification process and results. The experiments were focused on proving the security improvement inside the secure zone. Therefore, as a part of the process under the CFJ-DMZ scenario, we assumed that the source node has moved and that the friendly jamming drones have completed moving to form a secure zone.

#### **5. Experiment**

#### *5.1. Simulation*

#### 5.1.1. Effect of Friendly Jamming

The CFJ-DMZ model proposed in this paper uses three drones as friendly jammers to protect the confidentiality of D2D communication. At this time, the three drones are theoretically the smallest number to make a two-dimensional space, and they protect the communication of the two legitimate nodes by forming a cost-performance effective secure zone. The jamming effect on the eavesdropper is affected by the distance between the

source node and the eavesdropper (Source − Eve distance, dSE) and the distance between the jamming drones and the eavesdropper (Drone − Eve distance, dDE). Figure 3 shows the BER of the eavesdropper according to changes in dSE and dDE measured using the simulation. The BER of the eavesdropper was measured by changing each distance from 1 to 100 m. As a result, we found that when dDE is shorter than dSE, the BER of the eavesdropper increases, reducing the communication quality of the eavesdropper. source node and the eavesdropper (Source − Eve distance, dSE) and the distance between the jamming drones and the eavesdropper (Drone − Eve distance, dDE). Figure 3 shows the BER of the eavesdropper according to changes in dSE and dDE measured using the simulation. The BER of the eavesdropper was measured by changing each distance from 1 to 100 m. As a result, we found that when dDE is shorter than dSE, the BER of the eavesdropper increases, reducing the communication quality of the eavesdropper.

*Sensors* **2022**, *22*, x FOR PEER REVIEW 7 of 17

curity of D2D communication in the secure zone.

to form a secure zone.

5.1.1. Effect of Friendly Jamming

**5. Experiment**  *5.1. Simulation* 

Finally, S and D communicate inside the secure zone formed by the jamming-signal transmission of J. Here, because the distance between S and D is close, the transmission signal strength of S is reduced. The three units of J transmit cooperative jamming signals. As a result, the probability of success of the eavesdroppers decreases, improving the se-

In this study, we conducted simulations and field experiments to prove the security of the CFJ-DMZ model. Octave was used for the simulations, and Raspberry Pi 3 was used in the field experiments. Section 5 discusses the security verification process and results. The experiments were focused on proving the security improvement inside the secure zone. Therefore, as a part of the process under the CFJ-DMZ scenario, we assumed that the source node has moved and that the friendly jamming drones have completed moving

The CFJ-DMZ model proposed in this paper uses three drones as friendly jammers to protect the confidentiality of D2D communication. At this time, the three drones are theoretically the smallest number to make a two-dimensional space, and they protect the communication of the two legitimate nodes by forming a cost-performance effective secure zone. The jamming effect on the eavesdropper is affected by the distance between the

**Figure 3.** BER based on distance of Source-Eve and Drone-Eve.

#### **Figure 3.** BER based on distance of Source-Eve and Drone-Eve. 5.1.2. Evaluation Environments

5.1.2. Evaluation Environments The evaluation simulator was implemented using Octave v.6.1. The simulation was conducted in a PC environment with the Windows 10 operating system, 8 GB of RAM, and an Intel i5-7200U CPU. In the simulation, a free space of 200 m × 200 m was formed, and the location of nodes was randomly arranged to prove the effectiveness of the CFJ-DMZ. The experimental environment was set as a free space that did not take into account The evaluation simulator was implemented using Octave v.6.1. The simulation was conducted in a PC environment with the Windows 10 operating system, 8 GB of RAM, and an Intel i5-7200U CPU. In the simulation, a free space of 200 m × 200 m was formed, and the location of nodes was randomly arranged to prove the effectiveness of the CFJ-DMZ. The experimental environment was set as a free space that did not take into account the influence of air or other radio waves. In addition, the maximum transmission power of the transmitting node, receiving node, and friendly jamming drone in the simulation was 24 dBm each.

We configured four experimental settings to measure the leaked amount of information according to the mobility of the device and the friendly jamming technique. Figure 4 shows each experimental configuration.

After measuring the locations of the source node (S) and destination node (D), the locations of three drones (J) were calculated. The blue circles in Figure 4b,d represent the range in the cooperative jamming signals. A total of six eavesdropping nodes (eve1–6) were created, and the eavesdropping nodes (E) were located at arbitrary coordinates. The locations of the eavesdroppers were the same in all experiments for an effective comparison between the experiments. Table 2 shows a summary of the location of each node. In the simulation experiments, the distance from the source node to the destination source was set to 48.413 m in Figure 4a,b and to 2 m in Figure 4c,d.

was set to 48.413 m in Figure 4a,b and to 2 m in Figure 4c,d.

**Figure 4.** Network configuration for CFJ-DMZ simulation: (**a**) None mobility and None friendly jamming, (**b**) None mobility and Friendly jamming, (**c**) Mobility and None friendly jamming, and proposed scheme (**d**) Mobility and Friendly jamming. **Figure 4.** Network configuration for CFJ-DMZ simulation: (**a**) None mobility and None friendly jamming, (**b**) None mobility and Friendly jamming, (**c**) Mobility and None friendly jamming, and proposed scheme (**d**) Mobility and Friendly jamming.

the influence of air or other radio waves. In addition, the maximum transmission power of the transmitting node, receiving node, and friendly jamming drone in the simulation

We configured four experimental settings to measure the leaked amount of information according to the mobility of the device and the friendly jamming technique. Figure

After measuring the locations of the source node (S) and destination node (D), the locations of three drones (J) were calculated. The blue circles in Figure 4b,d represent the range in the cooperative jamming signals. A total of six eavesdropping nodes (eve1–6) were created, and the eavesdropping nodes (E) were located at arbitrary coordinates. The locations of the eavesdroppers were the same in all experiments for an effective comparison between the experiments. Table 2 shows a summary of the location of each node. In the simulation experiments, the distance from the source node to the destination source

was 24 dBm each.

4 shows each experimental configuration.



The test cases of the experimental settings in Figure 4 can be summarized as shown in Table 3. In the table, "mobility" refers to with or without movement of the source node. If the mobility is O, it indicates a case in which the source node sends information after moving to near the destination node. If the mobility is X, it indicates a case in which the source node sends information without moving to near the destination node. When the source node has mobility, and the distance between the source and destination nodes is close, the transmission signal strength of the source node is reduced. "Friendly jamming" refers to whether the friendly jamming technique of the drone is used. If friendly jamming

is O, it indicates a case in which the drones transmit cooperative jamming signals when sending information. If it is X, it indicates a case in which friendly jamming signals are not transmitted. "Source-Destination distance" refers to the distance between the source and destination nodes. The CFJ-DMZ model corresponds to Figure 4d in which both mobility and friendly jamming occur.

**Table 3.** Environment setting of Figure 4.


The design process of the simulation is as follows. First, it is assumed that there is a channel *h* ∗ *sd* in the form of a complex conjugate between the source and destination nodes. It is also assumed that *P<sup>s</sup>* at the transmitter and the receiver, and the maximum transmission power of *P<sup>j</sup>* of the friendly jamming drones, are both 24 (dBm). Under these assumptions, the following equations are defined to measure the BER at each object.

First, the signal that the destination node receives from the source node in Equation (3) can be expressed based on the distance between the source and destination nodes (*h* ∗ *sd*), the distance between the jammer drones and the destination node (*h* ∗ *jd*), the maximum transmission power of the receiver (*Ps*), the maximum transmission power of the jammer drones (*P<sup>j</sup>* ), and the noise (*n<sup>d</sup>* ) [28]:

$$y\_{\varepsilon} = \mathcal{G}\sqrt{P\_{\text{s}}}h\_{\text{sd}}^{\*}s + \sqrt{P\_{\text{j}}}h\_{\text{j}d}^{\*}q + n\_{d} \tag{3}$$

where the channel coefficient *h* is shown by Equation (4). In addition, *d* is the distance between the two communication nodes, *e* is a uniformly distributed random number *a* + *bi*, and *c* is the path loss exponent.

$$h = (d)^{\frac{-c}{2}}e\tag{4}$$

The amplification scale vector *G* can be shown through Equation (5), where *N* refers to the Gaussian noise.

$$G = \frac{1}{\sqrt{P\_s \left| h\_{sd}^\* \right|^2 + N}}\tag{5}$$

Finally, the signal received by the eavesdropper in Equation (6) can be represented by the distance between the source node and the eavesdropper (*h* ∗ *se*), the distance between the jammer drones and the eavesdropper (*h* ∗ *je*), the maximum transmission power of the receiver (*Ps*), the maximum transmission power of the jammer drones (*P<sup>j</sup>* ), and the noise (*n<sup>d</sup>* ).

$$y\_{\varepsilon} = \mathbb{G}\sqrt{P\_s}h\_{\text{se}}^\* + \sqrt{P\_j}h\_{j\varepsilon}^\*q + n\_{\varepsilon} \tag{6}$$

For the BER of the eavesdropper node in each experiment, we used an average of 1000 times, as shown in Equation (7). In Figure 4a,c, where there is no jamming signal, *G* p*Pjh* ∗ *je* ∗ *JamSymbols* is calculated as zero.

$$y\_{\varepsilon} = \mathcal{G}\sqrt{P\_{\sf s}}h\_{\mathcal{s}\varepsilon}^{\*} \ast \text{ TrustSymbols} + \mathcal{G}\sqrt{P\_{\sf j}}h\_{\mathcal{j}\varepsilon}^{\*} \ast \text{ImSymSymbols} \tag{7}$$

Table 4 shows the definitions of the parameters used in the simulation pseudo-code, and Algorithm 1 shows the simulation pseudo-code itself. The detailed operating principle of Algorithm 1 is as follows.


**Table 4.** Defined parameters for pseudo-code.



In line 1–5, the distance between the jamming drones and eves is measured, and the measured distance is used for the influence of jammer on eve. Line 6 refers to the number of simulation repetitions, and the average BER is evaluated by repeating 1000 times. The c of line 7 is the path loss index and is used to evaluate the channel coefficient. Line 8–24 is a function of calculating the average BER of the eavesdropper and is repeatedly performed by a value specified in line 6. This function receives the size of data to be transmitted, the path loss index, the maximum source power of jammer, and the distance between source node and eavesdropper, source node and destination node, and jammer and receiver as parameters. The e of line 10 is a random number in the form of a complex number and is used to evaluate the channel coefficient. In line 11–13, channel coefficients are calculated using c and e. The G of line 14 is a scaling coefficient amplified according to the distance between the source node and the destination node. In line 15–16, 0 and 1 are randomly generated as the size of the data to be transmitted and then mapped in a complex number form. In line 17–18, data to be used as jamming signals are randomly generated and then mapped in complex number form. Line 19 refers to a signal received by the eavesdropper, and line 20 refers to the bit formed as signals. In line 21, the BER is calculated using the

23: 24:

14: ←

10: ← (௧௦)

1

ට൫௧|ೞ|మ൯

ଶ ∗

ଶ ∗

15: ← ௧௦ 0,1

17: ← ௧௦ 0,1

20: ←

ଶ

11: ℎ௦ ← ି

12: ℎ௦ௗ ← ି

13: ℎ ← ି

bits sent by the source node and the bits received by eve. In line 22, the average BER is evaluated.

௧௦ ൗ ,

#### 5.1.3. Results of Simulation 5.1.3. Results of Simulation

16: ← ℎ

ℎ

22: ← ℎ ℎ ℎ ℎ ℎ

18: ← ℎ 19 ← ∗ ඥ௧ ∗ ℎ௦ ∗ + ඥ ∗ ℎ ∗

*Sensors* **2022**, *22*, x FOR PEER REVIEW 11 of 17

The source node transmitted 100,000 SignalSymbol data, and the BER was measured for the destination node and the eavesdroppers. The average BER was obtained by repeating the simulation 1000 times. In every case, the average BER of the destination node was 0, which means that all data sent by the source node were received. Figure 5 shows a graph that applies the average BER results of the eavesdroppers to the ILR metrics defined in Section 3. The source node transmitted 100,000 SignalSymbol data, and the BER was measured for the destination node and the eavesdroppers. The average BER was obtained by repeating the simulation 1000 times. In every case, the average BER of the destination node was 0, which means that all data sent by the source node were received. Figure 5 shows a graph that applies the average BER results of the eavesdroppers to the ILR metrics defined in Section 3.

**Figure 5.** Simulation results: conventional schemes (**a**–**c**) and proposed scheme (**d**). **Figure 5.** Simulation results: conventional schemes (**a**–**c**) and proposed scheme (**d**).

In Figure 4a, the ILR of every eavesdropper is 1. Therefore, if the devices have no mobility and the friendly-jamming technique is not used, the eavesdropping probability of the eavesdroppers is high. In Figure 4a, the ILR of every eavesdropper is 1. Therefore, if the devices have no mobility and the friendly-jamming technique is not used, the eavesdropping probability of the eavesdroppers is high.

In Figure 4b, the characteristics of Wang's Friendly UAV Jamming model (Fri-UJ) [19] are included. Fri-UJ reduces the eavesdropping probability by using multiple UAVs as jammers to transmit friendly jamming signals in a mobile communication network environment. However, unlike CFJ-DMZ, the mobility of the source node is not considered. At this time, the ILR of eve2 and eve5, which are within the cooperative jamming signal range of the drones, is 0.754 and 0.960, respectively. The ILR of the eavesdroppers outside the cooperative jamming signal range of the drones is 1. Therefore, if the friendly jamming In Figure 4b, the characteristics of Wang's Friendly UAV Jamming model (Fri-UJ) [19] are included. Fri-UJ reduces the eavesdropping probability by using multiple UAVs as jammers to transmit friendly jamming signals in a mobile communication network environment. However, unlike CFJ-DMZ, the mobility of the source node is not considered. At this time, the ILR of eve2 and eve5, which are within the cooperative jamming signal range of the drones, is 0.754 and 0.960, respectively. The ILR of the eavesdroppers outside the cooperative jamming signal range of the drones is 1. Therefore, if the friendly jamming technique is used without the mobility of the devices, the eavesdropping probability of eavesdroppers is high.

In Figure 4c, the ILR of every eavesdropper is 1. As the distance between the source and destination nodes is close, the signal strength of the source node is reduced. However, it does not have a significant impact on the communication quality of the eavesdroppers, because the simulation environment is a free space. Therefore, if the devices have mobility but the friendly jamming technique is not used, the eavesdropping probability of the eavesdroppers is high.

In Figure 4d, which is the case of the CFJ-DMZ model, the ILR of eve2 and eve5, which are within the cooperative jamming signal range of the drones, is 0.024 and 0.014, respectively. Furthermore, the ILR of the eavesdroppers outside the cooperative jamming signal range of the drones is close to zero, and the average ILR of all eavesdroppers is 0.03. Therefore, when the mobility of the devices and the friendly-jamming technique are used together, the communication quality of the eavesdropper decreases. Compared to Figure 4b that includes the characteristics of Fri-UJ, the average ILR is reduced by 92%, and compared to Figure 4a,c, the average ILR is reduced by 97%, facilitating secure communication in the secure zone.

#### *5.2. Field Experiment 5.2. Field Experiment*

in the secure zone.

eavesdroppers is high.

eavesdroppers is high.

#### 5.2.1. Effect of Friendly Jamming 5.2.1. Effect of Friendly Jamming

In this section, we conducted experiments using Raspberry Pi. The friendly jamming drones and the eavesdroppers were also implemented using Raspberry Pi. In the communication between Raspberry Pi devices, packets were sent using the D2D communication method. The source node became the AP using the host mode, and the destination node was connected to the AP of the source node. The maximum transmission power of each node was 24 dBm. Jamming signals were generated using the Ping of Death method. In this section, we conducted experiments using Raspberry Pi. The friendly jamming drones and the eavesdroppers were also implemented using Raspberry Pi. In the communication between Raspberry Pi devices, packets were sent using the D2D communication method. The source node became the AP using the host mode, and the destination node was connected to the AP of the source node. The maximum transmission power of each node was 24 dBm. Jamming signals were generated using the Ping of Death method.

technique is used without the mobility of the devices, the eavesdropping probability of

In Figure 4c, the ILR of every eavesdropper is 1. As the distance between the source and destination nodes is close, the signal strength of the source node is reduced. However, it does not have a significant impact on the communication quality of the eavesdroppers, because the simulation environment is a free space. Therefore, if the devices have mobility but the friendly jamming technique is not used, the eavesdropping probability of the

In Figure 4d, which is the case of the CFJ-DMZ model, the ILR of eve2 and eve5, which are within the cooperative jamming signal range of the drones, is 0.024 and 0.014, respectively. Furthermore, the ILR of the eavesdroppers outside the cooperative jamming signal range of the drones is close to zero, and the average ILR of all eavesdroppers is 0.03. Therefore, when the mobility of the devices and the friendly-jamming technique are used together, the communication quality of the eavesdropper decreases. Compared to Figure 4b that includes the characteristics of Fri-UJ, the average ILR is reduced by 92%, and compared to Figure 4a,c, the average ILR is reduced by 97%, facilitating secure communication

For the data, a string "1" consisting of a total of 256 bits was used, and as the preamble bits for synchronization, a string "a" consisting of 128 bits was used. Figure 6 shows how the preamble bits were processed. If "a" with less than 64 bits is received, it corresponds to a case in which the ILR is less than 0.5. Therefore, the data received through the corresponding packet are all processed as an unanalyzable state. If "a" with 64 or more bits is received, the number of error bits is obtained after removing the preamble data. The number of error bits is calculated by adding the number of lost bits and the number of unmatched bits. The number of lost bits is calculated by subtracting the number of receiving bits from the number of sending bits, and the number of unmatched bits is calculated by counting 1 after the sequential Xor. The BER of the destination node and the eavesdroppers is obtained using the error bits. For the data, a string "1" consisting of a total of 256 bits was used, and as the preamble bits for synchronization, a string "a" consisting of 128 bits was used. Figure 6 shows how the preamble bits were processed. If "a" with less than 64 bits is received, it corresponds to a case in which the ILR is less than 0.5. Therefore, the data received through the corresponding packet are all processed as an unanalyzable state. If "a" with 64 or more bits is received, the number of error bits is obtained after removing the preamble data. The number of error bits is calculated by adding the number of lost bits and the number of unmatched bits. The number of lost bits is calculated by subtracting the number of receiving bits from the number of sending bits, and the number of unmatched bits is calculated by counting 1 after the sequential Xor. The BER of the destination node and the eavesdroppers is obtained using the error bits.

*Sensors* **2022**, *22*, x FOR PEER REVIEW 12 of 17

**Figure 6.** Synchronization with preamble bit. **Figure 6.** Synchronization with preamble bit.

First, the friendly jamming model of Figure 1a was implemented to verify the effectiveness of the jamming signals. One jamming drone node was deployed based on the locations of the source and destination nodes. First, the friendly jamming model of Figure 1a was implemented to verify the effectiveness of the jamming signals. One jamming drone node was deployed based on the locations of the source and destination nodes.

The transmitted data of the source node, and the BER of both the destination node and the eavesdroppers, were measured. Table 5 shows the average BER obtained by repeatedly applying the model of Figure 1a 1000 times. The average BER of the destination node is zero. In other words, the destination node can communicate normally with the The transmitted data of the source node, and the BER of both the destination node and the eavesdroppers, were measured. Table 5 shows the average BER obtained by repeatedly applying the model of Figure 1a 1000 times. The average BER of the destination node is zero. In other words, the destination node can communicate normally with the source node because it is unaffected by the jamming signals. The average BER of the eavesdroppers located around the drone nodes (eve3, eve4, and eve5) is 0.975, 0.992, and 0.938, respectively, showing a result close to 1. This experimental result shows that the legitimate destination node can safely deliver information while reducing the amount of information leaked to the surrounding eavesdroppers by friendly jamming. Furthermore, the average BER of the eavesdroppers that are not located around the drones (eve1, eve2, and eve6) is 0.389, 0.012, and 0.562, respectively, demonstrating relatively low values. Therefore, if the friendly jamming technique is used, the eavesdropping probability of the eavesdroppers decreases, improving the communication security.

**Table 5.** Results of friendly jamming (field experiment).


5.2.2. Experimental Settings

The field experiment was executed using Raspberry Pi 3 Model B+. The experiment was conducted in an empty lot of 50 m × 50 m with a Quad-core 64-bit ARMv8 CPU and

1 GB of RAM. At this time, the empty lot was used to minimize the influence of other radio waves. The maximum transmission power of Raspberry Pi is 24 dBm, so the maximum transmission power of the source node, receiving node, and friendly jamming drone used in the experiment is 24 dBm each. The field experiments were conducted under the same environmental configuration as used in the simulations. That is, the devices were placed according to the experimental settings of Figure 4. Similar to the simulations, to achieve an effective comparison between the experiments, the locations of the eavesdropper were the same in every experiment. Furthermore, considering the signal ranges of the source node and the jamming nodes, we placed the eavesdroppers at the locations where the effects of the signals received according to the experimental environment were the same as those received by the eavesdroppers during the simulations. 1 GB of RAM. At this time, the empty lot was used to minimize the influence of other radio waves. The maximum transmission power of Raspberry Pi is 24 dBm, so the maximum transmission power of the source node, receiving node, and friendly jamming drone used in the experiment is 24 dBm each. The field experiments were conducted under the same environmental configuration as used in the simulations. That is, the devices were placed according to the experimental settings of Figure 4. Similar to the simulations, to achieve an effective comparison between the experiments, the locations of the eavesdropper were the same in every experiment. Furthermore, considering the signal ranges of the source node and the jamming nodes, we placed the eavesdroppers at the locations where the effects of the signals received according to the experimental environment were the same as those received by the eavesdroppers during the simulations.

#### 5.2.3. Experiment Results 5.2.3. Experiment Results

5.2.2. Experimental Settings

*Sensors* **2022**, *22*, x FOR PEER REVIEW 13 of 17

decreases, improving the communication security.

**Table 5.** Results of friendly jamming (field experiment).

source node because it is unaffected by the jamming signals. The average BER of the eavesdroppers located around the drone nodes (eve3, eve4, and eve5) is 0.975, 0.992, and 0.938, respectively, showing a result close to 1. This experimental result shows that the legitimate destination node can safely deliver information while reducing the amount of information leaked to the surrounding eavesdroppers by friendly jamming. Furthermore, the average BER of the eavesdroppers that are not located around the drones (eve1, eve2, and eve6) is 0.389, 0.012, and 0.562, respectively, demonstrating relatively low values. Therefore, if the friendly jamming technique is used, the eavesdropping probability of the eavesdroppers

**Node Destination Node eve1 eve2 eve3 eve4 eve5 eve6**  BER metric 0 0.389 0.012 0.975 0.992 0.938 0.562

The field experiment was executed using Raspberry Pi 3 Model B+. The experiment was conducted in an empty lot of 50 m x 50 m with a Quad-core 64-bit ARMv8 CPU and

The BERs of the destination node and the eavesdroppers, respectively, were measured when the source node sent the data. The average BER was obtained by measuring the BER 1000 times repeatedly. In every case, the average BER of the destination node was 0, which means that the data sent by the source node were all received. Figure 7 shows a graph applying the average BER results of the eavesdroppers to the ILR metrics defined in Section 3. The BERs of the destination node and the eavesdroppers, respectively, were measured when the source node sent the data. The average BER was obtained by measuring the BER 1000 times repeatedly. In every case, the average BER of the destination node was 0, which means that the data sent by the source node were all received. Figure 7 shows a graph applying the average BER results of the eavesdroppers to the ILR metrics defined in Section 3.

**Figure 7.** Field experiment results; conventional schemes (**a**–**c**) and proposed scheme (**d**). **Figure 7.** Field experiment results; conventional schemes (**a**–**c**) and proposed scheme (**d**).

In Figure 4a, the ILR of the eavesdroppers, except for eve1 and eve3, is 1. The ILR of eve1 and eve3 is 0.639 and 0.819, respectively. Therefore, if the devices have no mobility In Figure 4a, the ILR of the eavesdroppers, except for eve1 and eve3, is 1. The ILR of eve1 and eve3 is 0.639 and 0.819, respectively. Therefore, if the devices have no mobility and the friendly jamming technique is not used, the eavesdropping probability of the eavesdroppers is high. Furthermore, it was found that, as the distance between the eavesdropper and the source node increases, the eavesdropping probability of the eavesdropper decreases.

In Figure 4b, the ILR of every eavesdropper decreases because the cooperative jamming signals affect the communication quality of the eavesdroppers. The ILR of both eve2 and eve5, which are within the cooperative jamming signal range of the drones, is zero. However, the ILR of the eavesdroppers outside the cooperative jamming signal range of the drones is greater than or equal to 0.5. Therefore, if the friendly jamming technique is used without the mobility of the devices, the eavesdropping probability of the eavesdroppers outside the cooperative jamming signal range is high.

In Figure 4c, the transmission signal strength is reduced because the distance between the source and destination nodes is close. As a result, the ILR of eve1 and eve3 is 0.180 and 0.121, respectively. However, the ILR of eve2, eve4, eve5, and eve6, which are relatively close to the source node, is 0.732, 0.380, 0.433, and 0.338, respectively. Although the ILR of every eavesdropper decreases compared to that of Figure 4a, the ILR of some eavesdroppers is high. Therefore, if the devices have mobility but the friendly jamming technique is not used, the eavesdropping probability of the eavesdroppers is high.

In Figure 4d, which is the case of the CFJ-DMZ model, the ILR of every eavesdropper is zero. In other words, when the mobility of the devices and the friendly jamming technique are used together, the communication quality of the eavesdroppers decreases, reducing the eavesdropping probability. Therefore, secure communication is facilitated in the secure zone.

#### *5.3. Evaluation*

The results obtained from the simulations and the field experiments are as follows. Based on the ILR of eve1 and eve3 in Field Experiment Figure 4a, it was found that as the distance between the source node and the eavesdropper increases, the communication quality of the eavesdropper deteriorates.

Furthermore, based on the ILR of eve5 and eve6 in Figure 4b, it was found that as the eavesdropper reaches closer to the friendly jamming drones, the communication quality of the eavesdropper deteriorates.

A comparison of the average ILR of the eavesdroppers between Figure 4a–d shows the effect of the device mobility. It can be seen that when the source node moves to the receiving node to reduce the distance between the two devices and decreases the transmission signal strength, the communication quality of the eavesdroppers deteriorates.

In Figure 4d, where the mobility of the devices and the friendly jamming technique are used together, the average ILR of the eavesdroppers is 0.03 in the simulation results and zero in the field experiment results. This means that the CFJ-DMZ model can reduce the communication quality of eavesdroppers to reduce the eavesdropping probability in the zone where the possibility of eavesdropping is high. Therefore, the communication security in the secure zone improves.

#### **6. Conclusions**

In this paper, we proposed the CFJ-DMZ method, which improves the security of wireless communication by using the mobility of mobile IoT devices and jamming signals, and we verified its effectiveness in drone communication environments. The drones conducting the cooperative friendly jamming move to locations where a secure D2D communication will be performed and transmit jamming signals to form a secure zone. The formed secure zone can effectively prevent eavesdropping and is flexible because the location and size can be easily changed. Furthermore, because jamming signals are transmitted only when the data transmitter and receiver communicate, the effect of the jamming signals on other source and destination nodes is minimized, and the batteries are efficiently used. Through the CFJ-DMZ model-applied simulations and fields tests, the BER of the eavesdropping devices was measured, which confirmed that the receiving performance of the eavesdroppers deteriorated, reducing the normal packet reception rate. Furthermore, we defined the ILR as a metric for a security performance evaluation and confirmed experimentally that the information leakage decreased with the proposed scheme.

By integrating it with IoT networking environments across future social systems, including logistics, delivery, and unmanned moving objects, the proposed CFJ-DMZ method can be used not only for military drone communications but also as a model that can actually be commercialized. As a limitation of this study, the effects of jamming were examined in two-dimensional planes. To consider the intrinsic emission characteristics of RF, the effectiveness of the CFJ-DMZ method should also be verified in three dimensions. Furthermore, low latency is important for the proposed method to be applied to real-time systems, not only for delay-tolerant applications. However, in this study, time complexity could not be analyzed, and the study was conducted from the perspective of an information leakage ratio to verify whether confidential communication is possible using the proposed method. Therefore, experiments and verifications are required for various environments, including a case in which an eavesdropper enters the secure zone formed by the drones. In follow-up studies, we will make improvements on this limitation by applying actual environmental parameters to mathematical models and simulation models for assessment of

time and resource complexity. Based on these studies, we expect to design a more advanced security architecture and increase the level of security in confidential and complex zones.

**Author Contributions:** Conceptualization, Y.-J.L., J.-H.L. and I.-G.L.; methodology, H.-J.P., Y.-J.L. and J.-H.L.; software, Y.-S.S. and G.-H.J.; validation, Y.-S.S., H.-J.P. and Y.-J.L.; formal analysis, G.-H.J., Y.-S.S. and H.-J.P.; investigation, H.-J.P., Y.-J.L. and S.-W.Y.; resources, J.-H.L. and H.-J.P.; data curation, G.-H.J. and Y.-S.S.; writing—original draft preparation, Y.-J.L., H.-J.P. and Y.-S.S.; writing—review and editing, G.-H.J., J.-H.L., S.-W.Y. and I.-G.L.; visualization, Y.-S.S. and G.-H.J.; supervision, I.-G.L.; project administration, I.-G.L.; funding acquisition, I.-G.L. All authors have read and agreed to the published version of the manuscript.

**Funding:** This work was partly supported by a National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. 2020R1F1A1061107) and a Korea Institute for Advancement of Technology (KIAT) grant funded by the Korea Government (MOTIE) (P0008703, The Competency Development Program for Industry Specialist).

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

**Data Availability Statement:** Not applicable.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **Glossary/Nomenclature/Abbreviations**

The table summarizes the notation used in the paper.


#### **References**


**Tala Talaei Khoei \* , Shereen Ismail and Naima Kaabouch**

School of Electrical Engineering and Computer Science, University of North Dakota, Grand Forks, ND 58202, USA; shereen.ismail@ndus.edu (S.I.); naima.kaabouch@ndus.edu (N.K.)

**\*** Correspondence: tala.talaeikhoei@ndus.edu

**Abstract:** Unmanned aerial vehicles are prone to several cyber-attacks, including Global Positioning System spoofing. Several techniques have been proposed for detecting such attacks. However, the recurrence and frequent Global Positioning System spoofing incidents show a need for effective security solutions to protect unmanned aerial vehicles. In this paper, we propose two dynamic selection techniques, Metric Optimized Dynamic selector and Weighted Metric Optimized Dynamic selector, which identify the most effective classifier for the detection of such attacks. We develop a onestage ensemble feature selection method to identify and discard the correlated and low importance features from the dataset. We implement the proposed techniques using ten machine-learning models and compare their performance in terms of four evaluation metrics: accuracy, probability of detection, probability of false alarm, probability of misdetection, and processing time. The proposed techniques dynamically choose the classifier with the best results for detecting attacks. The results indicate that the proposed dynamic techniques outperform the existing ensemble models with an accuracy of 99.6%, a probability of detection of 98.9%, a probability of false alarm of 1.56%, a probability of misdetection of 1.09%, and a processing time of 1.24 s.

**Keywords:** unmanned aerial vehicles; global positioning system; GPS spoofing attacks; detection techniques; machine learning; dynamic selection; hyperparameter tuning

#### **1. Introduction**

The use of unmanned aerial vehicles (UAVs) in military and civilian applications has exponentially increased over the last decade. Military applications include inspection and patrol, surveillance, reconnaissance, area mapping, and strike and rescue missions. Civilian applications include multimedia shooting, agricultural monitoring, meteorological monitoring, disaster detection, traffic control, cargo transportation, delivery services, and emergency rescue. Middle and long-distance applications rely heavily on Global Positioning Systems (GPSs) for navigation and precise positioning tasks [1].

Huge technical advances in the design, control, and automation have been made over the last two decades; however, the security aspect of UAVs has been largely overlooked [2]. UAVs can be subject to several cyber-attacks, such as GPS spoofing and jamming, which can impact the safety of civilians and airspace. Several UAV security incidents were reported during warfare and conflicts in Iran, Ukraine, and Iraq. During these attacks, malicious users transmitted fake GPS signals with incorrect positional and timing data that could be easily detected, resulting in erroneous navigation. These signals are similar to those from by satellites and are indistinguishable from authentic GPS signals.

A number of techniques have been proposed to detect GPS spoofing attacks. These methods can be classified into three categories [3]: cryptography-based, signal processing methods, and external UAV characteristics. Cryptography techniques encrypt the GPS signals, which require a key to decrypt [4]. Techniques under the second category extract spatial and geometrical characteristics, or physical layer characteristics, such as angle-ofarrival, signal strength, signal phase, and discontinuities from legitime GPS signals. The

**Citation:** Talaei Khoei, T.; Ismail, S.; Kaabouch, N. Dynamic Selection Techniques for Detecting GPS Spoofing Attacks on UAVs. *Sensors* **2022**, *22*, 662. https://doi.org/ 10.3390/s22020662

Academic Editor: Omprakash Kaiwartya

Received: 21 November 2021 Accepted: 12 January 2022 Published: 15 January 2022

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

third category is based on external UAV characteristics, such as speed and acceleration, that that can be measured by the sensors of the UAV flight control system, such as barometer, inertial measurement unit (IMU), and compass.

These GPS spoofing detection methods have some drawbacks that limit their application. For instance, cryptographic methods are not practical for civil applications as they require encryption/decryption keys (GPS signals have to be unencrypted for these applications). Methods based on signal processing or external characteristics of UAVs may require additional hardware (sensors or antenna) or auxiliary equipment, apply changes to the interface specifications, or need extensive signal processing capabilities, which adversely affect real-time system performance or require additional communication overhead.

Inertial navigation systems (INS) techniques require continuous inertial sensor calibration as the error in position estimates and their covariance continuously grow without bounds. For the detection of GPS spoofing attacks, these techniques can be only used when the quality of sensors with respect of size and cost is high. Therefore, using such detection techniques for small drones is not possible [5]. In addition, using sensors, such as gyroscope and accelerator, also involves some limitations in detecting GPS spoofing attacks. For example, the accelerator can only measure changes in the velocity [6]. Furthermore, this sensor cannot measure the rotation around its own axis of movement. Therefore, it has to be used with a gyroscope to measure angular velocities. The accelerator is also sensitive to temperature, which makes it difficult for it to perform properly in different environmental situations of UAVs. Moreover, Gyroscope is a sensor that does not measure linear motion in any direction or any static angle of orientation [7]. Therefore, as these two sensors are the two main components of INS and IMU-based techniques, and based on their drawbacks, they cannot be used in detecting GPS spoofing attacks on UAVs [8].

Several studies based on traditional machine learning (ML) techniques have been proposed to classify and detect GPS spoofing attacks on UAVs. Examples of such techniques include artificial neural networks [2] and tree models [9]. Such models provide effective solutions for detecting these in the detection of GPS spoofing. Such models provide effective solutions in the detection of GPS spoofing [10].

Ensemble learning techniques are considered as one of the main developments in machine learning in the past decade as they perform better than traditional machine learning methods [11]. Examples of ensemble models that have been proposed for detecting cyber-attacks are bagging, boosting, and stacking [12]. The stacking classifiers apply meta-learning algorithms to select the best combinations of the base machine learning algorithms. The bagging methods use a combination of repetitive techniques to generate several sets from the original data and evaluate the performance simultaneously. In Boosting algorithms, the weight of observation is adjusted based on the last classification. Therefore, these three ensemble techniques can provide a better performance than a single conventional machine learning (ML) model. However, these models deal with some limitations, such as the difficulty of interpreting outputs and low or high rates of bias, which may lead to under or over-fitting issues.

Therefore, a holistic solution that can easily interpret and perform better than any single conventional ML model in detecting GPS spoofing attacks is known as multiple classifier systems (MCS). In this technique, a pool of classifiers is competing to provide the best prediction for a data sample, and the final result belongs to the most efficient base classifier. One example of MCS approach is the dynamic classifier selection (DCS) [13], which focuses on learning methods that automatically choose a subset of techniques in the prediction process. DCS focuses on fitting several ML classifiers on a training dataset and choosing the model that provides the best result in the prediction process based on specific proposed factors.

In this work, we propose two dynamic-based selection methods that detect GPS spoofing attacks on UAVs: Metric-Optimized Dynamic (MOD) selection and Weighted-Metric-Optimized Dynamic (WMOD) selection. We implement ten well-known supervised machine learning classifiers in both the proposed methods. These models are Support

Vector Machine, Naive Bayes, Decision Tree, K Nearest Neighbor, Linear Discriminative Analysis, Random Forest, Artificial Neural Network, Logistic Regression, Elastic Net, and AdaBoost. The two proposed classifier selection methods are trained and tested using a dataset with 13 GPS signals features built from real-time experiments and MATLAB attack simulations. The evaluation is conducted in terms of probability of detection (*P<sup>d</sup>* ), probability of false alarm (*Pf a*), probability of misdetection (*Pmd*), accuracy( *ACC*), and processing time.

The contributions of this paper are:


The remainder of this paper is organized as follows: Section 2 reviews the related works, while Section 3 illustrates the proposed architecture. Section 4 describes the materials applied in this work and highlights the methodology of the study. Section 5 discusses the simulation results. A conclusion is presented in Section 6.

#### **2. Related Work**

Several studies have been performed on GPS spoofing detection and mitigation methods. For instance, the authors of [7] proposed a GPS spoofing detection method that depends on the acceleration error calculated by estimating the acceleration from the GPS receiver and the acceleration measured from the IMU. In [9], the authors used IMU measurements (angle, velocity, and acceleration) and GPS data (longitude and latitude) in a two-step method that applies the XGBoost model and a Genetic Algorithm, to detect GPS-spoofing attacks. XGBoost was applied to learn the relationship between the IMU and GPS data, while the Genetic Algorithm was applied to tune the training parameters. An approach based on an artificial neural network was proposed in [2] to detect GPS spoofing signals. Several features, such as pseudo-range, doppler shift, and signal-to-noise ratio (SNR), were used to perform the GPS signal classification. Different neural network configurations were analyzed and tested. The proposed method revealed an acceptable efficiency in terms of probability of detection and probability of false alarm.

In [14], the authors proposed an anti-spoofing model that used linear regression to predict and model the optimal UAV route to its destination and used Long Short-Term Memory in the trajectory prediction. The model provides more than one detection scheme for GPS spoofing signals to improve UAV flight security and sensitivity to deception signal detection. Simulation experiments have determined that this method could enhance the ability to resist GPS spoofing without increasing hardware costs. Another GPS spoofing detection method was proposed in [15], based on the vision sensor combined with a UAV's sensors, monocular camera, and IMU. This method used vision sensors combined with IMU data to detect GPS spoofing. Another vision-based UAV spoofing detection method that utilized Visual Odometry was presented in [16], which uses the UAV camera since fake GPS signals would not alter its images. The UAV relative trajectory can be extracted from images using Visual Odometry. This extracted trajectory is compared with flight trajectory information obtained from GPS positions, to detect the spoofed signals.

In [17], the authors proposed a GPS spoofing-detection framework that needs minimal prior configuration and applies information fusion. The real-time detection scheme derives the current UAV location from IMU and compares it to the location information received by the GPS receiver to determine if the UAV system was experiencing a GPS spoofing attack. In [18], the authors proposed a new algorithm to handle GPS spoofing attacks that caused unknown sudden system state variable changes. The compensation of the GPS spoofing effect was manipulated using a prediction discrepancy based on a particle filter algorithm. The proposed algorithm decreases the effects of GPS spoofing errors and estimates the true position of the UAV in the presence of GPS spoofing attacks. In [19], the authors proposed a spoofing detection and classification algorithm based on Least Absolute Shrinkage and Selection Operator. They used some signal processing techniques to observe the decomposition of two code-phase values for authentic and spoofed signals using a certain threshold to mitigate false alarms. The proposed method achieves a promising detection error rate for a spoofer attack in nominal signal-to-noise ratio conditions.

In [20], the authors proposed a methodology that consists of several ML models with a set of values for K-folds where voting techniques are integrated to choose the learning model that achieves the highest accuracy. In [21], a hardware-based solution was proposed to detect GPS spoofing attacks. The authors demonstrated a simple method to detect hijacking based on gyroscopes measurements and GPS data. A switching mode resilient detection and estimation framework for GPS spoofing attacks has been studied in [22]. The authors tried to address the sensor drift issue by keeping the estimation errors to remain in a tolerable region with high probability.

Machine learning methods do not require additional hardware, which may be attractive for small civilian UAVs. For instance, in [23], the authors proposed an approach to detect UAV GPS spoofing attacks based on the analysis of state estimation using Support Vector Machine. The proposed method detects GPS spoofing attacks to some extent; however, the system experienced performance degradation during long attacks due to the interaction with the GPS sensor, especially with the Micro-Electro-Mechanical Systems sensors. In [24], a GPS spoofing detection method was proposed that leverages the uplink received signal strength measurements collected from base stations to identify the adaptive trustable residence area, which represents the trust region within which the UAV GPS position should be located to be classified as authentic or non-spoofed. In [3], the authors proposed a method for GPS spoofing attack detection based on a machine learning algorithm, Long Short-Term Memory, and compared the results to a method based on specifically designed UAV flight paths. This method can detect attacks well when the flight trajectory is not complicated. Table 1 provides a summary of existing studies in literature with their advantages and limitations.


**Table 1.** Existing Literature on Detecting GPS Spoofing on UAVs.


**Table 1.** *Cont.*


**Table 1.** *Cont.*

Although many spoofing detection techniques have been proposed in the literature, spoofers are continually evolving to produce new GPS spoofing attacks that are hard to detect, which increases the necessity to develop new mechanisms to prevent this kind of attack. Ensemble learning techniques can be a practical solution to address the limitations of the existing methods. In literature, there are no studies to investigate the performance of such approaches in detecting GPS spoofing attacks targeting UAVs; however, ensemble approaches, namely, bagging, boosting, and stacking, have been frequently utilized in detecting cyber-attacks in wireless communication systems. For instance, in [26], the authors proposed a stacked-based ensemble model to classify and detect attacks on wireless networks. The proposed approach consists of several base learning methods, namely, Support Vector Machine, Decision Tree, Random Forest, and Artificial Neural Network. The stacking approach outperforms the base learners. In [27], the authors compared different ensemble models, namely, bagging, boosting, and stacking, for predicting received signal power on UAVs. Their results demonstrate that the stacking model, including Support Vector Machine, Artificial Neural Network, and Gaussian Process, outperformed other base classifiers.

Dynamic classifier selection methods have been recently proposed as ensemble approaches that select the best performance ML model among all base models. To the best of our knowledge, no studies proposed such a technique for classifying and detecting GPS spoofing attacks on UAVs. Therefore, to fill the existing gap, two dynamic-based selection methods are proposed that use ten machine learning models. These methods select the ML method that provides best results to detect the presence or absence of an attack. To validate our proposed techniques and demonstrate that they provide optimal results, we compared our proposed methods with the three most known ensemble models, namely, bagging, boosting, and stacking, with our proposed techniques.

#### **3. Proposed Architecture**

The proposed system architecture is shown in Figure 1. This system consists of three phases: dataset building, data pre-processing and feature selection, and training and classification. For the dataset building, real-time experiments were conducted to collect real GPS signals, while attacks were generated through simulations. Features were identified and extracted from the real GPS signals, and the attack simulated signals [10]. The features for all samples are included in a dataset to be pre-processed. The second phase is the data pre-processing and feature selection, which focuses on missing value imputation, categorical data encoding, feature scaling, identifying correlated features, and discarding low importance features. In this study, we use feature scaling and transfer categorical feature values to numerical values to avoid any bias in the corresponding dataset.

Two feature selection techniques are applied: Spearman Correlation and Information Gain. The ensemble feature selection can simultaneously identify the correlated and low importance features and discard them from the corresponding dataset [28]. The primary aim of using ensemble feature selection is to decrease the dimensionality of the dataset and identify the most important features [29] that can enhance the performance of the proposed model.

For the training, testing, and classification phases, we implement ten traditional ML techniques: Support Vector Machine, Naive Bayes, Decision Tree, K Nearest Neighbor, Linear Discriminative Analysis, Random Forest, Artificial Neural Network, Logistic Regression, Elastic Net, and AdaBoost. To get the optimal results of each model, a hyperparameter tuning technique, Bayesian optimization, is used.

Two dynamic methods are implemented for detecting GPS spoofing attacks targeting UAVs. The proposed methods dynamically choose the classifier that achieves the best results for the considered performance metrics. Incoming signals are classified as authentic or spoofed in the prediction phase, and their probabilities is evaluated.

**Figure 1.** Overview of the Proposed Architecture.

#### **4. Methodology**

In the following, we discuss the dataset, data pre-processing, feature selection, description of the proposed models, and hyperparameter tuning that are used in this study, as follows:

#### *4.1. Dataset*

In this study, the used dataset was built in the work described was implemented in [10] . Real-time experiments and simulations were conducted to collect a dataset of authentic and spoofed signals at different dates in several locations. The hardware used in the implementation consisted of a universal software radio peripheral (USRP), a frontend active GPS antenna, and an I5-4300U laptop with 8 G RAM running with Ubuntu 16.04.7 LTS version. GPS attacks were simulated using MATLAB by considering three types of spoofing attacks with different complexity levels: simplistic, intermediate, and sophisticated. Each of these attacks impacts specific features of the GPS signals, such as Doppler Shift Measurement, Receiver Time, and Pseudo Range. In simplistic spoofing attacks, a fake GPS signals, which was unsynchronized with the authentic signals, was generated. In this case, higher Doppler Shift measurements were out of the normal range of ±20 Hz, leading to a signal drift. In this type of attacs, GPS spoofing signals are also transmitted at a higher power level, compared to that of authentic GPS signals, resulting in a higher Signal-to-Noise Ratio value.

In intermediate spoofing attacks, the attacker has a knowledge of UAV position. The intermediate attacker is able to control of the generated GPS signals. In this type of attack, the Doppler Shift Measurements and Pseudo Range values are kept within the normal ranges. In sophisticated attacks, the spoofer gains control over several channels of multiple synchronized antennas. This type of attack is the most threatening spoofing attack, due to the effect of multipath signals and the motion of the satellites and receiver.

Thirteen features were extracted from various receiver stages, starting from the tracking loop to the observable block. The extracted features from the received GPS signals with their short descriptions are listed in Table 2 . The corresponding dataset is balanced and contains 10,055 samples, of which 5028 are authentic signals and the remaining are equally divided between the three types of GPS spoofing attack signals. A sample of dataset is presented in Figure 2.


#### **Figure 2.** Sample of Dataset.

#### *4.2. Data Pre-Processing*

The dataset was previously pre-processed by identifying and removing any null, unknown, and noisy values during the missing value identification step [30]. The next step is to encode any categorical values to numerical values. There are only two categorical data values in our dataset, which represent the signals as attack or normal. For this purpose, we encode normal signals as 0 and spoofed signals as 1. Afterward, feature scaling is performed by applying normalization and standardization methods. Normalization can re-scale the values into ranges between 0 and 1. In this study, we use the power transformer technique based on the Yeo-Johnson transformer. Unlike other techniques, this method can

handle positive, negative, and zero data values. We also applied a simple standardization technique, which re-scaled the values to a mean of 0 and a standard deviation of 1.



#### *4.3. Feature Selection*

Ensemble feature selection techniques are widely used to enhance the robustness of feature selection techniques. These techniques are classified into two categories, namely, homogeneous and heterogeneous. In homogeneous ensemble feature selection, the same method is used with different sizes of training data, while heterogeneous ensemble feature selection mostly focuses on different feature selection methods with similar training datasets. This study employs a heterogeneous ensemble feature selection technique using two traditional feature selection techniques, namely, Spearman's Correlation and Information Gain. The goal of selecting these two feature-selection techniques is to remove correlated and unimportant features from the given dataset.

Spearman Correlation [31] primarily calculates the association and direction between each two features by calculating the score *τ* given by:

$$\pi = 1 - \frac{\text{6 } \sum\_{i=1}^{n} (d\_i)^2}{n(n^2 - 1)} \tag{1}$$

where *d<sup>i</sup>* is the difference between the two ranks of each observation, *i* is the index of the observation, and *n* is the number of observations. A feature is correlated if it attains a coefficient over 0.9. We consequently removed a feature from each pair of correlated features.

We also used the information-gain feature-selection technique, called mutual information [32], for feature importance to estimate the gain of each variable in terms of the target variable. The information gain, also known as entropy, is calculated for every feature; features with high entropy are selected as important features, and those with low entropy values are considered of low importance. Any feature that achieves an entropy less than 0.1 is discarded from the dataset in this work.

#### *4.4. Hyperparameter Tuning*

Several types of tuning techniques have been proposed in the literature; however, Bayesian optimization has emerged as an effective approach, outperforming other techniques such as random search and grid search since grid search suffers from the curse of dimensionality and random search is not suitable for training complex models [33–35]. Bayesian optimization can provide a practical solution to optimize functions using a computationally cheap surrogate model [36]. This approach can offer robust solutions for optimizing the black-box functions, applying a non-parametric Gaussian process to simulate unknown functions. A surrogate utility function, also known as the acquisition function, is another main component of Bayesian optimization, which is defined as a way to improve the optimality of the underlying function [37]. In this study, considering the benefits of Bayesian Optimization and shortcomings of other techniques, we employ this technique for optimization tuning.

#### *4.5. Description of the Proposed Methods*

Dynamic classifier selection techniques consist of a pool of homogenous or heterogeneous base classifiers. Homogenous classifiers are defined as using a set of classifiers that are of the same type built upon various data. In contrast, heterogeneous classifiers are designed using a group of classifiers belonging to various types built upon same data. In this work, we employed a set of heterogeneous base classifiers: Support Vector Machine (SVM), Naive Bayes (NB), Decision Tree (DT), K Nearest Neighbor (KNN), Linear Discriminative Analysis (LDA), Random Forest (RF), Artificial Neural Network (ANN), Logistic Regression (LR), Elastic Net (EN), and AdaBoost. The primary reason behind selecting heterogeneous classifiers is to increase the final model diversity without changing any model parameters [38].

We propose two dynamic selection methods for detecting GPS spoofing attacks on UAVs: MOD and WMOD classifiers, as shown in Figure 3. These methods focus on evaluating the ML models in terms of the probability of detection *P<sup>d</sup>* , probability of false alarm *Pf a*, probability of misdetection *Pmd*, and accuracy *ACC*. Figure 3a depicts the M base models and K performance metrics, i represents the base model index, and j represents the performance metric index. We initially calculate the performance metrics (*Ki*,*j*) for every base model *M<sup>i</sup>* to find the optimal results; then,we determine the count (*Ki*,*j*) where *M<sup>i</sup>* achieves the best results for every base model *M<sup>i</sup>* . As a model achieves higher *Pd*and ACC values and lower *Pf a* and *Pmd* values, the model is considered better at detecting GPS spoofing attacks; therefore, the model with higher *P<sup>d</sup>* and ACC, and lower *Pmd* and *Pf a*, will be selected for the final incoming GPS spoofing signal detection. This concept is implemented in the proposed MOD classifier approach since the algorithm will identify the model with the highest number of best metrics, using Max(*Ki*,*j*) for final detection.

MOD classifier is simpler to implement compared to other approaches, does not need extensive processing, does not require additional hardware, and has low time complexity. This classifier heavily depends on the selected base algorithm, which achieves the best metrics. As a result, no additional cost of processing or computational complexity was added to the overall algorithm. However, if two base models achieve the same number of best metrics , the MOD classifier will select one of the two classifiers trivially as the best model, in some cases when two base models achieve the same number of best metrics. To address this issue, we propose another approach: WMOD classifier. WMOD classifier, shown in Figure 3b, assigns a weight, w, for each performance metric to calculate a score using sum (*wi*,*j*) for each base model. In this model, a weight is assigned to each of the selected metrics based on their importance. We consider the importance of accuracy higher than the importance of *P<sup>d</sup>* , *Pf a*, and *Pmd*. Therefore, we assign a weight of 0.4 for accuracy, while other metrics, such as *P<sup>d</sup>* , *Pf a*, and *Pmd*, have each a weight of 0.2. We have to determine the count (*Ki*,*j*) for every model *M<sup>i</sup>* , where *M<sup>i</sup>* obtains the highest weights according to the defined weights. The model that achieves the best score is used for the final detection of any incoming signal.

**Figure 3.** Flowcharts of the Proposed Dynamic Selection Methods.

#### **5. Results**

We ran our simulation on intel core i7-10750H, CPU of 2.60 GHz, and 16.0 GB memory. We used four evaluation metrics to assess the proposed model's efficiency: the probability of detection (*P<sup>d</sup>* ), probability of false alarm (*Pf a*), probability of misdetection (*Pmd*), accuracy *(ACC)*, and processing time. These metrics were calculated using the following equations:

$$P\_d = \frac{T\_P}{T\_P + F\_N} \tag{2}$$

$$P\_{fa} = \frac{F\_P}{T\_F + F\_N} \tag{3}$$

$$P\_{md} = \frac{F\_N}{T\_N + F\_P} \tag{4}$$

$$A\text{CC} = \frac{T\_P + T\_N}{T\_P + T\_N + F\_P + F\_N} \tag{5}$$

where *T<sup>P</sup>* is the number of corrected predicted malicious flows, *T<sup>N</sup>* is the number of predicted normal flows, *F<sup>P</sup>* is the number of incorrectly predicted malicious flows, and *F<sup>N</sup>* is the number of incorrectly predicted normal flows.The processing time is defined as a time to train and test the classifiers. This metric highly depends on the ML model and dataset size. A three-fold cross-validation technique was applied to train 80% of the data and test 20% of the remaining dataset. The simulation analysis for the proposed dynamic methods was compared with the ten base selected classifiers in terms of the selected evaluation metrics.

Figure 4a,b show the results of the ensemble feature selection techniques, Spearman's Correlation and Information Gain, respectively. We discarded one feature of each pair of correlated features with a mutual coefficient > 0.9 as well as features of low importance with scores < 0.1 from the corresponding dataset. As one can see in Figure 4a, two pairs of features have a high correlation; the first pair, *D*O and *TCD*, has a correlation of 95%, and the second pair, TOW and RX, has a correlation of 94%. In addition, *DO* has a higher importance than *TCD*, and TOW has a higher importance than *RX*. Therefore, *TCD* and *RX* were discarded from the dataset. The remaining features are the relevant features selected to classify GPS signals.

**Figure 4.** Importance of Features based on Ensemble Feature Selection: Spearman's Correlation Coefficient and Information Gain.

Table 3 provides the parameter setting and best parameter results obtained after applying the Bayesian Optimization algorithm. As can be seen, we specify several parameter settings with multiple values to check the optimality of every ML model. The parameter setting for every ML model is selected based on the provided values in Scikit-learn. Scikit-learn tool is a simple and efficient library that provides the suitable implementation for training, testing , and validating ML models, along with parameter settings for every ML model. The lists of setting parameters are provided in the table. These values are applied to achieve the best performance for each of the individual models. For instance, the activation function in the NN model is set to identity, logistic, tanh, or relu, and our selected tuning technique identifies Tanh as the activation function that guarantees the highest performance for the NN model. The NN model also has other parameters, including solver and alpha, that are required to be provided by tuning technique to achieve optimal results. In addition, the NB model consists of a parameter, namely, var\_smoothing. This parameter is set to several values, provided in Scikit-learn tool. The Bayesian optimization technique identifies 1 <sup>×</sup> <sup>10</sup>−<sup>3</sup> as a hyperparameter, among other values, that ensures the best possible performance for the NB model. The same observations can be seen for the other selected models. To this end, these best parameters are used in training the selected models to ensure the optimality of the results.


**Table 3.** Parameter Setting Results.

Ten ML models are used with their best parameters' values in implementing the proposed dynamic classifiers. WMOD is proposed to handle a limitation of MOD. Such limitation occurs when two ML classifiers have the same number of metrics with the best results; therefore, two classifiers are selected as optimal. To address this issue, WMOD is proposed to return only the model with the best metric results.

Figure 5a provides the results of the proposed methods and the three ensemble models in terms of accuracy. As one can observe, the proposed MOD and WMOD dynamic methods provide the best results in terms of accuracy in comparison with bagging, boosting, and stacking-based ensemble models. As shown, the MOD and WMOD classifiers both have an accuracy of 99.8%. The stacking classifier has an accuracy of 99.7%, followed by bagging and boosting classifiers. Bagging model has an accuracy of 99.6%, while the boosting-based ensemble has the lowest accuracy of 99.56% compared to the other classifiers.

Figure 5b presents the results of the five models in terms of probability of detection. As can be seen, the proposed dynamic methods outperform the three ensemble models in terms of probability of detection with a slight difference. MOD and WMOD have the highest probability of detection of 99.9%, followed by the stacking, bagging, and boosting models. The stacking model has a probability of detection of 99.8%, bagging model has a probability of detection of 99.6%, and the boosting model has the lowest probability of detection of 99.35%.

**Figure 5.** Evaluation results of the selected methods in terms of accuracy, probability of detection, probability of misdetection, probability of false alarm, and processing time.

Figure 5c illustrates the results of the proposed dynamic selection methods and the three ensemble models in terms of the probability of misdetection. As one can observe, MOD and WMOD have an acceptable probability of misdetection; however, the lowest probability of misdetection belongs to the stacking model. The proposed dynamic selection methods have a probability of misdetection of 1.56%, while the stacking model has a probability of misdetection of 1.4%. The other two ensemble models, bagging and boosting, also provide acceptable probability of misdetection results of 1.67% and 1.76%, respectively.

Figure 5d provides the results of the 5 models in terms of the probability of false alarm. As can be seen, MOD and WMOD have the lowest and best probability of false alarm compared to the other ensemble models. These methods have a probability of false alarm of 1.09%, the bagging classifier has a probability of false alarm of 1.2%, and the stacking and boosting have a probability of false alarm of 1.6% and 1.64%, respectively.

Figure 5e provides the results of the selected models in terms of their processing time. As one can see, the MOD and WMOD classifiers require a processing time of 1.24 s, which is considered much lower in comparison with other techniques, such as bagging, boosting, and stacking. The bagging classifier has a processing time of 1.321 s, while the boosting classifier achieves a processing time of 1.987 s. The stacking classifier has a processing time of 5.432 s, which is significantly higher than MOD and WMOD.

The number of false positives (*fp*) is another important factor in evaluating models that compares the number of false positively predicted samples to total number of samples that are negatively predicted. Figure 6 provides the number of false positive for the highlighted methods. As one can observe, the MOD and WMOD provides the best number of false positives, followed by bagging, stacking, and boosting. The proposed dynamic selection methods have a number of false positives of 10.9 per second. In contrast, the bagging classifier has a number of false positives of 12 per secondm and the stacking and boosting classifiers have a number of false positives of 16 and 16.4 per second.

**Figure 6.** Number of False Positives for the Proposed Dynamic Selection Methods: MOD and WMOD Against the Classical Ensemble Techniques.

Table 4 provides the results of the proposed dynamic selection methods and the ensemble techniques, namely, bagging, boosting, and stacking. This table shows that MOD and WMOD have the best results in terms of accuracy, probability of detection, and probability of misdetection. In contrast, the stacking model provides the best result in terms of the probability of misdetection. It can be noticed that the proposed methods, MOD and WMOD, provide a probability of misdetection of 1.56%, which is higher than the stacking model by 0.16% considered as an insignificant difference. In contrast, the stacking model has a probability of false alarm of 1.6%, which is 0.51% higher than the probability of false alarm of these proposed methods. In addition, this stacking model has an accuracy of 99.7% and a probability of detection of 99.8%, which are 0.1% lower than the accuracy and probability of detection of the proposed dynamic selection methods.

As one can observe, the processing time of the proposed classifiers is 1.24 s, which is significantly lower than that of the other ensemble approaches. The bagging classifier has a processing time of 2.321 s, which is 1.081 s higher than that of MOD and WMOD. The boosting classifier has a processing time of 1.511 s, which is 0.271 s higher than that of the proposed classifiers. The stacking classifier has the worst processing time, which is 4.41 s higher than the processing time of MOD and WMOD.

To shed more light on the effectiveness of the proposed methods, we calculate the number of false positives (*fp*) for MOD and WMOD, followed by bagging, stacking, and boosting. The proposed methods provide a number of false positives of 10.9 per second, which is 1.1 lower than the bagging classifier, 5.5 lower than the boosting classifier, and 5.1 lower than the stacking classifier. To conclude, our proposed classifiers provide higher accuracy and probability of detection, and lower probability of misdetection, false alarm processing time, and the number of false alarms compared to the other classical ensemble techniques.


**Table 4.** Evaluation Results of the Proposed Dynamic Selection Methods and Ensemble Models.

In short, the key insights can be summarized as follows:


#### **6. Conclusions**

Interest in detecting GPS spoofing attacks on UAVs has increased significantly in the last decade, leading to considerable progress in different technologies. Several techniques have been proposed to identify and detect these vulnerabilities; however, this field of study still needs to address several challenges and limitations, such as high misdetection and false alarm rates. In this work, we used a one-stage heterogeneous ensemble feature selection to discard correlated and low importance features from the considered dataset using Spearman Correlation and Information Gain. As a result, two features, RX and TCD, were discarded from the given dataset. We implemented two dynamic selection methods, MOD and WMOD, which dynamically selected the best ML model among the ten implemented. However, MOD has a limitation when two ML classifiers have the same number of metrics with the best results. WMOD addresses this limitation and perfectly optimizes the selection criteria. The results show that MOD and WMOD have an accuracy of 99.6%, a probability of detection of 98.9%, a probability of false alarm of 1.56%, a probability of misdetection of 1.09%, and a processing time of 1.24%. These results outperform those of the existing ensemble learning models.

**Author Contributions:** Investigation, T.T.K. and S.I.; Methodology, T.T.K. and S.I.; Supervision, N.K.; Writing—original draft, T.T.K. and S.I.; Writing—review & editing, N.K. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research was funded by of the National Science Foundation (NSF), Award Number 2006674.

**Informed Consent Statement:** Not applicable.

**Data Availability Statement:** Dataset will be available in Github and other sites.

**Acknowledgments:** The authors acknowledge the support of the National Science Foundation (NSF), Award Number 2006674.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**

