A UAV-Swarm-Communication Model Using a Machine-Learning Approach for Search-and-Rescue Applications

Abstract

This paper presents a UAV-swarm-communication model using a machine-learning approach for search-and-rescue applications. Firstly, regarding the communication of UAVs, the receive signal strength (RSS) and power loss have been modeled using random forest regression, and the mathematical representation of the channel matrix has also been discussed. The second part consisted of swarm control modeling of UAVs; however, a dataset for five types of triangular swarm formations was generated, and K-means clustering was applied to predict the cluster. In order to obtain the correct swarm formation, the dendrogram of all types was investigated. Finally, the heat map and contour were plotted for all kinds of swarm clusters. Furthermore, it was observed that the RSS of proposed swarms had good agreement with swarm distances.

1. Introduction

The applications of unmanned aerial vehicles (UAVs) are increasing in different fields. They can be utilized for emergency services, life-saving missions and information-gathering systems. In order to ensure the reliability of a specific mission, a robust wireless communication channel is required. A number of technical committees have defined the basic standards for UAV radio communication links. Similarly, a regulatory body named the Radio Technical Commission for Aeronautics (RTCA) has standardized the minimum set of parameters for stable UAV operation [1]. Moreover, for the safe induction of UAVs in the airspace, another technical committee has developed the standards [2].

Generally, the UAVs are divided into three categories, such as rotary-wing, fixed-wing and vertical-takeoff ones. However, each type has its advantages and disadvantages in terms of hovering capabilities, take-off and landing, endurance length, loading capacity and operating radius. Specifically, in emergency response scenarios, the UAV relaying communications, emergency lighting and other rescue support can be used to collect on-site data and enable communication relay, night illumination and other support [3]. In the case of UAV swarms, a collection or group of UAVs carry out activities in a self-organized and self-adaptive manner to accomplish a task, which is the current trend in the developments of UAV technology. The self-organized part indicates that through local interactions, the UAV swarm spontaneously creates a totally decentralized or hybrid mode [4].

In the literature, numerous networking techniques and routing protocols have been developed and examined to improve networking performance. In order to guarantee the stability and dependability of UAVs, many researchers have already made significant progress in task allocation and cooperative execution. The idea of an avionics cloud based on a UAV swarm was proposed by [5], wherein the cloud model for avionics system integration was discussed. Regarding the communication mechanism of an intelligent UAV swarm, UAVs are highly mobile communication nodes that may function as transmitters or relays simultaneously. However, in the presence of interference, UAV swarms require multi-channel access and networking, and the omnidirectional antenna often employed for inter-UAV coordination may degrade spectrum efficiency [6]. Additionally, the cognitive-radio, broad-spectrum sensing capacity set up the learning process for reinforcement learning and produced better policy sets than conventional Q-Learning [7]. A real-time reinforcement learning system based on Q-Learning that utilized broadband spectrum sensing and greedy policy to improve real-time policy was presented by [8]. Additionally, multiple machine-learning types, their utilization and specific algorithms are provided in the alluvial map in Figure 1.

A detailed review paper related to swarm intelligence algorithms was published, where a number of optimization techniques are discussed [9]. The reallocation in emergent scenarios has been reported by [10]. In order to solve the problem of reallocation, Jun Tang et al. utilized the clustering of UAVs based on fuzzy C-means (FCM), along with ant colony optimization (ACO). The clustering head of a UAV swarm has been discussed by [11], wherein effective communication management has been addressed. Additionally, a multistage cluster of remotely located UAV swarm has also been addressed by Rang Ruan et al. [12]. The UAV swarm coordination protocol for supporting the assigned mission was reported by [13]. The problem of edge intelligence for UAV swarms has been discussed in [14]. Similarly, an adaptive weighted clustering algorithm is presented by [15]. Yongkun summarized the trends and application areas of UAV swarms; he also discussed the layer-based control of swarms [16]. Another paper was found related to UAVs’ collision avoidance, wherein, the author has discussed new approaches [17]. In addition, the uses of UAV swarms for modern applications are reported in [18]. Moreover, the survey paper related to remotely piloted aircraft systems (RPAS) is presented by [19]. Thereby, the research gap of communication models using machine learning was identified.

In this paper, a swarm of UAVs for rescue applications is discussed, as shown in Figure 2. The main objective of this work is to maximize the effectiveness of search and rescue activities during emergency. Furthermore, the UAV swarm can also be considered as a potential candidate for an emergency communication network. The Section 2, we present the machine-learning model for UAV communication based on the path-loss profile. The wireless channel matrix for unit swarm UAV is presented along with UAV cluster power. Furthermore, Section 3 describes the 2D swarm-control model with relative velocities ${\upsilon }_i^x\left(t\right)$ and ${\upsilon }_i^y\left(t\right)$ of UAVs. In Section 4, the generation of the cluster’s dataset is presented, based on multiple triangular swarm formation techniques. However, in Section 5, we present the application of K-means clustering to predict the swarm cluster’s formation, along with deprograms of UAVs. Finally, the prediction of received signal strength ($RSS$) based on UAVs’ reallocation in the cluster (swarm) is presented in a heat map and contour plot.

2. Path Loss Profile Additionally, Modeling for UAV Swarm

The section describes the UAV’s air-to-air link for swarm UAVs, whereas the communication channel is considered a line of sight (LOS), as shown in Figure 2. The Doppler frequency drift is taken as a dominant effect in the aerial link between the swarm of UAVs, and the effects increase with the moving speed of the swarm. Additionally, the Doppler effect can also be observed through a large-scale fading model, as shown below [20].

$$RSS\left(dB\right)=P_t+G_{UAV1}+G_{UAV2}+10{log}_{10}{\left(\frac{{\lambda }_0}{4\pi d_0}\right)}^{{\alpha }_0}$$

(1)

In Equation (1), the received signal strength is RSS, $P_t$ is taken as transmitted power and ${\lambda }_0$ is the wavelength. The distance between the UAVs is $d_0$, and the antenna gains of UAVs are $G_{UAV1}$ and $G_{UAV2}$. Referring to Equation (1), the $P_t$ and distance $d_0$ are constant for all swarm elements. The RSS vs. $d_0$ is plotted and shown in the Figure 3 [21]. However, there is a need to improve the gain of the antenna array in order to receive the $P_r$ from other UAVs. On the other hand, the relative velocities of swarm elements are assumed to be zero for the simplicity of the model. According to the reference [22], the Doppler effect of moving UAVs can be shown as below.

$$P_L\left(dB\right)=P_L\left(d_0\right)+10{\alpha }_o\left({log}_{10}\frac{d}{d_0}\right)-{log}_{10}\left(\frac{\delta h}{h_{opt}}\right)+C_p+10x{log}_{10}\frac{f_c+\delta f}{f_c}+\varsigma$$

(2)

where frequency-dependent path-loss is represented by x and $\delta f$ is considered as frequency drift [23]. One can understand that the aforementioned formula does not take antenna gain into account. During the flight of the swarm, the path-loss is calculated through [24]. Similarly, the power loss ($PL$) and $RSS$ vs. $d_0$ are shown in Figure 3.

The wireless channels (small-scale fading-model) are also taken into consideration to reflect the influences of the multi-path components while assessing the effectiveness of the deployed antenna design which was previously stated. Since the 3GPP/ITU model is the foundation for the 3D channel model created in [24,25], it is taken into consideration. The model provides information on the received multipath components such as amplitude, phase, time delay, angle of arrival (AoA) and angle of departure (AoD) in elevation and azimuth domains. As a result, the wireless channel matrix may be modeled using Equation (3).

Table 1. Variables used in the channel matrix equation.

Parameters	Description
$P_n$	Cluster power
$E_{Rx,i}^H,E_{Tx,j}^H$	Horizontal polarization of antenna
$E_{Rx,i}^V,E_{Tx,j}^V$	Vertical polarization of antenna
VV,HH	Received co-polarized signal
VH,HV	Received cross-polarized signal
${\Phi }_{n,m}^{VV},{\Phi }_{n,m}^{VH}$	Initial phases of a ray m in cluster n
${\Phi }_{n,m}^{HV},{\Phi }_{n,m}^{HH}$	Initial phases of a ray m in cluster n
${\overline{\psi }}_{n,m}$, ${\overline{\psi }}_{n,m}$	Tx and Rx antenna elements unit vectors
${\overline{r}}_s,{\overline{r}}_u$	Spatial Coordinate of Tx and Rx MIMO
${\lambda }_0$	wavelength of the carrier frequency
$\kappa$	Cross-polarization power ratio

provides a detailed list of the variables used in the equation below.

$$\begin{array}{c}\hfill G_{u,s,n}\left(t_0\right)=\sqrt{P_n}\sum _{m=1}^M{\left[\begin{array}{c}{E}_{tx0,s,V_i}\left({({\theta }_i,{\varphi }_i)}_{n,m,AOA}\right)\\ E_{tx0,s,H_i}\left({({\theta }_i,{\varphi }_i)}_{n,m,AOA}\right)\end{array}\right]}^T\left[\begin{array}{cc}{e}^{j{\Phi }_{n,m}^{vv}}& \sqrt{{\kappa }_{n,m}}{e}^{j{\Phi }_{n,m}^{vh}}\\ \sqrt{{\kappa }_{n,m}}{e}^{j{\Phi }_{n,m}^{hv}}& e^{j{\Phi }_{n,m}^{hh}}\end{array}\right]\\ \hfill {\left[\begin{array}{c}{E}_{rx0,u,V_i}\left({({\theta }_i,{\varphi }_i)}_{n,m,AOD}\right)\\ E_{rx0,u,H_i}\left({({\theta }_i,{\varphi }_i)}_{n,m,AOD}\right)\end{array}\right]}^T{e}^{j{\varpi }_{tx}}{e}^{j{\varpi }_{rx}}{e}^{2j{\pi \upsilon }_{n,m}t}\end{array}$$

(3)

The dataset of path loss and $RSS$ was extracted from Figure 3 [21]. For this purpose, the plot digitization technique (plot to CSV) was used and preprocessed in Python (Jupyter notebook). Thereafter, the supervised machine-learning algorithm known as random forest (RF) was applied to train the model based on available data points with (n_estimators = 10, random_state = 0). Accordingly, the prediction plots of path loss and $RSS$ has been presented in Figure 4 and Figure 5. As a result, the regression score of RF algorithm is observed around 99%.

3. Mathematical Model for Swarm Control

In this section, a mathematical model is formulated for a unit swarm (three UAVs). In order to make the model simple, it is assumed that all the UAVs are flying at the same altitude (2D flying pattern). The mathematical model for a 3D flying pattern is reserved for future work. In the presented unit swarm configuration, the leading UAV is regarded as the master and the two following UAVs are considered as slaves (see Figure 6). Accordingly, the moving parameters of slaves such as speed and direction are dependent on those of the master UAV. In order to avoid a collision, every element of a unit swarm must be at the safe distance. In a master–slave network configuration, there is need of a radar sensor on each element of the unit swarm. In fact, every master UAV of the unit swarm has the ability to communicate with two of its subsequent UAVs, as shown in Figure 7. Consequently, a sub-cluster is formed by joining multiple unit swarms. The expansion of the sub-cluster is the cluster swarm.

A unit swarm is represented in the rectangular coordinate system in Figure 6. The UAVs’ positions are $(x_i\left(t_0\right),y_i\left(t_0\right),z_i\left(t_0\right))$ while $i\in \{0,1,2\}$. For simplicity, it has been assumed that the UAVs are at the same height (z = 0) and the inter UAV distance or guard distance is ($x\left(t_0\right),y\left(t_0\right)$). According to [26], the equation of the swarm movement can be written as below.

$$\begin{array}{c}\hfill {\rho }_{0,1}^{*}=x_0\left(t_0\right)-x_1\left(t_0\right)-x_{0,1}\left(t_0\right)\\ \hfill {\rho }_{0,2}^{*}=x_0\left(t_0\right)-x_2\left(t_0\right)-x_{0,2}\left(t_0\right)\\ \hfill {\rho }_{1,2}^{*}=x_1\left(t_0\right)-x_2\left(t_0\right)-x_{1,2}\left(t_0\right)\end{array}$$

(4)

where the necessary x-axes spacing for swarm elements are $x_{0,1}\left(t_0\right)$, $x_{0,2}\left(t_0\right)$ and $x_{1,2}\left(t_0\right)$, and the spacing error function is ${\rho }^{*}\left(t_0\right)$. The velocity of UAVs may be expressed as:

$$\begin{array}{c}\hfill {\Delta }^{er}{v}_1=\nabla v_1\left(t_0\right)-{\overline{v}}_x\left(t_0\right)\\ \hfill {\Delta }^{er}{v}_2=\nabla v_2\left(t_0\right)-{\overline{v}}_x\left(t_0\right)\end{array}$$

(5)

The velocity equations (Equations (5)) belong to UAV${}_{s1}$ and UAV${}_{s2}$, whereas $\nabla v\left(t_0\right)$ and ${\overline{v}}_x\left(t_0\right)$ are the gradients of velocity and required velocities of UAVs. According to Figure 7 and Equation (5), the error function for velocities belonging to the $n^{th}$ swarm element may be expressed as follows:

$$\begin{array}{c}\hfill {\Delta }^{er}{v}_n=\nabla v_n\left(t_0\right)-{\overline{v}}_x{}_n\left(t_0\right)\end{array}$$

(6)

Referring to [27], the acceleration of UAVs in any swarm configuration must be depend on the error function (${\Delta }^e{v}_n$) along the x-axes. In addition, the dynamic control rule along the x and y axis are defined as mentioned in reference [26].

$$\begin{array}{c}\hfill {\upsilon }_a^x\left(t_0\right)={\alpha }_{ax}{\rho }_{0,a}^x+{\beta }_{ax}[v_0(t_0-{\tau }_{0,a}\left(t_0\right))-v_a\left(t_0\right)]+{\gamma }_{ax}{\rho }_{a,b}^x+{\zeta }_{ax}[v_0(t_0-{\tau }_{a,b}\left(t_0\right))-v_a\left(t_0\right)]\end{array}$$

(7)

On other hand, the velocity component is given as

$$\begin{array}{c}\hfill {\upsilon }_a^y\left(t_0\right)={\alpha }_{ay}{\rho }_{0,a}^y+{\beta }_{ay}[v_0(t_0-{\tau }_{0,a}\left(t_0\right))-v_a\left(t_0\right)]+{\gamma }_{ay}{\rho }_{a,b}^y+{\zeta }_{ay}[v_0(t_0-{\tau }_{a,b}\left(t_0\right))-v_a\left(t_0\right)]\end{array}$$

(8)

In the above two expressions ((7) and (8)) $a\ne b\in =\{1,2\}$, the parameters ${\alpha }_{ax}$, ${\beta }_{ax}$, ${\gamma }_{ax}$ and ${\zeta }_{ax}$ are considered as the gains in the x component. Furthermore, ${\alpha }_{ay}$, ${\beta }_{ay}$, ${\gamma }_{ay}$ and ${\zeta }_{ay}$ are y components in two dimensional space. Moreover, the variable ${\tau }_{a,b}$ is considered a time delaying function in the wireless communication.

4. Swarm Formation

In order to accelerate search and rescue operations, swarm formation is unavoidable. Many emergency applications require more than one UAV, such as those for highway road accidents, earthquakes, wild fires and floods. Consider a flood emergency scenario, since these are increasing due to environmental changes [28]. The search for humans during high water flow can be, more effectively carried out with the help of a swarm as compared to a single UAV. In addition, if one UAV carries one life jacket for a flood-flown person, there will be high significance of the UAV swarm for searching and rescuing. On the other hand, the swarm UAVs can share the search data (location and photos) with other UAVs for effective utilization of resource. Furthermore, a leading UAV be considered as a data acquisition terminal, and other UAVs can be assigned to drop the payloads. Consequently, the utilization of swarm of UAV during a hazard would be very effective for search-and-rescue applications.

The formation of UAVs with five different techniques based on triangles is discussed. However, the triangular formation is considered the most efficient for aerodynamic applications [29]. For this purpose, N UAVs were selected ($N=$ 10), which were ultimately divided into further sub-clusters and unit swarms. From Figure 8, Figure 9, Figure 10, Figure 11 and Figure 12, the mentioned types of triangular formation have been generated through Python code. According to Figure 8, the green dots represent the allocated space for the UAVs, whereas the random appointment of UAVs was taken for the worst case. The length of the UAV swarm is shown along the x-axis in Figure 8, and width along the y-axis.

According to Figure 8, the equilateral-swarm mode was presented with $N=$ 10 UAVs, wherein it can be observed that none of them crossed the boundary. It was considered that the UAVs were propagating along the direction of length. In order to avoid intra-swarm collision, very type of swarm formation was iterated multiple times to get a reasonable spatial distribution. Secondly, in isosceles-swarm formation proceeded with the starched top of the triangle. Thirdly, the obtuse-swarm data were generated from critically packed UAVs; most of them are seen in a linear arrangement and random uniform distribution. Furthermore, the acute- and right swarms are shown to have similar distributions.

5. Swarm Cluster Prediction

This section consists of two parts: firstly, the cluster prediction of previously generated swarms; secondly, the dendrogram plot, heat map and contour plot of the clustered swarm.

5.1. Machine-Learning Model

The dataset generated in the previous section is unlabeled, and UAVs are considered to be placed nearby. The optimum choice is to apply K-means clustering. The reason for the selection of K-means clustering is that the transmitted radiowave ($E_i=E_o{e}^{-2\pi z/\lambda })$ directly intersects with the receptor UAV. However, the diversity signal (multi-path) at the receiving UAV was ignored due to the near field. The K-means algorithm works with Euclidean distance between the UAVs, whereas the cosine and Manhattan algorithms do not use the direct distance approach. In order to satisfy Equation (1), d is considered as peer-to-peer distance between UAVs; therefore, only Euclidean distance could meet the requirement. According to Figure 6, the proposed mathematical model for swarm control has a two-dimensional (2D) assumption, and K-means with Euclidean distance can also fulfill this condition. However, the cosine, Manhattan, Minkowski and Chebyshev models can be used for higher dimensions. In fact, the distance and reallocation parameters are required for such applications.

The cluster process of UAVs is divided into three steps. Firstly, the optimum numbers of clusters are found through within-cluster sum of squares (WCSS), as shown in Figure 13. According to the elbow plot, $i=$ 4 for the given swarm datasets. Secondly, the Euclidean distance of each UAV is depicted with reference to $k_i$ centroids. In Figure 6, the coordinates of $UA{V}_1$ and $UA{V}_2$ are $(x_1\left(t\right),y_1\left(t\right))$ and $(x_2\left(t\right),y_2\left(t\right))$, respectively. The equation for euclidean distance can be expressed as below:

$$d(UA{V}_{s1},UA{V}_{s2})=\sqrt{{(x_2\left(t\right)-x_1\left(t\right))}^2+{(y_2\left(t\right)-y_1\left(t\right))}^2}$$

(9)

Thereafter, the centroids of UAV swarm can be found by

$$argmindist{(k_i,UA{V}_x)}^2$$

(10)

$k_i$ is the number of centroids ($k_i\in 1\dots 4$) which have been found through WCSS. Thirdly, we assign each UAV to the closest centroid, once the UAVs gather around the centroid; then, we again find the centroid by taking the average of each cluster. Finally, we repeat the steps of averaging cluster centroids till the centroid does not change. Likewise, the given below expressed as:

$$k_i=\frac{1}{|S_i|}\sum _{UA{V}_i\in S_i}^{}UA{V}_i$$

(11)

According to the Equation (11), the $S_i$ is set of all UAVs assigned to $k_i$ cluster. Finally, Figure 14, Figure 15, Figure 16, Figure 17 and Figure 18 present the K-means cluster for all proposed types of UAV swarms formation. It has been observed from Figure 15 that one UAV has been marked as a cluster which can be justified the inherent property of outlier in K-means clustering.

5.2. Deprogram for UAV formation

The dendrogram plots are presented for the above-mentioned clusters. In hierarchical clustering, the agglomerative algorithm is used, which has a bottom-to-top approach (unit swarm to cluster). The swarm-ability factor $f_{sw}$ is introduced with the minimum criteria of $f_{sw}=$ 0.5 for each unit cluster, and $f_{sw}=$ 1 for sub-clusters. Additionally, Figure 19, Figure 20, Figure 21, Figure 22 and Figure 23 present the dendrogram, and Figure 20 is considered the most suitable configuration for swarm formation. On the other hand, $f_{sw}<$ 0.5 generates the interference or highlights the other issues related to the physical movement (${\upsilon }_i^x\left(t\right)$ and ${\upsilon }_i^y\left(t\right)$) of swarm elements.

5.3. Prediction Data Visualization

In the path loss profiling and modeling, the predicted $RSS$ and K-means clustering data were visualized. However, the complete procedure is described in Figure 24. Whereas Figure 25 shows the head map with two predictions: UAV clusters and their corresponding $RSS$. In other words, the heat map can be considered to show the cluster power $P_n$ of parameters (mentioned in Table 1). The odd column shows the predicted $RSS$, and even columns are UAVs’ clustering distances. The rows in the heat map are the numbers of UAVs; $N=$ 10. Moreover, it has been observed that the UAVs, on there reallocation, receive power ($P_n$) in the range of −30 to −40 dB. In Figure 26, the contour plot shows the predicted power for the visualization of $RSS$. Finally, it is shown the whole swarm has a similar range of $RSS$ with normalized clustered distances.

In

Table 2. Comparison of UAVs and swarm parameters.

Ref.	UAVs	ML & AI	Algorithm	Model	Swarm	Application
[36]	4	Yes	OpenCV	Communication	No	S & R
[30]	10	No	–	Flux	No	Robotics
[37]	7	Yes	CNN	Network	No	S & R
[31]	2	No	PSO	Control	No	S & R
[38]	2	Yes	OpenCV	Collaboration	No	Robotics
[32]	1	No	Fusion	Collaboration	No	S & R
[33]	8	No	Maintenance	Communication	Yes	Mission Reliability
[34]	8	No	distributed algorithms	Communication	Yes	Wildfires
[35]	5	No	Leader-follower	Control	Yes	Robotics
[39]	1	Yes	Gaussian mixture	Communication	No	Mobile com.
Proposed work	10	Yes	R. Forest and K-means	Swarm control	Yes	S & R

, a comparison is presented wherein a number of research works are listed. According to that table, [30,31,32,33,34,35] used the conventional approaches towards communication, control, networking and collaboration for UAVs. As listed in Table 2, most reported works did not incorporate a UAV swarm, whereas the proposed UAV swarm technique has the potential to enhance the effectiveness of such applications. Few researchers have adopted swarm UAVs for other applications while using a machine-learning approach. However, the presented work is intelligent enough to utilize a machine-learning algorithm for UAV swarm formation. Additionally, a machine-learning-based communication model is also presented here for intra-swarm communication. As a result, the proper UAV swarm formations, along with received signal strength (−30 dB to −40 dB), have been achieved. Therefore, the presented work can be considered a good candidate for search-and-rescue applications.

6. Conclusions

This work presents a path-loss-profiling method for UAV swarms with machine learning, and it provides receive-signal-strength prediction. The UAV-swarm-communication problem for search-and-rescue applications has been solved by the proposed procedure machine-learning approach. Initially, the path-loss profile is generated though the training of a random forest model-based received-signal-strength dataset. In order to find the reallocation of UAVs, the dataset was generated for UAV clustering with the concept of triangular-based swarm formation. After that, K-means clustering was applied for the prediction of cluster parameters for a UAV swarm. Furthermore, the inter-swarm euclidean distance was found through the dendrogram plots. Finally, the results of communication model for the complete UAV swarm were presented in heap maps and contour plots. Future work could be related to radio-frequency-interference mitigation through machine learning.