Formation Control Algorithm of Multi-UAV-Based Network Infrastructure

Abstract

This paper addresses the analysis and the deployment of the network infrastructure based on multiple Unmanned Air Vehicles (UAVs). Despite the unprecedented potential to the mobility of the network infrastructure, there has been no effort to establish a mathematical model of the infrastructure and formation control strategies. We model the generic dynamics of the network infrastructure and derive the network throughput of the infrastructure. Through the parametrization of the model, we extract the generic factors of the network protocols and verify our model through the Network Simulator 3 (ns-3). By exploiting our network analysis model, we propose a novel formation control algorithm that determines the location of the UAVs to maximize the efficiency of the network. To achieve the objectives of the infrastructure, we define the formation-shaping effect as forces and elaborately design them using the generic factors. The formation algorithm continuously approaches to the optimized formation of a fleet of UAVs to enhance the overall throughput of the terrestrial devices. Our evaluations show that the algorithm guarantees remarkably higher throughput than the static formations. Through the dynamic transformation of the UAV formation, we believe that the multi-UAV-based network infrastructure could expand the boundary of the existing infrastructure while reducing the network traffic.

1. Introduction

As one of the promising solutions to the increasing wireless network traffic on the ground, a dynamic network infrastructure utilizing a number of Unmanned Air Vehicles (UAVs) was introduced [1,2]. The graphical representation of the network infrastructure is shown in Figure 1. The infrastructure is designed by two components: an ad hoc network among the UAVs, and a wireless access networks per UAV. The UAVs are spread in an area, and they establish an ad hoc wireless network with Ground Control Station (GCS) which is connected with the backbone network. At the same time, each UAV acts as an Access Point (AP) of the access network and connects the terrestrial mobile devices to the backbone network (i.e., Internet). Through the two-folded network architecture, the terrestrial mobile devices nearby the UAVs can connect to the gateway operating at the GCS. The network infrastructure has a number of unique features as follows.

• Mobility. UAV becomes one of the most versatile devices cooperating with various actuators such as camera [3,4], mechanic arms [5], and even weapons [6]. This trend is originated from the numerous studies in the network of the embedded computing board [7] and the UAV control research domain. While the previous infrastructures statically install a number of the base stations or APs on the ground, the UAVs can act as an AP and instantly deployed at the aerial space to provide the network to the ground. Providing network service by the multiple UAVs is a promising form of the Delay-Tolerant Network (DTN) concept described in RFC 4838 [8]. UAVs and their ad hoc network can act as a bundle layer, and provide functionality of the gateway to deliver the packets on the disrupted environments, such as a disaster area [9], wireless sensor networks [10] or highly congested traffic regions.
• Scalability. The fact that the UAVs are connected by an ad hoc network suggests the high flexibility of the network coverage. While the existing base stations or terrestrial APs should be plugged by wires that guarantee the connection to the backbone network, the multi-UAV-based network infrastructure can connect a new UAV so as to enlarge the service area of the access network. Also, with the possibility that the pre-installed network infrastructure could be act as a GCS, UAVs can be sparsely deployed in large area while augmenting the network-providing service.
• Wireless medium extensibility. Due to the immobility of the existing APs and base stations, the majority of the mobile devices share the wireless medium nearby the ground. Although Wireless LAN (WLAN) and cellular network use wired connection except one-hop wireless transmission, increment of the network traffic in a single AP or base station causes drastic decrement of the network throughput of the devices. Since UAVs are flying on the air, the wireless medium of the UAVs cannot be interfered by the channel of the existing networks. By extending the available wireless medium via multiple UAVs, the infrastructure has the potential to enhance the overall network throughput.

Compared to the multi-UAV-based network infrastructure having explicit possibilities, there has been lack of research for improving the system performance of the architecture. The reason for the lack of interest is mainly the doubt about the network efficiency of the infrastructure. Wireless ad hoc network among the UAVs grants the large scalability of network, but the limited range of communication and the throughput decrement of network are noticeable. However, it should be clarified that the multi-UAV-based network infrastructure is a non-replaceable solution in the situation when the previously deployed network infrastructure on the ground is disabled or degraded. To seamlessly provide the network by multiple UAVs in the emergency situations, their formation should be elaborately and instantly controlled in real time. In this paper, we aim to tune up the locations of the UAVs to efficiently provide the network traffic to the terrestrial mobile devices. Optimizing the formation of the UAVs according to the network requirement in a certain area is challengeable since the terrestrial devices are mobile, and tracking all of their exact location is generously hard. Our approach is a real-time feedback control where the UAVs exchange the information of the network (both access network and the ad hoc network) and location status with the neighbors and determine the proper direction to move. To determine the direction, the UAV should consider several aspects: the ad hoc connectivity with nearby UAVs, the extension of the network coverage area, and the traffic management of the ground. We model each concern as a force and each UAV determines a single direction from the sum of the forces. For example, if an UAV establishes excessively many connections with terrestrial devices, the UAV’s network status is reported to the neighbors and the formation is changed to disperse the traffic load of the UAV. However, since the UAV should not lose the ad hoc connection, so the UAV cannot move too far from the others. All of the behaviors are defined as the forces, derived from the information UAVs exchange.

To find the optimal direction of each UAV, we should consider the benefit of the overall network performance on the movement. We firstly analyze the dynamics of the multi-UAV-based network infrastructure on the side of the end-to-end communication between the terrestrial mobile devices and the backbone network. We model the generic phenomenon of the medium access of wireless network and propose the estimation equation of the aggregated throughput among the mobile devices. We verified our model by comparing the analytic result and the simulation result conducted by ns-3, and achieved above 98% accuracy with the simulation. Based on the analytic model, we propose a novel formation control algorithm that consistently enhances the overall network throughput of the multi-UAV-based network infrastructure. Due to the large-scale and extremely high cost feature of the practical experiment, we thoroughly simulated our algorithm and the network environment via ns-3 and shows approximately 41% increase in network throughput.

The contribution of the paper is listed as follows.

• We designed a network analysis model of the multi-UAV-based network infrastructure. During the derivation of the equations, we suggested a generic mathematical form of the multiple access behavior on the wireless access network. We proved that the generic form fits to the simulated behavior of the wireless access network, and applied the analysis results to the design of the algorithm.
• We proposed a multi-UAV control algorithm for the multi-UAV-based network infrastructure. The algorithm continuously transforms the formation of a fleet of UAV, while optimizing the network throughput of the infrastructure. Also, we proved that the result of the algorithm is a Nash equilibrium.
• We implemented a design of the multi-UAV-based network infrastructure, using ns-3. With the simulation tool, we evaluated our algorithm and proved that the algorithm outperforms the regular, static formation of the UAVs.

This paper is organized as follows. Section 2 investigates the former studies in UAV-based network-providing domain, and compares with our study to seize the motivation of this paper. Section 3 addresses the network analysis model of multi-UAV-based network infrastructure and Section 4 proposes a formation control algorithm of the UAVs. Then, we introduce the simulation environment and the evaluation of the model and the algorithm in Section 5. Finally, Section 6 concludes the paper.

2. Related Work

Multi-agent coordination problem has been studied in decades. As the application of the multi-robot system is varied and materialized, recent research focuses on the multi-agent control strategies to enhance the performance on the specific missions. Also, growing numbers of the mobile robots and the devices require more robust and reliable communication schemes on the lossy networks. This section briefly addresses the previous research on mobile networks and the formation control solutions that can be adapted to the multi-UAV management systems, and investigates the research in multi-UAV-based network service provisioning in detail to clarify the contribution of this paper.

Chatzigiannakis et al. [11] studied on the robustness of the mobile ad hoc network where the nodes change their connectivity frequently and unpredictably, and designed a framework for protocol classes to cope with the connection disturbances. In [12], the authors proposed the data collection strategies for a single mobile sink in wireless sensor networks. The authors demonstrated the strategies in several mobility model, and proved the performance of their work. Lee et al. [13] proposed a fast recoverable routing scheme for UAVs using geographic information. The research on the mobile nodes and the sinks is valuable, since most of the nodes in the multi-UAV-based network infrastructure are moving on 2D or 3D spaces. In this paper we focus on the formation control of the multiple UAVs, so we leave the deep consideration on the network protocol as future work.

Inalhan et al. [14] addressed a decentralized coordination problem of multiple aircrafts and proves the optimality of their work by Nash equilibrium. Our algorithm also locates the multiple UAVs in decentralized way, but the specific objective that the access network traffic management of our system reforms the formation to environment-aware one by adding the other equations. In [15], the authors showed an approach to the multi-vehicle formation system in graph rigidity manner and proposes the generic split/rejoin schemes for the stability of the system. This work is similar to a part of our algorithm constructing and maintaining the ad hoc network of multiple UAVs, except for the difference that our algorithm intends to enlarge the network coverage area while considering the traffic concentration. Ref [16] deeply worked on the aerodynamics of the multi-UAV environments and proposes a testbed for aerial control algorithm. While the testbed verifies the actual behavior of the multi-UAV when the formation control algorithm is applied, we evaluate the network performance of the multi-UAV network infrastructure with simulation study when our formation control algorithm is applied. Ref [17] arranged distributed multi-agent coordination systems while categorizing the sub-technologies into optimization, consensus, control, and estimation. Our algorithm consists of (i) the formation control in distributed manner, (ii) the information sharing between the nearby UAVs, (iii) optimality analysis by Nash equilibrium, (iv) estimation by performance modeling.

Calvo et al. [18] proposed a 3D positioning algorithm of the UAVs, considering the environment where the wireless charging stations are deployed. The mathematical models representing the wireless charging is notable, since extending the lifetime of the UAV is one of the crucial challenges in the UAV. However, this paper focuses on the performance improvement of the system using the static, position-optimized wireless charger. Our algorithm targets to the wide-spread mobile users, which move unpredictably.

Bor-Yaliniz et al. [19] worked on the path loss model of the UAV base station in the urban environments. By adopting International Telecommunication Union (ITU) channel model, the research reduces the path loss error from the former models. Adopting the study in [19] could be the promising solution for the environments with high-rise buildings, however, the priori knowledge containing the shape of the building could require the large amount of data, time, and expenditure. Our algorithm implicitly considers the variable path loss, by sharing the current number of connected mobile devices.

References [20,21] designed an UAV deployment problem where the UAV act as a flying base station while connecting to the mobile cellular users or the device-to-device (D2D) communication nodes. There is a difference between our work and this paper since the proposed system in [20] uses a single UAV. However, the approach to the solution which formulates to the disk covering problem [22] is similar to our research, since our algorithm adopts the sphere-shaped network range model.

In [23], the authors proposed the multi-UAV deployment algorithm optimizing the downlink coverage probability and the coverage lifetime. Unlike [20], the solution adopts disk packing problem [24] and derive the optimal position of each UAV. However, disk packing problem results in the blind spot of the network coverage region, which could drop the network performance. Also, optimizing the coverage lifetime might not be notable since the large amount of the power is consumed at the motors in UAV.

Challita et al. [25] designed a multi-UAV network formation game to optimize and stabilize the formation. Also, each UAV’s movements are determined by the virtual force, which let the UAVs maintain the multi-hop connection and avoid the physical collision. Even though the overall concept and approach are similar to our paper, there is a notable difference in terms of the amount of the formation research process. The main contribution of [25] is the expansion of the wireless backhauling area by modifying the node link status, which is addressed in our work called diffusion (See Section 4.4). In addition to the multi-hop network deployment, we applied an additional force to let the formation lean toward to the traffic concentrated area of the ground. In Section 5, we investigated the performance in the cases with and without the additional traffic-aware force.

3. Multi-UAV-Based Network Infrastructure Analysis

This section addresses the mathematical representation of the multi-UAV-based network infrastructure. Our approach is the establishment of the generic expression of the network throughput with variable parameters depending on the network protocol. We firstly present the assumptions and the notations for the analysis of the multi-UAV-based network infrastructure, and then propose the network throughput estimation equation for the mobile devices and the UAVs.

3.1. Assumptions and Notations

We assume that a number of the terrestrial mobile devices are randomly spread at a flat area S. M refers to the set of the mobile devices in S, and $m_i$ refers to the ith device where $1<i\le \left|M\right|$. At the same manner, D refers to the set of the UAVs composing the network infrastructure, and $d_j$ refers to the jth device where $1<j\le \left|D\right|$. Location of each device is referred to $p_{m_i}$, which is unknown on the side of the UAVs and the network infrastructure. Also, position of each UAV is referred to $p_{d_j}$, and we assume that $p_{d_j}\left(t\right)$ is periodically collected through the navigation devices (e.g., GPS) and shared by the UAVs’ ad hoc channel. Each device could have different amount of the network resource requirement $r_{m_i}$, depending on what application each device uses. We additionally assume that $r_{m_i}$ could not exceed the maximum data rate of the data transmission $r_{max}$. Each device is trying to access the Internet by scanning the nearby AP providers (UAVs) and connecting to the one of them.

For mobile devices to connect the Internet, they scan the APs around the locations. We refer the maximum range of the AP connection provided by the UAVs to the $R_{AP}$. We assume that the mobile devices select an AP (UAV) broadcasting the beacon signal with the best SINR, which can be expressed to

$$\mathrm{AP}\left(m_i\right)=\underset{d_j\in s_{m_i}}{arg\; max}SINR(m_i,d_j)$$

(1)

where $s_{m_i}$ refers to the set of the UAVs scanned by $m_i$ and $SINR(a,b)$ refers to the SINR of the a beacon signal from the device b. The assumption is valid since the existing AP selection mechanisms commonly shape the decision metric according to the received signal strength or SINR [26,27]. On the UAV’s side, a set of connected mobile devices are referred to $C\left(d_j\right)$, which can be obtained by

$$C\left(d_j\right)=\left\{i\right|\mathrm{AP}\left(m_i\right)=d_j\}.$$

(2)

We assume that the channel used by AP network of the UAVs and the channel used by ad hoc network are completely isolated, which means that there is no channel interference between the AP and ad hoc network. The assumption also can be accomplished by using the different frequency band (e.g., 2.4 GHz for AP and 5.0 GHz for ad hoc). The assumption is not only for the simplicity of the modeling, but the expected performance of the network infrastructure. If APs and ad hoc use the same channel, the medium is shared by the nearby UAVs as well as the devices connected in the same AP. Furthermore, since an expected network resource requirement of the ad hoc connection is the sum of resource requirement of the AP connections, AP network performance might be significantly lowered by the contention with the ad hoc connections.

Also, we assume that the ad hoc network routing protocol results in the optimized routing path while the UAVs are changing their locations. Since the network infrastructure specifies the GCS as a sync node, the UAVs spread directionally or omni-directionally around the GCS. We refer the number of connected UAVs with a certain UAV $d_j$ in a single hop to $N_{d_j}$, which can be calculated to

$$N_{d_j}=\left|\left\{d_k\right|dist(d_k,d_j)<R_{adhoc}\}\right|$$

(3)

where $R_{adhoc}$ refers to the maximum range of the one-hop communication of ad hoc network.

3.2. Wireless Access Network

This section addresses a generic form of the wireless access network model mathematically. Due to the resource limitation of the wireless medium and the randomized media access of the wireless devices, the network performance decreases when the number of the devices increase. As each UAV works as an AP, the UAVs might connect to the randomly spread users, which leads the large deviation of the number of connections between the UAVs. Some UAVs with relatively large number of the connections could suffer from the significant drop of the network performance, while some UAVs with relatively small number of the connections might waste the network resource. To elaborately distribute the network traffic to the UAVs, the mathematical model of the wireless access network and the numeric factors that affects the formation control should be designed. Also, there are valuable approaches to alleviate the risk of the packet collision such as Carrier Sense Multiple Access with Collision Detection (CSMA/CD) [28], variations of ALOHA protocol [29], and Orthogonal Frequency-Division Multiple Access (OFDMA) [30]. To give the generic view and enhance the flexibility to the model, we propose a mathematical expression of the aggregated throughput with a protocol-dependent parameter, which could be estimated by comparing the analytic solution and the simulated solution.

First, we define a network capacity R, which refers to the maximum data rate of a connection where only one mobile device is connected to a UAV, without any electrical interference and obstacle. Based on the R, the ratio of the network throughput degradation is exponentially increased as the number of mobile devices in the same network ($C\left(d_j\right)$) increases, due to the increased possibility of the collision. In a certain point of the time, the possibility of the collision when n devices are in the same medium is

$$\frac{2^n-n-1}{2^n-1}.$$

(4)

However, there are some previous studies allowing the concurrent transmissions by sub-channels [30] or code division [31], so the variable n could be divided into the approximated amount of the concurrent transmissions. Also, to reduce the collision probability, the mechanism that randomly backs off a number of time slots and detect the medium [28,32] and its variations [33,34,35] were studied. Considering these improvements, the throughput degradation ratio, $\mathcal{D}\left(n\right)$ can be expressed by augmenting Equation (4) to

$$\mathcal{D}\left(n\right)=1-\frac{1}{\beta }\left(\frac{2^{\frac{n}{\alpha }}-\frac{n}{\alpha }-1}{2^{\frac{n}{\alpha }}-1}\right)$$

(5)

where $\alpha$ and $\beta$ are named to concurrent transmission availability and congestion relaxation degree of the protocol respectively, and $n=\left|C\right(d_j\left)\right|$. As $\alpha$ or $\beta$ increase, the degradation of the throughput decreases. Note that $\alpha$ and $\beta$ do not represents the physical property or performance of the protocols, for instance, $\alpha =10$ does not suppose that an AP holds exactly 10 transmissions at maximum. Our generic model involves the common behavior of the wireless network and the parameters representing the evolution of the communication techniques. The verification of Equation (5) shows in Section 5.4. Since the objective of the paper is the improvement of the throughput of the infrastructure, we estimate $\alpha$ and $\beta$ of the WLAN protocol by the simulation and reflect to the formation algorithm.

By the assumption stated in Section 3.1, the devices could require different amount of the network resource. When the multiple devices try to access the same wireless channel, the actual amount of the network resource provided by the UAV is proportionate to the required resource. Reflecting the congestion of the medium expressed in Equation (5), the provided network capacity $r_{m_i}^{\prime }$ can be expressed to

$$r_{m_i}^{\prime }=max\left(R\times \frac{r_{m_i}}{{\sum }_{k\in C_i}{r}_{m_k}},r_{m_i}\right)\times \mathcal{D}\left(\right|C_i\left|\right)$$

(6)

where $C_i$ refers to the $C\left(d_{AP\left(m_i\right)}\right)$. The Equation (6) grants an insight of the importance of the congestion control and the direction of the throughput optimization. Full provision of the UAVs’ network capacity meets the original purpose of the network infrastructure, but the congestion occurred by the excessive connections could drop the performance of the access network. Our formation control algorithm leverages the mobility of the UAVs to mitigate the congestion of WLAN network and enhance the aggregated throughput of the network infrastructure. Finally, the network throughput carried by the UAV $T_{d_j}$ is calculated to

$$T_{d_j}=\sum _{k\in C_i}{r}_{m_k}^{\prime }.$$

(7)

Note that $T_{d_j}\ne R\times \mathcal{D}\left(\right|C_i\left|\right)$ since the total network resource amount for $m_i\in C_i$ could not be equal to the R.

3.3. Ad Hoc Network

The UAVs in the network infrastructure should forward the data from the mobile devices to the Internet gateway (uplink) and from the gateway to the mobile devices (downlink). In the multi-hop communication, the same phenomenon introduced in Section 3.2 should be considered in chain. For example, if the UAV 0 in Figure 2 tries to transmit the data to UAV 2, the UAV should contend with the UAV 1 to use the medium. Also, the contention between the UAV 2 and the UAV 3 affects the throughput of the data received from the UAV 0 or the UAV 1. If we let $P_{d_j}$ refer to the set of the UAVs included in the routing path to the GCS (e.g., $P_A=\{C,E\}$ in Figure 2), the actual throughput (received data rate of the GCS) $T_{d_j}^{\prime }$ becomes

$$T_{d_j}^{\prime }=T_{d_j}\prod _{m\in P_{d_j}}\mathcal{D}\left(N_{d_m}\right).$$

(8)

The analysis suggests that the expected routing path of the UAVs should be given to calculate the network throughput of the infrastructure. Our formation control algorithm estimates the expected routing path and the congestion of the wireless medium by the location information of the UAVs, collected periodically on the basis of the assumption stated in Section 3.1. Also, on the layer of ad hoc network, increasing $N_{d_j}$ and the hop count decreases the network throughput of the infrastructure.

Assuming that all UAVs are in the ad hoc network, the overall network performance of the multi-UAV-based network infrastructure $\mathcal{N}$ is calculated to

$$\mathcal{N}(D,M)=\sum _{d_j\in D}{T}_{d_j}^{\prime }.$$

(9)

As shown in Equation (9), optimizing D whose object function is a non-linear $\left|D\right|\times \left|M\right|$ dimension function is a remarkably complicated and might take high cost of time to estimate while following the changes of M. To cope with the unexpected change of the M, we periodically determine the flying direction of the UAVs to enhance the network throughput of the infrastructure in our formation control algorithm. The observations of the analysis are considered on the algorithm.

4. Formation Control Algorithm

The formation of the UAVs has a profound effect on the performance in the multi-UAV-based network infrastructure. Based on the design of the infrastructure stated in Section 3.1 and the model described in Section 3.2 and Section 3.3, we list the objectives of the UAVs’ formation as follows.

• All UAVs should maintain the ad hoc connectivity between each other.
• Congestions on the access networks and the ad hoc network should be reduced.
• The infrastructure should connect the mobile devices as many as possible.
• UAVs should not be redundantly deployed; Formation should use minimal UAVs to maximize the network performance.
• Formation should be changed in response to the change of M.

To achieve the multiple objectives in real time, we designed a novel feedback-based formation control algorithm of the network infrastructure. For each UAV in a specific start location, the algorithm determines the flying direction of each UAV in a short period, according to the periodically received information from the UAV itself and the neighbors. Our approach is to define the forces designed to fulfill the requirements of the network infrastructure and applied to the UAVs to change the direction of each [36]. By the vector sum of the forces, each UAV flies to the specific direction with a limited speed as shown in Figure 3.

The periodically received information of the UAV used by the algorithm includes

• The position of the gateway, ${\mathbf{p}}_{\mathbf{0}}$
• The position of the UAV, $\mathbf{p}$
• The positions of the nearby UAVs, ${\mathbf{p}}_i$
• The positions of the next hop UAVs, ${\mathbf{p}}_j$
• The number of the connected devices of the UAV, n
• The numbers of the connected devices of the nearby UAVs $n_i$

Which are collectable (We use the abbreviated notations in those variables for readability). The subsections listed in the below describe the forces applied to the UAVs and the control mechanism of our algorithm.

4.1. Ad Hoc Connectivity Force

The loss of the ad hoc connection for a UAV could lead to a serious tragedy, which is not only the drastic reduction of the performance but the loss of the control for the UAV. In our formation control design, ad hoc connectivity force has the most dominant magnitude if the connections come to weaken, mostly by the path loss due to the distance. We call neighbors that could be the next hop of a certain UAV in ad hoc network. With respect to the Euclidian distance to the next hop nodes, ad hoc connectivity force $F_{adhoc}$ is expressed to

$$F_{adhoc}=\sum _{H}max\left(exp\left(\right|{\mathbf{p}}_{\mathbf{j}}-\mathbf{p}|-A)-1,0\right)({\mathbf{p}}_{\mathbf{j}}-\mathbf{p})$$

(10)

where H is referred to the set of the next hop nodes and A is named to the ad hoc connectivity force threshold. Designing $F_{adhoc}$ as the distance to the next hop nodes is valid since the UAVs are flying above the flat area in the assumption stated in Section 3.1. $F_{adhoc}$ is applied to the UAV only the UAV is far enough to the next hop nodes to jeopardize the ad hoc connectivity. The threshold A is determined with respect to the ad hoc communication range $R_{adhoc}$. In our implementation, we observed that the performance is maximized when $A=0.8R_{adhoc}$.

4.2. Neighbor Distribution Force

If UAVs are too closed in a certain area, their network capacity could be redundant and the wireless medium could be overlapped. Neighbor distribution force is applied to spread out the clustered UAVs and expand the overall network coverage space of the infrastructure. In the same manner introduced in Section 4.2, neighbor distribution force $F_{distr}$ is expressed to

$$F_{distr}=\sum _{E}min\left(ln\frac{|\mathbf{p}-{\mathbf{p}}_{\mathbf{i}}|}{A},0\right)({\mathbf{p}}_{\mathbf{i}}-\mathbf{p}).$$

(11)

where E is referred to the set of the nearby UAVs, unlike $F_{adhoc}$.

In the UAV-to-UAV position control, $F_{adhoc}$ and $F_{distr}$ applies the exactly opposite direction of force to the UAV. However, the difference in the set of the vectors (H and E) let the UAVs to shape the formation properly. For example, although the neighbor is distributed enough and the ad hoc connectivity is kept, the network infrastructure could not be accomplished if any UAV cannot connect to the GCS. Since H could include the GCS, the UAVs can safely forward the data between the backbone network and their connected devices.

4.3. Congestion Equalization Force

$F_{adhoc}$ and $F_{distr}$ form the UAV fleet regularly; at the same altitude, and the same distance between the nodes. Although these two forces let UAVs be spread while maintaining the connection with the gateway, there is no concern about the traffic management of the end devices, which could make some UAVs redundant or cause the traffic concentration. In particular, variability of the altitude of the AP is one of the most essential advantages of the UAV-based network infrastructure, since the altitude determines the communication area projected at the ground (Equation (1)). As shown in Equation (5), excessive increment on the number of the connections in a single UAV degrades the network performance of the infrastructure. Under the influence of the aforementioned background forces, congestion equalization force is designed to let UAVs subdivide the traffic concentrated spots to equalize the congestion of the UAVs. We apply the force with direction to the $+z$ axis for the relatively more congested UAVs to explicitly narrow the coverage space and reduce the connections. Meanwhile, the relatively less congested UAVs are applied the force with the direction to the $-z$ axis and the direction toward the highly congested UAVs. Through this equalization force, the less congested UAVs take over a part of the devices which are excessively connected to another UAV. Also, to progressively seek the network resource requirement on the ground, relatively low congestion UAVs push the relatively high congestion UAVs until the expected congestion comes to the equilibrium. The equation of the congestion equalization force $F_{coneq}$ can be expressed to

$$F_{coneq}=\frac{1}{\beta }\sum _E\frac{n-n_i}{|n-n_i|}exp\left(\right|n-n_i\left|\right)(\mathbf{p}-{\mathbf{p}}_i)+{\mathbf{h}}_i$$

(12)

where

$${\mathbf{h}}_i=\left[\begin{array}{c}0\\ 0\\ \frac{1}{\beta }exp\left(\right|n-n_i\left|\right)\end{array}\right].$$

(13)

Congestion relaxation degree $\beta$ determines the weight of the $F_{coneq}$, since the congestion could be relaxed by the protocol and then the support of other UAVs is less needed. Figure 4a,b show an example of our $F_{coneq}$ design graphically. For better readability, we use the terms attractive and repulsive forces for every $F_{coneq}$ applied. Since UAV 2 located in the center of the traffic concentrated area, the altitude gets higher and the position is changed by the repulsive force by the nearby UAVs. On the case of UAV 3, the repulsive force is applied by UAV 4 and 5, and the attractive force is applied by UAV 2. By the sum of the forces, UAV 3 is dominantly powered by the repulsive force toward the center of the traffic concentrated area. UAV 0, 1, 4, and 5 receive the same direction of the power, but also receive the downward power by Equation (13). UAV 3 has more congestion than UAV 4 and UAV 5, and less congestion than UAV 2. As a result, the altitude change of UAV 3 is less than the other UAVs. One of the examples of the result formation effected by $F_{coneq}$ is shown in Figure 4b. Note that the excessive clustering among the UAVs by $F_{coneq}$ is prevented by $F_{distr}$, addressed in Section 4.2. On the basis of the formation maintenance forces $F_{distr}$ and $F_{adhoc}$, $F_{coneq}$ reforms the formation to be more traffic-aware.

4.4. Feedback-Based Control

Through the forces defined in Section 4.1, Section 4.2 and Section 4.3, we designed a novel feedback-based UAV formation control algorithm as Algorithm 1. In addition to the summation of the forces, we added the ad hoc network expansion procedure named diffusion from line 7 to 13. At the first step, all UAVs are directly connected to the GCS with a single hop to sustain the connectivity of the network infrastructure. At the state of the line 7, a UAV could guarantee the route path to the GCS by one of its neighbors in E. Then, the UAV cuts the farthest next hop connection and declares the diffusion to the nearest neighbor, to prevent the loss of the GCS connections of all UAVs. NeighborDiffused() function returns true if the UAV is informed the diffusion of another UAV.

Algorithm 1 Formation control algorithm

1: Collect ${\mathbf{p}}_{\mathbf{0}}$ 2: Initialize $H=\left\{{\mathbf{p}}_{\mathbf{0}}\right\},E=\varphi$ 3: while True do 4:     Refresh $\mathbf{p},{\mathbf{p}}_{\mathbf{i}},n,n_i,E$ 5:     $\mathbf{d}\leftarrow (0,0,0)$ 6:     $Calculate{F}_{adhoc},F_{distr},$ and $F_{coneq}$ 7:     if $F_{adhoc}\ne 0and|F_{distr}|+|F_{coneq}|>|F_{adhoc}|$ then 8:         if !NeighborDiffused() then 9:            $Remove{arg\; max}_j|\mathbf{p}-{\mathbf{p}}_{\mathbf{j}}|$ from H 10:            $H\leftarrow H\cup E$ 11:            Recalculate $F_{adhoc}$ 12:            Inform the diffusion to the nearest neighbor 13:         end if 14:     end if 15:     $\mathbf{d}\leftarrow F_{adhoc}+F_{distr}+F_{coneq}$ 16:     Set direction to $\frac{\mathbf{d}}{\left|\mathbf{d}\right|}$ 17: end while

On the congestion equalization, the force applied by a few differences of the connection counts could lead to the oscillation of the UAV. Since the frequent AP change of the mobile device could lead to the degradation of the performance, we utilize the concurrent transmission availability $\alpha$ as a threshold of the $F_{coneq}$ summation. The larger value of the $\alpha$ grants more sustainability of the UAV’s formation.

4.5. Algorithm Analysis

To validate our formation control algorithm, we prove that the output of the algorithm is a Nash equilibrium [37]. Let $S_i=vt{\mathbf{d}}_{\mathbf{i}}/\left|{\mathbf{d}}_{\mathbf{i}}\right|$ is the movement for the UAV $d_i$ in a time period t with the speed v, then $S=S_1\times S_2\times \cdots \times S_{\left|D\right|}$ is the set of strategy profiles. Since the objective of the algorithm is the enhancement of the network performance, payoff function could be expressed by the overall network throughput difference of the infrastructure such as

$$f\left(x\right)=\Delta \mathcal{N}(\Delta D,\Delta M)$$

(14)

where x refers to a certain direction (unit vector) of the UAV at t. On the circumstances that all $x^{\ast }=\mathbf{d}/\left|\mathbf{d}\right|$ is given, let $x_i^{\prime }$ satisfies $f(x_i^{\prime },x_{-i})>f(x_i^{\ast },x_{-i}^{\ast })$. If $x_i^{\prime }$ causes the loss of ad hoc connection of a UAV, $f(x_i^{\prime },x_{-i})\le f(x_i^{\ast },x_{-i}^{\ast })$ due to Equation (9) where $T_{d_j}^{\prime }\ge 0$. Otherwise, if $x_i^{\prime }$ causes the changes of the connected devices, there are total 4 cases as follows.

• Decrement of the $C\left(d_j\right)$ leads to the decrement of the $T_{d_j}$ except the case of the excessive $C\left(D_j\right)$, which is already reflected by $x_i^{\ast }$ in Equation (13). It leads to the decrement of the $T_{d_j}^{\prime }$ by Equation (8).
• Increment of the $C\left(d_j\right)$ leads to the decrement of the $\mathcal{D}$ except the case of the redundant R, which is already reflected by $x_i^{\ast }$ in Equation (12).
• Bringing already connected devices from other UAVs leads to the decrement of the $\mathcal{D}$ of i and the redundant R of the other UAVs, except the case of the congestion equalizing addressed in Section 4.3, which is already reflected by $x_i^{\ast }$.
• Yielding already connected devices to other UAVs leads to the redundant R of i and the increment of the $\mathcal{D}$ of the other UAVs, except the case of the congestion equalizing addressed in Section 4.3, which is already reflected by $x_i^{\ast }$.

Otherwise, $x_i^{\prime }$ induces the delay of the network coverage expansion and the reduction of the possibility to discover a new connection, while $x_i^{\ast }$ reflects the best possible point by Equation (11). In conclusion, the above statement is contradictory to the assumption that there is a $x_i^{\prime }$ which satisfies $f(x_i^{\prime },x_{-i})>f(x_i^{\ast },x_{-i}^{\ast })$. Hence, the algorithm is proved that the output S is a Nash equilibrium.

5. System Evaluation

We used our simulation tool for two objectives: verification of the analytic model expressed in Section 3, and evaluation of the formation control algorithm proposed in Section 4. In this section, we address the simulation design of the multi-UAV-based network infrastructure. We implemented the simulation using Python language and ns-3 library. Figure 5 overviews the simulation design and the ns-3 internal functions used, and the following subsections describe the design of the simulation in detail.

5.1. PHY and MAC Device Installation

We employed IEEE 802.11 Wi-Fi Physical Layer (PHY) and MAC model. The AP network has 3 channels (Channel number 1, 6, and 11) to select, for minimizing the medium overlap between the adjacent APs. The ad hoc network uses a single channel reflecting the native communication pattern. Ad hoc network channel and the AP channels are distinctly instantiated without any expected interference, to follow the assumption in Section 3.1. Mobile nodes can change the AP connection safely since all APs use the same channel. Unlike the mobile nodes and GCS have one network device for the station and the ad hoc respectively, UAVs have two network devices for the AP and the ad hoc. The UAVs broadcast a beacon signal per second.

5.2. Network and Transport Protocol

There are $\left|D\right|+1$ subnetworks, one for ad hoc and $\left|D\right|$ for APs. Internet Protocol (IP) addresses for the UAVs and the GCS is pre-assigned at the simulation environment install step. For the terrestrial nodes, IP addresses could be changed dynamically, no IP address is assigned in advance. However, it should be noted that ns-3 does not support automated Wi-Fi selection scheme internally. Mobile nodes cannot change the IP address dynamically by scanning the around APs and change the connection. To resolve the phenomenon, we explicitly probed the APs nearby each mobile node and reassigned the IP address in a certain period. In our implementation, the AP scan occurs per 3 s. Also, we employed Ad hoc On-demand Distance Vector (AODV) routing protocol [38] for the ad hoc network. In our previous work, we found that AODV outputs the best performance of the ns-3 supporting routing protocols for the ad hoc networking in terms of the connectivity and the data rate. Finally, to focus on the PHY to the Network layer performance, we employed User Datagram Protocol (UDP) protocol to reduce the transport layer overhead of the simulation.

5.3. Application Scenario

We employed a simple application model that consistently requests the fixed amount of data rate, which is referred to $r_{m_i}$ in this paper. Following the objective of the UAV-based network infrastructure, all the terrestrial devices try to connect to the backbone network by the gateway (i.e., GCS) and one of the UAVs. We installed PacketSink at the gateway and measured the overall throughput of the network.

5.4. $\mathcal{D}$ Model Verification

By using the simulation addressed in previous subsections, we verify the network throughput model of multi-UAV-based network infrastructure. At first, we observe the dynamics of $\mathcal{D}$, which is the basis of the network throughput models addressed in this paper. To estimate $\alpha ,\beta$ of $\mathcal{D}\left(n\right)$, we run the simulation with a single UAV and the varying number of the terrestrial devices $\left|M\right|$, and measure the network throughput. The devices are spread in the 20 m × 20 m area on the ground, and the GCS is located at the center of the ground, which is set to the origin. A single UAV is deployed at $(0,5,5)$, and the devices preliminary receive the IP address included at the subset of the UAV. GCS sends data to all of the devices in 1000 Kbps data rate equivalently, and the average network throughput is calculated by the average received data rate among the devices. Each case is simulated by 10 times with 300 s runtime, and $\left|M\right|$ varies to $1\le \left|M\right|\le 60$. Figure 6 shows the simulation result and the analytic solution where $\alpha =9.8170$ and $\beta =1.2044$.

When applying Equation (5), we observed that $\mathcal{D}\left(n\right)>1$ where $n<\alpha$. The phenomenon is due to the $\alpha$, since the equation calculates $\mathcal{D}$ in a case that a single connection has more sub-channels to occupy. However, the required network resource is limited to $r_{m_i}$, so we cut out these values into the maximum ratio 1. With the aforementioned $\alpha ,\beta$ values, the analytic solution achieves above 98% accuracy with the simulation. If we increase the repetition count, the fluctuation of the measured throughput in the range of $20\le \left|M\right|\le 25$ might be reduced. From the observation, we concluded that our throughput model forecasts the network throughput of the multi-UAV-based network infrastructure with acceptable accuracy, by fitting $\alpha$ and $\beta$ with respect to the desired network protocol.

To verify our formation control algorithm, we expanded the map size to 500 M × 500 M, and increase the number of devices to 100. The GCS is located at the center of the map and the UAVs are initially spread around the GCS with the altitude of 60 M. The parameters $\alpha ,\beta$ are applied to the 9.8170 and 1.2044 respectively, and the directions of the UAVs are calculated per 0.5 s. The terrestrial users are deployed with two-dimensional Gaussian distribution, centering a random location $(x,y)$ on the map. We observed the result of the algorithm working on 15 UAVs and graphically represented in Figure 7.

In Figure 7a,b, grey circles represent the communication range of the AP ($R_{AP}$) on the ground, and the number on the icon of the UAVs are the identifiers. Also, both figures show the stabilized (no oscillation by the forces) formation of the UAVs after 30 s. Even though the distribution of the UAVs in Figure 7a are regular, the actual connections to the terrestrial devices are not equalized (see Figure 7c,e) since the distribution of the devices are focused at nearby of UAV 3 and 4. UAV 10, 12 and 13 does not have any connection to the devices, which results in the redundant usage of the UAVs. On the other hand, the formation with $F_{coneq}$ shown in Figure 7b, the formation of UAVs is reformed toward the cluster of the devices, while each UAV providing the access network to at least 5 devices. Deploying UAV 2, 3, 4, and 6 nearby the cluster of the devices could more reduce the traffic concentration of the infrastructure. Also, $F_{coneq}$ dynamically adjusts the altitude of the UAVs to efficiently equalize the congestion as shown in Figure 7d. Through the observation, we verified that the algorithm adaptively controls the formation of the UAVs to reduce the network traffic of the multi-UAV based network infrastructure with various deployment cases of the devices.

To observe the dynamics of the formation algorithm, we performed numerical evaluation while varying the number of the terrestrial devices $\left|M\right|$ from 100 to 300, as shown in Figure 8. For each $\left|M\right|$ value, we performed the formation algorithm 50 times for the case with and without $F_{coneq}$ and recorded $|C_i|$ after the formation is stabilized. Successful coverage ratio in Figure 8a is calculated by $\sum |C_i|/|M|$, which results in the ratio of the connected mobile devices and the total devices. The case with $F_{coneq}$ outperforms the case without $F_{coneq}$ independent to $\left|M\right|$, since $F_{coneq}$ not only equalizes the WLAN congestion but pushes the edge-located UAVs out to search the unconnected devices. Also, the case with $F_{coneq}$ and 15 UAVs cover greater number of the devices than the case without $F_{coneq}$ and 20 UAVs, which means that $F_{coneq}$ can result the better coverage ratio with fewer UAVs. Figure 8b shows the standard deviation of $|C_i|$. As shown at the example of Figure 7, standard deviation of the case with $F_{coneq}$ is much less than the case without $F_{coneq}$. Also, as $\left|M\right|$ increases, standard deviation of the case without $F_{coneq}$ increases, but the case with $F_{coneq}$ does not change since $|C_i|$ keeps equalized. Estimated throughput refers to the average of the devices’ throughput calculated by Equation (5) with $|C_i|$, assuming that each device sends data packet with rate $1Mb$. Due to the network capacity of the AP and the congestion, estimated throughput has no choice but to decrease as $\left|M\right|$ increases. However, as previously verified in Figure 6, decreasing the number of $|C_i|$ can reduce the loss of the throughput. The case with $F_{coneq}$ and 20 UAVs results in the best throughput on the evaluation, while the difference grows as $\left|M\right|$ increases. To provide the better quality of the devices, increasing the number of the UAVs could be the easiest way, but it might not bring the meaningful changes in terms of the coverage (Figure 8a) or the standard deviation (Figure 8b). Dynamically adjusting the number of the UAVs can be the promising strategy to improve the UAV-based network infrastructure.

5.5. Network Performance Simulation

We also simulate the network throughput of the multi-UAV-based network infrastructure using the formation control algorithm. Since there is no existing method considering the formation of the UAVs of the network infrastructure, we compare the two cases where one applies $F_{coneq}$ and the other does not apply $F_{coneq}$. The comparison is valid since the formation without $F_{coneq}$ shows hexagonally patterned deployment of the UAVs, which is widely used in the placement of the wireless network infrastructure [39]. We firstly deployed the mobile devices with a cluster centered at $(-100,100)$ in the map, and let the devices move in the direction $(1/\sqrt{2},-1/\sqrt{2})$, as shown in Figure 9a. The devices start to move at 50 s from the start of the simulation, and stop to move at 70 s. The devices send packets to the GCS with 100 Kbps after the 20 s from the start of the simulation, and the network throughput is measured per second. Figure 9b shows the changes of the network throughput with respect to the time for two cases.

The simulation evaluates two aspects: the network throughput enhancement and the time-variant flexibility of the formation shaped by the algorithm. Figure 9b shows that our formation control algorithm with $F_{coneq}$ outperforms the network throughput at the whole time, except for the short intervals such as 25 to 26 s and 72 to 73 s. The reason of the temporal drop was the large number of the AP connection changes in a short period due to the change of the formation, which leaded to the update of the routing path. After the short interval, the network throughput is recovered and outperforms the case without $F_{coneq}$ again. The simulation verifies that the algorithm recognizes the dynamic location changes of the mobile devices by the exchange of the neighbors’ network status, and changes the formation of the UAVs to enhance the network throughput in real time. By the calculation of the average network throughput of 10 repetitions, the case with $F_{coneq}$ results in 41.2% better performance than the case without $F_{coneq}$ on average. Since the case without $F_{coneq}$ does not change the formation of the UAVs in time, the case is the representation of the deployment of the location-fixed network infrastructure, and then difference to the case with $F_{coneq}$ is the profit obtained by the proposed algorithm addressed in Section 4. Through the simulation, we verified that the proposed formation control algorithm enhances the network throughput of the multi-UAV-based network infrastructure on the fluctuation of the terrestrial network requirement.

6. Conclusions

In this paper, we study the design of the multi-UAV-based network infrastructure and propose a formation control algorithm. Based on the necessity of an instant and extensible network infrastructure, our algorithm dynamically transforms the network coverage of the infrastructure and leads to the enhancement of throughput. To grant the flexibility of the algorithm, we address the generic network model of the multi-UAV-based network infrastructure and derive protocol-specific parameters that could be applied to the algorithm. In addition, we have several research directions for future work:

• Network protocol. Formation algorithm equalizes the congestion by moving the UAVs, which could possibly incur handover between the APs more frequently. Handover procedures might decrease the throughput of the infrastructure, so the protocol that supports fast recovery from the connection change is needed. Also, we observed that the IPv6 Routing Protocol for low-power lossy networks (RPL) [40] is a promising routing scheme to adapt to the multi-UAV-based network infrastructure. The network model utilizing RPL in the network infrastructure should be analyzed and the pros and cons should be addressed.
• Terrain. This paper assumes that the ground is flat. Although the altitudes of the UAVs are sufficiently high to ignore some elevation of the terrain, some environments with high elevations or buildings could affect the formation of the UAVs. To manage the WLAN network traffic, z axis change could not be enough to approach optimal formation, which needs to improve our formation algorithm.
• Evaluation. We evaluate our formation control algorithm by ns-3. Even though ns-3 is one of the well-known network simulation tool and continually revised for improvements, it could suffer from precision issues due to lack of real-world projection. To evaluate our formation control algorithm on a more practical testbed, a dataset extracted from real-world environments [41] could be desirable.

We believe that our algorithm contributes to the increased availability of the multi-UAV-based network infrastructure in various situations, such as rescue or surveillance missions. Also, we hope that our novel approach to progressively controlling the formation of the mobile network nodes will present new possibilities for flexible network design.