Next Article in Journal
Numerical Study on Evaluating the Concrete-Bedrock Interface Condition for Hydraulic Tunnel Linings Using the SASW Method
Previous Article in Journal
An Attention-Based Recommender System to Predict Contextual Intent Based on Choice Histories across and within Sessions
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

A Coalition Formation Game Approach for Efficient Cooperative Multi-UAV Deployment

1
The College of Communications Engineering, Army Engineering University of PLA, Nanjing 210000, China
2
Key Embedded Technology and Intelligent System Laboratory, College of Information Science and Engineering, Guilin University of Technology, Guilin 541006, China
3
PLA 75836 Troops, Guangzhou 510400, China
*
Author to whom correspondence should be addressed.
Appl. Sci. 2018, 8(12), 2427; https://doi.org/10.3390/app8122427
Submission received: 6 September 2018 / Revised: 15 November 2018 / Accepted: 24 November 2018 / Published: 29 November 2018

Abstract

:
Unmanned aerial vehicle (UAV) cooperative control has been an important issue in UAV-assisted sensor networks, thanks to the considerable benefit obtained from the cooperative mechanism of UAVs being applied as a flying base station. In a coverage scenarios, the trade-off between coverage and transmission performance often makes deployment of UAVs fall into a dilemma, since both indexes are related to the distance between UAVs. To address this issue, UAV coverage and data transmission mechanism is analyzed in this paper; then, an efficient multi-UAV cooperative deployment model is proposed. The problem is modeled as a coalition formation game (CFG). The CFG with Pareto order is proved to have a stable partition. Then, an effective approach consisting of coverage deployment and coalition selection is designed, wherein UAVs can decide strategies cooperatively to achieve better coverage performance. Combining analysis of game approach, coalition selection and the position deployment algorithm based on Pareto order (CSPDA-PO) is designed to execute coverage deployment and coalition selection. Finally, simulation results are shown to validate the proposed approach based on an efficient multi-UAV cooperative deployment model.

1. Introduction

The past few decades have witnessed rapid development of application of unmanned aerial vehicles (UAVs), popularly known as drones. Given the characteristics of small volume, high mobility and low energy consumption, UAVs have been widely utilized in public, civil and military applications [1,2,3]. In particular, with proper deployment and operation, UAVs can provide reliable and efficient wireless communication solutions for various real-world scenarios. Furthermore, UAVs can act like self-organizing sensor nodes, and efficiently carry out the pre-assigned tasks [4]. For instance, in public safety scenarios (earthquake, volcano, military attack, etc.), UAVs can be utilized as flying base stations, and replace damaged communication infrastructure and guarantee information dissemination for UAV-assisted sensor networks [5,6].
Note that single UAV often fails to meet the need of complicated missions such as area coverage, communication connectivity and relay deployment. To address this issue, the large scale of UAVs are required to form groups (formations) to execute missions under different requirements. In [7], a multi-UAV cooperative model was presented to work out search and surveillance issues. Authors in [8] considered the information consensus under multi-UAVs communication noise, and constructed it as a signal-to-noise ratio model for UAV cooperative target tracking. The authors in [9] proposed a cooperative detection model which consisted of target identification and time control.
UAV area coverage problems were studied in [10,11,12,13,14]. Under the circumstance of cooperative multi-UAV coverage deployment, missions such as investigation, communication and mapping can be accomplished well. The most important part of the UAV area coverage problem is the evaluation of coverage ability. In [10], the authors described the ratio of coverage area and given area as the evaluation of UAV coverage ability, which is one of the most common evaluations. Notably, characteristics of mobility and line-of-sight (LOS) enable UAVs to support terrestrial networks for information dissemination and connectivity enhancement [15,16,17]. Given the fact that the ground communication infrastructure will be destroyed during natural disasters, it is critical for UAVs to perform efficient coverage deployment and execute searching and rescue operations.
In the above scenarios, UAVs arrive at the designated position according to task demand and perform coverage tasks such as data collection. Next, the information (data, etc.) will be forwarded to the central UAV through the UAV-to-UAV link. Note that UAVs can carry limited energy, and the energy consumed by information transmission must be minimized based on mission requirements. Therefore, the data transmission mechanism should be well constructed in UAV communication. In general terms, as the transmitting distance grows, the energy consumption of UAVs will increase exponentially. To avoid high energy consumption, an effective technical solution is proposed in [18], utilizing some UAVs as relays. However, it remains challenging to deploy UAVs and design transmission mechanism due to the complex cooperative behaviors among UAVs.
Motivated by the design of the cooperative mechanism, with which authors significantly reduce the overhead of spectrum acquisition in [19,20], we adopt the coalition formation game (CFG) to describe the relationships between UAVs. Further work about data transmission mechanism design is introduced in [21]. However, unlike the studies above, the coalition is constituted by considering coverage reward and corresponding energy consumption among UAVs in this work. The object of CFG is to study the network coalition structure to optimize performance [22], which can be the basis for investigating cooperative behaviors of UAVs.
Research has been done to study CFG in UAV networks. The authors in [4] proposed a novel CFG model in data collection and transmission, where UAVs are sent to serve randomly located missions, collect data and transmit it to a central command through wireless communication. In [23], the authors took the aerial back haul scheme among UAVs as the mechanism of network formation game, and proposed a model that utilizes UAVs as an on-demand flying network, where UAVs seek to form a multi-hop back haul network in the air. However, the current research of CFG has focused on connectivity enhancement, data transmission improvement and routing, but without considering the trade-off between the coverage reward and energy consumed by transmission in UAV networks. In that situation, we jointly consider a communication coverage and data transmission scenario in the UAV-assisted sensor network, and propose an efficient cooperative UAV deployment model based on coalition formation. The key of coalition formation is to describe the relationships between coverage reward and energy consumption.
The main contribution in this paper can be summarized as follows:
  • An efficient cooperative multi-UAV deployment model based on coalition formation is proposed, describing the trade-off between the coverage reward and energy consumption in UAV-assisted network. Coverage utility function is characterized as a reference for coalition formation to accurately depict the transmission model.
  • Through game analysis, the stable solution of the proposed system model is proved to exist. The proposed UAV coverage and data transmission approach can achieve at least one stable coverage deployment and coalition formation scheme to maximize the coverage utility of the whole UAV network.
  • Coalition selection and position deployment algorithm based on Pareto order (CSPDA-PO) is designed to perform coalition selection and position deployment. The CSPDA-PO algorithm is able to achieve the stable state of our system model. Meanwhile, convergence of the proposed algorithm is also shown in the simulation.
Note that authors in [7,24] did some works that are relevant to our work, especially in constructing the UAV coverage model, including evaluation of coverage ability and deployment mechanism. The main differences can be summed up to: (i) the UAV deployment is not only concerned with the coverage ability, but also the coalition formation under the comprehensive coverage reward and energy consumption; and (ii) the energy consumption model for transmitting and receiving through radio frequency (RF) signals is introduced into coalition formation mechanism, which is associated with coverage reward under current deployment.
The rest of the paper is organized as follows: Section 2 shows the system model of multi-UAV cooperative deployment and problem formulation. In Section 3, a coalition formation game approach for the system model is analyzed, and the learning algorithm is designed to converge the proposed problem to the stable state. Simulation results and analysis are presented in Section 4. Finally, the concluding remarks are given in Section 5.

2. System Model and Problem Formulation

We consider a UAV-assisted sensor network employing N distributed UAVs and one central control UAV, where UAV groups perform reconnaissance and data collection tasks under disaster scenarios. UAVs communicate with each other through radio frequency (RF) signals of wireless sensors. The sensor devices are equipped on UAV’s fuselage. Sensor devices have limited battery power and data processing capabilities. They can also acquire the location of other sensor nodes (UAV). The UAV groups need to collect data information by covering the ground. Here, UAV’s coverage capability is determined by observation thresholds. Then, the data traffic collected by distributed UAVs is sent to the central UAV for processing. Due to the huge energy consumption caused by long distance transmission, it requires forming groups and utilizes some UAV sensor nodes as relays to carry out data transmission.
For the cooperative multi-UAV deployment example showed in Figure 1, there are seven UAVs performing coverage mission and uploading the collected information data to the upper central UAV. Due to the limited energy of sensor devices, UAVs need to form groups to cooperate. In this case, some UAVs are used as relays to optimize the energy consumption of data transmission. In particular, the high transmission energy consumption between UAVs that are too far apart (such as UAV 1 and UAV 7) prevents the formation of the grand coalition. Therefore, it makes sense to consider the coalition formation by optimizing the coverage utility of the UAV network.
To facilitate computation, we discretize the continuous area I R 2 , which is uniformly meshed and divided into cells, and each cell i’s width is l w and is identified by the position of its center g i * = { x i , y i , h i } . Denote the number of available coalitions as M, 1 M N , and the set of UAVs as N , i.e., N = { 1 , 2 , , N } , and the set of available coalition as M , i.e., M = { 1 , 2 , , M } . The serial number of the central UAV is set to 0. For each n N , let c n denote UAV n’s coalition selections, g n = { x n , y n , h n } denotes UAV n’s three-dimensional position. Hence, the UAV n’s state is defined as s n = { g n , c n } . For each m M , let C O m denote the set of UAVs which select coalition m for data transmission, i.e, C O m = { n N : c n = m } . In addition, the central UAV’s position is denoted as g 0 = { x * , y * , h * , h * > h } .
For each i I , suppose that σ ( i ) is the importance (traffic, etc.) of mission cell i, and is denoted by the form of a normalized density function. For simplicity, we set U ξ as the reward for UAV set ξ completing the coverage of mission area I successfully, which is given as follows [1]:
U ξ = I h min n ξ g n g i * σ ( i ) d i ,
where
h ( g n g i * ) ) = 1 , g n g i * ) R n , 0 , o t h e r w i s e .
Equation (2) indicates that, as long as area i is within the detection range of UAV n in the network, it will be considered successfully covered by UAV n with all data traffic collected. Hence, U ξ represents the coverage performance (says reward) of mission area I under UAV set ξ . In particular, the overall coverage reward of UAV network is U N .
From the above description, data traffic collected by distributed UAVs will be transmitted through the UAV-to-UAV link to the central control UAV for processing. In the UAV-assisted sensor network, most of the power energy consumed by wireless sensors is derived extensively from a transmission operation that consists of sending and receiving. Therefore, motivated by the work in [25], we adopt expressions to estimate the energy consumption for transmitting and receiving of each sensor node through RF signals, which are given by:
R e c e i v i n g : E r = E 0 , T r a n s m i t t i n g : E t ( d i , j ) = E 0 + ε s · d i , j 2 , d i , j < d 0 , E 0 + ε l · d i , j 4 , d i , j d 0 ,
where d i , j represents the distance between UAV i and j, E 0 is the energy consumed by radio when performing transmitting and receiving. ε s and ε t represents the energy consumed by transmitting one bit of data at a shorter distance and at a longer distance, respectively.
Thus, consider the data transmission through link of UAV i and UAV j, the overall energy consumption per bit of data is derived as follows:
E ( d i , j ) = E r + E t ( d i , j ) .
In a coalition, cluster head UAV plays an important role in connecting coalition members and the upper central UAV, so as to achieve data transmission. Suppose that the overall energy consumption function of the whole network is defined as follows:
E 0 = m = 1 M n C O m ( E ( d n , c l m ) · U N ) + E ( d 0 , c l m ) · U C O m ,
where c l m represents the coalition m’s cluster head UAV, U n and U C O m is UAV n and UAV set C O m ’s mission reward, indicating the data traffic they collect. Equation (5) contains two items: the first item represents the energy consumption of data transmission, which is collected from coalition m’s cluster member UAV to m’s cluster head UAV. The second item represents the energy consumption of data transmission, which is transmitted from coalition m’s cluster head UAV to the upper central UAV. Therefore, Equation (5) reflects the overall energy consumption of the whole UAV network when all data are transmitted to the central UAV.
From the above formulation, it can be concluded that UAV deployment is affected by the importance of ground data traffic distribution, while the energy consumption of the path is also limited by the communication distance between UAVs. When the battery can’t support long distance UAV communication, the deployment of UAV coverage can be severely affected, making our proposed problem fall into a dilemma. It is meaningful to accurately characterize the relationships between coverage reward and energy consumption. Therefore, according to Equations (1) and (5), we have the whole UAV network’s coverage utility:
U 1 = U N ρ · E 0 ,
where ρ represents the weight coefficient, indicating the importance of reward and transmission energy consumption. Here, ρ is calculated and measured from parameters of UAVs and mission requirements environment. Therefore, the setting of ρ decides the mission requirements and has a strong reference value of weighing the gain and cost. It should be pointed out that some constraints for both variables can be set under the condition of driven mission, such as a guarantee of minimal reward and limitation of maximal energy consumption. For intuitive comparison, the object of this model is maximizing the whole UAV network’s coverage utility through adjusting UAVs’ state:
G : s n o p t = arg max U 1 , n N .
Here, solving G can obtain an efficient multi-UAV deployment and coalition transmission mechanism, which makes UAVs achieve better coverage reward without producing much energy consumed by data transmitting and receiving.

3. A Coalition Formation Game Approach for Coverage Utility Maximization

Notably, solving G is challenging due to the enormous strategies (positions and coalitions), an effective method is required to simplify the process: Step 1 (Reward optimization): Carry out coverage optimization deployment under current coalition formation. Step 2 (Coverage utility maximization): Under the fixed deployment of UAVs, the coalition selection method is considered to maximize U 1 . Step 3: Repeat the above steps until U 1 reaches maximization. This approach can not only avoid the computing complexity caused by multi-strategies, but also be able to work out the optimal solution of G .
In distributed multi-agent control system, players interact with each other and make self-determined strategies. Motivated by that, the proposed UAV efficient deployment problem can be formulated as a game. According to Equation (6), the utility function consists of two items, and we focus on achieving global optimization. Formally, the game is denoted by G = ( N , { S n } n N , { u n } n N ) , where N = { 1 , 2 , , N } is a set of UAVs, S n = G n C n (⊗ is Cartesian product) represents a set of available joint state profiles, where G n and C n = { 1 , 2 , , M } represent a set of the available positions and coalitions for each UAV n, respectively. Denote an element of G n and C n as g n and c n . u n represents the utility function and can be expressed as u n ( s n , s n ) , where s n S n represents the state selection of UAV n and s n S 1 S 2 S n 1 S n + 1 S N is the state profiles of all the UAVs excluding n, the same is true with g n and c n .
Definitions are given in the following to analyze the properties of the coalition formation game.
Definition 1 (Coalition formation game, CFG [22]).
A (hedonic) coalition formation game is given by a pair ( N , P ) , where P = ( 1 , 2 , , n ) denotes the preferences profile, specifying for each player i N its preference relation i .
It should be noted that a coalition partition is denoted as a set Π = { C O m } m = 1 M which partitions N, where the subsets C O m are called coalitions and are disjoint. Let C O Π ( i ) denote the set C O m i.e., if
i N : C O m Π , i C O m .
Apparently, G is a typical CFG. UAVs in G make decisions through different preference relations, and form different coalition partitions. However, first of all, the UAV deployment should be discussed first given the coalition partition Π .

3.1. Analysis of the Mission Reward Maximization

(1) Utility Function: As discussed before, each UAV prefers a higher coverage performance and less energy consumption of transmission. Accordingly, we consider the UAV n’s marginal coverage contribution and measure the coverage utility of mission area I of UAV n as follows:
u n * ( g n , g n ) = I h min k N ( g k g i * σ ( i ) d i I h min k N n ( g k g i * σ ( i ) d i ρ · E 0 ( C O n ) .
In Step 1, given the method that coverage optimization deployment should be done before coalition selection each loop, we have the following equation:
P 1 : max u n * ( g n , g n ) .
(2) Analysis of G 1 : Considering that the cooperative behaviours among UAVs should be well depicted, a potential game is adopted, which applies to a distributed multi-agent system and can associate with the local utility and global utility of each participant [26]. Hence, combining with the distributed self-organizing characteristics of UAVs, the whole system can make decisions to achieve more efficient performance. Here, the definition of Nash equilibrium (NE) is shown in the following, indicating the steady state of a noncooperative game.
Definition 2 (Nash equilibrium (NE) [27]).
A position selection profile g * = ( g n * , , g n * ) is a pure strategy NE of game G 1 = ( N , { S n } n N , { u n * } n N ) if and only if no player n can improve its utility by changing its states, i.e.,
u n * ( g n * , g n * ) u n * ( g n , g n * ) , n N , g n G n , g n g n * .
Definition 3 (Exact potential game (EPG) [28]).
For utility function u n * ( g n , g n ) in a game G 1 = ( N , { S n } n N , { u n * } n N ) , if there exists a potential function ϕ : G n R , for arbitrary position strategy selection changes from g n to g n , the following equation is true:
u n * ( g n , g n ) u n * ( g n , g n ) = ϕ ( g n , g n ) ϕ ( g n , g n ) , n N , G n , g n G n .
Then, this game is called exact potential game and has at least one NE point.
Lemma 1.
Given that the current coalition partition Π, G 1 is an EPG and has at least one NE point.
Proof. 
Here, we construct the potential function as follows:
ϕ ( g n , g n ) = I h min k N g k g i * ) σ ( i ) d i ρ · E 0 ( C O ) ,
which also represents the global coverage utility of UAV network. Assume that an arbitrary UAV n changes its position profile from g n to g n ; then, we have the following formulation:
u n * ( g n , g n ) u n * ( g n , g n ) = I h min k N ( g k g i * σ ( i ) d i I h min k N , g n = g n ( g k g i * σ ( i ) d i I h min k N n ( g k g i * σ ( i ) d i + I h min k N n , g n = g n ( g k g i * σ ( i ) d i + ρ · ( E 0 ( C O n ) E 0 ( C O n ) ) .
Intuitively, for each UAV k N n , its individual coverage performance is completely unaffected by the above action changes, in addition to the energy consumption of UAV set C O k , k N c n . Thus, the results of the last four items from Equation (14) are identical: zero. Then, we derive the following equation according to Equations (13) and (14):
u n * ( g n , g n ) u n * ( g n , g n ) = ϕ ( g n , g n ) ϕ ( g n , g n ) .
The work for the proof has been done in [7]. Therefore, according to Definition 3, G 1 is an EPG with coverage utility U I serving as the potential function. It can be concluded that P 1 has at least one NE point. Note that the designed potential function refers to overall coverage utility. Hence, the NE solution of P 1 turns to be the pure strategy NE point of G 1 . which can be effectively helpful for the proof of the stability in the proposed model G . □
Motivated by the learning algorithm design in [7], we introduce a coverage optimization deployment algorithm based on binary log-linear learning (CODA-BLL) (Algorithm 1) to execute coverage maximization deployment. The proposed algorithm is mainly based on the binary log-linear learning, which was proved to guarantee convergence of Nash equilibrium [7,29]. Given the current coalition selection profiles, the algorithm is executed one time to explore an optimal deployment for coverage utility per loop. Here, we design Equation (16) as the UAVs’ position selection probability function, where β is the learning parameter ( β > 0):
q n g = e x p { β u n * ( g n , g n ) } e x p { β u n * ( g n , g n ) } + e x p { β u n * ( g n , g n ) } .
Algorithm 1. One time coverage optimization deployment algorithm based on Binary log-linear learning (CODA-BLL).
Initialization: Input UAVs’ current state profiles { s n } n N .
Step 1: Randomly select UAV n, calculate its coverage performance u n * ( g n , g n ) of current UAV deployment state according to Equation (9).
Step 2: For selected UAV n, choose an expected action g n from the G n . The selected UAV n computes its current coverage performance u n * ( g n , g n ) and the expected utility u n * ( g n , g n ) according to Equation (6).
Step 3: UAV n choose a position deployment according to Equation (16) and update its state g n .
Step 4: Output UAVs’ state { s n } n N .

3.2. Analysis of the Stable Coalition Partition

(1) Utility Function: In Step 2, under fixed deployment, UAV n’s individual energy consumption can be expressed as follows:
T 1 ( c n ) = k C O Π ( n ) ( E ( d k , c l c n ) · U k ) + E ( d 0 , c l c n ) · U C O Π ( n ) ,
where g c n c l represents the three-dimensional position of coalition c n ’s cluster head UAV. T 1 ( c n ) represents the coalition c n ’s data traffic energy consumption from the terrestrial to the central controller.
Then, for each UAV n N , it makes coalition selection through its individual coverage utility, which can be derived as follows:
u n ( c n , c n ) = u n * ( g n , g n ) ρ · E 0 ( c n ) .
In Definition 1, the preference relation is introduced, which decides whether a player prefers to join or leave the coalition. Notably, the preference relation can affect the convergence and the stable state of the final structure of CFG [30]. Next, an ordinary preference order called Pareto order is adopted to analyze the property of the proposed game model.
Definition 4 (Pareto order).
In CFG, its preference relation n satisfies Pareto order if for each player n N and all coalition partition C O , C O Π , there exists the following formula:
C O n C O u n ( C O ) > u n ( C O ) u i ( C O ) > u i ( C O n ) , i C O { n } , u i ( C O ) < u i ( C O n ) , i C O { n } .
In Pareto order, each player n obeys the principle that they won’t make strategy damage the utility of other players i in the original or new coalition. Hence, the profit of all coalition partition will not decrease. Due to the limited profit players can get, Pareto order can be a very strong proof of stable coalition partition.
The UAVs’ coalition selection satisfies the definition of preference profiles, which also indicates that G 2 is a typical CFG according to Definition 1. Typically, in a CFG, the most important part is the formation criteria, which determines the structure of coalitions. Here, we consider the Pareto order [31], which rests with the preference on the individual payoffs of the players rather than the coalition value [22]. Afterwards, the coalition formation rule is required when the preference order is determined.
Authors in [32] introduced a merge and split rule, which is used for forming or breaking coalition. Both rules focus on the profits of all the players in the coalition. For example, any pair of coalition C O and C O can be merged into one coalition when all the players have an increased profit. On the contrary, a coalition can be split into coalition C O and C O since each players in their new coalition can achieve a better profit. It is noted that, according to Equation (18), for each UAV n, the first item of its individual coverage utility function is fixed due to the stability of G 1 .
(2) Analysis of CFG: In this part, we analyze the stability of the proposed CFG through the given preferences and rules, and then solve the problem.
Definition 5 (Stable coalition partition [19]).
A partition Π is said to be stable if no player can improve its utility by arbitrarily change its strategy, i.e., if
u n ( c n , c n ) u n ( c n , c n ) , n N , c n , c n C n , c n c n ,
then Π is thought to have a stable coalition partition.
Theorem 1.
With the preference relation of Pareto order, the CFG G 2 can be converged to the stable coalition partition.
Proof. 
(1) Note that, given the coalition selection profiles { c n } n N of UAVs in Step 1, the reward maximization model G 1 is proved to be an EPG and has at least one NE point through Lemma 1. According to Definition 2, there exists at least one position selection profile for UAVs if and only if no UAV n can improve its utility by changing its state under the current coalition selection.
(2) In the coverage utility maximization step, the utility function can be derived as u n ( c n , c n ) since UAVs’ deployment is determined. Based on the above description, Pareto order is used for coalition selection while each UAV n take u n ( c n , c n ) as its payoff. In the above description, the sets of players (UAVs) and strategies ( { C n } n N ) are limited, and Pareto order can improve current coalition’s utility without damage any other UAVs, so that the profit of all coalitions can finally converge to a peak. Denote C O * as the final coalition partition. If C O * is not stable, then there exists a UAV, says n, whose arbitrary change will improve u n ( C O * ) , which contradicts the previous argument of limited profit. Finally, we can conclude from Definition 5 that there exists a stable coalition partition Π in CFG G 2 with Pareto order. □

3.3. Algorithm Design of CFG Model

Based on the existence of the stable state in the above section, next we are going to solve the stable stable of G by designing an algorithm. However, due to the diversity of UAV set and strategy sets (position and coalition), the kind of optimal selection approach often falls into a trap loop, that is, local optimum.
In that case, learning algorithms need to be applied to explore the stable state of game. Motivated by the exploring mechanism of a distributed learning algorithm called spatial adaptive play (SAP) in [27], we designed a coalition selection and position deployment algorithm based on Pareto order (CSPDA-PO) to converge the proposed model to the stable coalition state. In each iteration of the algorithm, the chosen UAV updates its coalition selection and makes a comparative update under Pareto order, while all the other UAVs maintain their current selection strategy. Learning from Theorem 1, the stable solution of P 1 can be solved out. In the following table, Algorithm 2 demonstrates procedures of CSPDA-PO. In Equation (16), q n g ( t ) represents the coalition selection probability function of UAV n. β is the learning parameter ( β > 0).
Combining the existence of NE in G 1 with the existence of stable coalition partition in G 2 , our proposed method can converge the problem P 1 to a stable state solution.
Algorithm 2. Coalition selection and position deployment algorithm based on Pareto order (CSPDA-PO).
Initialization: Set j = 1 and the position of central UAV g c e n , initialize UAVs’ state s n = { g n , c n , n N } and mission area I’s state σ ( i ) , i I . Specially, each UAV n N chooses different coalition from C n .
Loop:
Step 1: All UAVs in the network exchange information (coverage deployment and coalition selection) with each other.
Step 2: Randomly select one UAV at each iteration, says n. Input UAVs’ current position profiles { s n } n N to Algorithm 1 and obtain optimal coverage deployment { s n o p t } n N under given coalition selections of UAVs. { s n } n N { s n o p t } n N .
Step 3: All the other UAVs repeat the previous coalition selection, i.e., c k ( j + 1 ) = c k ( j ) , k N n . For the choosing UAV n, it changes its coalition selection to c n C n c n . Update s n = { g n o p t , c n } and input { s n } n N to Algorithm 1, obtain optimal coverage deployment s n o p t under a given coalition selection for UAV n. { s n } n N { s n o p t } n N .
Step 4: Set C O = C O c n , C O = C O c n ; then, UAV n calculates its coverage utility u n ( j ) on the original and expected coalition and updates its strategy according to Definition 4 (Pareto order), where the relationships of utility are explored using probability formula according to Equation (21):
q n c ( j ) = e x p { β u n ( c n , c n ) ( j ) } e x p { β u n ( c n , c n ) ( j ) } + e x p { β u n ( c n , c n ) ( j ) } ,
where β is the learning parameter ( β > 0 ) and can be used to adjust convergence performance and speed.
Step 5: j = j + 1 . If the stop criterion* is satisfied, output UAVs’ state { s n } n N ; otherwise, go to Step 1.
* Stop criterion can be described as the following rules: q n c ( j ) is more than a certain value like 0.98 or j reaches a predefined maximum number of iteration steps.

4. Simulation Results and Discussion

In this section, we carry out simulations to verify the convergence and effectiveness of our proposed approach. Given the simulation parameters in [25], we set E 0 = 50 nJ/bit, ε s = 10 ( nJ / bit / m 2 ) , ε l = 10 ( nJ / bit / m 4 ) . The mission area is uniformly divided into 50 × 50 cells (each cell’s width is 10 m). We consider a UAV network consisting of one central control UAV g c e n = ( 15 , 15 , 40 ) ( × 10 m ) and a certain number of UAVs for data collecting and transmission, whose flying altitudes are all set to 30 (×10 m). All UAVs’ detection ranges are set to be 50m. The amount of data traffic per unit coverage reward is denoted as 1000 MB.

4.1. Diagram Form

Assume that the prior information of the mission area is known. In particular, a probability density function (PDF) which obeys the normal distribution is intended to describe the importance (data traffic) in the mission area. Figure 2a shows the color-coded display of the density map for the mission area, where the length of each mission cell is 10 m, and σ i is denoted as the normalized density of overall data traffic.
Figure 2b–d illustrates deployment and coalition formation diagram of 10 UAVs under different values of weight coefficient ρ . The coloured circles represent the UAVs’ detection range, that is, coverage ability. These figures depict the formation of UAV coalitions and the way data are transmitted. Coalition member UAVs (black dots) send their data to the coalition head UAVs (green triangles), which upload the data to the central UAV (red star). It can be seen that, as ρ increases, the UAVs’ state is more inclined to forming a coalition to minimize the energy consumption, while a mission reward becomes lower. We analyze that, in Equation (18), UAVs’ whole coverage utility is determined by mission reward and energy consumption. After taking the partial derivative of Equation (18), ρ is the indicator that determines the marginal utility of mission reward and energy consumption.

4.2. Basic Performance

In order to better reflect the performance of the proposed approach in UAVs’ coverage deployment and coalition formation, we introduce the coverage reward based deployment approach for comparison. In the comparison approach, the coverage utility only depends on the mission reward and there is no coalition formation mechanism. In order to achieve better contrast, we assume that the coverage reward under the comparison approach converges to that of the proposed approach. Under this premise, we evaluate the overall energy consumption and coverage utility for the specific mission area showed in Figure 2a. As mentioned above, the weight coefficient ρ is the parameter that needs to be calculated and measured from parameters of UAVs and mission environment. For convenience, performance comparisons considering different weight coefficients are paid as reference. To prevent contingency, both algorithms are performed on 100 independent trials; then, the mean results are acquired. Here, we observe some important results from the figures and draw conclusions from the analysis.
Figure 3a,b demonstrates the comparison results. In Figure 3a, as the weight coefficient ρ scales up in the beginning, the overall coverage reward U N of the proposed approach decreases due to the increased importance of energy consumption. Meanwhile, the overall reward of comparison approach tends to be the same as U N , as can be expected. In particular, in the late stage ( ρ = 3.5 × 10 5 4.5 × 10 5 ), UAVs are no longer keen to cover high-yielding areas since the importance of energy consumption is extremely high, which makes UAVs form a stable deployment state.
In Figure 3b, the comparison result shows that the proposed approach can achieve a lower energy consumption E 0 . We analyse that the existence of coalition formation mechanism makes the energy consumption of the proposed approach much less than that of the comparison approach. As the weight coefficient scales up, E 0 decreases rapidly at first. It suggests that the whole UAV network has a bigger tendency to form coalitions when the weight coefficient increases. Another discovery is that, as weight coefficient ρ scales up, energy consumption under the coalition formation mechanism tends to be steady due to the limited data traffic UAV collection.
According to the above results and Equation (6), the comparison of coverage utility is shown in Figure 4, which is an important indicator of the whole UAV network. This figure shows that coverage utility under all approaches decreases as the weighting coefficient increases in the beginning, while the proposed approach can achieve higher coverage utility under the same weight coefficient. This suggests that, without a guarantee of coalition formation mechanism, the system will obtain a fair reward, but, at the same time, needs more energy consumption, so that the coverage utility is less. Likewise, in the late stage ( ρ = 3.5 × 10 5 4.5 × 10 5 ), the coverage utility of the proposed approach tends to be stable since the coverage reward and energy consumption is in a stable state.
Figure 5a,b shows changes in overall coverage reward and overall coverage utility for different numbers of UAVs, respectively. From the figure, we observe that the coverage performance improves as the number of UAVs increases, which are consistent with the fact that more UAVs are able to collect more data information from the ground. However, when the number of UAVs reaches a certain number, the increase in coverage utility will slow down until eventually declining. The main reason says that there is an upper bound for coverage reward due to the limited data information. Meanwhile, in the coalition structure, more UAVs mean more transmission paths, and the high-energy consumption of the increasing transmission paths will make it worthless. Note that the model with the increase of weighting coefficients will also make the model focus more on energy consumption, so coverage utility will decrease with the increase of the weighting coefficient, even reaching the “saturated” state in advance (e.g., eleven UAVs with ρ = 1.3 × 10 5 and twelve UAVs with ρ = 1 × 10 5 ). Therefore, the performance shows such a fluctuant shape.

4.3. Convergence Performance

In order to deeply investigate convergence performance, we choose ρ = 1 × 10 5 as a contrastive index of both approaches. Figure 6a shows the diagram which shows the diagram of the convergence state of UAV deployment and coalition formation. Convergence performance of different algorithms is shown in Figure 6b–d. We observe that all the indicators converge after a certain number of iterations, which indicates that our proposed method can achieve a stable state solution of the system model. The coverage reward of the comparison algorithm converges to the same value. However, since there is no coalition formation mechanism for a comparison approach, its energy consumption and coverage utility converge to a worse level, which is consistent with the model analysis. It is an issue when facing complex environment, such as disaster, which requires UAVs to make accurate and fast self-decision for rapidly changing scenario parameters. Authors in [33] utilized reinforcement learning (RL) to solve the problem of anti-jamming communications in dynamic jamming environment. The application of RL can significantly help UAVs adapting to dynamic environment and make deployment and parameters adjustment accordingly. This gives us inspiration and will be our next step of the work outlook.
To summarize, the simulation results illustrate that the proposed cooperative multi-UAV deployment model based on coalition formation can describe UAVs’ cooperative behaviors well. Moreover, the setting of the weight coefficients can provide a theoretical reference for different task requirements. Finally, the proposed approach can achieve higher coverage utility compared with a comparison approach, reflecting the good performance of coalition formation mechanism and efficiency of the proposed model.

5. Conclusions

In this paper, we proposed an efficient cooperative UAV deployment model in the UAV-assisted sensor network. A multi-UAV coverage deployment and transmission mechanism was established, and the coverage utility and corresponding energy consumptions under the network structure were studied based on cooperation. The system model was transformed into a coalition formation game (CFG). The CFG with Pareto order was proved to have at least a stable coalition partition. Then, coalition selection and a position deployment algorithm based on Pareto order (CSPDA-PO) were designed to perform coalition selection. Finally, simulations were performed to validate the effectiveness of our proposed approach, indicating that the proposed model can describe UAVs’ cooperative behavior well and has better performance compared with other approaches in UAV networks.

Author Contributions

Formal Analysis, Y.Z. and D.L.; Validation, J.C., H.J., Y.Z. and D.L.; Visualization, J.C., Q.G. and H.J.; Writing—Original Draft, L.R.; Writing—Review and Editing, L.R., Y.Z. and D.L.

Funding

This work was supported by the National Natural Science Foundation of China under Grant No. 61771488, No. 61801492, in part by the Natural Science Foundation for Distinguished Young Scholars of Jiangsu Province under Grant No. BK20160034, and the Guang Xi Universities Key Laboratory Fund of Embedded Technology and Intelligent System (Guilin University of Technology).

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Zou, Y.; Jiang, X.; Yan, P.; Zhu, J. Next-generation unmanned aerial vehicle (UAV) cooperative communications. J. Nanjing Univ. Posts Telecommun. (Nat. Sci. Ed.) 2017, 37, 43–51. [Google Scholar] [CrossRef]
  2. Di, B.; Zhou, R.; Duan, H. Potential field based receding horizon motion planning for centrality-aware multiple UAV cooperative surveillance. Aerosp. Sci. Technol. 2015, 46, 386–397. [Google Scholar] [CrossRef]
  3. Mozaffari, M.; Saad, W.; Bennis, M.; Debbah, M. Drone Small Cells in the Clouds: Design, Deployment and Performance Analysis. In Proceedings of the IEEE Global Communications Conference, San Diego, CA, USA, 6–10 December 2015; pp. 1–6. [Google Scholar] [CrossRef]
  4. Saad, W.; Zhu, H.; Basar, T.; Debbah, M. A selfish approach to coalition formation among unmanned air vehicles in wireless networks. In Proceedings of the IEEE International Conference on Game Theory for Networks, Istanbul, Turkey, 13–15 May 2009; pp. 142–149. [Google Scholar] [CrossRef]
  5. Arabi, S.; Sabir, E.; Elbiaze, H.; Sadik, M. Data Gathering and Energy Transfer Dilemma in UAV-Assisted Flying Access Network for IoT. Sensors 2018, 18, 1519. [Google Scholar] [CrossRef] [PubMed]
  6. Li, J.; Chen, J.; Wang, P.; Li, C. Sensor-Oriented Path Planning for Multiregion Surveillance with a Single Lightweight UAV SAR. Sensors 2018, 18, 548. [Google Scholar] [CrossRef] [PubMed]
  7. Li, P.; Duan, H. A potential game approach to multiple UAV cooperative search and surveillance. Aerosp. Sci. Technol. 2017, 68, 403–415. [Google Scholar] [CrossRef]
  8. Fu, X.; Liu, K.; Gao, X. Multi-UAVs Communication-Aware Cooperative Target Tracking. Appl. Sci. 2018, 8, 870. [Google Scholar] [CrossRef]
  9. Alidaee, B.; Wang, H.; Landram, F. On the flexible demand assignment problems: case of unmanned aerial vehicles. IEEE Trans. Autom. Sci. Eng. 2011, 8, 865–868. [Google Scholar] [CrossRef]
  10. Chen, Y.; Zhang, H.; Xu, M. The coverage problem in UAV network: A survey. In Proceedings of the International Conference on Computing, Communication and Networking Technologies, Hefei, China, 11–13 July 2014; pp. 142–149. [Google Scholar] [CrossRef]
  11. Yanmaz, E.; Yanmaz, E.; Muzaffar, R. Survey on unmanned aerial vehicle networks for civil applications: A communications viewpoint. IEEE Press 2016, 18, 2624–2661. [Google Scholar] [CrossRef]
  12. Zhu, Q.; Zhou, R.; Zhang, J. Connectivity Maintenance Based on Multiple Relay UAVs Selection Scheme in Cooperative Surveillance. Appl. Sci. 2017, 7, 8. [Google Scholar] [CrossRef]
  13. Doherty, P. Advanced research with autonomous unmanned aerial vehicles. In Proceedings of the Ninth International Conference on Principles of Knowledge Representation and Reasoning, Whistler, BC, Canada, 2–5 June 2004; pp. 731–732. [Google Scholar]
  14. Acevedo, J.J.; Arrue, B.C.; Maza, I.; Ollero, A. Cooperative perimeter surveillance with a team of mobile robots under communication constraints. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, Tokyo, Japan, 3–8 November 2013; pp. 5067–5072. [Google Scholar] [CrossRef]
  15. Liu, D.; Xu, Y.; Wang, J.; Xu, Y.; Anpalagan, A.; Wu, Q.; Wang, H.; Shen, L. Self-organizing relay selection in UAV communication networks: A matching game perspective. arXiv, 2016; arXiv:1805.09257v1. [Google Scholar]
  16. Zeng, Y.; Zhang, R.; Teng, J.L. Wireless communications with unmanned aerial vehicles: Opportunities and challenges. IEEE Commun. Mag. 2016, 54, 36–42. [Google Scholar] [CrossRef]
  17. Orsino, A.; Ometov, A.; Fodor, G.; Moltchanov, D.; Militano, L.; Andreev, S.; Yilmaz, O.N.; Tirronen, T.; Torsner, J.; Araniti, G.; et al. Effects of heterogeneous mobility on D2D-and drone-assisted mission-critical MTC in 5G. IEEE Commun. Mag. 2017, 55, 79–87. [Google Scholar] [CrossRef]
  18. Fan, R.; Cui, J.; Jin, S.; Yang, K.; An, J. Optimal Node Placement and Resource Allocation for UAV Relaying Network. IEEE Commun. Lett. 2018, 22, 808–811. [Google Scholar] [CrossRef]
  19. Zhang, Y.; Xu, Y.; Anpalagan, A.; Wu, Q.; Xu, Y.; Sun, Y.; Feng, S.; Luo, Y. Context-aware Group Buying in Ultra-dense Small Cell Networks: Unity is Strength. arXiv, 2018; arXiv:1807.08426v1. [Google Scholar]
  20. Ruan, L.; Chen, J.; Qiu, Y.; Liu, X.; Zhang, Y.; Zhu, X.; Xu, Y. Context-aware group buying in D2D networks: An overlapping coalition formation game approach. In Proceedings of the 2017 IEEE 17th International Conference on Communication Technology (ICCT), Chengdu, China, 27–30 October 2017; pp. 142–149. [Google Scholar] [CrossRef]
  21. Ruan, L.; Chen, J.; Guo, Q.; Zhang, X.; Zhang, Y.; Liu, D. Group Buying-Based Data Transmission in Flying Ad-Hoc Networks: A Coalition Game Approach. Information 2018, 9, 253. [Google Scholar] [CrossRef]
  22. Saad, W.; Han, Z.; Debbah, M.; Hjørungnes, A.; Başar, T. Coalitional game theory for communication network. IEEE Signal Process. Mag. Spec. Issue Game Theory 2009, 26, 77–97. [Google Scholar] [CrossRef] [Green Version]
  23. Challita, U.; Saad, W. Network Formation in the Sky: Unmanned aerial vehicles for multi-hop wireless backhauling. arXiv, 2017; arXiv:1707.09132v1. [Google Scholar]
  24. Ruan, L.; Wang, J.; Chen, J.; Xu, Y.; Yang, Y.; Jiang, H.; Zhang, Y.; Xu, Y. Energy-Efficient Multi-UAV Coverage Deployment in UAV Networks: A Game-theoretic Framework. China Commun. 2018, 10, 142–149. [Google Scholar] [CrossRef]
  25. Say, S.; Inata, H.; Liu, J.; Shimamoto, S. Priority-Based Data Gathering Framework in UAV-Assisted Wireless Sensor Networks. IEEE Sens. J. 2016, 16, 5785–5794. [Google Scholar] [CrossRef]
  26. Xu, Y.; Wang, J.; Wu, Q.; Zheng, J.; Shen, L.; Anpalagan, A. Dynamic spectrum access in time-varying environment: Distributed learning beyond expectation optimization. IEEE Trans. Commun. 2017, 65, 5305–5318. [Google Scholar] [CrossRef]
  27. Xu, Y.; Wang, J.; Wu, Q.; Anpalagan, A.; Yao, Y.D. Opportunistic Spectrum Access in Cognitive Radio Networks: Global Optimization Using Local Interaction Games. IEEE J. Sel. Top. Signal Process. 2012, 6, 180–194. [Google Scholar] [CrossRef] [Green Version]
  28. Monderer, D.; Shapley, L.S. Potential games. Games Econ. Behav. 1996, 14, 124–143. [Google Scholar] [CrossRef]
  29. Arslan, G.; Marden, J.R.; Shamma, J.S. Autonomous vehicle-target assignment: A game-theoretical formulation. J. Dyn. Syst. Meas. Control 2007, 129, 584–596. [Google Scholar] [CrossRef]
  30. Zhang, Y.; Xu, Y.; Wu, Q.; Luo, Y.; Xu, Y.; Chen, X.; Anpalagan, A.; Zhang, D. Context Awareness Group Buying in D2D Networks: A Coalition Formation Game-theoretic Approach. IEEE Trans. Veh. Technol. 2018. [Google Scholar] [CrossRef]
  31. Bogomolnaia, A.; Jackson, M.O. The Stability of Hedonic Coalition Structures. Games Econ. Behav. 2002, 38, 201–230. [Google Scholar] [CrossRef] [Green Version]
  32. Apt, K.R.; Witzel, A. A generic approach to coalition formation. Int. Game Theory Rev. 2009, 11, 347–367. [Google Scholar] [CrossRef]
  33. Liu, X.; Xu, Y.; Jia, L.; Wu, Q.; Anpalagan, A. Anti-jamming Communications Using Spectrum Waterfall: A Deep Reinforcement Learning Approach. IEEE Commun. Lett. 2018, 22, 998–1001. [Google Scholar] [CrossRef]
Figure 1. A diagram of unmanned aerial vehicle (UAV) coverage in a mission area.
Figure 1. A diagram of unmanned aerial vehicle (UAV) coverage in a mission area.
Applsci 08 02427 g001
Figure 2. The diagram of 10 UAV deployment and coalition formations under the proposed algorithm: (a) Color-coded display of the density map for the mission area; (b) UAV deployment and coalition formations with ρ = 0.1 × 10 5 ; (c) UAV deployment and coalition formations with ρ = 1 × 10 5 ; (d) UAV deployment and coalition formations with ρ = 2.5 × 10 5 .
Figure 2. The diagram of 10 UAV deployment and coalition formations under the proposed algorithm: (a) Color-coded display of the density map for the mission area; (b) UAV deployment and coalition formations with ρ = 0.1 × 10 5 ; (c) UAV deployment and coalition formations with ρ = 1 × 10 5 ; (d) UAV deployment and coalition formations with ρ = 2.5 × 10 5 .
Applsci 08 02427 g002
Figure 3. Performance comparison of algorithms based on weight coefficients (10 UAVs).
Figure 3. Performance comparison of algorithms based on weight coefficients (10 UAVs).
Applsci 08 02427 g003
Figure 4. Overall coverage utility U 1 in UAV network.
Figure 4. Overall coverage utility U 1 in UAV network.
Applsci 08 02427 g004
Figure 5. Coverage performance considers the number of UAVs and weighting coefficient.
Figure 5. Coverage performance considers the number of UAVs and weighting coefficient.
Applsci 08 02427 g005
Figure 6. The performance convergence graph of the proposed model ( ρ = 1 × 10 5 ). (a) Diagram of UAV deployment and coalition formations; (b) Convergence performance of overall coverage reward; (c) Convergence performance of overall energy consumption; (d) Convergence performance of overall coverage utility.
Figure 6. The performance convergence graph of the proposed model ( ρ = 1 × 10 5 ). (a) Diagram of UAV deployment and coalition formations; (b) Convergence performance of overall coverage reward; (c) Convergence performance of overall energy consumption; (d) Convergence performance of overall coverage utility.
Applsci 08 02427 g006

Share and Cite

MDPI and ACS Style

Ruan, L.; Chen, J.; Guo, Q.; Jiang, H.; Zhang, Y.; Liu, D. A Coalition Formation Game Approach for Efficient Cooperative Multi-UAV Deployment. Appl. Sci. 2018, 8, 2427. https://doi.org/10.3390/app8122427

AMA Style

Ruan L, Chen J, Guo Q, Jiang H, Zhang Y, Liu D. A Coalition Formation Game Approach for Efficient Cooperative Multi-UAV Deployment. Applied Sciences. 2018; 8(12):2427. https://doi.org/10.3390/app8122427

Chicago/Turabian Style

Ruan, Lang, Jin Chen, Qiuju Guo, Han Jiang, Yuli Zhang, and Dianxiong Liu. 2018. "A Coalition Formation Game Approach for Efficient Cooperative Multi-UAV Deployment" Applied Sciences 8, no. 12: 2427. https://doi.org/10.3390/app8122427

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop