Prioritized User Association for Sum-Rate Maximization in UAV-Assisted Emergency Communication: A Reinforcement Learning Approach
Abstract
:1. Introduction
2. Related Work
2.1. Contributions
2.2. Reproducible Research
3. System Model
4. Problem Formulation
5. Proposed Prioritized User Association Algorithm
Algorithm 1: Prioritized User Association for Sum-Rate Maximization Algorithm. |
|
6. Simulation Results
- Benchmark: we tried every possible combinations of ABS user associations, which resulted in the highest mean sum rate compared to other schemes.
- SINR-based user association: users are associated with the ABS from which they received the maximum SINR.
- Distance-based user association: users are associated with the nearest ABS.
- Random user association: users are associated randomly with any ABS in the vicinity without caring about any requirement.
7. Conclusions
Author Contributions
Funding
Conflicts of Interest
References
- Boccardi, F.; Heath, R.W.; Lozano, A.; Marzetta, T.L.; Popovski, P. Five disruptive technology directions for 5G. IEEE Commun. Mag. 2014, 52, 74–80. [Google Scholar] [CrossRef] [Green Version]
- Zeng, Y.; Zhang, R.; Lim, T.J. Wireless communications with unmanned aerial vehicles: Opportunities and challenges. IEEE Commun. Mag. 2016, 54, 36–42. [Google Scholar] [CrossRef] [Green Version]
- Li, Y.; Zhang, J.; Sun, G.; Lu, D. The sparsity adaptive reconstruction algorithm based on simulated annealing for compressed sensing. J. Electr. Comput. Eng. 2019, 2019, 1–8. [Google Scholar] [CrossRef] [Green Version]
- Shakoor, S.; Kaleem, Z.; Baig, M.I.; Chughtai, O.; Duong, T.Q.; Nguyen, L.D. Role of UAVs in Public Safety Communications: Energy Efficiency Perspective. IEEE Access 2019, 7, 140665–140679. [Google Scholar] [CrossRef]
- Al-Hourani, A.; Kandeepan, S.; Lardner, S. Optimal LAP altitude for maximum coverage. IEEE Wirel. Commun. Lett. 2014, 3, 569–572. [Google Scholar] [CrossRef] [Green Version]
- Ueyama, J.; Freitas, H.; Faical, B.S.; Filho, G.P.; Fini, P.; Pessin, G.; Gomes, P.H.; Villas, L.A. Exploiting the use of unmanned aerial vehicles to provide resilience in wireless sensor networks. IEEE Commun. Mag. 2014, 52, 81–87. [Google Scholar] [CrossRef]
- Xi, X.; Cao, X.; Yang, P.; Chen, J.; Quek, T.; Wu, D. Joint User Association and UAV Location Optimization for UAV-Aided Communications. IEEE Wirel. Commun. Lett. 2019, 8, 1688–1691. [Google Scholar] [CrossRef]
- Qiu, C.; Wei, Z.; Feng, Z.; Zhang, P. Joint Resource Allocation, Placement and User Association of Multiple UAV-Mounted Base Stations With In-Band Wireless Backhaul. IEEE Wirel. Commun. Lett. 2019, 8, 1575–1578. [Google Scholar] [CrossRef]
- Li, Q.; Ding, M.; Ma, C.; Liu, C.; Lin, Z.; Liang, Y.C. A Reinforcement Learning Based User Association Algorithm for UAV Networks. In Proceedings of the 2018 28th International Telecommunication Networks and Applications Conference (ITNAC), Sydney, Australia, 21–23 November 2018; pp. 1–6. [Google Scholar] [CrossRef]
- Alzenad, M.; El-Keyi, A.; Lagum, F.; Yanikomeroglu, H. 3-D placement of an unmanned aerial vehicle base station (UAV-BS) for energy-efficient maximal coverage. IEEE Wirel. Commun. Lett. 2017, 6, 434–437. [Google Scholar] [CrossRef] [Green Version]
- Mozaffari, M.; Saad, W.; Bennis, M.; Debbah, M. Efficient deployment of multiple unmanned aerial vehicles for optimal wireless coverage. IEEE Commun. Lett. 2016, 20, 1647–1650. [Google Scholar] [CrossRef]
- Lu, X.; Xiao, L.; Dai, C.; Dai, H. UAV-aided cellular communications with deep reinforcement learning against jamming. IEEE Wirel. Commun. 2020, 27, 48–53. [Google Scholar] [CrossRef]
- Shakhatreh, H.; Khreishah, A.; Ji, B. Providing wireless coverage to high-rise buildings using UAVs. In Proceedings of the 2017 IEEE International Conference on Communications (ICC), Paris, France, 21–25 May 2017; IEEE: Piscataway, NJ, USA, 2017; pp. 1–6. [Google Scholar]
- Azari, M.M.; Rosas, F.; Chen, K.C.; Pollin, S. Ultra reliable UAV communication using altitude and cooperation diversity. IEEE Trans. Commun. 2017, 66, 330–344. [Google Scholar] [CrossRef] [Green Version]
- Qian, Y.; Wang, F.; Li, J.; Shi, L.; Cai, K.; Shu, F. User association and path planning for UAV-aided mobile edge computing with energy restriction. IEEE Wirel. Commun. Lett. 2019, 8, 1312–1315. [Google Scholar] [CrossRef] [Green Version]
- Wang, L.; Huang, P.; Wang, K.; Zhang, G.; Zhang, L.; Aslam, N.; Yang, K. RL-based user association and resource allocation for multi-UAV enabled MEC. In Proceedings of the 15th International Wireless Communications & Mobile Computing Conference (IWCMC), Tangier, Morocco, 15–19 June 2019; IEEE: Piscataway, NJ, USA, 2019; pp. 741–746. [Google Scholar]
- Arani, A.H.; Azari, M.M.; Melek, W.; Safavi-Naeini, S. Learning in the sky: Towards efficient 3D placement of UAVs. In Proceedings of the 2020 IEEE 31st Annual International Symposium on Personal, Indoor and Mobile Radio Communications, London, UK, 31 August–3 September 2020; IEEE: Piscataway, NJ, USA, 2020; pp. 1–7. [Google Scholar]
- Bor-Yaliniz, R.I.; El-Keyi, A.; Yanikomeroglu, H. Efficient 3-D placement of an aerial base station in next generation cellular networks. In Proceedings of the 2016 IEEE International Conference on Communications (ICC), Kuala Lumpur, Malaysia, 23–27 May 2016; IEEE: Piscataway, NJ, USA, 2016; pp. 1–5. [Google Scholar]
- ur Rahman, S.; Kim, G.H.; Cho, Y.Z.; Khan, A. Positioning of UAVs for throughput maximization in software-defined disaster area UAV communication networks. J. Commun. Netw. 2018, 20, 452–463. [Google Scholar] [CrossRef]
- Shakoor, S.; Kaleem, Z.; Do, D.T.; Dobre, O.A.; Jamalipour, A. Joint Optimization of UAV 3-D Placement and Path-Loss Factor for Energy-Efficient Maximal Coverage. IEEE Internet Things J. 2021, 8, 9776–9786. [Google Scholar] [CrossRef]
- Jiao, S.; Xie, X.; Ding, Z. Deep Reinforcement Learning Based Optimization for IRS Based UAV-NOMA Downlink Networks. arXiv 2021, arXiv:2106.09616. [Google Scholar]
- Shakhatreh, H.; Alenezi, A.; Sawalmeh, A.; Almutiry, M.; Malkawi, W. Efficient Placement of an Aerial Relay Drone for Throughput Maximization. Wirel. Commun. Mob. Comput. 2021, 2021, 1–11. [Google Scholar] [CrossRef]
- Lee, W.; Jeon, Y.; Kim, T.; Kim, Y.I. Deep Reinforcement Learning for UAV Trajectory Design Considering Mobile Ground Users. Sensors 2021, 21, 8239. [Google Scholar] [CrossRef] [PubMed]
- Gopi, S.P.; Magarini, M. Reinforcement Learning Aided UAV Base Station Location Optimization for Rate Maximization. Electronics 2021, 10, 2953. [Google Scholar] [CrossRef]
- Amiri, R.; Mehrpouyan, H.; Fridman, L.; Mallik, R.K.; Nallanathan, A.; Matolak, D. A machine learning approach for power allocation in HetNets considering QoS. In Proceedings of the 2018 IEEE International Conference on Communications (ICC), Kansas City, MO, USA, 20–24 May 2018; IEEE: Piscataway, NJ, USA, 2018; pp. 1–7. [Google Scholar]
- Kumar, S.; Suman, S.; De, S. Backhaul and delay-aware placement of UAV-enabled base station. In Proceedings of the IEEE INFOCOM 2018-IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS), Honolulu, HI, USA, 15–19 April 2018; IEEE: Piscataway, NJ, USA, 2018; pp. 634–639. [Google Scholar]
- Mahmood, A.; Usman, M.Q.; Shahzad, K.; Saddique, N. Evolution of Optimal 3D placement of UAV with Minimum Transmit Power. Int. J. Wirel. Commun. Mob. Comput. 2019, 7, 13–18. [Google Scholar] [CrossRef]
- Kim, M.; Lee, J. Outage Probability of UAV Communications in the Presence of Interference. In Proceedings of the IEEE Global Communications Conference (GLOBECOM), Abu Dhabi, United Arab Emirates, 9–13 December 2018; pp. 1–6. [Google Scholar] [CrossRef] [Green Version]
Paper | Problem Statement | Technique/Scheme Used | Improvement Observed |
---|---|---|---|
[5] | An optimal ABS platform to maximize coverage by finding the optimum altitude | Analytical approach | Maximum coverage was achieved at optimal low altitude |
[10] | A 3D placement of UAV-BS by decoupling vertical from horizontal dimensions to maximize the coverage of users by minimizing the transmit power | Optimal placement algorithm | Savings in transmit power and maximized coverage were achieved |
[11] | Deployment of multiple UAVs having directional antennas and optimization of the altitude of UAVs to maximize the coverage area and lifetime of UAVs | Circle-packing theory | The optimum altitude can be obtained on the basis of the number of UAVs and beam width of directional antennas |
[12] | UAV-aided cellular communication network against jamming | Reinforcement learning | A minimized bit error rate and energy saving for the cellular network |
[13] | Single UAV to provide wireless coverage for indoor users when cellular network goes down | Gradient descent algorithm | A minimum transmit power with maximum path loss was obtained |
[14] | Optimizing the height of a UAV to maximize coverage and minimizing outage probability | Decode and forward-relaying method | Maximum coverage with minimum outage was obtained by finding the optimum height of a UAV |
[17] | A 3D deployment of UAV to improve throughput in overloaded and outage situations | Reinforcement learning | A maximum performance gain in terms of throughput was achieved |
[18] | A 3D deployment of UAV to maximize revenue of the network | Bisection search algorithm | The maximized revenue of the network was achieved |
[19] | Throughput maximization by adjusting the position of a UAV in software-defined disaster areas | Centralized algorithm | The throughput was improved by 26% |
[26] | Channel model of backhaul and delay-aware was taken into account to minimize delay by finding the optimum height of a UBS | Backhaul and delay-aware positioning of UBS (BaDPU) algorithm | It was observed that the delay was less for low arrival rates and increased for high arrival rates |
[27] | Optimal UAV placement to maximize the sum rate by using a minimum transmit power | Genetic algorithm | The optimal placement of UAV was achieved with minimum transmit power and minimum path loss |
[21] | Deployment of a UAV equipped with intelligent reflecting surface (IRS) to maximize the sum rate by optimizing the power allocation of a base station (BS), phase shift of the intelligent reflecting surface (IRS), and horizontal position of the UAV | Deep reinforcement learning | An enhanced sum rate was obtained |
[22] | Efficient placement of a UAV-BS serving as a relay node to maximize throughput | Equal power allocation method, water filling method and modified water filling method | Results showed that water filling method gives better results as compared to other two methods |
[23] | Deployment of a UAV by optimizing its trajectory to maximize the mean opinion score (MOS) | Deep Q-learning | The maximized mean opinion score (MOS) was achieved |
Notations | Description |
---|---|
ABS | Aerial base station |
GBS | Ground base station |
N | Number of users connected with ABS |
ABS coordinates | |
Users coordinates | |
ABS transmit power | |
Elevation angle | |
Environmental parameters | |
Carrier frequency | |
d | Distance between ABS and ground user |
Mean additional loss for LoS | |
Mean additional loss for NLoS | |
SINR threshold | |
Reward, state, action at time t | |
Next state and action | |
Discount factor | |
Learning rate |
Parameters | Value |
---|---|
Users (U) | 40 |
ABS (M) | 16 |
−20 dBm | |
25 dBm | |
Step size | 1.5 |
100 | |
600 | |
(R) | 250 m |
0.5 | |
0.9 | |
maxIteration | 50,000 |
Algorithms | Mean Sum Rate (bps/Hz) |
---|---|
Benchmark | 30.5 |
Proposed prioritized user association | 24 |
SINR-based user association | 1.5 |
Distance-based user association | 1.3 |
Random user association | 1.1 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Siddiqui, A.B.; Aqeel, I.; Alkhayyat, A.; Javed, U.; Kaleem, Z. Prioritized User Association for Sum-Rate Maximization in UAV-Assisted Emergency Communication: A Reinforcement Learning Approach. Drones 2022, 6, 45. https://doi.org/10.3390/drones6020045
Siddiqui AB, Aqeel I, Alkhayyat A, Javed U, Kaleem Z. Prioritized User Association for Sum-Rate Maximization in UAV-Assisted Emergency Communication: A Reinforcement Learning Approach. Drones. 2022; 6(2):45. https://doi.org/10.3390/drones6020045
Chicago/Turabian StyleSiddiqui, Abdul Basit, Iraj Aqeel, Ahmed Alkhayyat, Umer Javed, and Zeeshan Kaleem. 2022. "Prioritized User Association for Sum-Rate Maximization in UAV-Assisted Emergency Communication: A Reinforcement Learning Approach" Drones 6, no. 2: 45. https://doi.org/10.3390/drones6020045