Task Offloading and Resource Allocation for Augmented Reality Applications in UAV-Based Networks Using a Dual Network Architecture
Abstract
:1. Introduction
- This work considers a novel system model for AR applications using UECs to enable efficient task offloading and resource allocation.
- The challenge of dependent tasks in AR applications is addressed. Specifically, this work models the dependencies between tasks and formulates an optimization problem to determine optimal task offloading decisions and resource allocation considering these dependencies.
- This work proposes the DNA-TO algorithm for task offloading decisions, leveraging the DNA framework to mitigate noise in reinforcement learning. Additionally, we proposes the KKT-RA algorithm for optimal resource allocation.
- Numerous simulations with various scenarios, using multiple performance metrics, were conducted to verify the performance of our proposed algorithms. The results show that the proposed algorithms achieved better results than existing methods.
2. Related Work
3. The System Model
- User equipment (UEs): UEs represent mobile devices capable of handling AR tasks. These tasks can be processed locally on the device or offloaded for execution on edge servers to improve performance or handle computationally intensive tasks.
- UAVs with edge computing servers (UECs): UECs are responsible for communicating with UEs in their designated blocks, and for processing offloaded AR tasks upon receiving requests from those UEs.
- The video source: This module acts as the initial stage, capturing raw video frames from the user’s camera. The captured frames serve as the foundation for subsequent processing.
- The tracker: Playing a crucial role in determining the user’s position and orientation relative to the surrounding environment, the tracker analyzes the video frames to identify and track key features, enabling accurate placement of virtual elements within the real world.
- The mapper: This task constructs a digital representation of the environment, often referred to as a map. By analyzing the video data and using information from the tracker, the mapper builds a model that includes feature points (details about the real world). This digital map is essential for aligning virtual objects and ensuring they interact and behave realistically within the user’s physical space.
- The object recognizer: After analyzing video frames to identify known objects and estimating their 3D positions, this information is the input for the renderer.
- The renderer: The final stage of the processing pipeline, the renderer takes the processed data from previous tasks and prepares the frames for display on the user’s device. This may involve overlaying virtual elements onto the video frames, incorporating information from the object recognizer, and ensuring seamless integration of the virtual content with the real world.
3.1. The Communication Model
3.2. The Computation Model
3.2.1. Local Computing
3.2.2. Computing in a UEC
4. Problem Formulation
5. Problem Decomposition
6. Algorithm
6.1. DNA-Based Task Offloading
6.1.1. Markov Decision Process Formulation
- (a)
- State space : At the beginning of each time slot t, the state that includes the environment information is collected by the UEC. Let be the state in time slot t. It is defined as follows:
- represents the set of offloading decisions made for tasks in time slot t. The value of indicates whether task i is executed locally () or offloaded and processed by the UEC ().
- is the set of offloading decisions for tasks in the previous time slot .
- represents the set of relationships between tasks in time slot t and the tasks in time slot for all UEs. Specifically, captures the relationship between the tasks of UE i. A value of indicates that the current task of UE i is the subsequent task in the processing pipeline of its previous task; otherwise, .
- represents the set of computation costs for tasks in the current time slot; the values are normalized to fall between 0 and 1.
- is the set of input sizes for all tasks in the current time slot. The values are also normalized to fall between 0 and 1.
- is the set of location coordinates for all UEs in the system; is the location of UE i. Both x and y coordinates are normalized to fall between 0 and 1.
- (b)
- Action space : This defines the set of all possible actions the agent can take in a single time slot. Let be the action in time slot t, where is the number of tasks in time slot t. Each action is represented by a single integer value within the range . The value of determines whether a task is offloaded or executed locally.
- Offloading: If , it means the task with index in was chosen for offloading to the UEC.
- Local execution: If , it indicates the task with index in is executed locally on the device.
- (c)
- Immediate reward function : To minimize the objective, is designed to negatively correlate with the objective value such that actions leading to lower objective values receive higher rewards. Additionally, to satisfy constraints (19) and (20) in P2, two penalty terms are considered. For constraint (19), let be a binary variable indicating whether the remaining energy of UE i violates constraint (19) or not. It is defined as follows:We define as representing the number of UEs that violate constraint (19). It is calculated as follows:For constraint (20), is a binary variable to indicate whether constraint (20) is violated or not. is defined as:As penalty terms, and are used to discourage the agent from taking actions that violate constraints (19) and (20), respectively. Let s, a, and be the current state, the action taken, and the next state, respectively. denotes the value of the objective function for the next state. The reward denoted by , received by the agent for taking action a in state s and transitioning to state , is calculated as follows:
6.1.2. The DNA-TO Algorithm
- Policy phase: The policy phase optimizes the network. It uses the clipped surrogate objective from proximal policy optimization (PPO) [34] with an entropy bonus, and the policy loss is defined as
- Value phase: The value phase optimizes the network. It uses the squared-error value loss. The loss function is defined as follows:
- Distillation phase: In this phase, knowledge from value network is transferred to the policy network, , through a constrained distillation update [35]. This is achieved using the mean squared error while softly constraining the network’s policy. The distillation loss function is
6.2. Karush–Kuhn–Tucker-Based Resource Allocation
6.3. Algorithm Complexity
7. Evaluation Results
- Greedy algorithm: This method selects the processing device with the lowest weighted sum of energy consumption and execution latency for each task. The KKT-RA algorithm, the same algorithm in our proposal, was used to calculate these metrics and determine the final resource allocation solution.
- Genetic algorithm (GA): A modified genetic algorithm [45] determined the task offloading decision, while the KKT-RA algorithm was used to compute the objective function value and the final resource allocation solution.
- Soft actor critic (SAC): This approach is similar to DNA-TO. However, instead of using the DNA for task offloading, a soft actor critic [46] determines the task offloading decision. Like other methods, KKT-RA was used to calculate the reward for each state and find the final resource allocation solution.
7.1. Simulation Results
7.1.1. Results with a Single UEC
- (a)
- The effect from different numbers of UEs
- (b)
- The effect of different task data sizes
- (c)
- The effect of different CPU cycle requirements
7.1.2. Results with Multiple UECs
- (a)
- The effect from different task data sizes
- (b)
- The effect of different CPU cycle requirements
8. Conclusions
Author Contributions
Funding
Data Availability Statement
Conflicts of Interest
References
- Mendoza-Ramírez, C.E.; Tudon-Martinez, J.C.; Félix-Herrán, L.C.; Lozoya-Santos, J.d.J.; Vargas-Martínez, A. Augmented reality: Survey. Appl. Sci. 2023, 13, 10491. [Google Scholar] [CrossRef]
- Boos, U.C.; Reichenbacher, T.; Kiefer, P.; Sailer, C. An augmented reality study for public participation in urban planning. J. Locat. Based Serv. 2023, 17, 48–77. [Google Scholar] [CrossRef]
- Mao, C.C.; Chen, C.H. Augmented reality of 3D content application in common operational picture training system for army. Int. J. Hum.–Comput. Interact. 2021, 37, 1899–1915. [Google Scholar] [CrossRef]
- Al-Ansi, A.M.; Jaboob, M.; Garad, A.; Al-Ansi, A. Analyzing augmented reality (AR) and virtual reality (VR) recent development in education. Soc. Sci. Humanit. Open 2023, 8, 100532. [Google Scholar] [CrossRef]
- Ashwini, K.B.; Savitha, R.; Ananya, H. Application of Augmented Reality Technology for Home Healthcare Product Visualization. ECS Trans. 2022, 107, 10921. [Google Scholar] [CrossRef]
- Davis, L.; Aslam, U. Analyzing consumer expectations and experiences of Augmented Reality (AR) apps in the fashion retail sector. J. Retail. Consum. Serv. 2024, 76, 103577. [Google Scholar]
- Villagran-Vizcarra, D.C.; Luviano-Cruz, D.; Pérez-Domínguez, L.A.; Méndez-González, L.C.; Garcia-Luna, F. Applications analyses, challenges and development of augmented reality in education, industry, marketing, medicine, and entertainment. Appl. Sci. 2023, 13, 2766. [Google Scholar] [CrossRef]
- Huynh, L.N.; Lee, Y.; Balan, R.K. Deepmon: Mobile gpu-based deep learning framework for continuous vision applications. In Proceedings of the 15th Annual International Conference on Mobile Systems, Applications, and Services, Niagara Falls, NY, USA, 19–23 June 2017; pp. 82–95. [Google Scholar]
- Liu, Q.; Huang, S.; Opadere, J.; Han, T. An edge network orchestrator for mobile augmented reality. In Proceedings of the IEEE INFOCOM 2018-IEEE Conference on Computer Communications, Honolulu, HI, USA, 16–19 April 2018; pp. 756–764. [Google Scholar]
- Dustdar, S.; Avasalcai, C.; Murturi, I. Invited Paper: Edge and Fog Computing: Vision and Research Challenges. In Proceedings of the 2019 IEEE International Conference on Service-Oriented System Engineering (SOSE), San Francisco, CA, USA, 4–9 April 2019; pp. 96–9609. [Google Scholar] [CrossRef]
- Wang, C.; Zhang, S.; Qian, Z.; Xiao, M.; Wu, J.; Ye, B.; Lu, S. Joint server assignment and resource management for edge-based MAR system. IEEE/ACM Trans. Netw. 2020, 28, 2378–2391. [Google Scholar] [CrossRef]
- Chen, X.; Liu, G. Joint optimization of task offloading and resource allocation via deep reinforcement learning for augmented reality in mobile edge network. In Proceedings of the 2020 IEEE International Conference on Edge Computing (EDGE), Beijing, China, 19–23 October 2020; pp. 76–82. [Google Scholar]
- Akter, S.; Duong, D.V.A.; Kim, D.Y.; Yoon, S. Task Offloading and Resource Allocation in UAV-aided Emergency response Operations via Soft Actor Critic. IEEE Access 2024, 12, 69258–69275. [Google Scholar] [CrossRef]
- Sun, G.; He, L.; Sun, Z.; Wu, Q.; Liang, S.; Li, J.; Niyato, D.; Leung, V.C.M. Joint Task Offloading and Resource Allocation in Aerial-Terrestrial UAV Networks with Edge and Fog Computing for Post-Disaster Rescue. IEEE Trans. Mob. Comput. 2024, 23, 8582–8600. [Google Scholar] [CrossRef]
- Morshed Alam, M.; Moh, S. Joint Optimization of Trajectory Control, Task Offloading, and Resource Allocation in Air-Ground Integrated Networks. IEEE Internet Things J. 2024, 11, 24273–24288. [Google Scholar] [CrossRef]
- Wu, G.; Liu, Z.; Fan, M.; Wu, K. Joint Task Offloading and Resource Allocation in Multi-UAV Multi-Server Systems: An Attention-based Deep Reinforcement Learning Approach. IEEE Trans. Veh. Technol. 2024, 73, 11964–11978. [Google Scholar] [CrossRef]
- An, X.; Fan, R.; Hu, H.; Zhang, N.; Atapattu, S.; Tsiftsis, T.A. Joint Task Offloading and Resource Allocation for IoT Edge Computing with Sequential Task Dependency. IEEE Internet Things J. 2022, 9, 16546–16561. [Google Scholar] [CrossRef]
- Yan, J.; Bi, S.; Zhang, Y.J.; Tao, M. Optimal Task Offloading and Resource Allocation in Mobile-Edge Computing with Inter-User Task Dependency. IEEE Trans. Wirel. Commun. 2020, 19, 235–250. [Google Scholar] [CrossRef]
- Aitchison, M.; Sweetser, P. DNA: Proximal policy optimization with a dual network architecture. Adv. Neural Inf. Process. Syst. 2022, 35, 35921–35932. [Google Scholar]
- Zheng, Y.; Fu, H.; Xie, X.; Ma, W.Y.; Li, Q. Geolife GPS Trajectory Dataset—User Guide, Geolife GPS trajectories 1.1 ed.; Microsoft: Redmond, WA, USA, 2011. [Google Scholar]
- Fan, R.; Liang, B.; Zuo, S.; Hu, H.; Jiang, H.; Zhang, N. Robust Task Offloading and Resource Allocation in Mobile Edge Computing with Uncertain Distribution of Computation Burden. IEEE Trans. Commun. 2023, 71, 4283–4299. [Google Scholar] [CrossRef]
- He, Y.; Zhai, D.; Zhang, R.; Du, J.; Aujla, G.S.; Cao, H. A Mobile Edge Computing Framework for Task Offloading and Resource Allocation in UAV-assisted VANETs. In Proceedings of the IEEE INFOCOM 2021-IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS), Vancouver, BC, Canada, 10–13 May 2021; pp. 1–6. [Google Scholar] [CrossRef]
- Xu, D.; Xu, D. Cooperative task offloading and resource allocation for UAV-enabled mobile edge computing systems. Comput. Netw. 2023, 223, 109574. [Google Scholar] [CrossRef]
- Consul, P.; Budhiraja, I.; Garg, D.; Kumar, N.; Singh, R.; Almogren, A.S. A Hybrid Task Offloading and Resource Allocation Approach for Digital Twin-Empowered UAV-Assisted MEC Network Using Federated Reinforcement Learning for Future Wireless Network. IEEE Trans. Consum. Electron. 2024, 70, 3120–3130. [Google Scholar] [CrossRef]
- Zhang, P.; Su, Y.; Li, B.; Liu, L.; Wang, C.; Zhang, W.; Tan, L. Deep Reinforcement Learning Based Computation Offloading in UAV-Assisted Edge Computing. Drones 2023, 7, 213. [Google Scholar] [CrossRef]
- Elgendy, I.A.; Meshoul, S.; Hammad, M. Joint task offloading, resource allocation, and load-balancing optimization in multi-UAV-aided MEC systems. Appl. Sci. 2023, 13, 2625. [Google Scholar] [CrossRef]
- Seid, A.M.; Boateng, G.O.; Mareri, B.; Sun, G.; Jiang, W. Multi-Agent DRL for Task Offloading and Resource Allocation in Multi-UAV Enabled IoT Edge Network. IEEE Trans. Netw. Serv. Manag. 2021, 18, 4531–4547. [Google Scholar] [CrossRef]
- Hao, J. Machine Learning for Road Active Safety in Vehicular Networks. Ph.D. Thesis, Institut Polytechnique de Paris, Palaiseau, France, 2024. [Google Scholar]
- Lin, N.; Tang, H.; Zhao, L.; Wan, S.; Hawbani, A.; Guizani, M. A PDDQNLP algorithm for energy efficient computation offloading in UAV-assisted MEC. IEEE Trans. Wirel. Commun. 2023, 22, 8876–8890. [Google Scholar] [CrossRef]
- Yan, J.; Zhao, X.; Li, Z. Deep Reinforcement Learning Based Computation Offloading in UAV-Assisted Vehicular Edge Computing Networks. IEEE Internet Things J. 2024, 11, 19882–19897. [Google Scholar] [CrossRef]
- Bor-Yaliniz, R.I.; El-Keyi, A.; Yanikomeroglu, H. Efficient 3-D placement of an aerial base station in next generation cellular networks. In Proceedings of the 2016 IEEE International Conference on Communications (ICC), Kuala Lumpur, Malaysia, 22–27 May 2016; pp. 1–5. [Google Scholar] [CrossRef]
- Shakya, A.K.; Pillai, G.; Chakrabarty, S. Reinforcement learning algorithms: A brief survey. Expert Syst. Appl. 2023, 231, 120495. [Google Scholar] [CrossRef]
- Sutton, R.S. Learning to predict by the methods of temporal differences. Mach. Learn. 1988, 3, 9–44. [Google Scholar] [CrossRef]
- Schulman, J.; Wolski, F.; Dhariwal, P.; Radford, A.; Klimov, O. Proximal policy optimization algorithms. arXiv 2017, arXiv:1707.06347. [Google Scholar]
- Hinton, G.; Vinyals, O.; Dean, J. Distilling the knowledge in a neural network. arXiv 2015, arXiv:1503.02531. [Google Scholar]
- Gordon, G.; Tibshirani, R. Karush-kuhn-tucker conditions. Optimization 2012, 10, 725. [Google Scholar]
- Al-Hourani, A.; Kandeepan, S.; Lardner, S. Optimal LAP Altitude for Maximum Coverage. IEEE Wirel. Commun. Lett. 2014, 3, 569–572. [Google Scholar] [CrossRef]
- Sadia, R.; Akter, S.; Yoon, S. Ellipsoidal Trajectory Optimization for Minimizing Latency and Data Transmission Energy in UAV-Assisted MEC Using Deep Reinforcement Learning. Appl. Sci. 2023, 13, 12136. [Google Scholar] [CrossRef]
- Tian, J.; Wang, D.; Zhang, H.; Wu, D. Service Satisfaction-Oriented Task Offloading and UAV Scheduling in UAV-Enabled MEC Networks. IEEE Trans. Wirel. Commun. 2023, 22, 8949–8964. [Google Scholar] [CrossRef]
- Zheng, J.; Cai, Y.; Wu, Y.; Shen, X. Dynamic Computation Offloading for Mobile Cloud Computing: A Stochastic Game-Theoretic Approach. IEEE Trans. Mob. Comput. 2019, 18, 771–786. [Google Scholar] [CrossRef]
- Akter, S.; Kim, D.Y.; Yoon, S. Task offloading in multi-access edge computing enabled UAV-aided emergency response operations. IEEE Access 2023, 11, 23167–23188. [Google Scholar] [CrossRef]
- Romli, R.; Razali, A.F.; Ghazali, N.H.; Hanin, N.A.; Ibrahim, S.Z. Mobile augmented reality (AR) marker-based for indoor library navigation. IOP Conf. Ser. Mater. Sci. Eng. 2020, 767, 012062. [Google Scholar] [CrossRef]
- Gherghina, A.; Olteanu, A.C.; Tapus, N. A marker-based augmented reality system for mobile devices. In Proceedings of the 2013 11th RoEduNet International Conference, Sinaia, Romania, 17–19 January 2013; pp. 1–6. [Google Scholar] [CrossRef]
- Siriwardhana, Y.; Porambage, P.; Liyanage, M.; Ylianttila, M. A Survey on Mobile Augmented Reality with 5G Mobile Edge Computing: Architectures, Applications, and Technical Aspects. IEEE Commun. Surv. Tutorials 2021, 23, 1160–1192. [Google Scholar] [CrossRef]
- Alhijawi, B.; Awajan, A. Genetic algorithms: Theory, genetic operators, solutions, and applications. Evol. Intell. 2024, 17, 1245–1256. [Google Scholar] [CrossRef]
- Haarnoja, T.; Zhou, A.; Hartikainen, K.; Tucker, G.; Ha, S.; Tan, J.; Kumar, V.; Zhu, H.; Gupta, A.; Abbeel, P.; et al. Soft actor-critic algorithms and applications. arXiv 2018, arXiv:1812.05905. [Google Scholar]
For the UEC | |
The remaining energy of the UEC | |
The available computation resource on the UEC | |
The computation resource assigned to task l by the UEC | |
The energy consumed per CPU cycle of the UEC | |
The power consumption per time unit of the UEC when receiving data | |
h | The height of the UEC |
For the UEs | |
The set of UEs | |
The remaining energy of UE i | |
The transmission power of UE i | |
The uplink transmission rate from UE i to the UEC | |
The computation resource of UE i | |
The energy consumed per CPU cycle of UE i | |
Other parameters | |
The set of task types | |
The set of tasks in time slot t | |
The type of task l | |
The computation cost of task l | |
The input data size of task l | |
a and b | Constants that depend on the propagation environment in Equation (1) |
c | The speed of light |
B | The sub-channel bandwidth |
The completion time of task l when executing locally | |
The energy consumption by UE i for locally processed task l | |
The time delay for transmitting the input of task l from to the UEC | |
The energy consumption by UE i when uploading task l to the UEC | |
The energy consumption by the UEC when receiving input data for task l | |
The execution time of task l when executed by the UEC | |
The energy consumption when executing task l at the UEC | |
The completion time of task l when offloaded to the UEC | |
The total energy consumption when offloading task l to the UEC |
Parameter | Value |
---|---|
Number of UECs | {1, 3} |
Number of UEs | 20 |
Bandwidth B (in megahertz) | 20 |
Data size of task (in megabytes): Video source Tracker Mapper Object Recognizer Renderer | [0.5, 8] [0.5, 8] [0.5, 16] [0.5, 8] [0.5, 8] |
Required CPU cycles to complete task: Video source Tracker Mapper Object Recognizer Renderer | [0.1 × 109, 0.5 × 109] [0.1 × 109, 0.5 × 109] [0.1 × 109, 1 × 109] [0.1 × 109, 0.5 × 109] [0.1 × 109, 0.5 × 109] |
Energy capacity of UECs (in megajoules) | 6 |
Energy capacity of UEs (in kilojoules) | [10, 20] |
Computation resource of UECs (in gigahertz) | 50 |
Computation resource of UEs (in gigahertz) | [1.8, 2] |
The carrier frequency f (in gigahertz) | 2 |
The transmission power of UEs (in decibel-milliwatt) | 20 |
Path loss parameter a | 9.61 |
Path loss parameter b | 0.16 |
Path loss parameter | 1.0 |
Path loss parameter | 20 |
Tunable parameter | 0.5 |
The clipping coefficient | 0.2 |
The entropy bonus coefficient | |
The policy constraint coefficient | 1 |
Task | Setup 1 | Setup 2 | Setup 3 | Setup 4 |
---|---|---|---|---|
Video source | [0.5, 2] | [2, 4] | [4, 6] | [6, 8] |
Tracker | [0.5, 2] | [2, 4] | [4, 6] | [6, 8] |
Mapper | [0.5, 4] | [2, 8] | [4, 12] | [6, 16] |
object Recognizer | [0.5, 2] | [2, 4] | [4, 6] | [6, 8] |
Renderer | [0.5, 2] | [2, 4] | [4, 6] | [6, 8] |
Task | Setup 1 | Setup 2 | Setup 3 | Setup 4 |
---|---|---|---|---|
Video source | [0.1 × 109, 0.2 × 109] | [0.2 × 109, 0.3 × 109] | [0.3 × 109, 0.4 × 109] | [0.4 × 109, 0.5 × 109] |
Tracker | [0.1 × 109, 0.2 × 109] | [0.2 × 109, 0.3 × 109] | [0.3 × 109, 0.4 × 109] | [0.4 × 109, 0.5 × 109] |
Mapper | [0.1 × 109, 0.4 × 109] | [0.2 × 109, 0.6 × 109] | [0.3 × 109, 0.8 × 109] | [0.4 × 109, 1 × 109] |
Object Recognizer | [0.1 × 109, 0.2 × 109] | [0.2 × 109, 0.3 × 109] | [0.3 × 109, 0.4 × 109] | [0.4 × 109, 0.5 × 109] |
Renderer | [0.1 × 109, 0.2 × 109] | [0.2 × 109, 0.3 × 109] | [0.3 × 109, 0.4 × 109] | [0.4 × 109, 0.5 × 109] |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Van Anh Duong, D.; Akter, S.; Yoon, S. Task Offloading and Resource Allocation for Augmented Reality Applications in UAV-Based Networks Using a Dual Network Architecture. Electronics 2024, 13, 3590. https://doi.org/10.3390/electronics13183590
Van Anh Duong D, Akter S, Yoon S. Task Offloading and Resource Allocation for Augmented Reality Applications in UAV-Based Networks Using a Dual Network Architecture. Electronics. 2024; 13(18):3590. https://doi.org/10.3390/electronics13183590
Chicago/Turabian StyleVan Anh Duong, Dat, Shathee Akter, and Seokhoon Yoon. 2024. "Task Offloading and Resource Allocation for Augmented Reality Applications in UAV-Based Networks Using a Dual Network Architecture" Electronics 13, no. 18: 3590. https://doi.org/10.3390/electronics13183590