Joint Optimization of Service Migration and Resource Allocation in Mobile Edge–Cloud Computing
Abstract
:1. Introduction
1.1. Motivation
1.2. Our Contributions
- Comprehensive optimization of service migration and resource allocation: We provide a thorough problem formulation that simultaneously optimizes service migration and resource allocation within MECC frameworks. Addressing the challenges of heterogeneous ES environments, our study tackles the intertwined issues of user mobility and fluctuating computational demands. The optimization strategically aims to minimize average response times, substantially enhancing QoS while meeting rigorous temporal constraints.
- Strategic transformation into a Markov decision process (MDP): Moving from theoretical models, this paper adeptly transforms the joint optimization challenge into an MDP. We introduce a novel deep reinforcement learning (DRL)-based algorithm to tackle this MDP, autonomously adapting migration and resource allocation strategies without relying on prior knowledge of system states.
- Rigorous evaluation through simulation: The efficacy and robustness of our proposed DRL-based dynamic migration and resource allocation strategy are rigorously tested through comprehensive simulations. Performance metrics, including task failure rate and average task response delay, serve as benchmarks. The results demonstrate that our DRL-based approach sustains high service quality and markedly reduces average response delays, thereby outperforming established benchmarks in this paper.
2. Related Works
2.1. Joint Optimization of Dynamic Computation Offloading and Resource Allocation
2.2. Optimization of Service Migration Strategy
2.3. Joint Optimization of Service Migration and Resource Allocation
- Acknowledgment of migration delays and impacts: Unlike existing studies that often overlook the critical impact of the reduction in computational time caused by migration processes, our research takes these factors into account. We analyze the direct consequences of migration processes on the operational efficiency of systems, ensuring a more comprehensive understanding of the migration dynamics.
- Exploration of service migration and resource allocation interplay: The interaction between service migration strategies and resource allocation decisions has not been thoroughly examined in prior research. Our study delves into this interplay, aiming to establish a balanced approach that optimizes both elements to improve overall system performance.
- Consideration of incomplete tasks during migration: While a few studies have begun to address the interdependencies between service migration strategies and resource allocation decisions, they rarely consider scenarios where tasks are not completed before migration. This oversight can lead to significant challenges, such as the need to migrate unfinished task data and service contexts together, which can further complicate resource allocation strategies. Our work addresses this gap by incorporating these scenarios into our optimization model, aiming to minimize disruptions and enhance service quality.
3. System Model and Problem Definition
3.1. System Model
3.2. Migration Model
3.3. Computation and Communication Model
3.4. Problem Formulation
4. Proposed A2C-Based Algorithm
4.1. Problem Transformation
4.2. Dynamic Migration and Resource Allocation Algorithm Based on A2C
Algorithm 1: Training of A2C-based dynamic migration and resource allocation algorithm. |
5. Performance Evaluation
5.1. Simulation Settings
5.2. Comparison Experiments
- Follow-Avg scheme: This scheme targets the user’s current location for migration if a task remains incomplete and there is residual time. It then allocates computing resources equally among all tasks on the same server.
- PSO scheme: In this scheme, migration targets and resource allocation decisions are treated as particles within a particle swarm optimization (PSO) algorithm, using average response delay as the fitness function. Decisions are made for each time slot state.
- PPO scheme: Proximal policy optimization (PPO) is employed, a method from online reinforcement learning within the DRL spectrum, to determine service migration and resource allocation.
- DDPG: Deep deterministic policy gradient (DDPG) utilizes the actor–critic framework of DRL to derive migration and resource allocation strategies.
5.3. Simulation Results
5.4. Evaluation of Algorithm Overhead
6. Conclusions and Future Work
Author Contributions
Funding
Data Availability Statement
Conflicts of Interest
References
- Chen, W.; Wang, D.; Li, K. Multi-User Multi-Task Computation Offloading in Green Mobile Edge Cloud Computing. IEEE Trans. Serv. Comput. 2019, 12, 726–738. [Google Scholar] [CrossRef]
- Chen, Z.; Zhou, Z.; Chen, C. Code Caching-Assisted Computation Offloading and Resource Allocation for Multi-User Mobile Edge Computing. IEEE Trans. Netw. Serv. Manag. 2021, 18, 4517–4530. [Google Scholar] [CrossRef]
- Alkhalaileh, M.; Calheiros, R.N.; Nguyen, Q.V.; Javadi, B. Data-intensive application scheduling on Mobile Edge Cloud Computing. J. Netw. Comput. Appl. 2020, 167, 102735. [Google Scholar] [CrossRef]
- Kong, X.; Wu, Y.; Wang, H.; Xia, F. Edge Computing for Internet of Everything: A Survey. IEEE Internet Things J. 2022, 9, 23472–23485. [Google Scholar] [CrossRef]
- Siriwardhana, Y.; Porambage, P.; Liyanage, M.; Ylianttila, M. A Survey on Mobile Augmented Reality With 5G Mobile Edge Computing: Architectures, Applications, and Technical Aspects. IEEE Commun. Surv. Tutor. 2021, 23, 1160–1192. [Google Scholar] [CrossRef]
- Liu, S.; Yu, Y.; Lian, X.; Feng, Y.; She, C.; Yeoh, P.L.; Guo, L.; Vucetic, B.; Li, Y. Dependent Task Scheduling and Offloading for Minimizing Deadline Violation Ratio in Mobile Edge Computing Networks. IEEE J. Sel. Areas Commun. 2023, 41, 538–554. [Google Scholar] [CrossRef]
- Liu, S.; Liu, L.; Tang, J.; Yu, B.; Wang, Y.; Shi, W. Edge Computing for Autonomous Driving: Opportunities and Challenges. Proc. IEEE 2019, 107, 1697–1716. [Google Scholar] [CrossRef]
- Shakarami, A.; Ghobaei-Arani, M.; Shahidinejad, A. A survey on the computation offloading approaches in mobile edge computing: A machine learning-based perspective. Comput. Netw. 2020, 182, 107496. [Google Scholar] [CrossRef]
- Duan, S.; Wang, D.; Ren, J.; Lyu, F.; Zhang, Y.; Wu, H.; Shen, X. Distributed Artificial Intelligence Empowered by End-Edge-Cloud Computing: A Survey. IEEE Commun. Surv. Tutor. 2023, 25, 591–624. [Google Scholar] [CrossRef]
- Chen, X.; Bi, Y.; Chen, X.; Zhao, H.; Cheng, N.; Li, F.; Cheng, W. Dynamic Service Migration and Request Routing for Microservice in Multicell Mobile-Edge Computing. IEEE Internet Things J. 2022, 9, 13126–13143. [Google Scholar] [CrossRef]
- Li, X.; Chen, S.; Zhou, Y.; Chen, J.; Feng, G. Intelligent Service Migration Based on Hidden State Inference for Mobile Edge Computing. IEEE Trans. Cogn. Commun. Netw. 2022, 8, 380–393. [Google Scholar] [CrossRef]
- Wang, P.; Ouyang, T.; Liao, G.; Gong, J.; Yu, S.; Chen, X. Edge intelligence in motion: Mobility-aware dynamic DNN inference service migration with downtime in mobile edge computing. J. Syst. Archit. 2022, 130, 102664. [Google Scholar] [CrossRef]
- Liu, C.F.; Bennis, M.; Debbah, M.; Poor, H.V. Dynamic Task Offloading and Resource Allocation for Ultra-Reliable Low-Latency Edge Computing. IEEE Trans. Commun. 2019, 67, 4132–4150. [Google Scholar] [CrossRef]
- Wang, X.; Ye, J.; Lui, J.C. Decentralized Task Offloading in Edge Computing: A Multi-User Multi-Armed Bandit Approach. In Proceedings of the IEEE INFOCOM 2022—IEEE Conference on Computer Communications, London, UK, 2–5 May 2022; pp. 1199–1208. [Google Scholar] [CrossRef]
- Yang, J.; Yuan, Q.; Chen, S.; He, H.; Jiang, X.; Tan, X. Cooperative Task Offloading for Mobile Edge Computing Based on Multi-Agent Deep Reinforcement Learning. IEEE Trans. Netw. Serv. Manag. 2023, 20, 3205–3219. [Google Scholar] [CrossRef]
- Liang, J.; Xing, H.; Wang, F.; Lau, V.K.N. Joint Task Offloading and Cache Placement for Energy-Efficient Mobile Edge Computing Systems. IEEE Wirel. Commun. Lett. 2023, 12, 694–698. [Google Scholar] [CrossRef]
- Fang, J.; Qu, D.; Chen, H.; Liu, Y. Dependency-Aware Dynamic Task Offloading Based on Deep Reinforcement Learning in Mobile Edge Computing. IEEE Trans. Netw. Serv. Manag. 2023, 1403–1415. [Google Scholar] [CrossRef]
- Zhu, Y.; Mao, B.; Kato, N. A Dynamic Task Scheduling Strategy for Multi-Access Edge Computing in IRS-Aided Vehicular Networks. IEEE Trans. Emerg. Top. Comput. 2022, 10, 1761–1771. [Google Scholar] [CrossRef]
- Liu, L.; Zhao, M.; Yu, M.; Jan, M.A.; Lan, D.; Taherkordi, A. Mobility-Aware Multi-Hop Task Offloading for Autonomous Driving in Vehicular Edge Computing and Networks. IEEE Trans. Intell. Transp. Syst. 2023, 24, 2169–2182. [Google Scholar] [CrossRef]
- Dang, X.; Su, L.; Hao, Z.; Shang, X. Dynamic Offloading Method for Mobile Edge Computing of Internet of Vehicles Based on Multi-Vehicle Users and Multi-MEC Servers. Electronics 2022, 11, 2326. [Google Scholar] [CrossRef]
- Liao, L.; Lai, Y.; Yang, F.; Zeng, W. Online computation offloading with double reinforcement learning algorithm in mobile edge computing. J. Parallel Distrib. Comput. 2023, 171, 28–39. [Google Scholar] [CrossRef]
- Huang, J.; Wan, J.; Lv, B.; Ye, Q.; Chen, Y. Joint Computation Offloading and Resource Allocation for Edge-Cloud Collaboration in Internet of Vehicles via Deep Reinforcement Learning. IEEE Syst. J. 2023, 17, 2500–2511. [Google Scholar] [CrossRef]
- Xu, X.; Liu, K.; Dai, P.; Jin, F.; Ren, H.; Zhan, C.; Guo, S. Joint task offloading and resource optimization in NOMA-based vehicular edge computing: A game-theoretic DRL approach. J. Syst. Archit. 2023, 134, 102780. [Google Scholar] [CrossRef]
- Liu, L.; Feng, J.; Mu, X.; Pei, Q.; Lan, D.; Xiao, M. Asynchronous Deep Reinforcement Learning for Collaborative Task Computing and On-Demand Resource Allocation in Vehicular Edge Computing. IEEE Trans. Intell. Transp. Syst. 2023, 24, 15513–15526. [Google Scholar] [CrossRef]
- Liang, Z.; Liu, Y.; Lok, T.M.; Huang, K. A Two-Timescale Approach to Mobility Management for Multicell Mobile Edge Computing. IEEE Trans. Wirel. Commun. 2022, 21, 10981–10995. [Google Scholar] [CrossRef]
- Bozkaya, E. Digital twin-assisted and mobility-aware service migration in Mobile Edge Computing. Comput. Netw. 2023, 231, 109798. [Google Scholar] [CrossRef]
- Peng, Y.; Tang, X.; Zhou, Y.; Li, J.; Qi, Y.; Liu, L.; Lin, H. Computing and Communication Cost-Aware Service Migration Enabled by Transfer Reinforcement Learning for Dynamic Vehicular Edge Computing Networks. IEEE Trans. Mob. Comput. 2024, 23, 257–269. [Google Scholar] [CrossRef]
- Xu, M.; Zhou, Q.; Wu, H.; Lin, W.; Ye, K.; Xu, C. PDMA: Probabilistic service migration approach for delay-aware and mobility-aware mobile edge computing. Softw. Pract. Exp. 2022, 52, 394–414. [Google Scholar] [CrossRef]
- Wang, H.; Li, Y.; Zhou, A.; Guo, Y.; Wang, S. Service migration in mobile edge computing: A deep reinforcement learning approach. Int. J. Commun. Syst. 2023, 36, e4413. [Google Scholar] [CrossRef]
- Li, C.; Zhang, Y.; Gao, X.; Luo, Y. Energy-latency tradeoffs for edge caching and dynamic service migration based on DQN in mobile edge computing. J. Parallel Distrib. Comput. 2022, 166, 15–31. [Google Scholar] [CrossRef]
- Chen, W.; Chen, Y.; Wu, J.; Tang, Z. A multi-user service migration scheme based on deep reinforcement learning and SDN in mobile edge computing. Phys. Commun. 2021, 47, 101397. [Google Scholar] [CrossRef]
- Chen, W.; Chen, Y.; Liu, J. Service migration for mobile edge computing based on partially observable Markov decision processes. Comput. Electr. Eng. 2023, 106, 108552. [Google Scholar] [CrossRef]
- Liang, Z.; Liu, Y.; Lok, T.M.; Huang, K. Multi-Cell Mobile Edge Computing: Joint Service Migration and Resource Allocation. IEEE Trans. Wirel. Commun. 2021, 20, 5898–5912. [Google Scholar] [CrossRef]
- Liu, F.; Yu, H.; Huang, J.; Taleb, T. Joint Service Migration and Resource Allocation in Edge IoT System Based on Deep Reinforcement Learning. IEEE Internet Things J. 2024, 11, 11341–11352. [Google Scholar] [CrossRef]
- Sutton, R.; Barto, A. Reinforcement Learning, Second Edition: An Introduction; Adaptive Computation and Machine Learning Series; MIT Press: Cambridge, MA, USA, 2018. [Google Scholar]
Symbol | Definition |
---|---|
S | The set of ESs |
U | The set of users |
T | The set of time slots |
The duration of the time slot t | |
Number of task types generated by users | |
A list of tasks offloaded by users at different time slots | |
List of offloaded task information for a single time slot t | |
The state about task k during time slot t | |
The response time constraints of application W | |
Remaining time of in time slot t | |
Remaining size of in time slot t | |
The total computing resources possessed by server | |
The ES associated with the user who offloaded | |
The allocation of on ESs in time slot t | |
Indicator of task completion | |
ES where the is located | |
Migration delay of in time slot t | |
Suspension delay of in time slot t | |
Synchronization delay of in time slot t | |
Restoration delay of in time slot | |
The computational resources allocated to | |
The computational intensity required to suspend service W | |
The computational intensity required to restore service W | |
The context size to be migrated for service W | |
The wired bandwidth between service and target service | |
The average migration delay for services migrating to server | |
The average migration delay for services migrating out of server | |
The average migration delay of server | |
Propagation delay between ESs and cloud server | |
Wireless transmission rate of user j in time slot t | |
Average task data size for application W | |
The uplink transmission delay of | |
The ratio of the result data size to the | |
The delay in returning task results to the ES | |
The delay in downlinking the task results to the user | |
The computational intensity of the task generated by application W |
Parameters | Value |
---|---|
Bandwidth between ESs and cloud | [20, 100] Mbps |
Total computing capacity in ES | [1, 3] GHz |
Suspension processing intensity | [1.0, 2.0] cycles/bit |
State context size | [100, 200] KByte |
Bandwidth between ESs | [100, 200] Mbps |
The ratio of the result data size to initial size | [0.001, 0.005] |
Propagation delay from ESs to the cloud | 400 ms |
Computational intensity requirements | [20, 40] cycles/bit |
Average task data size for W | [2.0, 3.0] MByte |
The bandwidth between user and RANs | 10 MHz |
Transmission power | 10 dBm |
Noise power spectrum density | dBm/Hz |
Network Scale (Number of Users) | Memory Usage (MB) | Number of Iterations | Training Duration per Iteration (s) |
---|---|---|---|
10 | 282.5 | 40,000 | 0.240 |
40 | 312.8 | 40,000 | 0.938 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
He, Z.; Li, L.; Lin, Z.; Dong, Y.; Qin, J.; Li, K. Joint Optimization of Service Migration and Resource Allocation in Mobile Edge–Cloud Computing. Algorithms 2024, 17, 370. https://doi.org/10.3390/a17080370
He Z, Li L, Lin Z, Dong Y, Qin J, Li K. Joint Optimization of Service Migration and Resource Allocation in Mobile Edge–Cloud Computing. Algorithms. 2024; 17(8):370. https://doi.org/10.3390/a17080370
Chicago/Turabian StyleHe, Zhenli, Liheng Li, Ziqi Lin, Yunyun Dong, Jianglong Qin, and Keqin Li. 2024. "Joint Optimization of Service Migration and Resource Allocation in Mobile Edge–Cloud Computing" Algorithms 17, no. 8: 370. https://doi.org/10.3390/a17080370