Online Service-Time Allocation Strategy for Balancing Energy Consumption and Queuing Delay of a MEC Server

Abstract

MEC servers (MESs) support multiple queues to accommodate the delay requirements of tasks offloaded from end devices or transferred from other MESs. The service time assigned to each queue trades off the queue backlog and energy consumption. Because multiple queues share the computational resources of a MES, optimally scheduling the service time among them is important, reducing the energy consumption of a MES and ensuring the delay requirement of each queue. To achieve a balance between these metrics, we propose an online service-time allocation method that minimizes the average energy consumption and satisfies the average queue backlog constraint. We employ the Lyapunov optimization framework to transform the time-averaged optimization problem into a per-time-slot optimization problem and devise an online service-time allocation method whose time complexity is linear to the number of queues. This method determines the service time for each queue at the beginning of each time slot using the observed queue length and expected workload. We adopt a long short-term memory (LSTM) deep learning model to predict the workload that will be imposed on each queue during a time slot. Using simulation studies, we verify that the proposed method strikes a better balance between energy consumption and queuing delay than conventional methods.

1. Introduction

A key characteristic of 5G systems is that most data are generated and consumed locally [1]. To accommodate this characteristic, a multi-access edge computing (MEC) system should be an integral part of 5G systems [2,3,4]. By enabling devices to offload their tasks to nearby MEC servers (MESs) deployed at the edge of a network, a MEC system is expected to reduce the task service delay. In addition, a MEC system can decrease the traffic load to be imposed on the backhaul network if devices use a cloud server in a data center located far from the devices.

The offloading decision problem that determines whether a device offloads a task to a MES or processes it locally has drawn much attention from researchers. To save the energy of a device and reduce the task completion time, various methods for a device to make an offloading decision have been proposed [5,6,7,8,9]. After a device offloads a task to a MES, it can move out of the service area of the MES. Then, the delay requirement of the task is likely to be violated. To resolve this issue, task migration methods have been proposed in the literature [10,11]. They attempt to overcome the limited coverage of a MES and support the user mobility without task QoS degradation.

Once a MES receives tasks from devices or other MESs, the MES processes the tasks by appropriately managing its computation and storage resources. The methods managing the resources of a MES mainly focus on solving the MES congestion problem to increase the utility of a MEC system. In other words, they attempt to prevent a MES from being congested by balancing loads among MESs or transferring tasks from a congested MES to a cloud server. The load-balancing methods proposed in [12,13] redistribute loads among MESs by driving devices to offload their tasks to the MES that can serve them the best. However, the load of a MES is influenced not only by the tasks offloaded directly from devices, but also the tasks migrated from other MESs. Thus, in [14], the authors attempted to balance the loads among MESs by moving the tasks in an overloaded MES to an underloaded MES. A MES congestion problem can also be resolved by an efficient collaboration between a MES and a cloud server [15,16]. A MES determines whether to process a task or offload it to a cloud server to reduce overall energy consumption and latency while satisfying the task’s service requirement [15]. The authors in [16] proposed a task offloading method at a MES to minimize the task processing cost of a MES while guaranteeing queue stability.

Previous works on MEC system resource management considered all the resources in a path from a device to a MES. Thus, the formulated problem involves not only the local information, but also the information of the other nodes. Sometimes, they even require the system-wide global information. Then, it is assumed implicitly or explicitly that all the information needed to resolve the formulated problem is known to a device or a MES whenever it makes a resource management decision. However, in a practical MEC system, the nodes in a MEC system are supposed to be autonomous. Thus, the state information of one node is hardly known to the other node. For example, in cellular networks, user devices do not exchange information among themselves. Moreover, a user device and a MES are recommended not to exchange their state information for security reasons. Therefore, to implement a practical MEC system, it is important for a node to make a resource management decision by using only its local information.

As noted in [17], there are various MEC application scenarios and each application task has its specific QoS requirements. It has been assumed in the literature that a MES provides a separate FIFO queue for each device. A device may offload several tasks having different QoS requirements. Then, to satisfy the specific QoS requirement of each task, a MES has to maintain a separate queue for each task offloaded by each device and exert a sophisticated computation scheduling method. Thus, the resource management burden of a MES increases as the number of devices and the number of tasks with different QoS requirements increase. To resolve such an issue, the class-based QoS concept has often been adopted in wireless networks and the Internet. The basic idea of the class-based QoS concept is to group various applications into a set of classes according to their QoS characteristics and guarantee the QoS specified for each class, instead of handling the specific QoS requirement of each application task separately. For example, the LTE network assigns a QoS class identifier to each carrier type. Various application services are classified according to their priority, packet error rate, and packet delay budget [18]. Applications with similar QoS requirements in terms of the priority, packet error rate, and packet delay budget are classified into the same class. A similar concept is adopted in the Internet as the DiffServ service classes [19]. In this paper, we follow the class-based QoS concept. A MES classifies tasks according to their delay requirements and manages a queue for each class. A MES manages the service-time allocated to each queue to provide differentiated services among the queues in terms of the average queuing latency.

Another important criterion that must be considered in devising a MES resource management method is the energy consumption of a MES. The amount of energy consumed by a MES to serve a set of class queues must be minimized to reduce both the operational expenditure and the amount of carbon dioxide emissions. A MEC system is expected to reduce the energy consumption of a device. Therefore, the issue of minimizing the energy consumption of a device has been inspected thoroughly. However, comparatively, the problem of minimizing the energy consumption of a MES has not been explored extensively. Thus, in this paper, we address the service-time allocation problem of a MES in the situation where the task input process to a MES varies in a probabilistic way and its distribution is not known in advance. In this environment, we devise a service-time allocation strategy by using only the task queue length and the recent sequences of the task input rates, which can be measured by a MES without any aid of other nodes. Specifically, we aim to minimize the time-averaged energy consumption of a MES, which is not overloaded while guaranteeing the average task queue length below a given threshold for all the queues. The contributions of this study are as follows:

• We consider a service-time allocation problem at a MES. We formulate a time-averaged energy-minimization problem with an average queuing delay constraint. The formulated problem is complex because it involves information on the input dynamics for future time slots when determining an optimal allocation strategy. Thus, using the Lyapunov approach, we transform it into a per-time-slot optimization problem and devise an online service time allocation method.
• Unlike other Lyapunov-based methods that use only the observed queue length at the beginning of a time slot, we explicitly consider the dynamics of an input process by predicting the amount of workload that will be imposed on each queue during each time slot. For the prediction, we use a long short-term memory (LSTM) model [20], which is a deep learning model widely used in predicting time series data [21,22]. Through simulation studies, we show that we can strike a better balance between the average energy consumption and average queuing delay than with a conventional Lyapunov-based method.
• We devise a service-time allocation algorithm whose complexity is $O(|N|)$, where N is the set of queues managed by a MES. Most optimization-based methods use an iterative process to resolve the formulated problem. However, the iterative process requires time to converge. For example, the convergence time of a subgradient method with a constant step size is known to be $O(1/ϵ)$[23], where $0<ϵ\ll 1$ is the difference between the termination point and an optimal point. Thus, the time complexity of the conventional method is $O(|N|/ϵ)$, which is greater than that of ours.
• By providing a weight parameter on the importance of the energy consumption, our method enables a MEC system operator to control the correct balance between the energy consumption and queuing delay according to his/her operational purpose.

The remainder of this paper is organized as follows. In Section 2, we present the related works. In Section 3, we describe the system model and formally state the service-time allocation problem. In Section 4, we present our online service-time allocation method. After validating the proposed method by comparing its performance to those of conventional methods in Section 5, we conclude the paper with future research directions in Section 6. We present the notations used in this paper in Abbreviations for the readers’ convenience.

2. Related Works

2.1. Resource Management Methods in a MEC System

The resource management problem in a MEC system has been investigated in various scenarios. In [6], the authors formulated a long-term system energy-minimization problem with average end-to-end delay constraints in a MEC system composed of one MES, multiple APs, and multiple devices, all exploiting low-power sleep mode. By using the Lyapunov optimization framework, they separated the problem into a CPU scheduling problem at a MES and a device–AP association problem at a device. They solved the first problem by an iterative algorithm and the second problem based on multi-agent reinforcement learning (MARL). In [7], an optimal secondary sensing strategy for saving the energy consumption of a device was designed in an environment where devices opportunistically use a licensed spectrum as the secondary users. A cost function was defined as the sum of the total energy consumption of the devices and the total queue length, which is the sum of the queue length of the devices and the queue length at a MES for each device. Each device employs a deep Q network (DQN) agent to minimize the long-term average cost by making a sensing decision for each channel and selecting both the local CPU frequency and the transmission power. In [8], the authors studied the task offloading problem and the resource allocation problem in multiple devices and a multi-edge network scenario. To reduce the resource cost for task offloading and improve the utilization of server resources, they proposed an online predictive offloading algorithm based on deep reinforcement learning (DRL) and LSTM. LSTM was used to predict the next arriving task, which was used by a DQN agent in a device for making an offloading decision. When a device decides to offload a task, the DQN agent also determines an optimal transmission power to minimize the offloading cost. However, in these works [6,7,8], a deep learning agent resides in a device and jointly determines its local decision parameters and the parameters related to the service-time allocation in a MES. Therefore, a device requires the local information, the information in a MES, and the global information to determine the optimal parameters. However, it is not clear how an agent in a device acquires the information of the other nodes and how the decision made by a device is transferred to a MES. In addition, they are device-centric methods. Since decisions are made by a device, their main goal is not to minimize the energy consumption of a MES, but to minimize the energy consumption of a device.

MES-centric methods were proposed in [9,16]. The authors in [16] considered a MEC system composed of one cloud server, one MES, and multiple mobile devices. They focused on the task offloading problem from a MES to a cloud server to minimize the time-averaged penalty. They cast the resource allocation problem at a MES as a Lyapunov optimization problem and proposed a DRL-based approach to solve the problem by incorporating the queue stability constraint into the DRL formulation. They devised a DRL agent by using a soft actor–critic (SAC) algorithm, which learns an optimal policy for resource allocation at a MES while stabilizing the queues. By using the agent, a MES optimally determines whether to offload a task to a cloud server or process the task by itself under the task queue stability condition. Basically, the problem addressed by the authors is a MES congestion resolution problem. In this paper, we deal with the energy consumption minimization problem when a MES is not overloaded. However, we note that our method can work with their method. When a MES detects an impending congestion while using our method, it can invoke the DRL agent to offload some tasks to a cloud server. In [9], an optimal scheduling problem was formulated, which aimed to minimize the long-term average weighted sum of energy consumption and delay of all mobile devices in an environment with one 5G small-cell BS with multiple antennas, one MES, and multiple mobile devices. Since the formulated problem requires information that is generally not available in a practical system, the authors proposed a DDPG-based solution that learns an optimal task offloading decision from mobile devices to a MES, the transmission power of each mobile device, and CPU frequencies allocated by a MES to each task. However, to lean an optimal policy, an agent still needs system-wide information such as the tasks selected by all mobile devices and the channel condition between a BS and all mobile devices. In addition, the agent has to distribute the result of the offloading and resource allocation decision to all the mobile devices so that they take actions accordingly. Since the signaling overhead increases with the number of mobile devices, it may cause a scalability problem. Furthermore, the signaling procedures between devices and a MES is not clear.

2.2. Technical Approaches

A resource management problem in a MEC system is often formulated as an optimization problem with constraints. Although the specific system models are different, the formulated optimization problems are often NP-hard or NP-complete. Various technical approaches were adopted to reduce the complexity of the formulated problem. The authors in [24] cast the virtual network function (VNF) instance deployment and user request assignment problem as an integer linear programming problem and proposed two heuristic algorithms with limited complexity. In [14], the authors proposed a task-redirection method to balance the loads among MESs. By lexicographically minimizing the MES load vector, they increased the resource efficiency of a MEC system and reduced the average task blocking rate. In [25], a utility maximization problem was formulated to jointly optimize the video segment cache, transcoding strategy, and wireless resource-allocation strategy. Because this is a mixed-integer nonlinear programming problem, they decomposed the original problem into multiple simpler subproblems and proposed a low-complexity heuristic method to resolve each subproblem. To simplify the objective function, auxiliary variables were used in [26] to minimize the maximum task execution delay. Bilevel programming and matching algorithms were used in [27] to optimize the task assignment, power allocation, and user association jointly. To reduce the complexity of the formulated problem, these methods transform the original problem into a more tractable form by simplifying the objective function or decomposing it into multiple simpler subproblems. They then devised an iterative process to find an optimal solution to the converted problem. However, the iterative process requires time before finding an optimal solution. Thus, applying them to make online resource-allocation decisions may be inappropriate.

Game theory is also used for resource management in MEC systems [28,29,30]. Methods based on game theory are scalable and easily adapt to unexpected behaviors. However, implementing a lightweight algorithm with limited complexity using theoretical game models is not easy [31]. For example, a cooperative game requires information exchanges among players, which incurs high signaling costs in a MEC system. Machine learning models have been used to resolve resource management problems. In [32], three regression models were used to obtain an optimal subcarrier allocation. However, because they used an exhaustive search to find optimal labels when constructing training datasets, the computational complexity of their method was high. In addition, they did not consider the input load dynamics or queuing delay when making a resource allocation decision. Therefore, the performance achieved using these methods is not expected to exceed a certain threshold. In [33], a deep deterministic policy gradient (DDPG) managed the available spectrum, computing, and caching resources for MEC-mounted base stations and unmanned aerial vehicles. DDPG was also used in [34] to solve the optimal user association and video quality selection problems. However, DDPG is sensitive to hyperparameters and often converges to a poor solution [35]. The authors in [36] used the Lyapunov optimization approach to optimally offload tasks in a dense cloud radio access network (RAN). The methods using the Lyapunov approach manage resources using only the observed queue length at the start of each time slot. However, because they do not consider input dynamics, these methods are suboptimal, in particular when the input load changes widely over time.

3. System Model and Problem Formulation

Figure 1 shows the system model that we consider in this paper. Tasks arriving at a MES can be classified into two groups. One group is composed of the tasks offloaded directly from devices, and the other group comprises the tasks sent by other MESs by means of the task migration and the load-balancing. In this paper, we focus on the service-time allocation problem occurring when a MES serves the received tasks. Therefore, we do not distinguish the former from the latter. We assumed that once a MES receives a task, it classifies the task according to its type and buffers its workload in a queue arranged for that type of task. To devise an efficient service-time allocation strategy in a MES, we considered a MES that supports a set N of task classes. The time is divided into equally sized slots. We denote $\tau$ as the length of a time slot and $f_{mes}$ as the CPU frequency of a MES. A task imposes a workload in terms of the number of CPU cycles required to complete the task. For each task class i, a MES maintains a queue i, which stores the workloads imposed by the class tasks i. We assumed a stochastic fluid queue for each class [37]. That is, we considered the workload imposed by a task to be a continuous fluid, although the workload is a discrete unit. Thus, we do not delineate task boundaries in each queue. We denote the workload stored in queue i at the start of time slot t as $Q_i\left(t\right)$.

During each time slot t, the CPU of a MES is shared by the set of queues. If we denote the proportion of time that a MES serves a queue i during a time slot t as ${\alpha }_i\left(t\right)$, the number of CPU cycles used to serve a queue i becomes

$$b_i\left(t\right)={\alpha }_i\left(t\right)\tau f_{mes},0\le {\alpha }_i\left(t\right)\le 1,\forall i\in N,\forall t>0.$$

(1)

We adopted a popular cubic model to quantify the energy consumption of a MES [38]. Thus, during time slot t, the amount of energy consumed by a MES to serve queue i is

$$e_i\left(t\right)={\alpha }_i\left(t\right)\tau \kappa f_{mes}^3,$$

(2)

where $\kappa$ denotes the effective switching capacity of the processor. Therefore, the amount of energy consumed by a MES during time slot t is obtained as:

$$E\left(t\right)=\sum _{i\in N}{e}_i\left(t\right).$$

(3)

If we denote the workload newly imposed on queue i during time slot t as $w_i\left(t\right)$, the dynamics of $Q_i\left(t\right)$ can be described as

$$Q_i(t+1)=max\{0,Q_i\left(t\right)-b_i\left(t\right)+w_i\left(t\right)\}.$$

(4)

According to Little’s law, if the data arrival rate to a stationary queuing system is given as ${\overline{w}}_i=\mathbb{E}[w_i\left(t\right)/\tau ]$, the long-term time-averaged queuing latency is given as

$${\overline{L}}_i=\underset{T\to \infty }{lim}\frac{1}{T}\sum _{t=1}^T\mathbb{E}[\frac{Q_i\left(t\right)}{{\overline{w}}_i}].$$

(5)

${\overline{L}}_i$ is proportional to the long-term time-averaged queue length. To provide differentiated services among the queues in terms of the queuing latency, the long-term time-averaged queue length should be maintained under the queue length threshold ${\delta }_i={\overline{w}}_i{\overline{L}}_i$. That is,

$$\underset{T\to \infty }{lim}\frac{1}{T}\sum _{t=1}^T\mathbb{E}[Q_i\left(t\right)]\le {\delta }_i,\forall i\in N.$$

(6)

Since ${\delta }_i$ is determined by the delay characteristics of the tasks belonging to the class i, we assumed that ${\delta }_i$ is pre-defined for each class queue.

As seen in Equations (2) and (4), $e_i\left(t\right)$ increases with ${\alpha }_i\left(t\right)$, while $Q_i\left(t\right)$ decreases as ${\alpha }_i\left(t\right)$ increases. Thus, to strike the correct balance between energy consumption and queuing delay, we require a systematic method to determine ${\alpha }_i\left(t\right)$. To address the trade-off between the energy consumption and queuing delay, we intend to minimize the long-term time-averaged energy consumption of a MES while satisfying the queue length constraint in Equation (6) by optimally controlling the proportion of time that a MES serves each queue. We formulated our problem as the following optimization problem.

$$\begin{array}{c}\hfill \mathrm{P}1:min\underset{T\to \infty }{lim}\frac{1}{T}\sum _{t=1}^T\mathbb{E}[E\left(t\right)]\end{array}$$

(7)

$$\begin{array}{c}\hfill s.t.\underset{T\to \infty }{lim}\frac{1}{T}\sum _{t=1}^T\mathbb{E}[Q_i\left(t\right)]\le {\delta }_i,\forall i\in N.\end{array}$$

(8)

$$\begin{array}{c}\hfill 0\le {\alpha }_i\left(t\right)\le 1,\forall i\in N.\end{array}$$

(9)

$$\begin{array}{c}\hfill \sum _{i\in N}{\alpha }_i\left(t\right)\le 1.\end{array}$$

(10)

Problem P1 is difficult to solve because we cannot know the statistics of the task arrivals for all time slots when we must determine ${\alpha }_i\left(t\right)$s. To tackle this difficulty, we adopted the Lyapunov optimization framework and a deep learning model to devise an online service-time allocation method that determines $\mathsf{\Omega }\left(t\right)=\{{\alpha }_i\left(t\right):i\in N\}$ at the beginning of each time slot t.

4. Online Resource Allocation Method

In this section, we convert Problem P1 into a per-time-slot optimization problem using the Lyapunov optimization framework. We then design an online service-time allocation strategy that can solve the converted problem in $O(|N|)$.

4.1. Per-Time-Slot Optimization Problem

To control the delay constraint in P1, we define the virtual queue $Z_i\left(t\right)$ for each task queue that evolves as follows:

$$Z_i(t+1)=max[0,Z_i\left(t\right)+Q_i(t+1)-{\delta }_i],\forall i\in N.$$

(11)

A virtual queue is said to be mean rate stable if

$$\underset{T\to \infty }{lim}\frac{\mathbb{E}[Z_i\left(T\right)]}{T}=0.$$

(12)

From the Lyapunov optimization framework in [39], the delay constraint in P1 is satisfied when the virtual queue is mean rate stable. To guarantee the mean rate stability of the virtual queue, we define the following Lyapunov function:

$$L\left(Z\left(t\right)\right)=\frac{1}{2}\sum _{i\in N}{Z}_i{\left(t\right)}^2,\forall t.$$

(13)

We also define the Lyapunov drift, which is the conditional expected changes in the Lyapunov function over one time slot.

$$\Delta L\left(Z\right(t\left)\right)=\mathbb{E}[L(Z(t+1))-L(Z\left(t\right))|Z\left(t\right)].$$

(14)

By minimizing $\Delta L\left(Z\right(t\left)\right)$ for each time slot, we can make $Z_i\left(t\right)$s mean rate stable. However, minimizing only $\Delta L\left(Z\right(t\left)\right)$ may result in unnecessary energy consumption. Thus, following the approach in [39], we define the drift-plus-penalty function ${\Delta }_{p}L\left(Z\left(t\right)\right)$ to consider the energy consumption of a MES.

$${\Delta }_{p}L\left(Z\left(t\right)\right)=\Delta L\left(Z\left(t\right)\right)+V\mathbb{E}[E\left(t\right)|Z\left(t\right)],$$

(15)

where V is a weight parameter that controls the importance of the MES energy consumption on ${\Delta }_{p}L\left(Z\left(t\right)\right)$. For example, as V increases, the energy consumption becomes more important than the delay constraint for each queue. From the concept of opportunistically minimizing the expectation, our problem becomes minimizing ${\Delta }_{p}L\left(Z\left(t\right)\right)$ for each time slot. To achieve this goal, we introduce Proposition 1, which provides the upper bound of ${\Delta }_{p}L\left(Z\left(t\right)\right)$.

Proposition 1.

${\Delta }_{p}L\left(Z\left(t\right)\right)$ is upper bounded by

$${\Delta }_{p}L\left(Z\left(t\right)\right)\le B+\mathbb{E}[\sum _{i\in N}{b}_i{\left(t\right)}^2-b_i\left(t\right)(2Q_i\left(t\right)+2w_i\left(t\right)+Z_i\left(t\right))+V{b}_i\left(t\right)\kappa f_{mes}^3|Z\left(t\right)],$$

(16)

where $b_i\left(t\right)$ = ${\alpha }_i\left(t\right)\tau f_{mes}$ and B = ${\sum }_{i\in N}{Q}_i{\left(t\right)}^2+w_i{\left(t\right)}^2+2Q_i\left(t\right)w_i\left(t\right)+Z_i\left(t\right)(Q_i\left(t\right)+w_i\left(t\right))+{\delta }_i^2$ is a positive constant irrelevant to ${\alpha }_i\left(t\right)$s.

Proof. 

Since $Z_i(t+1)=max[0,Z_i\left(t\right)+Q_i(t+1)-{\delta }_i]$, $Z_i\left(t\right)\le Z_i{\left(t\right)}^2+2Z_i\left(t\right)(Q_i(t+1)-{\delta }_i)+{(Q_i(t+1)-{\delta }_i)}^2$. Let ${\Delta }_{Z_i\left(t\right)}=\frac{1}{2}(Z_i{(t+1)}^2-Z_i{\left(t\right)}^2)$. Then,

$${\Delta }_{Z_i\left(t\right)}\le \frac{{(Q_i(t+1)-{\delta }_i)}^2}{2}+Z_i\left(t\right)(Q_i(t+1)-{\delta }_i).$$

Since ${(x+y)}^2\le 2x^2+2y^2$,

$${\Delta }_{Z_i\left(t\right)}\le Q_i{(t+1)}^2+Z_i\left(t\right)Q_i(t+1)+{\delta }_i^2.$$

(17)

Because $Q_i(t+1)=max[0,Q_i\left(t\right)-b_i\left(t\right)+w_i\left(t\right)]$, $Q_i{(t+1)}^2\le Q_i{\left(t\right)}^2+b_i{\left(t\right)}^2+w_i{\left(t\right)}^2+2Q_i\left(t\right)(w_i\left(t\right)-b_i\left(t\right))-2w_i\left(t\right)b_i\left(t\right)$. By rearranging the terms, we obtain

$$Q_i{(t+1)}^2\le B_i\left(t\right)+b_i{\left(t\right)}^2-2b_i\left(t\right)(Q_i\left(t\right)+w_i\left(t\right)),$$

(18)

where $B_i\left(t\right)=Q_i{\left(t\right)}^2+w_i{\left(t\right)}^2+2Q_i\left(t\right)w_i\left(t\right)$ is a positive constant unrelated to $b_i\left(t\right)$. By placing Equation (18) into Equation (17), we obtain

$${\Delta }_{Z_i\left(t\right)}\le B_i\left(t\right)+b_i{\left(t\right)}^2-2b_i\left(t\right)(Q_i\left(t\right)+w_i\left(t\right))+Z_i\left(t\right)(Q_i\left(t\right)-b_i\left(t\right)+w_i\left(t\right))+{\delta }_i^2.$$

(19)

If we let $C_i\left(t\right)=B_i\left(t\right)+Z_i\left(t\right)(Q_i\left(t\right)+w_i\left(t\right))+{\delta }_i^2$, we obtain:

$${\Delta }_{Z_i\left(t\right)}\le C_i\left(t\right)+b_i{\left(t\right)}^2-b_i\left(t\right)(2Q_i\left(t\right)+2w_i\left(t\right)+Z_i\left(t\right)).$$

(20)

As $\Delta L\left(Z\left(t\right)\right)=\mathbb{E}[L\left(Z(t+1)\right)-L\left(Z\left(t\right)\right)|Z\left(t\right)]=\mathbb{E}[{\sum }_{i\in N}{\Delta }_{Z_i\left(t\right)}|Z\left(t\right)]$,

$$\Delta L\left(Z\left(t\right)\right)\le \sum _{i\in N}{C}_i\left(t\right)+\mathbb{E}[\sum _{i\in N}{b}_i{\left(t\right)}^2-b_i\left(t\right)(2Q_i\left(t\right)+2w_i\left(t\right)+Z_i\left(t\right))|Z\left(t\right)].$$

(21)

Recall that $b_i\left(t\right)={\alpha }_i\left(t\right)\tau f_{mes}^3$ and $e_i\left(t\right)={\alpha }_i\left(t\right)\tau \kappa f_{mes}^3=b_i\left(t\right)\kappa f_{mes}^2$. Because $E\left(t\right)={\sum }_{i\in N}{e}_i\left(t\right)={\sum }_{i\in N}{b}_i\left(t\right)\kappa f_{mes}^2$, we obtain:

$$V\mathbb{E}[E\left(t\right)|Z\left(t\right)]=V\mathbb{E}[\sum _{i\in N}{b}_i\left(t\right)\kappa f_{mes}^2|Z\left(t\right)].$$

(22)

From Equations (21) and (22), the ${\Delta }_{p}L\left(Z\left(t\right)\right)$ is upper bounded as

$${\Delta }_{p}L\left(Z\left(t\right)\right)\le B+\mathbb{E}[\sum _{i\in N}{b}_i{\left(t\right)}^2-b_i\left(t\right)(2Q_i\left(t\right)+2w_i\left(t\right)+Z_i\left(t\right))+V{b}_i\left(t\right)\kappa f_{mes}^2|Z\left(t\right)],$$

where $B={\sum }_{i\in N}{C}_i\left(t\right)$ is a positive constant unrelated to $b_i\left(t\right)$. □

Note that the space defined in Equation (10) is the subset of the space defined by Equation (9). Then, from Proposition 1 and the concept of opportunistically minimizing the expectation, we can translate Problem P1 as the following per-time-slot optimization problem.

$$\begin{array}{c}\hfill \mathbf{P}\mathbf{2}:argmi{n}_{\mathsf{\Omega }\left(t\right)=\{{\alpha }_i\left(t\right):i\in N\}}\mathbb{E}[\sum _{i\in N}{b}_i{\left(t\right)}^2-b_i\left(t\right)(2Q_i\left(t\right)+2w_i\left(t\right)+Z_i\left(t\right))+V{b}_i\left(t\right)\kappa f_{mes}^2].\\ \hfill s.t.\sum _{i\in N}{\alpha }_i\left(t\right)\le 1.\end{array}$$

(23)

4.2. Online Service Time Allocation Method

Problem P2 is a quadratic programming problem with a constraint that is not a separable inequality. Therefore, we may use a projected subgradient method to obtain the solution. However, ref. [23] showed that the convergence time of a subgradient method with a constant step size is $O(1/ϵ)$, where $0<ϵ\ll 1$ is the difference between the termination point and the optimal point. Thus, the time complexity of the subgradient method is $O(|N|/ϵ)$, which may be too long for online decisions. Therefore, in this section, we propose a heuristic algorithm that determines $\mathsf{\Omega }\left(t\right)$ in $O(|N|)$ time at the beginning of each time slot and is optimal while the sum of the determined ${\alpha }_i\left(t\right)$s is not larger than 1. Let us denote $g_i\left(t\right)=b_i{\left(t\right)}^2-(2Q_i{\left(t\right)}^2+2w_i{\left(t\right)}^2+Z_i\left(t\right)-V\kappa f_{mes}^2)b_i\left(t\right)$ and ignore Constraint (23). Because $min\left({\sum }_{i\in N}{g}_i\left(t\right)\right)\ge {\sum }_{i\in N}min\left(g_i\left(t\right)\right)$, we can obtain a solution to Problem P2 by finding an ${\alpha }_i\left(t\right)$ that satisfies $\frac{\partial g_i\left(t\right)}{\partial {\alpha }_i\left(t\right)}=0$ for all $i\in N$. As $b_i\left(t\right)={\alpha }_i\left(t\right)\tau f_{mes}$, $\frac{\partial g_i\left(t\right)}{\partial {\alpha }_i\left(t\right)}=2{\tau }^2{f}_{mes}^2{\alpha }_i\left(t\right)-\tau f_{mes}(2Q_i\left(t\right)+2w_i\left(t\right)+Z_i\left(t\right)-V\kappa f_{mes}^2)$. Therefore, if we denote an ${\alpha }_i\left(t\right)$ that minimizes $\partial g_i\left(t\right)/\partial {\alpha }_i\left(t\right)$ as $x_i\left(t\right)$, it becomes

$$x_i\left(t\right)=\frac{2Q_i\left(t\right)+2w_i\left(t\right)+Z_i\left(t\right)-V\kappa f_{mes}^2}{2\tau f_{mes}}.$$

(24)

A MES must decide ${\alpha }_i\left(t\right)$s at the start of each time slot. However, $x_i\left(t\right)$ involves $w_i\left(t\right)$, which a MES knows only at the end of the time slot t. To resolve this, we adopted an LSTM model to predict $w_i\left(t\right)$ at the beginning of a time slot t. To train the LSTM model for each queue, a MES measures the workload imposed during each time slot. At the start of each time slot k, a MES maintains h past workload arrival histories $h_i\left(k\right)=\{w_i(k-h),\dots ,w_i(k-1)\}$ for each queue i and feeds $h_i\left(k\right)$ to the LSTM model to obtain ${\widehat{w}}_i\left(k\right)$ as its output. Because a MES can measure $w_i\left(k\right)$ at the end of time slot k, it updates the LSTM model using ${(w_i\left(k\right)-{\widehat{w}}_i\left(k\right))}^2$ as the loss function. Once a MES finishes training the LSTM model, it uses the model to predict the amount of workload that will be imposed on each queue during a time slot t at the beginning of time slot t. Specifically, at the start of a time slot t, a MES feeds $(w_i(t-h),\dots ,w_i(t-1))$ to the LSTM and obtains ${\widehat{w}}_i\left(t\right)$ as the workload to be imposed on the queue i during time slot t.

By replacing $w_i\left(t\right)$ with ${\widehat{w}}_i\left(t\right)$ in Equation (24), we obtain:

$${\widehat{x}}_i\left(t\right)=\frac{2Q_i\left(t\right)+2{\widehat{w}}_i\left(t\right)+Z_i\left(t\right)-V\kappa f_{mes}^2}{2\tau f_{mes}}.$$

(25)

Let us denote the fraction of time that a MES allocates to each queue during a time slot t as ${\alpha }_i^{\ast }\left(t\right)$. Because $0\le {\alpha }_i\left(t\right)\le 1$, when ${\sum }_{i\in N}{\alpha }_i^{\ast }\left(t\right)\le 1$, ${\alpha }_i^{\ast }\left(t\right)$ is determined as follows.

$${\alpha }_i^{\ast }\left(t\right)=\left\{\begin{array}{cc}1,\hfill & 1\le {\widehat{x}}_i\left(t\right)\hfill \\ {\widehat{x}}_i\left(t\right),\hfill & 0<{\widehat{x}}_i\left(t\right)<1\hfill \\ 0,\hfill & {\widehat{x}}_i\left(t\right)\le 0\hfill \end{array}\right.$$

(26)

If ${\sum }_{i\in N}{\alpha }_i^{\ast }\left(t\right)>1$, then the optimal solution to P2 is outside the space defined by the constraint in Equation (23). That is, because the workload imposed on queue i (i.e., $Q_i\left(t\right)+w_i\left(t\right)$) is temporarily larger than the capacity of a MES, a MES cannot make $Q_i(t+1)<{\delta }_i$. Because the MES resource is insufficient to handle the total workload imposed on it, we adjust the fraction of time assigned to each queue in proportion to ${\alpha }_i^{\ast }\left(t\right)$s in Equation (26). Specifically,

$${\alpha }_i^{\ast \ast }\left(t\right)=\frac{{\alpha }_i^{\ast }\left(t\right)}{{\sum }_{j\in N}{\alpha }_j^{\ast }\left(t\right)},\forall i\in N.$$

(27)

From Equations (26) and (27), a MES determines ${\alpha }_i\left(t\right)$s at the start of each time slot t as follows.

$${\alpha }_i\left(t\right)=\left\{\begin{array}{cc}{\alpha }_i^{\ast }\left(t\right),\hfill & \sum _{i\in N}{\alpha }_i^{\ast }\left(t\right)\le 1\hfill \\ {\alpha }_i^{\ast \ast }\left(t\right),\hfill & otherwise\hfill \end{array}\right.$$

(28)

We summarize our online resource allocation method in Algorithm 1.

Algorithm 1 Online service time allocation algorithm

1: At the start of each time slot t: 2:  Observe $Q_i\left(t\right)$s, $\forall i\in N$. 3:  Predict ${\widehat{w}}_i\left(t\right)$ using an LSTM, $\forall i\in N$. 4:  $y=0$. 5: while$i\le |N|$do 6:   ${\widehat{x}}_i\left(t\right)=\frac{2Q_i\left(t\right)+2{\widehat{w}}_i\left(t\right)+Z_i\left(t\right)-V\kappa f_{mes}^2}{2\tau f_{mes}}$. 7:   ${\alpha }_i^{\ast }\left(t\right)=\left\{\begin{array}{cc}1,\hfill & 1\le {\widehat{x}}_i\left(t\right)\hfill \\ {\widehat{x}}_i\left(t\right),\hfill & 0<{\widehat{x}}_i\left(t\right)<1\hfill \\ 0,\hfill & {\widehat{x}}_i\left(t\right)\le 0\hfill \end{array}\right.$ 8:   y += ${\alpha }_i^{\ast }\left(t\right)$ 9:   $i++$ 10: if$y\le 1$then 11:     ${\alpha }_i\left(t\right)={\alpha }_i^{\ast }\left(t\right)$ 12: else 13:   while $i\le |N|$ do ${\alpha }_i\left(t\right)=\frac{{\alpha }_i^{\ast }\left(t\right)}{{\sum }_{j\in N}{\alpha }_j^{\ast }\left(t\right)}$

4.3. Properties

We state the characteristics of our algorithm as Proposition 2.

Proposition 2.

If the drift function satisfies the drift condition for all time slots and all possible $Z\left(t\right)$ and the expected penalty function $\mathbb{E}[E(t)]$ is lower bounded by a finite value ${\zeta }_{min}$, the long-term time-averaged expected energy a MES consumes when it uses our online service time allocation algorithm satisfies

$$\underset{T\to \infty }{lim}\frac{1}{T}\sum _{t=0}^{T-1}\mathbb{E}[E\left(t\right)]\le {\zeta }_{opt}+\frac{B}{V},$$

(29)

where ${\zeta }_{opt}$ is the optimal time-averaged penalty value and B a positive constant. In addition, the long-term time-averaged expected virtual queue length satisfies

$$\underset{T\to \infty }{lim}\frac{1}{T}\sum _{t=0}^{T-1}\sum _{i\in N}\mathbb{E}[|Z_i\left(t\right)|]\le \frac{B+V({\zeta }_{opt}-{\zeta }_{min})}{ϵ},$$

(30)

where ϵ is a non-negative constant.

Proof. 

We followed the approach in [39]. Suppose there are constants $B>0$ and $ϵ\ge 0$ such that the following drift condition holds for all time slots and possible $Z\left(t\right)$s:

$$\Delta L\left(Z\left(t\right)\right)\le B-ϵ\sum _{i\in N}|Z_i\left(t\right)|.$$

(31)

Let us assume that the expected penalty function $\mathbb{E}[E(t)]$ is lower bounded by a finite value ${\zeta }_{min}$. Then, for all time slots and all possible resource allocation actions, we have

$$\mathbb{E}[E\left(t\right)]\ge {\zeta }_{min}.$$

(32)

We denote the optimal time-averaged penalty value as ${\zeta }_{opt}$. That is, for all time slots and all possible resource allocation actions, we obtain

$$\mathbb{E}[E\left(t\right)]\le {\zeta }_{opt}.$$

(33)

Then, for a time slot t, the following inequality holds.

$$\Delta L\left(Z\left(t\right)\right)+V\mathbb{E}[E\left(t\right)]\le B+V{\zeta }_{opt}-ϵ\sum _{i\in N}|Z_i\left(t\right)|.$$

(34)

Taking expectations on both sides and applying the law of iterated expectations for $t=0,\dots ,T-1$, we have

$$\mathbb{E}[L\left(Z\left(T\right)\right)]-\mathbb{E}[L(Z\left(0\right)]+V\sum _{t=0}^{T-1}\mathbb{E}[E\left(t\right)]\le T(B+V{\zeta }_{opt})-ϵ\sum _{t=0}^{T-1}\sum _{i\in N}|Z_i\left(t\right)|.$$

(35)

Dividing both side of Equation (35) by $TV$, we obtain

$$\frac{{\sum }_{t=0}^{T-1}\mathbb{E}[E\left(t\right)]}{T}\le {\zeta }_{opt}+\frac{B}{V}+\frac{\mathbb{E}[L(Z\left(0\right))]-\mathbb{E}[L(Z\left(T\right)]}{TV}.$$

(36)

Then, by taking $T\to \infty$, we obtain

$$\underset{T\to \infty }{lim}\frac{1}{T}\sum _{t=0}^{T-1}\mathbb{E}[E\left(t\right)]\le {\zeta }_{opt}+\frac{B}{V}.$$

Because $0\le V{\sum }_{t=0}^{T-1}\mathbb{E}[E\left(t\right)]\le \infty$ and $0\le L\left(Z\right(T\left)\right)\le \infty$, by dividing both sides of Equation (35) by $ϵT$ and rearranging terms, we also obtain the following inequality.

$$\begin{array}{cc}\hfill \frac{1}{T}\sum _{t=1}^{T-1}|Z_i\left(t\right)|& \le \frac{B+V{\zeta }_{opt}}{ϵ}+\frac{V{\sum }_{t=0}^{T-1}\mathbb{E}[E\left(t\right)]}{ϵT}+\frac{\mathbb{E}[L(Z\left(0\right))-\mathbb{E}[L\left(Z\right(T\left)\right)]}{ϵT}\hfill \end{array}$$

(37)

$$\begin{array}{cc}\hfill & \le \frac{B+V{\zeta }_{opt}}{ϵ}+\frac{VT{\zeta }_{min}}{ϵT}+\frac{\mathbb{E}[L(Z\left(0\right))]}{ϵT}\hfill \end{array}$$

(38)

$$\begin{array}{cc}\hfill & =\frac{B+V({\zeta }_{opt}-{\zeta }_{min})}{ϵ}+\frac{\mathbb{E}[L(Z\left(0\right))]}{ϵT}.\hfill \end{array}$$

(39)

By taking $T\to \infty$, we obtain

$$\underset{T\to \infty }{lim}\frac{1}{T}\sum _{t=1}^{T-1}|Z_i\left(t\right)|\le \frac{B+V({\zeta }_{opt}-{\zeta }_{min})}{ϵ}.$$

(40)

□

5. Performance Evaluation

In this section, we evaluate the performance of the proposed method using simulation studies. The default parameter values are as follows, if not specified otherwise. A MES is equipped with a $f_{mes}=1.0$ GHz CPU core, whose effective switching capacity is $\kappa ={10}^{-27}$. The length of a time slot is $\tau =1$ s and $V={10}^{15}$. A MES supports $|N|=3$ classes, whose delay requirement is configured as ${\delta }_i=10i$ ms for each $i\in N$. To emulate the task arrival process for each class, we used the traffic traces from a cellular network containing the time that a device generates a call. We used the dataset that results from a computation over the call detail records (CDRs) generated by the Telecom Italia cellular network for nine days from the city [40]. Assuming that the call generation pattern of a device is similar to the task offloading pattern of the device, we divided the real traffic traces into time slots and calculated the number of tasks generated during each time slot. The workload imposed by a task is influenced by the applications to which a task belongs [41]. In this study, for each task class i, the workload imposed by a task is randomly selected in $[w_i^{min},w_i^{max}]$ according to the uniform distribution. For each class $i\in N$, we set $[w_i^{min},w_i^{max}]=[i\times 10.0,i\times 31.6]$ kilocycles. To focus on the service-time allocation problem at a MES, we considered the situation where a MES is not overloaded such that ${lim}_{T\to \infty }1/T{\sum }_{t=1}^T{\sum }_{i\in N}{w}_i\left(t\right)<\tau f_{mes}$.

5.1. Task Arrival Prediction Using LSTM

To predict the number of tasks arriving at each queue during a time slot, we used a single-layer LSTM with five hidden units for a given sequence input. We add

Table 1. The parameters for LSTM training.

Parameters	Values
Learning rate	0.01
Number of epochs	3000
Input size	3 to 5
Hidden size	5
Activation function	ReLU
Optimization function	Adam

, which shows the important parameters and their values used for the LSTM training.

To compare the prediction performance according to the length of the input sequence (i.e., h), we measured the prediction error in terms of ${\zeta }_i\left(t\right)={((w_i\left(t\right)-{\widehat{w}}_i\left(t\right))/w_i\left(t\right))}^2$. Then, we compared the mean and the standard deviation of the root-mean-squared (RMS) prediction error in

Table 2. Performance comparison of LSTM prediction in terms of RMS.

	Class 1		Class 2		Class 3
	Avg.	Std. Dev.	Avg.	Std. Dev.	Avg.	Std. Dev.
$h=3$	0.1022	0.0875	0.1182	0.1069	0.0738	0.0662
$h=4$	0.0957	0.0767	0.1152	0.1016	0.0889	0.0795
$h=5$	0.1057	0.0836	0.1249	0.0948	0.0931	0.0709

. If we denote the number of time steps used to calculate the RMS as n, the RMS for a class i queue is calculated as $\sqrt{\frac{1}{n}{\sum }_{t=1}^n{\zeta }_i\left(t\right)}$.

We observe in Table 2 that the best performance is obtained when the input size is four. Figure 2 shows the differences in the actual and predicted values over time for each class when the input size is four. The accuracy of the predicted values is quite high compared to the actual values.

5.2. Performance Comparisons

We compared the performance of the proposed method with those of three alternatives. The first alternative is EVEN, which determines ${\alpha }_i\left(t\right)=1/|N|$. The second alternative determines ${\alpha }_i\left(t\right)$ in proportion to the queue-length threshold ${\delta }_i$. Specifically, if we denote ${\eta }_i=1/{\delta }_i$, the second alternative determines ${\alpha }_i\left(t\right)={\eta }_i/{\sum }_{j\in N}{\eta }_j$. Henceforth, we refer to this as TPRA. The third alternative, WoPred, is similar to our method in that it uses a Lyapunov optimization framework. However, WoPred uses only the observed queue length without considering the input workload dynamics when determining ${\alpha }_i\left(t\right)$s at the start of each time slot. We also show the results obtained when our method uses $w_i\left(t\right)$s instead of ${\widehat{w}}_i\left(t\right)$s, which indicate the upper performance bound of our method.

First of all, we analyzed the time complexity of these methods. Algorithm 1 describes our online resource allocation method. The method is composed of two main parts—the prediction of the workload to be imposed on the queue i during time slot t and the determination of the time fraction that a MES allocates to each queue during a time slot t. To analyze the time complexity of the workload prediction, we considered the time consumed by the LSTM deep learning model to predict the workload. Since the deep learning technique is used for prediction after going through the training phase, the required time for the prediction is $O\left(1\right)$. Next, the time complexity of the time fraction determination is $O\left(N\right)$, which is proportional to the set of queues managed by a MES. Therefore, the time complexity of our method is $O\left(N\right)$. Meanwhile, in the case of the ideal method, the time complexity is $O\left(N\right)$ because it has only the time fraction determination process. In EVEN, TPRA, and WoPred, only the time fraction determination process is included. However, the difference among these three methods is in how they determine the fraction of the time that a MES allocates to each queue during a time slot t (${\alpha }_i^{\ast }\left(t\right)$). Since EVEN and TPRA allocate ${\alpha }_i^{\ast }\left(t\right)$s statically, their time complexity is $O\left(1\right)$. In contrast, since WoPred dynamically allocates service-time at each time slot by calculating ${\alpha }_i^{\ast }\left(t\right)$s, its time complexity is $O\left(N\right)$.

In Figure 3, we show how each method changes $Q_i\left(t\right)$s over time. We can see in Figure 3a that WoPred makes $Q_1\left(t\right)$ fluctuate widely and does not make $Q_1\left(t\right)<{\delta }_1$ most of thetime. We also observe similar behaviors in Figure 3b,c. This is because WoPred uses only the current queue length when it allocates service-time for each queue. When EVEN and TPRA are used, $Q_1\left(t\right)$ is maintained under ${\delta }_1$. However, $Q_2\left(t\right)$ and $Q_3\left(t\right)$ are not always maintained below ${\delta }_2$ and ${\delta }_3$. This is attributed to the fact that they statically allocate service-time. If the workloads imposed on the Class 2 queue and Class 3 queue are temporarily larger than the service-time allocated to them, $Q_2\left(t\right)$ and $Q_3\left(t\right)$ rise sharply. In this figure, we can observe that our method makes $Q_i\left(t\right)<{\delta }_i$ for all i all the time by dynamically allocating service-time to each queue according to the current queue length and the expected amount of workload.

To further examine this phenomenon, we show the changes in ${\alpha }_i\left(t\right)$s over time in Figure 4.

When the workload arriving at each queue is small and the length of each queue is small, all methods maintain $Q_i\left(t\right)<{\delta }_i$ ($\forall i\in N$). However, when the workload fed to a queue i is small and the workload arriving at another queue j is high, the MES must decrease ${\alpha }_i\left(t\right)$ and increase ${\alpha }_j\left(t\right)$ to avoid the temporal overload of queue j. However, EVEN and TPRA statically assign ${\alpha }_i\left(t\right)$s without considering the input workload dynamics and states of the queues. Because they cannot adjust the amount of CPU resources allocated to each queue, they cannot prevent the temporal overload of the queue. By contrast, our method dynamically adjusts ${\alpha }_i\left(t\right)$s according to $Q_i\left(t\right)$s, $w_i\left(t\right)$s, and ${\delta }_i$. Because our method can increase ${\alpha }_j\left(t\right)$ more than that allocated by TPRA and decrease ${\alpha }_i\left(t\right)$ less than that allocated by TPRA, it can maintain the queue length below its target threshold.

In Figure 5, we compare $e_i\left(t\right)$s over time. We observe that the energy consumed by a MES is high when static methods (EVEN and TPRA) are used. The results are attributed to the manner in which each method determines ${\alpha }_i\left(t\right)$s. We observe in Figure 4 that the static methods assign more service time to each queue than the dynamic methods (our method and WoPred) during times when the queues are not temporarily overloaded. That is, static methods make a MES in an active state unnecessarily longer than dynamic methods. A MES in an active state consumes energy, even when a queue is empty. According to [42], the idle energy consumed by a MES corresponds to 50∼$70\%$ of the peak power consumption when a MES is fully used. In contrast, our method and WoPred dynamically determine ${\alpha }_i\left(t\right)$s at the start of each time slot by considering $Q_i\left(t\right)$s, reflecting the historical resource allocations. Therefore, a MES can be turned into a low-power mode for $(1-{\sum }_{i\in N}{\alpha }_i\left(t\right))\tau$. Thus, they consume less energy than the static methods.

Because WoPred does not consider a workload arrival process (i.e., ${\widehat{w}}_i\left(t\right)$), it determines ${\alpha }_i\left(t\right)$s in a reactive manner. Thus, when we examine the ${\alpha }_i\left(t\right)$s obtained by WoPred, the ${\alpha }_i\left(t\right)$s lag because WoPred reacts only to the previous situation reflected in $Q_i\left(t\right)$s. That is, WoPred allocates the ${\alpha }_i\left(t\right)$s using only $Q_i\left(t\right)$s without considering ${\widehat{w}}_i\left(t\right)$s. Thus, the ${\alpha }_i\left(t\right)$s determined by WoPred may be larger or smaller than the ${\alpha }_i\left(t\right)$s required to maintain $Q_i(t+1)\le {\delta }_i$. Note that $Q_i(t+1)$ is affected by $w_i\left(t\right)$. Because WoPred considers $w_i\left(t\right)$ when it determines ${\alpha }_i(t+1)$ using $Q_i(t+1)$, WoPred reacts to $w_i\left(t\right)$. In contrast, our method predicts $w_i\left(t\right)$ at the beginning of a time slot t and considers ${\widehat{w}}_i\left(t\right)$ when it determines ${\alpha }_i\left(t\right)$. Thus, by considering $Q_i\left(t\right)$ and ${\widehat{w}}_i\left(t\right)$ at the start of time slot t, our method can determine ${\alpha }_i\left(t\right)$s more optimally than WoPred, which results in smaller $Q_i\left(t\right)$s and $e_i\left(t\right)$s than WoPred.

5.3. Parameter Effect

In Figure 6 and Figure 7, we show the average MES energy consumption and an average queue length after the 140th time slot with different Vs. These two figures show the influence of V, which trades off energy consumption and queue length. V determines the importance of the penalty term (i.e., a MES energy consumption) in the drift-plus-penalty function. Thus, as V increases, the amount of energy consumed by a MES dominates ${\Delta }_{p}L\left(Z\left(t\right)\right)$. As V increases, our method attempts to further reduce the energy consumption. That is, the service time allocated to each queue decreases as V increases. Thus, the average queue length increases, whereas the amount of energy consumed by a MES decreases with V. Therefore, by controlling V during the service-time allocation process, a MES operator can balance the energy consumption and queuing delay. In these figures, we also observe that, compared to WoPred, our method both consumes less energy and achieves a smaller average queue length for all Vs because our method considers the previous workload history reflected in $Q_i\left(t\right)$ and the workload expected during the current time slot ${\widehat{w}}_i\left(t\right)$ when determining ${\alpha }_i\left(t\right)$s. In addition, the average energy consumption and average queue length obtained using our method are comparable to those achieved in an ideal case, in which $w_i\left(t\right)$ is known in advance.

5.4. Scalability

In Figure 8 and Figure 9, we show the average queue lengths of the different methods when the number of task classes (N) is 6 and 9, respectively. When $N=6$, the average queue lengths obtained by TPRA are much larger than those achieved by other methods in Classes 4–6. For example, the average queue lengths obtained by TPRA are 11–25-times larger than those achieved by EVEN or WoPred. Moreover, in Class 6, the average queue length obtained by EVEN is much larger than those achieved by other methods. Therefore, in these subfigures, we omit the cases when TPRA and EVEN are applied to show the differences among the other methods visually. When $N=9$, the average queue lengths obtained by TPRA are much larger than those achieved by other methods in Classes 7–9. Thus, in Classes 7–9, we omit the cases when TPRA is applied to show the differences among the other methods visually. In these figures, we also observe that the average queue lengths obtained by our method look identical to those obtained by the ideal method that knows $w_i\left(t\right)$s in advance. This is because they achieve similar average queue lengths for all Vs. For example, when $N=6$ and $lo{g}_{10}V=16$ (i.e., $V={10}^{16}$), the differences in the average queue length between our method and the ideal method ranges from 0.0053 to 0.097. When $N=9$ and $V={10}^{16}$, the differences range from 0.0027 to 0.073. From Figure 8 and Figure 9, we can observe that unlike the other methods, our method guarantees the average queue length below a given threshold for all the queues.

In Figure 10, we show the average energy consumption of a MES when $N=6$ (Figure 10a) and $N=9$ (Figure 10b) with different Vs. We can see that the energy consumed by our method is comparable to that consumed by the ideal method. Moreover, we can observe that our method consumes the least energy. We can also observe that there is a trade-off between the average energy consumption and the average queue length, which we can control by using the parameter V.

6. Conclusions and Future Works

In this study, we proposed an online service-time allocation method that can reduce the energy consumption of a MES while guaranteeing the queuing latency requirement for each class queue. We combined the Lyapunov optimization framework with a deep learning model and devised a lightweight algorithm that can quickly determine an optimal solution at the beginning of each time slot using the observed queue length and predicted workload. Using simulation studies, we verified that the proposed method is superior to conventional methods in reducing the energy consumption of a MES while maintaining the queuing delay below a given threshold value.

Since our method deals with the service-time allocation problem when a MES is not overloaded, it can complement a MES congestion resolution method and a task migration method. If a MES detects that the average queue length cannot be guaranteed because the task input rate to a MES exceeds the capacity of the MES, it can invoke a MES congestion resolution method to move some tasks to the other MESs or to a cloud server. In addition, when a MES detects that a device moves out of its service area, it invokes a task migration method to move tasks to other MESs.

As a future work, we plan to extend our resource management method into more complex scenarios where MESs migrate tasks and the backhaul network capacity is limited. To further improve the system performance and QoS requirements of tasks, we will jointly optimize a service-time allocation strategy, a MES congestion resolution policy, and a task migration algorithm.