1. Introduction
In the 5G era, the concept of vehicular networks has extended to the Internet of Vehicles (IoV), in which intelligent and interactive vehicular applications such as autonomous vehicles, path planning and navigation are enabled by communication and computation technologies [
1,
2,
3]. These computation-intensive applications frequently require ultra-reliability and low-latency (uRLLC). For example, autonomous driving applications need to process huge amounts of data in real time (as high as 2 GB/s) within about 10 ms, which is a very low delay constraint [
2]. Applications for efficient driving, e.g., path planning or navigation generate less data than autonomous driving, but tasks from the applications should be performed within a still-low delay constraint of about 100 ms [
4]. Despite improving the computing power of vehicles, processing such applications on vehicular terminals is still difficult while ensuring high computing power and quality-of-service (QoS) requirements, such as low latency.
To overcome these limitations and facilitate efficient application processing, multi-access edge computing (MEC) has emerged as a promising technology that can satisfy the demand for the heavy computation of vehicles by providing rapid and sufficient computational resources to vehicles [
5]. Thus, by offloading tasks from vehicles to MEC servers (MECSs), the MEC system can facilitate computation-intensive applications with low latency in vehicles with limited computing resources. Meanwhile, compared with conventional centralized cloud computing, offloading computation-intensive tasks to MECSs, which are geographically closer to vehicles, by deploying them at the network edge, can further reduce the network delay.
However, because MECSs have limited computing power, the computing power that the vehicle can request from the MECS depends on the MECS load level, i.e., the number of tasks already allocated to the MECS [
6]. Due to the mobility of vehicles, vehicles are not evenly distributed across a MEC system; thus, the number of tasks offloaded to a MECS may not be evenly distributed among MECSs. Therefore, some MECSs are heavily congested, whereas others are lightly loaded. If a task is offloaded to a congested MECS, the task may be blocked or the delay will be prolonged. Thus, the delay requirements may be violated. Moreover, the mobility of vehicles causes uncertainty in downloading results [
7]. Owing to the limited coverage of MECSs, the vehicle can travel out of the coverage of a MECS that offloads a task during a service session, resulting in a service interruption. Service interruption occurs because of dynamically changing radio association, which can significantly increase the total service latency, particularly in urban areas with highly dense infrastructure.
To improve the satisfaction with QoS requirements, references [
8,
9,
10,
11,
12,
13] studied service migration. In [
8,
9,
10], a service migration that migrates services according to a vehicle’s trajectory was proposed to minimize the average completion time of tasks. As they only considered the trajectory of vehicles without considering the load of MECSs, tasks can simultaneously migrate to a specific MECS. Thus, this MECS becomes overloaded, which can result in uneven loads among MECSs and reduce system throughput. Service migration with load balancing was addressed in [
11,
12,
13]. In [
11], a service migration method was proposed that considered the load difference and migration cost in balancing the loads of MECSs in a MEC system. In [
12], service migration was determined using the traffic conditions and loads of each MECS to minimize the migration costs and travel times of vehicles. The load of each MECS in [
11,
12] was measured using the number of tasks in the MECS. In [
13], the method distributed the load from an overloaded cell to a lightly loaded cell from its neighbors to increase the utilization of radio resources. The load is represented by the resource block utilization ratio of the cells. The delay constraint of a task was not considered in [
11,
12,
13] when defining the load. In this paper, we define the load as a computational workload required to complete a task, considering delay constraint. In addition, unlike in [
11,
12,
13], in which tasks are migrated to one MECS, tasks in overloaded MECS may be migrated to one or more underloaded MECSs to distribute the loads evenly in the MEC system. The authors of [
14,
15] proposed a load balancing/resource allocation method in a framework, called dew computing, that integrates smart mobile devices into distributed and high-performance computing platforms. In [
16], their method determined whether to offload a task or not in order to reduce the processing time, energy consumption of the smartphone and monetary cost. However, we determine whether/where to migrate to and partition among MECSs once the vehicles offload their tasks them in order to reduce the task execution delay and improve the system throughput. Therefore, we address a method in the collaborative edge computing framework that enables tasks to be processed outside of the communication coverage of the MECS offloaded from the vehicle.
Thus, we consider task partitioning in which a task can be divided into a set of subtasks and computed by multiple MECSs in parallel. This can improve service reliability by reducing the task execution delay. In other words, by reducing the execution delay, a vehicle can obtain computing results without service interruption before it moves out of the coverage of a MECS. Resource utilization or energy consumption costs are additionally incurred in the process of partitioning or gathering the computing results, but there are more benefits, such as reducing execution time and improving system throughput.
In contrast to the methods used in [
8,
9,
10,
11,
12,
13] that migrate an entire task to the MECS without partitioning, the methods used in [
17,
18,
19,
20] addressed task migration with partitioning. In [
17], the partitioned tasks were migrated to the serving MECS and cloud server to reduce the average total delay of tasks. In [
18], to reduce the service delay, the partitioned tasks were migrated from the MECS in which the task was offloaded to the geographically nearest MECS. The task considered in [
17,
18] consisted of independent or sequentially dependent subtasks. In practice, a task is generally composed of multiple threads. For example, navigation applications frequently involve threads of graphics, camera preview, and video processing [
21]. Each thread can be considered a subtask, so that computation tasks of the applications can be partitioned and the task can be modeled using a directed acyclic graph (DAG). In [
19,
20], methods were investigated to optimize performance in terms of delay and energy consumption. However, if the set of subtasks of a task migrates to the MECS without considering the loads of the MECSs, the total service delay may increase. This becomes useless if the subtasks migrate to a congested MECS or a specific MECS simultaneously. By selecting an appropriate MECS to which to migrate the subtasks and determining the amount of workload required of subtasks, the satisfaction of the task’s QoS requirements can be improved. Task migration and partitioning methods based on machine learning were proposed in [
6,
7,
11,
14,
22]. Generally, methods using machine learning require a huge amount of data for learning and analysis. However, since we assume a small clustered area with its own characteristics, the proposed MEC system is built on each area to optimize the performance, reflecting the characteristics of each area.
Therefore, we propose a method of task migration with partitioning using a heuristic algorithm to distribute loads among MECSs and execute the task in parallel in a collaborative edge computing framework. The set of subtasks in an overloaded MECS is migrated to one or more underloaded MECSs depending on the load difference between the overloaded and underloaded MECSs. Balancing the loads in the MEC system through task partitioning and migration can increase both the satisfaction of QoS requirements, such as low latency and service reliability, and MEC system throughput.
The remainder of this paper is organized as follows. In
Section 2, we describe the system model and the problem formulation. In
Section 3, we introduce the proposed method in detail. In
Section 4, we present and discuss the simulation results. Finally,
Section 5 concludes the study.
2. System Model and Problem Formulation
The MEC system proposed in this paper is shown in
Figure 1. We consider a MEC system comprising a set of MECSs,
, located in roadside units (RSUs) and a set of vehicles,
. A MECS can provide seamless communication and computing service coverage for vehicles on the road. We assume that the MECSs communicate with each other through wired links, so communication latency and bandwidth can be ignored [
20,
23]. Each MECS uses two queues: a waiting queue and computation queue. The waiting queue uses a limited task buffer to accommodate tasks offloaded from vehicles, and the computation queue operates in a first-in-first-out manner to process tasks. A MECS controller acts as a centralized controller with global knowledge of the MECSs in a system and is responsible for making the decisions for migration and partitioning. Time is divided into slots
, and each time slot has an equal duration of Δ
s. We assume that the vehicle travels at speed
s during the simulation.
During each time slot, vehicle
offloads its task in which the data size is
(in bits), the computation workload is
(in CPU cycles), and the delay constraint is
, which can be divided into several interdependent subtasks. The dependency among subtasks is modeled by a DAG, i.e.,
, where
is the set of dependent subtasks in task
and
is the set of dependencies between the subtasks in task
. Each subtask
in task
is associated with a computation workload
. In this paper, we only consider the tree-structured task graph, where the out-degree of each vertex is equal to 1, except for that of the first vertex. The first and last subtasks compute at the serving MECS, to complete the first subtask and migrate other subtasks simultaneously [
18], and to gather and transmit computing results at the end of the service session [
13]. For example, in an autonomous driving application, recognizing a situation can be the first subtask and driving guidance can be the last subtask.
We assume that at the beginning of each time slot
, a vehicle offloads its tasks to the serving MECS, giving it the highest signal, as MECS
. The MECS receives the task offloaded from the vehicle, then the task is placed at the waiting queue of the MECS to be partitioned and migrated. At the end of time slot
, the MECS controller gathers the status information from the MECSs in the system. Based on the information, the MECS controller makes a decision of task migration and partitioning whether/where to migrate or partition tasks in the waiting queues of servers. According to the decision made by a MECS controller, at time slot
, the tasks stored in the waiting queues are placed into the computation queues of the servers to which they belong or the computation queues of other servers. Once tasks or subtasks are placed in the computation queue, they cannot be partitioned or migrated to another MECS additionally. Assuming that some subtasks in MECS
are migrated to another MECS
, the subtasks are performed in MECS
and
respectively, the computation results of subtasks are merged by MECS
m, and sent to the vehicle. These operation of a MEC network are shown in
Figure 2.
The data rate for forwarding the task offloaded from a vehicle to a MECS at time slot
can be calculated as
where
is the bandwidth allocated to a channel,
is the transmission power of the vehicle,
is the channel gain between vehicle
and MECS
, and
is the received noise power. The transmission delay to offload the task from the vehicle to its serving MECS is
The load imposed on the MECS by the task is the computational workload required to complete the task within the delay constraint. As the vehicle has already offloaded its task to the serving MECS and the task is placed in its waiting queue, the task must be completed within a time, excluding the transmission delay under the delay constraint. Therefore, the CPU cycles required to complete the tasks in the waiting queue are
The load of each MECS is equal to the sum of the workloads of tasks in the waiting queue
of MECS
at the end of time slot
.
To measure whether the load of each MECS is evenly distributed among the MECSs in the MEC system, the MECS controller calculates the average load difference among the MECSs in the MEC system.
As the number of vehicles associated with each MECS is not evenly distributed, the number of tasks offloaded to the MECSs is not even. Therefore, because the workloads imposed on MECSs differ, the loads of some MECSs are high and those of others are low. If tasks are migrated to an overloaded MECS or a particular MECS simultaneously, the delay requirements of tasks may be violated because of the increased time spent in the waiting queue of the MECS. Thus, to increase the satisfaction of QoS requirements and system throughput, we propose a task migration with partitioning method to balance the loads among MECSs in the MEC system, as follows:
4. Simulation Results and Discussion
In this section, we verify the performance of the proposed method compared with other methods. It is assumed that the MEC system is deployed in a clustered area grouped by the units of administrative districts in the urban environments where MECSs are densely deployed and vehicles are not driving at high speed. We randomly distributed 10 MECSs within a MEC system. The capacity of each MECS was 10 GHz. Vehicles were distributed in a Poisson distribution on the road. All vehicles had mobility within a random walk model in an area of 1000 m × 1000 m using the SUMO simulator [
25]. The input size of the task followed a random distribution with 2–10 Kbits. The task graph used in the simulation was randomly generated, where the workload of each subtask followed a random distribution with 0.1–0.2 Gcycles. The mobility speed of the vehicle also followed a random distribution of 1–30 m/s. The parameters used in the simulation are summarized in
Table 1 according to [
26,
27,
28]. The path loss between the RSU and vehicle was modeled as
. When vehicles offloaded their tasks simultaneously, the vehicles performed data transmission in different spectral bands via orthogonal frequency-division multiple access technology, and there was mutual interference among different subcarrier allocations.
To demonstrate the reason for using task partition and migration in our method, we compared the Proposed method with No partition and No migration strategies. The Proposed method uses both task partitioning and migration. The No partition method migrates an entire task without any task partitioning to the target MECS, and the No migration method executes a task on the MECS in which the vehicle’s task is offloaded without any task migration.
The load variance among the MECSs is shown in
Figure 4. It shows the load variance of each time slot when the vehicle arrival rates were 0.3, 0.6 and 0.9. In other words, it is an indicator of the distribution of the load among MECSs during the simulation. In the
Proposed method, the set of task subtasks in an overloaded MECS was migrated to underloaded MECSs. Under
No partition method, the all tasks in the overloaded MECS were migrated to the underloaded MECS. Thus, the
Proposed method further reduced the load difference between overloaded and underloaded MECSs compared with the
No partition method.
No migration method had a large load variance, because vehicles were not evenly distributed geographically and had high mobility; thus, the loads may have been concentrated in specific MECSs.
The task completion rates within the delay constraints and task reliability rates are shown in
Figure 5 and
Figure 6, respectively. The rate of task completion within the delay constraint is defined as the ratio of the number of completed tasks while satisfying the delay constraint to the total number of offloaded tasks. The task reliability rate is defined as the ratio of the number of completed tasks to the total number of offloaded tasks while receiving the result without the service interruption. The task completion rates within the delay constraints and task reliability rates are affected by the task execution delay. For example, for a congested MECS, the load of the MECS becomes overloaded. This can increase the waiting time of the MECS queue, resulting in an increase in a task execution delay. As shown in
Figure 5, depending on the load balancing among MECSs, the task completion rates within the delay constraints had a high rate. Thus, if the task execution delay is prolonged, service interruption occurs when receiving its result. To receive the result without service interruption, the vehicle must obtain the result quickly before it moves out of the coverage of the MECS to which the task is offloaded. If the load balancing among MECSs is appropriate, the task execution delay will be reduced, and the reliability rate will be high. Therefore, the
Proposed method had the highest rate, and the
No migration method had the lowest rate in terms of the task completion rates within the delay constraints and task reliability rates. Therefore, it was better to perform migration than not, and migration with partitioning performed better than full migration. Therefore, in this study, we considered task migration with partitioning.
To verify the performance of our proposed method, we compared it with four other methods:
IPSO [
20],
Nearest,
Least and
Random.
IPSO aims to minimize the service failure probability by considering task execution delay. It is a method of migrating tasks to MECSs that meet the latency constraint of the subtasks.
IPSO defines the workload of the subtasks as the ratio of the sum of the required CPU cycles to the sum of its input and output data.
Nearest is a method of migrating tasks in overloaded MECSs to the nearest MECS among underloaded MECSs.
Least is a method of migrating tasks in overloaded MECSs to the least loaded MECS among underloaded MECSs.
Random is a method of migrating tasks in overloaded MECSs to randomly selected MECSs among underloaded MECSs.
The task completion rates within the delay constraints under different values of delay constraints are shown in
Figure 7 for a vehicle arrival rate of 0.6.
Proposed achieved the highest completion rate for different values of the delay constraint, a value of approximately 35–60% higher than that of other methods. The task reliability rate under different values of mobility speed of vehicles is shown in
Figure 8 for a vehicle arrival rate of 0.6. Service interruption occurs when the vehicle can travel out of the coverage of the MECS that is offloaded during a service session, which significantly increases the total service latency. We observed that
Proposed achieved the highest task reliability rate for different values of mobility speed, approximately 20–51% higher than that of other methods. A comparison of the environment with high and low speeds indicated differences of approximately 8% for
Proposed, 11% for
IPSO, 15% for
Nearest, 17% for
Least, and 13% for
Random. Under conditions with different delay constraints and mobility speed,
Proposed performed better than that of others by optimizing the load balancing, which reduces the task execution delay. Therefore, we conducted the experiment by setting the delay constraints to 0.1–0.6 s and the mobility speed to 1–30 m/s.
Figure 9 shows the load variance among the MECSs when the arrival rates were 0.3, 0.6 and 0.9. In
Proposed, the set of subtasks in the task was migrated to one or more target MECSs depending on the load difference. In
Nearest and
Random, the set of subtasks in the task was migrated to the target MECSs based on distance and randomness. In
Least, the set of subtasks in the task is migrated to the least loaded MECS, which indicated a scenario in which the least loaded MECS was concentrated. Therefore, depending on which MECS was selected as the target MECS, the loads of the MECSs varied. In
IPSO, the set of subtasks was migrated based on delay constraint.
Proposed shows better performance than
IPSO in terms of load variance because proposed considers load difference to determine the task migration.
IPSO shows better performance than
Nearest,
Random and
Least, because it is affected by the delay, especially the queueing delay associated with the load of the MECS.
Figure 10 depicts the average blocking rate and standard deviation of the blocking rate with varying arrival rates. The blocking rate is defined as the ratio of the number of tasks blocked by the MECS to the total number of offloaded tasks. The task is blocked by the MECS when the task is offloaded to a MECS whose queue is full. In
Nearest and
Random, the target MECS may be another overloaded MECS or may become overloaded owing to concentrating tasks on it. In
Least, because tasks in overloaded MECSs are migrated to the least loaded MECS, the least-loaded MECS becomes overloaded. Therefore, because some MECSs have a high load, the task may be blocked when it is offloaded to the MECS with a high load. In
IPSO, when all MECSs in MEC system do not meet the delay constraints for the subtasks, the subtasks migrate to the MECS that minimizes the computing delay. In this case, since only the processing delay is considered, there is likely that the overloaded MECS is selected.
Proposed not only had the lowest blocking rate but also the smallest standard deviation compared with the other methods because the load was relatively well distributed. The system throughput with varying arrival rates is shown in
Figure 11. The system throughput is defined as the multiplication of the number of tasks completed per time slot by the average task size. When the loads are balanced among the MECSs, the block rate decreases. In addition, if the block rate decreases, more tasks can be performed; thus, the system throughput increases.
Figure 12 and
Figure 13 show the rate of task completion within the delay constraints and task reliability rate, respectively. The task completion rate within the delay constraints is affected by the task execution delay consisting of transmission, queueing, execution, and migration delay. Similarly, task reliability is affected by the task execution delay. A migration delay is the time a task is migrated to another MECS, which does not account for most of the total delay because it uses wired links. The queueing delay is affected by the load of the MECS, that is, the length of the queue of the MECS. Increasing the parallelism of task execution reduces the execution delay. Therefore, the better the loads of MECSs are distributed, the less the queuing delay affected by the MECS load and queue length, increasing the task completion rate within the delay constraints. The task completion rate of
Proposed was approximately 18–58% higher than that of other methods. If the task execution delay is prolonged, the vehicle may move out of the coverage of the MECS, which is offloaded to the task during a service session. Thus, service interruption may occur more often in receiving its result. To receive the result without the service interruption, the loads of the MECS must be appropriately distributed to reduce the task execution delay. The reliability rate of
Proposed was approximately 12–46% higher than that of the other methods because it distributed the loads of MECSs well.