EdgeUP: Utilization and Priority-Aware Load Balancing in Edge Computing

Abstract

This paper introduces a dynamic context-aware load balancing mechanism called EdgeUP, which leverages dynamic contexts of both edge nodes and IoT devices to balance tasks on edge nodes optimally. To perform this, we devised a lightweight load balancer to allocate tasks optimally based on CPU utilization across edge nodes while satisfying the task priority of IoT devices. We implemented EdgeUP on real edge devices (i.e., Raspberry Pi 400) and evaluated it in terms of the average CPU utilization of edge nodes, idle CPU distance, and execution time. The experimental results show that EdgeUP outperforms three conventional load balancing algorithms, including round-robin, hill climbing, and K-Means, by 5.6×, 2.1×, and 3.6×, respectively.

1. Introduction

Prominence and necessity of edge computing. A massive amount of sensor data are generated and collected in Internet of Things (IoT) everyday [1,2,3,4,5,6]. Processing the massive sensor data has become one of the most prominent challenges in IoT [7,8,9,10] since it causes high latency [11,12,13]. Accordingly, cloud computing and fog/edge computing have emerged to efficiently handle this challenge in IoT [1]. Edge computing is the computer power that exists on the edge of a connected ecosystem [14,15,16,17,18,19,20]. The computing is close to endpoint devices generating data (e.g., sensors, mobile devices, etc.). The main advantages of edge computing are the reduction in latency, improvement in security, operational cost savings, increase in reliability, and scalability. Therefore, edge computing is a proper solution for massive data processing in IoT [21]. However, edge nodes in edge computing have limitations on resources (e.g., network, memory, processing, etc.) [22]. As a result, they can be overloaded or overworked when they exceed or run out of their resources. Load balancing mechanisms, which prevent edge nodes from being overloaded, play a critical role in edge computing [23,24,25].

Load balancing at application layer in edge computing. Load balancing mechanisms in edge computing exhibit a diversity based on the layers of the TCP/IP model to which they are applied, such as network access, network, transport, and application layers. Numerous studies are dedicated to proposing load balancing mechanisms across the layers. These studies design load balancing mechanisms to enhance the distribution of network traffic, aiming to maximize or minimize specific network metrics [25,26,27,28], such as latency, throughput, bandwidth, etc. It is clear to observe that load balancing mechanisms mostly focus on network metrics. Giving more consideration to computing metrics is crucial, as the computing resource is also one of the constrained resources in edge computing. Moreover, there are limited studies at the application layer compared to the three lower layers (i.e., transport, network, and network access layers). Therefore, designing load balancing mechanisms while considering computing metrics at the application layer and also conducting practical experiments are essential in edge computing.

Prior work. Numerous studies have explored load balancing strategies for IoT systems. These studies differ widely in terms of the load balancing metrics employed, the IoT layer targeted, the system layers considered, and the solution approaches employed.

Table 1. Comparison between existing studies and EdgeUP (NA: computing node-aware; DA: user device-aware; ML: machine learning-based approach; Opt: optimization approach; Greedy: greedy approach).

Study	Static Context	Dynamic Context	IoT Layer	System Layer	ML	Opt	Greedy
Fan et al. [25]		NA	Fog	Physical		✔
Li et al. [29]	✔	NA	Edge	Physical		✔
Dong et al. [27]		NA	Edge	Physical		✔
Yu et al. [30]		NA	Cloud	Application		✔
Li et al. [31]		NA, DA	Edge	Physical	✔
Baek et al. [32]		NA	Fog	Physical	✔
Esmaeili et al. [33]	✔	NA, DA	Edge	Physical	✔
EdgeUP		NA, DA	Edge	Application			✔

presents a comparative analysis of previous works and EdgeUP, focusing on their load balancing metrics, i.e., static (a static context refers to an object’s state that remains constant over time ) and dynamic (a dynamic context refers to an object’s state that can change over time) contexts, IoT layers, system layers, and algorithm methods.

For load balancing metrics, most recent studies on load balancing metrics focus on dynamic contexts, which reflect the changing statuses of computing nodes [25,27,29,30,31,32,33], as indicated by ‘Computing node-aware (NA)’ in the table, and users’ devices [31], as indicated by ‘User device-aware (DA)’ in the table. In terms of IoT layers, load balancing mechanisms can be implemented at three levels of IoT: cloud [30], fog [25,32], and edge [27,29,31,33]. In terms of system layers, most recent load balancing mechanisms are deployed at the physical layer [25,27,29,31,32,33]. Load balancing mechanisms implemented at the physical layer rely on physical data, such as precise locations, data rates, transmission power, and signal-to-noise ratios, making the deployment more complex. To address challenges associated with physical layer deployment, the study [30] proposed shifting load balancing mechanisms to the application layer, offering a simpler and faster implementation. Regarding approaches of load balancing mechanisms, load balancing mechanisms often employ three popular approaches: machine learning-based [31,32,33], optimization-based [25,27,29,30], and greedy approaches. Both machine learning-based and optimization-based approaches can provide optimal load balancing solutions. However, they are computationally intensive and time-consuming. In contrast, the greedy approach ensures computational efficiency.

Our study, EdgeUP, focuses on load balancing in edge computing, specifically considering the edge layer in IoT. Recognizing the impact of dynamic contexts on load balancing methods, we support both types of dynamic contexts in IoT: computing node and user device contexts. Furthermore, our system is designed at the application layer for more efficiency and flexibility than the physical layer. Lastly, we employ a greedy approach to provide efficient computation on resource-constrained devices.

To perform this, EdgeUP employs a novel load balancing metric, ServingValue, as an objective computing metric to allocate tasks to appropriate edge nodes. To handle the dynamic nature of an edge computing environment, ServingValue considers two key dynamic contexts: the dynamic contexts of IoT devices and edge nodes. The proposed load balancing mechanism operates as a module at the application layer of the edge layer in IoT, making optimal decisions regarding task assignments to edge nodes to maximize the ServingValue metric. Our study offers the following contributions:

• First, we introduce a novel load balancing metric, ServingValue, which incorporates two dynamic contexts in edge computing: IoT device context (i.e., task priority) and edge node context (i.e., CPU utilization). Our goal is to maximize this metric by optimizing task assignments (i.e., pairs of tasks and corresponding edge nodes).
• Second, we employ a lightweight greedy approach, EdgeUP, to address the task assignment problem, making it suitable for edge nodes with limited resources.
• Third, we provide a practical implementation of a load balancing module at the application layer.
• Finally, our proposal outperforms three well-known load balancing methods, including round-robin, hill climbing, and K-Means.

2. Related Works

EdgeUP introduces a novel load balancing metric (i.e., ServingValue), which takes into account dynamic contexts in edge computing, such as IoT device and edge node contexts. Not only does it provide a dynamic context-aware load balancing mechanism using the proposed metric, but EdgeUP also guarantees a lightweight execution, making it suitable for edge computing by utilizing a greedy load balancing strategy.

Fan et al. [25] proposed a load balancing method at the fog layer in IoT, using CPU capacity, one of the dynamic contexts of computing nodes, as a key metric in their optimization-based mechanism to reduce user response latency. Dong et al. [27] took into account the dynamic contexts of computing nodes at the edge layer in IoT, specifically memory and CPU utilization, to assign tasks to either edge or cloud nodes. Their optimization-based load balancing mechanism aims to reduce task completion time. Li et al. [29] utilized both static and dynamic load balancing metrics. The static metrics include physical memory size, CPU frequency, number of cores, and disk size, while the dynamic metrics involve memory and CPU utilization. Using both types of contexts, their optimization-based load balancing mechanism in edge computing classifies edge nodes into different states (i.e., light, normal, or heavy) and assigns tasks to the most suitable nodes to reduce task completion time. Our study, EdgeUP, aligns with previous studies [25,27,29] in considering dynamic contexts of computing nodes (e.g., memory and CPU utilization) for load balancing. Additionally, EdgeUP shares similarities with [27,29] in addressing load balancing at the edge layer in IoT. However, while the load balancing mechanisms in [25,27,29] are deployed at the physical layer, EdgeUP is deployed at the application layer. Moreover, in addition to considering dynamic contexts of computing nodes like these studies, EdgeUP also takes into account another type of dynamic context, a dynamic context of user devices (i.e., user requests). Lastly, whereas these studies use optimization-based approaches for load balancing, EdgeUP employs a greedy approach to ensure lightweight execution of the mechanism.

Li et al. [31] incorporated both types of dynamic contexts for load balancing in edge computing, including the dynamic context of computing nodes (i.e., computing power of edge nodes) and dynamic context of user devices (i.e., task demands). Their approach aimed to enhance resource utilization while minimizing computational latency for tasks. Baek et al. [32] used both static and dynamic contexts for load balancing in fog computing. They considered CPU frequency as the static context and the number of remaining tasks in queues of computing nodes as the dynamic context to reduce task response latency. Esmaeili et al. [33] considered both static and dynamic contexts for load balancing in edge computing. They used memory and disk capacity of edge nodes as static contexts, and CPU utilization of edge nodes and task demands as dynamic contexts (which are dynamic contexts of computing nodes and user devices, respectively). Their approach aimed to reduce user response latency. EdgeUP aligns with [31,32,33] by considering the dynamic contexts of computing nodes, such as memory and CPU utilization of edge nodes. Furthermore, EdgeUP is in line with [31,33] by addressing both types of dynamic contexts (i.e., computing node and user device contexts) and focusing on load balancing at the edge layer in IoT. In contrast to these studies that are deployed at the physical layer, EdgeUP operates at the application layer. Additionally, EdgeUP introduces a novel load balancing metric, ServingValue, which integrates both types of dynamic contexts (i.e., computing node and user device contexts), to efficiently enhance CPU utilization across all computing nodes. Moreover, unlike these studies that depend on machine learning-based methods for solving load balancing problems, EdgeUP employs a greedy approach to maintain the lightweight execution.

Yu et al. [30] addressed load balancing in microservice applications deployed in the cloud by incorporating dynamic contexts. Their study focuses on the dynamic contexts of computing nodes, particularly the external and internal demands within the microservice nodes. Our study, EdgeUP, aligns with this study in considering dynamic contexts of computing nodes and deploying the load balancing mechanism at the application layer. In contrast, our study tackles the load balancing problem at the edge layer in IoT, while Yu et al.’s study [30] concentrated on the cloud layer in IoT. Additionally, regarding load balancing metrics, while Yu et al. [30] focused only on the dynamic contexts of computing nodes, EdgeUP introduces a novel load balancing metric by combining the dynamic contexts of both computing nodes and user devices. Furthermore, while Yu et al. [30] proposed an optimization-based approach for their load balancing problem, our study employed a greedy approach to ensure the mechanism’s lightweight execution.

3. Background and Motivation

3.1. Load Balancing-Aware Edge Computing in IoT

In IoT, the network architecture is typically divided into three layers, such as cloud layer, fog layer, and edge/device layer [34,35,36], based on their computing, storage capabilities, and their distance to end-users. The load balancing challenge in edge computing also pertains to the three-layer network architecture [28,29]. Figure 1 presents load balancing-aware edge computing in IoT based on the three-layer network architecture. In this study, we focus on load balancing within edge computing. As shown in Figure 1, we emphasize the edge/device layer in the three-layer network architecture by specifying it, while merging the fog and cloud layers into single upper layers.

Load balancing-aware edge computing in IoT is structured into three main layers:

• Device layer: This layer comprises IoT devices such as mobile phones, sensor nodes, autonomous cars, smart traffic lights, and robots. In the load balancing-aware edge computing architecture, we consider scenarios where IoT devices request services from the upper layers. For example, they can request tasks, such as data processing, data caching/storage, and IoT management, from the edge layer.
• Edge layer: This layer consists of a cluster of edge nodes, which are capable of computing and storage, and are located close to IoT devices. Edge nodes in the cluster are classified into two types: one master node (i.e., controller node) and multiple edge nodes (i.e., worker nodes). In this architecture, the master node works as a load balancer, collecting task requests from IoT devices, as well as information (e.g., data, status, etc.) about IoT devices and edge nodes. It then executes load balancing strategies to assign the IoT devices’ task requests to the appropriate edge nodes. Edge nodes report their information (e.g., status (ACTIVE / NONACTIVE), memory, and CPU utilization, etc.) to the load balancer, receive task assignments from the load balancer, and execute the assigned tasks. As depicted in Figure 1, the red arrows indicate the task assignments progressing from the load balancer to edge nodes. Dotted lines present task offloading from edge nodes to others if they are incapable of performing the assigned tasks. In the case of any significant tasks that cannot be processed at the edge layer, they are offloaded to the upper layer (i.e., fog/cloud layer).
• Fog and Cloud layers (upper computing layer): This layer consists of machines with higher computing and storage capabilities than the lower layer (i.e., edge layer). If the significant tasks cannot be processed at the edge layer, they will be offloaded to and executed at this layer.

3.2. Load Balancing Methods in Edge Computing

The concept of load balancing in edge computing is similar to that in fog and cloud computing. Since computing nodes in edge computing clusters contend with resource constraints (e.g., network, memory, processing, etc.) [23,25,37], load balancing mechanisms play a pivotal role in preventing computing nodes from becoming overloaded.

Load balancing in edge computing employs various metrics, such as resource utilization, processing time, throughput, availability, security, energy consumption, cost, and response time [23,25,38]. It is obvious that conventional load balancing metrics mostly focus on the dynamic contexts of computing nodes (e.g., CPU utilization, memory usage, etc.), while omitting or not directly addressing the dynamic contexts of IoT devices, such as the volume of task requests and their computational requirements. The volume of task requests from IoT devices fluctuates over time. Additionally, the computational requirements of these task requests can vary significantly. These contexts of IoT devices directly impact the operation of computing nodes. For instance, the types of task requests from IoT devices can indicate how tasks affect the operation of computing nodes during or after their execution.

To underscore the limitation of conventional load balancing metrics in handling the dynamic context of IoT devices, we conducted an experiment using three load balancing mechanisms that only consider computing node context (e.g., CPU utilization), as shown in Figure 2.

In the experiment, the dynamic context of IoT devices is represented by three types of tasks (i.e., reading, writing, and model training) generated randomly. Considering only computing node context (i.e., CPU utilization), load balancing mechanisms mostly fail to prevent edge nodes from becoming overloaded. As shown in Figure 2a, the CPU utilization of Edge Node 4 reaches approximately $100\%$ with the round-robin load balancing mechanism. As shown in Figure 2b,c, even when employing K-Means and hill climbing, significant gaps in CPU utilization among edge nodes persist, making it easy to push some edge nodes into an overloaded state.

In conclusion, a novel load balancing metric, which is aware of both the dynamic contexts of computing nodes and IoT devices, should be efficiently proposed for load balancing mechanisms in edge computing. To address this challenge, this study introduces EdgeUP, which takes into account the dynamic contexts of both computing nodes and IoT devices to make decisions of task assignments while ensuring its lightweight execution for resource-constrained devices in edge computing.

4. EdgeUP: Utilization and Priority-Aware Load Balancing in Edge Computing

4.1. Overview of EdgeUP

Our load balancing mechanism aims to improve task assignment within edge computing. Our load balancing mechanism, EdgeUP, addresses the dynamic change in edge computing by taking into account the dynamic contexts of both IoT devices and edge nodes at the application layer. Regarding IoT devices, we consider priorities of tasks requested by users for execution on edge nodes. Regarding edge nodes, we consider their computational resource: CPU utilization. The product of these factors (i.e., priorities of tasks and CPU utilization) is introduced as a new metric called ServingValue. Eventually, our load balancing algorithm identifies the optimal (task, edge node) pairs to maximize the total ServingValue. By maximizing ServingValue with each task assignment, EdgeUP balances CPU utilization across edge nodes, preventing them from becoming overloaded. EdgeUP directly emphasizes the task assignments rather than offloading tasks between edge nodes.

Figure 3 provides an overview of our load balancing system at the application layer. The system consists of four primary components:

• IoT devices: Send task requests to “Task” topic on the message broker.
• Message broker: A module that facilitates communication among components in the system, allowing them to exchange information through message queue topics. It includes predefined topics containing message queues. The “Task” topic holds messages about task requests from IoT devices. The “Utilization” topic stores messages about CPU utilization, which are reported by the edge nodes. The “Task Assignment” topic contains messages about the assignment results, specifically the (task, edge node) pairs determined by the load balancing algorithm.
• Balancer: Receives task requests and CPU utilization from the corresponding topics on the message broker, executes the load balancing algorithm (i.e., EdgeUP), and sends the assignment results to the “Task Assignment” topic on the message broker.
• Edge nodes: Receive task assignments from the “Task Assignment” topic on the message broker and execute the assigned tasks. Additionally, edge nodes periodically report their CPU utilization to the “Utilization” topic on the message broker.

• In the following part, we formulate the load balancing problem in EdgeUP.

4.2. Problem Formulation

The system comprises a cluster $\mathcal{E}$ of E edge nodes with computational capabilities, $\mathcal{E}=\{1,2,\dots ,E\}$, a load balancer $\mathcal{B}$, and a set $\mathcal{N}$ of N IoT devices, $\mathcal{N}=\{1,2,\dots ,N\}$. Communication among IoT devices, edge nodes, and the balancer is facilitated through a message broker $\mathcal{Q}$.

• Each IoT device n at time t has the following:

• A binary variable $v_n^t\in \{0,1\}$, which denotes whether the IoT device n requests a task at time t or not.
• Priority value $V_n^t\in [0,1]$ represents the importance or criticality of a task requested by IoT device n at time t. The priority values should be predefined, with higher values indicating tasks of greater importance and requiring more computational resources for execution.

• Each edge node e at time t has the following:

• $A_e^t$, called “idle CPU” of edge node e at time t, represents a normalized value in the range $[0,1]$ of idle CPU percentage (i.e., idle CPU percentage is the complement of CPU utilization percentage, idle CPU = 100% - CPU utilization).
• A binary variable $a_e^t\in \{0,1\}$, which denotes whether edge node e at time t is assigned to a task or not.

As shown in Figure 3, at step , edge nodes periodically perform their CPU utilization by reporting to the “Utilization” topic on the message broker. At step , IoT devices send task requests (e.g., reading, writing files, training models, etc.) to the message broker. Task requests from IoT devices are sent to the corresponding topic on the message broker, “Task”. Each time, there are task request messages from IoT devices; the balancer will receive them by subscribing the “Task” topic on the message broker as shown at step . It also obtains the latest CPU utilization message from the “Utilization” topic as shown at step . After consuming task requests and CPU utilization messages, the balancer performs its load balancing algorithm as shown at step . The results of the load balancing algorithm from are sent to the “Task Assignment” topic on the message broker as shown at step . Edge nodes always subscribe the “Task Assignment” topic, so every time there is a new message (i.e., task assignment message from the balancer), edge nodes will receive and execute the task assigned to them as shown at step .

Particularly, tasks from IoT devices $\mathcal{N}$ are pushed into the “Task” topic. CPU utilization from edge nodes $\mathcal{E}$ are pushed into the “Utilization” topic. Balancer $\mathcal{B}$ uses two this information for the load balancing algorithm to output task assignments. As the result, we obtain pairs of (task, edge node), $(v_n^t,a_e^t)$, at each time t. We denote a new load balancing metric as ServingValue. It indicates the relationship between a pair comprising a task and an edge node.

• The serving value at time t of the pair (i.e., task n and edge node e, $(v_n^t,a_e^t)$) is

  $$S(v_n^t,a_e^t)=a_e^t{A}_e^t{v}_n^t{V}_n^t$$

  (1)
• The total serving value at time t is

  $$S^t=\sum _{\mathcal{E}}\sum _{\mathcal{N}}S(v_n^t,a_e^t)=\sum _{\mathcal{E}}\sum _{\mathcal{N}}{a}_e^t{A}_e^t{v}_n^t{V}_n^t$$

  (2)
• With a constraint of serving limitation of each edge node at each time,

  $$\sum _{\mathcal{E}}\sum _{\mathcal{N}}{a}_e^t\le E$$

  (3)
• The objective of this study was to maximize the total serving value. This objective helps to obtain optimal task assignments to prevent edge nodes from being overloaded.
• The problem formulation is

  $$max\sum _{\mathcal{E}}\sum _{\mathcal{N}}{a}_e^t{A}_e^t{v}_n^t{V}_n^t$$

  (4)

  $$st.\sum _{\mathcal{E}}\sum _{\mathcal{N}}{a}_e^t\le E$$

  (5)
• To maximize the total serving value at each timestamp, it is necessary to determine the optimal task and edge node pairs at each moment. Due to the limitation on the computation of edge nodes, we employ a lightweight approach to solve (4). The EdgeUP algorithm is presented in Algorithm 1.

Firstly, sets of edge nodes, IoT devices, and task priority values are set up (line 1). Three empty arrays are initialized for edge node indexes after an idle CPU value-sorting step (i.e., A), for task indexes after a priority value-sorting step (i.e., V), and for task assignment results (i.e., P), respectively (line 2). In a running process of T requesting times, at time t, the balancer collects idle CPU values of E edge nodes and priority values of N IoT devices (line 4 and line 5). Next, these values of idle CPU and task priority are sorted in ascending order into the A and V arrays, respectively (line 6 and line 7). Items with the same indexes in both sorted arrays from the previous step are paired together, forming pairs (i.e., P) of edge node index and task index (line 9). These pairs are the task assignments at the tth requesting time to edge nodes, $P^t={\{e,n\}}^N$ (line 11). It means that the edge node e is assigned to perform the task n for each pair of $P^t$. The assignment is produced to the corresponding topic (i.e., “Task Assignment” topic) on the message broker (line 11). Consequently, all edge nodes can receive task assignment messages. Next, the three temporary arrays $A,V,P$ are emptied (line 12), and the process continues with the next request time (line 13).

The total serving value at each timestamp is maximized in Algorithm 1 by applying the rearrangement inequality, which is detailed in Appendix A. The maximization problem in Equation (4) involves two key components: idle CPU values of edge nodes (represented as array A in Algorithm 1) and task priority values of IoT devices (represented as array V in Algorithm 1). According to the rearrangement inequality, the highest serving value at each timestamp is achieved by pairing the largest elements of A with the largest elements of V. Thus, sorting both arrays A and V in ascending order and pairing elements with the same indices ensure the maximization described in Equation (4).

Algorithm 1 EdgeUP: Utilization and priority-aware load balancing in edge computing.

1: Initialize $\mathcal{E}\left(\mathrm{Set}\mathrm{of}\mathrm{edge}\mathrm{nodes}.\right)=\{1,\dots ,E\}$, $\mathcal{N}\left(\mathrm{Set}\mathrm{of}\mathrm{IoT}\mathrm{devices}.\right)=\{1,\dots ,N\}$, and $\{V_n{\left(\mathrm{Task}\mathrm{priority}\mathrm{value}\mathrm{of}\mathrm{IoT}\mathrm{device}\mathrm{n}.)\right\}}^N$ 2: Initialize A (Array of edge node indexes after sorting idle CPU of edge nodes in ascending order.) = [$empty$], V (Array of task indexes after sorting priority of tasks requested from IoT devices in ascending  order.) = [$empty$], P (Array of (e, n) pairs; The variables e and n represent the indexes of the edge node and IoT device,  respectively.) = [$empty$] 3: for $t=0$ to $(T-1)$ do 4:       Collect set of idle CPU of edge nodes ${\left\{A_e^t\right\}}^E$ 5:       Collect set of priority values of tasks ${\left\{V_n^t\right\}}^N$ 6:       $A^t$ = Sort(${\left\{A_e^t\right\}}^E$) in ascending order 7:       $V^t$ = Sort(${\left\{V_n^t\right\}}^N$) in ascending order 8:       for $i=1$ to N do 9:             $P^t\left[i\right]=[e$ from $A^t\left[i\right]$, n from $V^t\left[i\right]]$ 10:       end for 11:       Produce task assignments at t with $P^t={\left\{(e,n)\right\}}^N$ 12:       $A^{t+1}=\left[empty\right],V^{t+1}=\left[empty\right],P^{t+1}=\left[empty\right]$ 13:       $t++$ 14: end for

4.3. EdgeUP Procedure

The EdgeUP procedure is presented in Figure 4. The three primary stages of the EdgeUP procedure, derived from the seven steps outlined in Figure 3, are illustrated in Figure 4. Each stage reflects the status of the three key components of the system (i.e., edge nodes, message broker, and balancer). The numbered sequence from Figure 3 is preserved, demonstrating the procedural flow of EdgeUP. The orange color is used to show the active states of functions and messages in each stage.

• At stage (1, 2, 3), edge nodes report their utilization to the “Utilization” topic on the message broker . Simultaneously, IoT devices send task requests to the “Task” topic on the message broker, represented by . Subsequently, the balancer retrieves task requests and utilization reports from the “Task” and “Utilization” topics on the message broker, as shown at and , respectively. The retrieved data serve as input for the load balancing function, illustrated at .
• At stage (4, 5, 6), the balancer uses the results from the previous step as inputs and executes the LoadBalancing function to determine task assignments, as indicated by . The task assignments, represented as pairs of (edge node index, task index), are processed by SendAssignment in preparation for delivery to the message broker, shown at . For example, in Figure 4, at stage (4, 5, 6), task assignments are EN_1:1, …, EN_E:N, meaning that the edge node 1 is assigned to task 1 and the edge node E is assigned to task N. Subsequently, these assignments are sent to the “Task Assignment” topic by the SendAssignment function, as illustrated at . Finally, the ReadAssignment functions at the edge nodes retrieve the assignments by subscribing to the topic, as shown at .
• At stage (7), edge nodes execute tasks assigned to them, as depicted at . For example, EdgeNode1 carries out Task1, while EdgeNodeE executes TaskN. During this stage, edge nodes continue to report their utilization to the “Utilization” topic on the message broker at regular intervals, as indicated by . Simultaneously, task requests are continuously generated, represented by . The balancer retrieves task requests and utilization reports through the ReadTaskRequest and ReadUtilizationReport functions, which read from the relevant topics on the message broker, as shown at and , respectively.

4.4. Task Generator

A task generator is a critical component in computing frameworks and simulators that samples events/tasks in an interval of time. The generating process is under the control of probability models, which can be Gaussian, Poisson, Dirichlet probability models, etc. Tasks in cloud, fog, and edge computing experiments have been popularly simulated as Poisson-based events or Poisson processes [39,40,41]. The usage of the Poisson process for events/task generators is in cloud, fog, and edge simulators, such as Cloudsim [42], CloudSimPy [43], EdgeCloudSim [44], etc. A Poisson process is defined as follows.

Suppose that events/tasks occur at a fixed rate $\lambda >0$ throughout an interval. The interval can be divided into sub-intervals t, expressed as $t\in [0,\infty )$. The counting process, resulting in the number of events/tasks at the sub-intervals, is called a Poisson process and denoted as $\left\{N\right(t),t\in [0,\infty \left)\right\}$. This process must satisfy all three conditions:

• $P\left(0\right)=0:$ The probability of events/tasks occurring at the first sub-interval is zero;
• An event/task occurring in any given sub-interval is independent of other sub-intervals;
• $P(t,\tau )\propto Poisson\left(\lambda \tau \right):$ The probability of one event occurring in a sub-interval is proportional to the length $\tau$ of the sub-interval.

The word “process” is used to suggest that the simulation takes place over time. The random variable X representing the actual number of events/tasks occurring in the interval has a Poisson distribution. According to the Poisson model, the probability of r occurrences in the interval is given as follows:

$$P(X=r)=e^{-\lambda }{\displaystyle \frac{{\lambda }^r}{r!}},r=0,1,2,3,\dots$$

(6)

This research adopts methods in simulation from previous works in cloud, fog, and edge computing, so the Poisson model is used to sample events/tasks in the experiment.

5. Experiments and Results

5.1. Experimental Setup

In our experiment, we set up a cluster of 5 edge computing nodes with 1 master node working as a balancer and 4 edge nodes working as workers. All 5 edge computing nodes used in our experiment were Raspberry Pi 400. Configuration and installation of the edge computing nodes are shown in

Table 2. Configuration and installation.

	Master Node	Edge Node
Quantity	1	4
Configuration	Raspbian OS	Raspbian OS
	4 CPUs	4 CPUs
Tools	Apache Kafka	Apache Kafka,
		Python Threading,
		TensorFlow Keras
Modules	Kafka broker,	Kafka producer,
	Kafka producer,	Kafka consumer,
	Kafka consumer,	Tasks (reading, writing,
	LB mechanisms,	MLP training)
	Task generator

.

We simulated an edge computing system by adhering to a widely recognized reference architecture for edge computing [45,46]. This architecture specifies that both storage and intelligence operations are conducted at the edge layer. Key operations include reading and writing for storage and machine learning model training for intelligence. To reflect this, we simulated three types of tasks: reading, writing, and machine learning model training. Specifically, reading and writing tasks involve accessing and modifying rows within a time-series dataset, a common format in IoT applications. For our experiments, we utilized SensorScope datasets [47], which contain time-series data rows comprising timestamp, temperature, and light with the data types string, float, and float.

Each reading task retrieves a single data row, while each writing task appends a new data row to the file. For the model training task, we trained a basic neural network, specifically a multi-layer perceptron (MLP). The training configuration included an MLP with a (12, 8) structure (e.g., 12 input features fed into the network; the network with 1 hidden layer with 8 neurons; the output layer with Softmax activation function for classification), 1 training round, the Adam optimizer, 500 epochs, and a batch size of 10. Approximately 2800 KB of time-series data from the SensorScope dataset [47] were used as the training dataset.

The task generator in our simulation is responsible for generating task requests at regular intervals. The rationale behind incorporating a task generator is detailed in Section 4.4. To simulate task requests originating from IoT devices, we employed a Poisson-based request generator at the load balancer, ensuring realistic task generation patterns.

We used Apache Kafka as the message broker. Furthermore, at the edge nodes, we utilized the Threading module in Python to provide them with asynchronous task running. Since tasks assigned to each edge node occur randomly and have lengthy execution times, each edge node was capable of running tasks asynchronously. Additionally, edge nodes were required to perform tasks such as reading, writing, and multi-layer perceptron (MLP) training. For the MLP training task, the edge nodes installed TensorFlow Keras to create and perform the training model. To conclude, 5 functional modules were installed on the master node, while 3 functional modules were installed on edge nodes as shown in Table 2.

The master node runs 5 modules:

• Kafka broker: Works as a message broker in Figure 3.
• Task generator: Generates 5 tasks of 3 task types (i.e., reading, writing, and MLP training) within each interval based on the Poisson process at the rate $\lambda =0.5$.
• Balancer: Performs 4 load balancing mechanisms (i.e., EdgeUP, round-robin, hill climbing, K-Means). Priorities of tasks are predefined (e.g., $V_{reading}=0.2,V_{writing}=0.2,V_{training}=0.6$).
• Kafka producer: Produces messages to their topics on the Kafka broker. In particular, after obtaining tasks from the task generator, the Kafka producer on the master node produces tasks to the “Task” topic as shown in Figure 3. The balancer obtains messages from the “Task” topic as one of two inputs. Additionally, after the load balancing mechanisms are executed and the task assignment results are provided, the Kafka producer produces those task assignments to the “Task assignment” topic.
• Kafka consumer: The master node can also work as a message consumer. With this role, it consumes messages on the "Utilization" topic as the other input of the balancer.

Edge nodes run 3 main modules:

• Task: Executes tasks (i.e., reading, writing, and MLP training).
• Kafka producer: Periodically conducts CPU utilization reporting by gathering CPU utilization values and subsequently sending them to the “Utilization” topic on the Kafka broker. In experiments, we set up various reporting periods of edge nodes, such as $p=5,p=15,p=30,p=50$ in seconds.
• Kafka consumer: Consumes the “Task assignment” topic on the Kafka broker to identify the tasks assigned to edge nodes by the master node.

In our experiments, we focused on evaluating how well our proposal could improve idle CPU distance and the CPU utilization of edge nodes across various reporting periods of edge nodes and also be compared to 3 load balancing mechanisms with a conventional load balancing metric (i.e., CPU utilization). Additionally, we compared the execution time of our proposal to that of other load balancing mechanisms.

5.2. Results and Analysis

5.2.1. Impact of Reporting Period on ServingValue

First, we examined how the reporting periods of edge nodes affect average serving values (i.e., proposed load balancing metric) in our proposal. Figure 5 shows how the reporting period affects ServingValue.

ServingValue reflects the effectiveness of a load balancing method in our study. A higher ServingValue indicates better performance of the load balancing method. Figure 5 presents the average ServingValue across different reporting periods. As shown, EdgeUP achieves the highest ServingValue at $p=5$. This suggests that shorter reporting periods lead to improved ServingValue.

This can be explained by the fact that shorter reporting periods provide more up-to-date information on the CPU utilization of edge nodes within load balancing mechanisms. In other words, load balancing mechanisms can leverage the latest CPU utilization of edge nodes for their operations, being aware of the dynamic status of CPU utilization. Conversely, a prolonged reporting period can degrade the performance of load balancing mechanisms, as it prevents the accurate observation of the dynamic CPU utilization status of edge nodes. Consequently, for subsequent experiments, $p=5$ is employed as the reporting period setup.

5.2.2. Comparison of CPU Utilization

The primary focus of this study is to introduce an effective load balancing mechanism that minimizes CPU utilization discrepancies among edge nodes. This is achieved by maximizing the proposed load balancing metric, ServingValue, thereby assisting edge nodes in avoiding overloaded states.

Figure 6 shows the CPU utilization of edge nodes over 50 rounds of task generation across different load balancing mechanisms (i.e., EdgeUP, round-robin, hill climbing, K-Means) with $p=5$. The figure demonstrates that EdgeUP successfully maintains a high level of similarity in CPU utilization among edge nodes within a computing cluster. In contrast, the round-robin method results in the worst similarity in CPU utilization (Figure 6a). With K-Means and hill climbing, the similarity in CPU utilization is more improved than with the round-robin. However, it still raises up to over 30%, 15% of CPU utilization gaps between 2 edge nodes with K-Means (Figure 6b) and hill climbing (Figure 6c), respectively, whereas, as shown in Figure 6d, EdgeUP limits CPU utilization gaps to no more than $10\%$.

The large disparities in CPU utilization among edge nodes when using the three conventional methods stem from their failure to account for the current status of edge nodes (i.e., CPU utilization) and the nature of incoming task requests (i.e., reading, writing, and MLP training). For instance, in our experiments, MLP training demands the highest CPU utilization compared to the other two task types. In the worst-case scenario, an edge node already engaged in an MLP training task is assigned an additional MLP training task, causing its CPU utilization to rise further. This scenario frequently occurs with the round-robin method, as it assigns tasks sequentially without considering the edge nodes’ current workload or the type of task. As shown in Figure 6a, Edge Node 4 exhibits an increasing CPU utilization, nearing an overloaded state. This is because Edge Node 4, already handling an MLP training task with high CPU demand, continues to receive similar tasks, compounding its CPU load.

Additionally, EdgeUP can consistently maintain the similarity in CPU utilization among edge nodes. The higher the similarity in CPU utilization, the more effective the load balancing mechanism. This is because a high similarity in utilization among edge nodes is equal to minimize CPU utilization discrepancies by maximizing ServingValue, reducing the likelihood of edge nodes becoming overloaded. ServingValue is a combination of idle CPU (i.e., the complementary part of CPU utilization, idle CPU = 100% − CPU utilization) and task priority. EdgeUP tries to assign tasks with high priority to edge nodes with high idle CPU. Because tasks with high priority will assume high utilization, they should be assigned to edge nodes with high idle CPU. As the result, the CPU utilization values of edge nodes converge towards each other. In other words, whenever an edge node has a low idle CPU, EdgeUP strives to balance the load by assigning heavier tasks to edge nodes with higher idle CPU or by avoiding assigning heavy tasks to those with low idle CPU.

To better discern the similarity in CPU utilization among edge nodes, we provide a comparison of average idle CPU distance among edge nodes over 50 rounds of task generation across multiple load balancing mechanisms in Figure 7.

The idle CPU distance among edge nodes is calculated by

$$d\left(\mathcal{E}\right)=\sum d(e_i,e_j),$$

(7)

with

$$e_i,e_j\in \mathcal{E}$$

(8)

$$i<j$$

(9)

$$d(e_i,e_j)=|idleCP{U}_{e_i}-idleCP{U}_{e_j}|$$

(10)

where $\mathcal{E}$ is the set of edge nodes. $e_i,e_j$ are edge node i and edge node j.

The idle CPU distance between 2 edge nodes is calculated as Equation (10). The lower the idle CPU distance, the higher the similarity of CPU utilization. Figure 7 shows that EdgeUP raises the lowest idle CPU distance among the load balancing mechanisms across various reporting periods. For example, with the shortest reporting period $p=5$, EdgeUP outperforms round-robin, hill climbing, and K-Means by 5.6x, 2.1x, and 3.6x, respectively.

5.2.3. Execution Time

Finally,

Table 3. Execution time of load balancing mechanisms.

	Round-Robin	K-Means	Hill Climbing	EdgeUP
Execution time	0.21 μs	1.1 ms	0.03 s	0.6 ms

presents a comparison of the execution time between EdgeUP and other load balancing methods.

The table shows the execution time for each load balancing mechanism to produce a task assignment on a Raspberry Pi 400. Among the methods, round-robin demonstrates the shortest execution time, with EdgeUP following closely behind. Although EdgeUP takes slightly longer to execute than round-robin, the difference is minimal. Notably, EdgeUP outperforms other load balancing mechanisms (i.e., K-Means and hill climbing) in terms of execution time, underscoring its lightweight nature.

This outcome aligns with expectations, as round-robin is a fundamental load balancing approach that requires minimal computation to generate task assignments. In contrast, K-Means and hill climbing exhibit longer execution times compared to round-robin and EdgeUP. As learning and optimization-based methods, they involve more complex operations to achieve optimal results, contributing to their extended execution time.

6. Conclusions

This study proposes EdgeUP, a lightweight load balancing mechanism for edge computing in IoT applications. EdgeUP introduces a new load balancing metric, named ServingValue, which considers both dynamic contexts of edge nodes (i.e., CPU utilization) and IoT devices (i.e., task priority). EdgeUP allocates tasks to edge nodes by identifying pairs of tasks and edge nodes that result in the highest ServingValue. By maximizing ServingValue at each task assignment, EdgeUP converges the CPU utilization of edge nodes, avoiding the overloaded state of edge nodes in edge computing. EdgeUP outperforms load balancing mechanisms using conventional load balancing metrics in terms of CPU utilization similarity, idle CPU distance, and execution time.

In addition to considering the dynamic context of IoT devices (i.e., task priority), EdgeUP currently prioritizes a single computing context (i.e., CPU utilization) when balancing workloads. To improve its effectiveness, it could benefit from incorporating additional computing factors such as memory consumption, input/output performance, and other aspects of the load balancing, including response latency, task completion time, cost, etc.