**1. Introduction**

From the recent advancement of Internet of Things (IoT) devices with high computing power, complicated computation can be handled without remote servers [1]. However, the development speed of batteries for IoT devices is inferior to that of computing module, and thus the limited battery capacity of IoT devices is being a major deployment barrier for IoT-based computing systems. Therefore, there is an increasing interest on the energy harvesting technique that converts wasted energy to electricity [2,3]. With this technique, IoT devices do not need to recharge and/or replace their batteries anymore, and thus the operating expenditure of IoT-based computing systems can be reduced [4]. However, the energy that can be harvested from external energy sources is generally uncontrollable and intermittent. Moreover, the harvested energy volume has temporal and spatial variations. Therefore, it is difficult to provide a reliable power supply to IoT devices. In this situation, if an IoT device cannot harvest energy for a long time and it processes lots of tasks requiring high computing power, its energy can be depleted. To mitigate this problem and improve the energy efficiency of harvesting IoT devices, a number of works (e.g., sleep scheduling, CPU cycle adjustment, and so on) have been investigated in the literature [5–10]. One of the possible solutions is offloading tasks to nearby IoT devices [5–7]. IoT-based computing systems have advantages compared to remote servers-based offloading systems. For example, offloading to remote servers consumes huge resources in networks when IoT devices generate lots of tasks. In addition, longer latency is needed when offloading to remote servers. Especially when an energy-scarce IoT device offloads tasks to a nearby energy-abundant IoT device, energy depletion of the energy-scarce IoT devices probably does not occur. However, unplanned offloading can cause another energy depletion. For example, when an offloader

(i.e., an energy-scarce IoT device) always offloads all tasks to an offloadee (i.e., an energy-abundant IoT device) having lots of own tasks and/or small harvesting rate, the energy of the offloadee can be depleted within a short duration. Moreover, tasks cannot be completed within their deadlines due to the high load of the offloadee. Therefore, a sophisticated offloading algorithm should be devised.

In this paper, we propose an energy efficient cooperative computation algorithm (EE-CCA). In an EE-CCA, each IoT device is paired to its partner and a pair of IoT devices conduct cooperative computing. Specifically, a centralized controller collects information such as the distribution about temporal and spatial variations of external energy sources, the task occurrence rates of IoT devices, and the energy levels of IoT devices. Based on this information, the controller can construct and distribute offloading decision tables to IoT devices. Then, when a task occurs in IoT devices, they decide whether to offload some parts of the task to the opponent by following the decision tables. To minimize the energy outage probability while completing most of the tasks before their deadlines, we formulate a constraint Markov decision process (CMDP) problem, and the optimal offloading strategy is obtained by linear programming (LP). Meanwhile, an optimization problem of finding pairs of IoT devices (i.e., IoT device pairing problem) is formulated under the optimal offloading strategy. Evaluation results demonstrate that the EE-CCA can reduce the energy outage probability up to 78% compared with the random offloading scheme while completing tasks before their deadlines with high probability. In addition, the EE-CCA operates adaptively even when the operating environment (e.g., inter-task occurrence rate) changes.

The contribution of this paper can be summarized as follows: (1) we develop the cooperative computation algorithm called EE-CCA for IoT devices, while optimizing the EE-CCA by means of CMDP formulation; (2) optimal pairs of IoT devices are decided based on the optimization problem; and (3) extensive evaluation results are presented and analyzed under various environments, providing valuable guidelines for the design of cooperative computing in energy harvesting IoT.

The remainder of this paper is organized as follows. Related works are summarized in Section 2, and the EE-CCA is described in Section 3. The CMDP model for cooperative computing and the optimization problem for the IoT device pairing are developed in Section 4. Evaluation results are given in Section 5, and followed by the concluding remarks in Section 6.

## **2. Related Works**

A number of studies on the computation offloading have been conducted to improve the energy efficiency of energy-constraint devices [11–22]. These can be categorized into: (1) framework design [11–16]; and (2) algorithm design [17–22].

Wang et al. [11] proposed an evolutionary mobile network architecture called MobiScud that integrates the cloud services into the mobile networks by means of software defined networking (SDN) and network function virtualization (NFV) technologies in a backwards compatible fashion. Tong et al. [12] organized edge cloud servers into a hierarchical architecture which enables aggregation of the peak loads across different tiers of cloud servers. Specifically, when the loads exceed the capacities of lower tiers of edge cloud servers, they can be aggregated and offloaded by other servers at higher tiers in the edge cloud hierarchy to maximize the amount of mobile workloads being served. Taleb and Ksentini [13] introduced a follow me cloud concept that enables mobile cloud services to follow their respective mobile devices by migrating services to the optimal cloud. Liu et al. [14] proposed convergence of cloud and cellular systems, abbreviated as CONCERT, based on a concept of control/data plane decoupling and hierarchically placement of the resources within the network to manage flexibly and elastically networks and cloud services. Puente et al. [15] presented a seamless approach for the deployment of edge clouds where conventional mobile traffic and computing related traffic are segregated and handled individually at base stations. However, since these works do not consider the device-to-device offloading, IoT devices with high computing power cannot be exploited efficiently. Shukla and Munir [16] proposed a computation offloading architecture where an IoT device first tries to offload tasks to another IoT device instead of directly offloading to the cloud to process the

huge amount of data while guaranteeing the task completion before the deadline. However, they did not provide any optimization method.

Ko et al. [17] proposed a spatial and temporal computation offloading decision algorithm where an energy-constraint device decides where and when to process tasks by means of a Markov decision process (MDP) by considering the energy consumption and the transmission cost. Zhao et al. [18] developed an optimization problem whose objective function is to maximize the probability that task execution satisfies the given delay bound. The problem was proved to be concave, and an optimal algorithm was proposed. Tang and Chen [19] studied a social-aware computation offloading game and designed a distributed computation offloading algorithm to achieve the Nash equilibrium. Similarly, Chen et al. [20] modeled a multi-user computation offloading game and designed a distributed computation offloading algorithm that can achieve the Nash equilibrium of the game. Zheng et al. [21] formulated the mobile users' offloading decision process under a dynamic environment as a stochastic game. Then, they proposed a multi-agent stochastic learning algorithm that can run in a fully distributed manner without any information exchange. Yu et al. [22] developed an optimal collaborative offloading strategy under a distributed caching scenario. Specifically, they formulated a problem of users' allocation as a coalition formation game with the consideration of relationships between the offloading and caching and then proposed an optimal offloading with a caching-enhancement scheme. These works improve the performance of computation offloading; however, no previous studies optimize the performance of IoT-based computing systems.

#### **3. Energy Efficient Cooperative Computation Algorithm (EE-CCA)**

Figure 1 shows the system model of this paper. In our system model, there are *N* IoT devices with the energy harvesting capability. We assume heterogeneous IoT environment, where IoT devices have different computing power and current energy level. In real systems, some IoT devices may not have sufficient computing power and/or harvesting capability. To support this situation, a study on the robustness of the proposed algorithm (e.g., a resilient multiscale coordination control [23]) should be conducted, which is one of our future works. In addition, since they are installed at different spots and the condition of external energy sources is volatile, energy volumes that can be harvested at each IoT device are different from each other. In addition, with different rates, these IoT devices periodically generate tasks that can be abstracted into the input data and the completion deadline before which the task should be completed [24]. In addition, we consider an application where the input data do not have dependency, e.g., binomial classification that determines whether each input is larger than a given threshold or not, and therefore the input data can be partitioned and offloaded. When the task occurs in a particular IoT device, it decides whether to offload some parts of the task (i.e., input data) to a neighbor IoT device or not with the consideration of the energy level and the deadline of the task. Note that, even though the formulation of this paper is based on the assumption where IoT devices can offload all or half of the tasks, it can be easily extended to consider other portions of the task.

**Figure 1.** System model.

Intuitively, when an IoT device offloads some parts of the task to a neighbor IoT device, the task can be processed in a distributed manner, which can reduce the energy consumption of the task owner. Since IoT devices offload their tasks to nearby IoT devices by exploiting transmission technologies with low power consumption (e.g., Bluetooth), the energy consumption for transmission of the task can be neglected compared to that for processing the task. Moreover, if the neighbor IoT device does not have its own task, the task completion time can be shortened. However, if the task is offloaded to an energy-scarce IoT device, it causes the energy depletion of the IoT device, and then the offloaded task cannot be processed due to the energy depletion. In addition, when the neighbor IoT device has its own task, it should process its own task and the offloaded task simultaneously, and thus its processing time can increase. Then, both tasks may not be completed within their deadlines. To prevent these situations, we propose the EE-CCA and its flow chart is shown in Figure 2. First, the controller collects and/or maintains information such as the distribution about temporal and spatial variations of external energy sources, the task occurrence rates of IoT devices, and the energy levels of IoT devices (Step 1 in Figure 2). Based on this information, the controller determines appropriate partners of IoT devices for cooperative computation and transmits the pairing information to IoT devices (Step 2 in Figure 2). In addition, the controller constructs an offloading decision table consisting of the current status and the operation in a centralized manner. If some parameters (e.g., the task occurrence rates of IoT devices) are changed, the offloading decision table can be reconstructed by the controller and transmitted to IoT devices again. Therefore, if the parameters are frequently changed, extra signaling overhead can occur. To mitigate the signaling overhead, several techniques such as aggregation and delta encoding can be exploited [25]. Note that the offloading decision table can be obtained by CMDP, which will be elaborated in Section 4. After that, the controller transmits the optimal offloading decision table to IoT devices (Step 3 in Figure 2). On the basis of this table, IoT devices can conduct the cooperative computing (i.e., decide whether to offload some parts of the task or not) (Step 4 in Figure 2). By means of table deployments in IoT devices, the CMDP model can be applied to resource-constrained IoT devices without any high computation overhead in IoT devices [26].

**Figure 2.** Flow chart.
