*4.2. State Space*

We define the overall state space **S** as (The state space is constructed based on the assumption where IoT devices can offload all or half of the task. However, it can be easily extended to consider other portions of the task to add elements to **T<sup>M</sup> <sup>i</sup>** , **<sup>T</sup><sup>O</sup> <sup>i</sup>** , **<sup>T</sup><sup>M</sup> <sup>j</sup>** , and **<sup>T</sup><sup>O</sup> j** .)

$$\mathbf{S} = \mathbf{T\_i^M} \times \mathbf{T\_i^O} \times \mathbf{E\_i} \times \mathbf{D\_i} \times \mathbf{T\_j^M} \times \mathbf{T\_j^O} \times \mathbf{E\_j} \times \mathbf{D\_j} \tag{1}$$

where **T<sup>M</sup> <sup>i</sup>** and **<sup>T</sup><sup>M</sup> <sup>j</sup>** are the states for representing the occurrence and processing status for the task of IoT devices *i* and *j*, respectively. **T<sup>O</sup> <sup>i</sup>** and **<sup>T</sup><sup>O</sup> <sup>j</sup>** denote the states for the processing status of the offloaded task of IoT devices *i* and *j*, respectively. **Ei** and **Ej** are the states for the energy level of IoT devices *i* and *j*, respectively. **Di** and **Dj** represent the states for denoting whether the timers for the deadline of the task of IoT devices *i* and *j* expire or not, respectively.

**T<sup>M</sup> <sup>i</sup>** , **<sup>T</sup><sup>O</sup> <sup>i</sup>** , **Ei**, and **Di** are the states for IoT device *i*, and these states are defined as follows. First, **T<sup>M</sup> <sup>i</sup>** is given by

$$\mathbf{T\_i^M} = \{0, 1, 2, 3, 4\}\,,\tag{2}$$

where *T<sup>M</sup> <sup>i</sup>* represents the occurrence and processing status for the task of IoT device *i*. In other words, *T<sup>M</sup> <sup>i</sup>* = 0 denotes that the task does not occur in IoT device *<sup>i</sup>*, whereas *<sup>T</sup><sup>M</sup> <sup>i</sup>* = 1 refers to the situation immediately after the task occurs in IoT device *i*. *T<sup>M</sup> <sup>i</sup>* = 2 represents the situation where IoT device *i* processes all of the task by itself. Meanwhile, *T<sup>M</sup> <sup>i</sup>* = 3 and *<sup>T</sup><sup>M</sup> <sup>i</sup>* = 4 represent the situations where half of the task and all of the tasks are offloaded to IoT device *j*, respectively. Note that, when *T<sup>M</sup> <sup>i</sup>* = 3, IoT device *i* processes the remaining half of the task.

**T<sup>O</sup> <sup>i</sup>** is represented by

$$\mathbf{T}\_{\mathbf{i}}^{\mathbf{O}} = \{0, 1, 2\}, \tag{3}$$

where *T<sup>O</sup> <sup>i</sup>* represents the processing status of the offloaded task of IoT device *<sup>i</sup>*. Specifically, *<sup>T</sup><sup>O</sup> <sup>i</sup>* = 0 describes the situation where any task is not offloaded to its partner (i.e., IoT device *j*). Meanwhile, *T<sup>O</sup> <sup>i</sup>* = 1 and *<sup>T</sup><sup>O</sup> <sup>i</sup>* = 2 represent the situation where half of the task and all of the tasks are offloaded to IoT device *j*, respectively, and it is being processed in IoT device *j*.

**Ei** is represented as [29]

$$\mathbf{E}\_{\mathbf{i}} = \{0, 1, 2, \dots \\ E\_{MAX}\}\_{\mathbf{i}'} \tag{4}$$

where *EMAX* is the maximum battery capacity of an IoT device.

$$\mathbf{D}\_{\mathbf{i}} = \{0, 1\},$$

$$\mathbf{D}\_{\mathbf{i}} = \begin{bmatrix} 0, 1 \\ \end{bmatrix},$$

where *Di* denotes whether the timer for the deadline of the task of IoT device *i* expires or not. In other words, *Di* = 0 and *Di* = 1 represent that the timer for the deadline of the task of IoT device *i* does not expire and expires, respectively.

**T<sup>M</sup> <sup>j</sup>** , **<sup>T</sup><sup>O</sup> <sup>j</sup>** , **Ej**, and **Dj** are the states for IoT device *j*, and these states can be defined as similar with the states for IoT device *i*. These definitions are omitted in this paper due to the page limitation and for simple descriptions, which can be found in [30].
