**4. Constraint Markov Decision Process (CMDP)**

To obtain the optimal offloading strategy, we formulate a CMDP model with five elements. (Since the CMDP model that is a mathematical framework to model decision-making when outcomes need to be constrained, and they are partially random and under the control of the decision maker [27], it is suitable to construct the optimal offloading decision table.): (1) decision epoch; (2) state; (3) action; (4) transition probability; and (5) cost and constraint functions. Subsequently, we convert the CMDP model to an equivalent LP problem to obtain the optimal policy. After that, the IoT device pairing problem is formulated under the optimal offloading policy. Important notations for the CMDP model and IoT device pairing problem are summarized in Table 1.


