*4.1. Decision Epoch*

Figure 3 shows the timing diagram for the CMDP model. A sequence *T* = {1, 2, 3, ...} represents the time epochs when successive decisions are made [28]. *St* and *At* denote the state and the action chosen at the decision epoch *t* ∈ *T*, respectively. *τ* represents the duration of each decision epoch.

$$\underbrace{\begin{array}{c} \mathsf{A}\_{\mathsf{A}}, \mathsf{A}\_{\mathsf{A}} \\ \xleftarrow{\iota\_{\mathsf{A}}} \mathsf{A}\_{\mathsf{A}} \end{array}}\_{\begin{array}{c} \mathsf{A}\_{\mathsf{A}}, \mathsf{A}\_{\mathsf{A}} \\ \vdots \\ \xleftarrow{\iota\_{\mathsf{A}}} \mathsf{A}\_{\mathsf{A}} \end{array}} \xrightarrow{\iota\_{\mathsf{A}}, \mathsf{A}\_{\mathsf{A}}} \underbrace{\begin{array}{c} \mathsf{A}\_{\mathsf{A}}, \mathsf{A}\_{\mathsf{A}} \\ \xleftarrow{\iota\_{\mathsf{A}}} \mathsf{A}\_{\mathsf{A}}, \mathsf{A}\_{\mathsf{A}} \end{array}}\_{\begin{array}{c} \mathsf{A}\_{\mathsf{A}}, \mathsf{A}\_{\mathsf{A}} \\ \xleftarrow{\iota\_{\mathsf{A}}} \mathsf{A}\_{\mathsf{A}}, \mathsf{A}\_{\mathsf{A}} \end{array}}$$

**Figure 3.** Timing diagram.
