*3.3. Semi TMC-GMM*

In the Markov model considered in our previous work [7], the remaining time of the sojourn of the hidden state *v<sup>n</sup>* is of geometric distribution. While considering *V* as semi-Markovian seems to better suited to our problematic, as in general, *v<sup>n</sup>* has no geometric remaining sojourn time. For example, the gait phase has a minimum duration, while in geometric distribution the maximal probability is for null duration. In real applications of classic hidden semi-Markov model (HSMM) [26,27], there is a fixed maximum sojourn time for each possible value of *vn*. When *v<sup>n</sup>* switches to a new value, the maximal possible random sojourn time is shorter than a fixed value *M*. Once the sojourn time has elapsed at time *n*, the hidden state must change to a different value, i.e., *p*(*vn*+<sup>1</sup> = *vn*) = 0. This implies that the maximum sojourn time should be large enough to cover the largest possible sojourn time, which appears as a drawback in our application. In another semi-Markov approach described in [28] that we adopt here, the random sojourn time (just after having switched) is not the exact duration of the state, but the minimum sojourn time. This means that once the sojourn time elapsed, the next hidden state is possible to stay the same. This character allows make the maximum value *M* significantly smaller than the one in classic HSMM, which accelerates the entire method since the dimension of transition matrix is reduced.

*Sensors* **2019**, *19*, 4242

To be more precise, consider a new stochastic process *<sup>D</sup>* = (*D*1, ··· , *DN*) that represents the minimum remaining sojourn time in a given hidden state *vn*, and the possible realization of each *Dn* (denoted by *dn*) takes its value in *L* = {0, 1, ··· , -}. Thus for *<sup>V</sup><sup>n</sup>* <sup>=</sup> *<sup>v</sup><sup>n</sup>* and *Dn* <sup>=</sup> *dn*, we have *<sup>v</sup><sup>n</sup>* <sup>=</sup> *<sup>v</sup>n*+<sup>1</sup> <sup>=</sup> ··· <sup>=</sup> *<sup>v</sup>n*+*dn* . And *<sup>v</sup>n*+*dn* is obtained *w.r.t. <sup>p</sup>*(*vn*+*dn*+1|*vn*+*dn* ), which is a transition similar to the ones in the TMC and TMC-GMM. Thus, *<sup>v</sup>n*+*dn*+<sup>1</sup> is possible to be the same as *<sup>v</sup>n*+*dn* . Once *<sup>v</sup>n*+<sup>1</sup> is set, a new minimum sojourn time *dn*+<sup>1</sup> is obtained in *L* = {0, 1, ··· , -}. Please notice that for *Dn* = *dn* = 0, there is *Dn*+<sup>1</sup> = *dn*+<sup>1</sup> = *dn* − 1, which is specified in Equation (10).

Finally, SemiTMC-GMM is extended from TMC-GMM *Z* via the couple (*Z*, *D*), which follows the transition probabilities:

$$p(\mathbf{z}\_{n+1}, d\_{n+1} | \mathbf{z}\_n, d\_n) = p(\mathbf{v}\_{n+1} | \mathbf{z}\_n, d\_n) p(h\_{n+1} | \mathbf{v}\_{n+1}) p(d\_{n+1} | \mathbf{v}\_{n+1}, d\_n) p(\mathbf{y}\_{n+1} | \mathbf{v}\_{n+1}, h\_{n+1}), \tag{8}$$

$$p(\boldsymbol{\sigma}\_{n+1}|\boldsymbol{z}\_{n},d\_{n}) = \begin{cases} \delta\_{\boldsymbol{\sigma}\_{n}}(\boldsymbol{\sigma}\_{n+1}), & d\_{n} > 0 \\ p^{\*}(\boldsymbol{\sigma}\_{n+1}|\boldsymbol{\sigma}\_{n}), & d\_{n} = 0 \end{cases} \tag{9}$$

$$p(d\_{n+1}|\boldsymbol{\upsilon}\_{n+1}, d\_n) = \begin{cases} \delta\_{d\_n - 1}(d\_{n+1}), & d\_n > 0 \\ p(d\_{n+1}|\boldsymbol{\upsilon}\_{n+1}), & d\_n = 0 \end{cases} \tag{10}$$

where *δ* is the Kronecker function (*δa*(*b*) = 1 for *a* = *b* and *δa*(*b*) = 0 for *a* = *b*).

The properties of the four terms on the right side of Equation (8) are clarified in the following:


Now, the Equations (9) and (10) together describe how the hidden states, *Vn* and *Dn*, transfer in SemiTMC-GMM.

The dependency graphs of the three models, i.e., TMC, TMC-GMM, and SemiTMC-GMM, are shown in Figure 1. The couple *V* = (*X*, *U*) is regarded as one hidden state for reducing the complexity of the graphs, as well as reminding that the total number of processes involved in the three models are 3, 4, and 5, respectively.

**Figure 1.** Dependency graphs. TMC = triplet Markov chain; GMM = Gaussian mixture model.

Estimating the individual *xn* and *un* is different from both TMC and TMC-GMM, for the sense of introducing the sojourn state *Dn*. The probabilities of *xn* can be obtained by

$$\begin{split} p(\mathbf{x}\_{n}|\mathbf{y}\_{1}^{n}) &= \sum\_{\mathbf{u}\_{n}} \sum\_{d\_{\mathrm{u}}} p(\mathbf{x}\_{\mathrm{n}}, \boldsymbol{\mu}\_{\mathrm{n}}, d\_{\mathrm{u}} | \mathbf{y}\_{1}^{n}), \\ p(\mathbf{x}\_{n}|\mathbf{y}\_{1}^{N}) &= \sum\_{\mathbf{u}\_{n}} \sum\_{d\_{\mathrm{u}}} p(\mathbf{x}\_{\mathrm{n}}, \boldsymbol{\mu}\_{\mathrm{n}}, d\_{\mathrm{u}} | \mathbf{y}\_{1}^{N}). \end{split} \tag{11}$$

The probabilities *<sup>p</sup>*(*xn*, *un*, *dn*|*y<sup>n</sup>* <sup>1</sup> ) and *<sup>p</sup>*(*xn*, *un*, *dn*|*y<sup>N</sup>* <sup>1</sup> ) are the filtering and smoothing probability of the hidden state in SemiTMC-GMM, respectively. Likewise, the probabilities *<sup>p</sup>*(*un*|*y<sup>n</sup>* <sup>1</sup> ) and *<sup>p</sup>*(*un*|*y<sup>N</sup>* 1 ) are obtained in a similar way. Finally, the estimated hidden state *x*ˆ*<sup>n</sup>* and *u*ˆ*<sup>n</sup>* can be obtained by Equation (5).

To summarize, the proposed SemiTMC-GMM is a model contains five stochastic processes *X*, *U*, *D*, *H*, *Y*, with Markov distribution of *Z*<sup>∗</sup> = (*X*, *U*, *D*, *H*, *Y*). The process *X* models the activities we are looking for, *Y* models the observation, *U* models the introduced gait or leg phase, *D* models the semi-Markovianity of *V* = (*X*, *U*), and *H* models the presence of Gaussian mixtures. Thus, *Z*<sup>∗</sup> = (*V*,*W*, *Y*) can be regarded as a classic TMC with hidden state *V*, observed *Y*, and an additional latent *W* = (*D*, *H*).
