*5.2. LMFIMU Dataset*

For this dataset, the size of sliding window for computing features is set to 15, and it is also determined by our experience. Firstly, the batch mode recognition is performed using LOOCV strategy. Figure 5a shows the recognition accuracy *w.r.t.* different *κ*. The accuracy of SemiTMC-GMM when *κ* = 9 is 95.16%, while the one of TMC-GMM is 92.57%. Meanwhile, the choice of *κ* has less impact on accuracy for SemiTMC-GMM. The recognition accuracy obtained by TMC-HIST is 80.42%, which is lower than the ones of TMC-GMM and SemiTMC-GMM when *κ* > 1. The ROC shown in Figure 5b implies that SemiTMC-GMM outperforms the TMC-GMM in a general view. Table 2 shows the sensitivity, specificity, F1 score, and MCC of each individual activity. By comparing the batch mode recognition shown in Tables 1 and 2, both TMC-GMM and SemiTMC-GMM outperform TMC-HIST. It means that considering the observation correlation improves the recognition performance.

**Figure 5.** The batch mode recognition performance of the Locomotion of Foot-mounted IMU (LMFIMU) dataset, SemiTMC-GMM, and TMC-GMM models, according to different GMM mixture number *κ*.

As a matter of fact, Figures 4a, 5a and 6a show that introducing semi-Markov structure into the TMC model can improve the accuracy. Meanwhile, using GMM with *κ* > 1 also improves the recognition significantly. But it does not mean that using a larger *κ* allows higher accuracy to be achieved. In Figure 4a, the accuracy when *κ* = 9 is slightly lower than that obtained when *κ* = 6, it is because the observation of SDA dataset is more closer to a GMM mixture of 6 densities. A too large value of *κ* may probably lead to an overfitting problem. It is sure that *κ* can be automatically acquired through the methods, such as BIC [34] and AIC [35], to make *κ* consistent with different activities. For simplification in this paper, we manually set *κ* to 6 for all the activities based on the experimental results.

Then, the on-line EM algorithm is performed to validate the adaptive on-line recognition performances. The proposed algorithm is implemented in MATLAB code, running on a 64-bit system computer with 3.2GHz CPU and 32G RAM. In the dataset, the average experiment time is 32.33 min, while the computing time of SemiTMC-GMM when *κ* = 1, 3, 6, 9 are 9.72, 14.72, 21.53, and 27.65 min, respectively. Thus, using on-line EM is applicable in on-line scenarios. The window length *Wl* for updating the parameters is set to 1000, which means that parameters are updated every 10 s.

**Table 2.** The sensitivity, specificity, F1 score, MCC value of the batch mode recognition, for each activity of LMFIMU dataset, using the sensor placed on right shoe. **Up**: TMC-HIST; **middle**: TMC-GMM when *κ* = 9; and **down**: SemiTMC-GMM when *κ* = 9.

**Figure 6.** The on-line mode recognition performance of the two experiment sections in LMFIMU dataset, according to different GMM mixture number *κ*.

Figure 6a shows the recognition accuracy obtained by LOOCV strategy. The solid lines are higher than the dashed lines which means that the on-line EM algorithm can improve the recognition performance. In addition, the GMM with *κ* > 1 can significantly improve the accuracy. When *κ* = 9, SemiTMC-GMM has an accuracy improved from 95.48% in the first section to 96.93% in the second section, TMC-GMM achieves an improvement from 93.83% to 95.04%. By contrast, in the adaptive on-line algorithm using TMC-HIST in our previous work, the accuracy was improved from 95.32% to 96.93%. However, this high accuracy is mainly because of the gait cycle complete detection in the adaptive on-line algorithm, which manually set the activity of all the samplings in one gait cycle to be identical. If the gait cycle complete detection is not used, TMC-HIST will fail in the on-line recognition, with the accuracies of 78.32% in the first section and 65.20% in the second section. Comparing SemiTMC (when *κ* = 1) and TMC-HIST, we can conclude that semi-Markov structure is more robust for recognizing the hidden states which have sojourn time. Figure 6b,c show the ROC of the two experiment sections, SemiTMC-GMM obtains better performance than TMC-GMM in both the two sections, and SemiTMC-GMM obtains a bigger improvement from the first to second section than the

other one. Therefore, the results indicate that both GMM density and semi-Markov structure improve the on-line recognition, and the combination the two improves the performance the most.

By comparing the two different sensor placements in SDA and LMFIMU dataset, the proposed algorithm shows that the sensor is not necessary to be placed at a specific place of the lower limb. In fact, the sensor can be placed in any position that implies the introduced gait phase and leg phase.

In order to understand the dynamic performance of the parameter updating, Figure 7 shows the recognition accuracy computed during the latest 10 s. Notice that the accuracies when *κ* = 1 are not displayed in Figure 7a,e because TMC obtains accuracies lower than 70% for D2A1 and D2A3. SemiTMC-GMM obtains a relatively fast convergence rate when *κ* equals to 6 and 9. The activities D2A1 and D2A2 reach high accuracy within 20 s in the first section of the experiment, 97.77% and 99.02%, respectively. By contrast, D2A3 and D2A4 are slower (take about 50 s) than the former two activities, and obtain lower accuracies of 92.04% and 89.48%, respectively. The main reason of this phenomenon is that the activity patterns of D2A3 and D2A4 vary much more differently among the subjects. But in a general view, we can still state that the on-line EM algorithm can dynamically improve the recognition accuracy to a reasonable level.

**Figure 7.** Recognition accuracy computed in the latest 10 s *w.r.t.* each activity of LMFIMU dataset. (**Left column**) TME-GMM; (**right column**) SemTMC-GMM.

Figure 8 displays the estimated gait cycles of each activity, when the model converged, obtained by TMC-GMM and SemiTMC-GMM, *κ* is set to 1 and 6. *ωx*, *ω<sup>y</sup>* and *ω<sup>z</sup>* are the sliding mean of angular rate along the three axes of sensor. The features are 12-dimensional, but here we only display the acceleration of along the three axes to show how the gaits proceed. Hence, the estimated gait cycles are displayed *w.r.t.* four models, i.e., TMC, SemiTMC, TMC-GMM, and SemiTMC-GMM. In fact, the gait phases or leg phases are introduced in the model to improve the recognition accuracy of the lower limb locomotion activity. The figure shows that SemiTMC-GMM obtains the most regular gait cycle, with no fluctuation in short period and no missing detection. As a consequence, the well estimated gait or leg cycle obtained from SemiTMC-GMM leads to a higher activity recognition performance.

**Figure 8.** Estimated gait cycle of each activity. The blue, cyan, black, and magenta represent the gait obtained by TMC, SemiTMC, TMC-GMM, and SemiTMC-GMM, respectively.

## **6. Conclusions**

In this paper, we proposed a wearable IMU-based algorithm for recognizing lower limb locomotion activities, with the help of introducing gait cycle or leg cycle into the model. The algorithm is based on the developed SemiTMC-GMM model, which better replicate the real motion. Our experiments show that semi-Markov structure and GMM density can better recover gait or leg cycles, which in return improve the activity recognition significantly. The adopted on-line EM algorithm can gradually improve the accuracy to a high level. The proposed algorithm is not only developed for the applications which require run-time activity recognition, but is also helpful to those applications that require gait cycles. For example, if using two sensors placed on both left and right legs of impaired people, then it is possible to develop a special SemiTMC-GMM model that detects their imbalanced gait phases. This can be beneficial for exoskeleton equipment to better assist impaired people in performing locomotion activities, by providing precise information to the equipment.

While there are still some limitations, the proposed algorithm only takes periodic lower limb locomotion into consideration; neither the static activity nor non-periodic lower limb locomotion activity is involved in our current work, such as standing and making turn. To distinguish static and motion activities, it is possible to include specific features into the observations. For example, standard deviation will be close to zero when a person is in static; otherwise, it will vary according to the motion. Distinguishing periodic and non-periodic can be accomplished by periodic pattern mining method, such as fast Fourier transform-based [36] and principle component analysis-based [37] approaches. Our future work will focus on adopting more types of activities, including static activity and non-periodic locomotion activities.

**Author Contributions:** Conceptualization, H.L.; Methodology, H.L., S.D., W.P.; Software, H.L.; Validation, H.L., S.D. and W.P.; Formal analysis, H.L., S.D.; Investigation, H.L., S.D.; Resources, S.D., W.P.; Data curation, H.L.; Writing—original draft preparation, H.L.; Writing—review and editing, S.D., W.P.; Visualization, H.L.; Supervision, S.D., W.P.; Project administration, S.D.

**Funding:** This research received no external funding.

**Acknowledgments:** We would like to thank the Chinese Scholarship Council (CSC) which partially supports this research.

**Conflicts of Interest:** The authors declare no conflict of interest.
