Advanced Hand Gesture Prediction Robust to Electrode Shift with an Arbitrary Angle
Abstract
:1. Introduction
- the proposed model enables advanced predictive capability via an improved artificial neural network (ANN) substantially, which could output predicted results in 338 ms from hand gestures start, with above 94% accuracy;
- the developed method allows electrode displacement detection at random angles rather than conventional fixed coarse resolution, capable of satisfying actual requirements;
- this system can make simplified rapid correction according to electrode shift.
2. Materials and Methods
2.1. Electrode Registration and Data Acquisition
2.1.1. Standard Configuration and Initial Position Designation
2.1.2. Employed and Synchronous Gesture Definition
2.1.3. Training and Testing Dataset Organization
2.2. Electrode Shift Detection Based on IPL and Synchronous Gesture
2.3. Data Rearrangement
2.4. Training Set Selection
2.5. ANN-Based Hand Gesture Prediction
Algorithm 1. ANN-based prediction model. | |
Input: | |
Output: | |
1: | initialize s; // where s denotes stride length |
2: | initialize ; // where denotes time-domain features bag function; |
3: | training process: extract features from using to form |
// where Fi and Li denote feature vector and corresponding label vector, respectively | |
4: | compute weight decay && employ regularization to |
5: | apply to classifier, and form classifier.predict |
6: | for each windowed , feed to classifier for predicting gesture code |
7: | count number of different generated code to form n |
8: | if n == preset threshold of outputting gesture, then |
get the predicted gesture code, else return None |
3. Experimental Results
3.1. Prediction Accuracy with Electrode Shift Correction
3.2. Accuracy Improvement on Electrode Shift
3.3. Synchronous Gesture Selection Varies Accuracies
4. Discussion
4.1. Governing Parameters Varies Accuracies
4.2. Performance of Electrode Shift Correction
4.3. Performance with Gesture Prediction
5. Conclusions
Author Contributions
Funding
Conflicts of Interest
References
- Li, K.; Cheng, J.; Zhang, Q.; Liu, J. Hand gesture tracking and recognition based human-computer interaction system and its applications. In Proceedings of the IEEE ICIA, Wuyishan, China, 11–13 August 2018; pp. 667–672. [Google Scholar]
- Kaur, H.; Rani, J. A review: Study of various techniques of hand gesture recognition. In Proceedings of the IEEE ICPEICES, Delhi, India, 4–6 July 2016. [Google Scholar]
- Sayin, F.S.; Ozen, S.; Baspinar, U. Hand gesture recognition by using semg signals for human machine interaction applications. In Proceedings of the SPA, Poznan, Poland, 19–21 September 2018; pp. 27–30. [Google Scholar]
- Shin, S.; Tafreshi, R.; Langari, R. Emg and imu based real-time hci using dynamic hand gestures for a multiple-dof robot arm. J. Intell. Fuzzy Syst. 2018, 35, 861–876. [Google Scholar] [CrossRef]
- Al-Shamayleh, A.S.; Ahmad, R.; Abushariah, M.A.M.; Alam, K.A.; Jomhari, N. A systematic literature review on vision based gesture recognition techniques. Multimed. Tools Appl. 2018, 77, 28121–28184. [Google Scholar] [CrossRef]
- Cheok, M.J.; Omar, Z.; Jaward, M.H. A review of hand gesture and sign language recognition techniques. Int. J. Mach. Learn. Cybern. 2019, 10, 131–153. [Google Scholar] [CrossRef]
- Leone, F.; Gentile, C.; Ciancio, A.L.; Gruppioni, E.; Davalli, A.; Sacchetti, R.; Guglielmelli, E.; Zollo, L. Simultaneous semg classification of hand/wrist gestures and forces. Front. Neurorobotics 2019, 13, 42. [Google Scholar] [CrossRef] [PubMed]
- Li, Q.X.; Chan, P.P.K.; Zhou, D.; Fang, Y.; Liu, H.; Yeung, D.S. Improving robustness against electrode shift of semg based hand gesture recognition using online semi-supervised learning. In Proceedings of the ICMLC, Jeju, Korea, 10–13 July 2016; pp. 344–349. [Google Scholar]
- Motoche, C.; Benalcázar, M.E. In Real-time hand gesture recognition based on electromyographic signals and artificial neural networks. In Proceedings of the ICANN, Rhodes, Greece, 4–7 October 2018; pp. 352–361. [Google Scholar]
- Vimos, V.H.; Benalcázar, M.; Oña, A.F.; Cruz, P.J. A novel technique for improving the robustness to sensor rotation in hand gesture recognition using semg. In Proceedings of the CSEI, Ambato, Ecuador, 24 October 2019; pp. 226–243. [Google Scholar]
- Kim, J.; Mastnik, S.; André, E. Emg-based hand gesture recognition for realtime biosignal interfacing. In Proceedings of the IUI, Gran Canaria, Spain, 13–16 January 2008; pp. 30–39. [Google Scholar]
- Zhang, Z.; Yang, K.; Qian, J.; Zhang, L. Real-time surface emg pattern recognition for hand gestures based on an artificial neural network. Sensors 2019, 19, 3170. [Google Scholar] [CrossRef] [Green Version]
- Young, A.J.; Hargrove, L.J.; Kuiken, T.A. Improving myoelectric pattern recognition robustness to electrode shift by changing interelectrode distance and electrode configuration. IEEE Trans. Biomed. Eng. 2012, 59, 645–652. [Google Scholar] [CrossRef] [Green Version]
- Zhang, Y.; Chen, Y.; Yu, H.; Yang, X.; Lu, W.; Liu, H. Wearing-independent hand gesture recognition method based on emg armband. Pers. Ubiquitous Comput. 2018, 22, 511–524. [Google Scholar] [CrossRef]
- Li, Z.; Wang, F.; Zhao, X.; Ding, Q.; Zhang, D.; Han, J. The method for gestures recognition based on myo rotation shifts estimation and adaptive correction. Acta Automatica Sinica 2019, 1–11. [Google Scholar]
- Steinhardt, C.R.; Bettthauser, J.; Hunt, C.; Thakor, N. Registration of emg electrodes to reduce classification errors due to electrode shift. In Proceedings of the IEEE BioCAS, Cleveland, OH, USA, 17–19 October 2018; pp. 1–4. [Google Scholar]
- Farina, D.; Sartori, M. Surface electromyography for man-machine interfacing in rehabilitation technologies. In Surface Electromyography: Physiology, Engineering, and Applications; IEEE Press: Piscataway, NJ, USA, 2016; pp. 540–560. [Google Scholar]
- Pancholi, S.; Joshi, A.M. Electromyography-based hand gesture recognition system for upper limb amputees. IEEE Sens. Lett. 2019, 3, 1–4. [Google Scholar] [CrossRef]
- Javaid, H.A.; Rashid, N.; Tiwana, M.I.; Anwar, M.W. Comparative analysis of emg signal features in time-domain and frequency-domain using myo gesture control. In Proceedings of the ICMRE, Valenciennes, France, 7–11 February 2018; pp. 157–162. [Google Scholar]
- Chen, H.; Tong, R.; Chen, M.; Fang, Y.; Liu, H. A hybrid cnn-svm classifier for hand gesture recognition with surface emg signals. In Proceedings of the ICMLC, Chengdu, China, 15–18 July 2018; pp. 619–624. [Google Scholar]
- Qi, J.; Jiang, G.; Li, G.; Sun, Y.; Tao, B. Surface emg hand gesture recognition system based on pca and grnn. Neural Comput. Appl. 2019. [Google Scholar] [CrossRef]
- Qi, J.; Jiang, G.; Li, G.; Sun, Y.; Tao, B. Intelligent human-computer interaction based on surface emg gesture recognition. IEEE Access 2019, 7, 61378–61387. [Google Scholar] [CrossRef]
- Pinzón-Arenas, J.O.; Jiménez-Moreno, R.; Herrera-Benavides, J.E. Convolutional neural network for hand gesture recognition using 8 different emg signals. In Proceedings of the STSIVA, Bucaramanga, Colombia, 24–26 April 2019; pp. 1–5. [Google Scholar]
- Neacsu, A.A.; Cioroiu, G.; Radoi, A.; Burileanu, C. Automatic emg-based hand gesture recognition system using time-domain descriptors and fully-connected neural networks. In Proceedings of the TSP, Budapest, Hungary, 1–3 July 2019; pp. 232–235. [Google Scholar]
- Lu, Z.; Chen, X.; Zhang, X.; Tong, K.-Y.; Zhou, P. Real-time control of an exoskeleton hand robot with myoelectric pattern recognition. Int. J. Neural Syst. 2017, 27, 1750009. [Google Scholar] [CrossRef] [PubMed]
- Crepin, R.; Fall, C.L.; Mascret, Q.; Gosselin, C.; Campeau-Lecours, A.; Gosselin, B. Real-time hand motion recognition using semg patterns classification. In Proceedings of the IEEE EMBC, Honolulu, HI, USA, 18–21 July 2018; pp. 2655–2658. [Google Scholar]
- Farina, D.; Merletti, R. Comparison of algorithms for estimation of emg variables during voluntary isometric contractions. J. Electromyogr. Kinesiol. 2000, 10, 337–349. [Google Scholar] [CrossRef]
- Han, Y.; Zhao, J. Accurate substrate analysis based on a novel finite difference method via synchronization method on layered and adaptive meshing. IEEE Trans. Comput-Aided Des. Integr. Circuits Syst. 2013, 32, 1520–1532. [Google Scholar] [CrossRef]
- Benalcázar, M.E.; Anchundia, C.E.; Zea, J.A.; Zambrano, P.; Jaramillo, A.G.; Segura, M. Real-time hand gesture recognition based on artificial feed-forward neural networks and emg. In Proceedings of the EUSIPCO, Rome, Italy, 3–7 September 2018; pp. 1492–1496. [Google Scholar]
- Ruta, D.; Gabrys, B. Classifier selection for majority voting. Inf. Fusion 2005, 6, 63–81. [Google Scholar] [CrossRef]
- Mizuno, H.; Tsujiuchi, N.; Koizumi, T. Forearm motion discrimination technique using real-time emg signals. In Proceedings of the EMBC, Boston, MA, USA, 30 August–3 September 2011; pp. 4435–4438. [Google Scholar]
- Boschmann, A.; Platzner, M. Reducing classification accuracy degradation of pattern recognition based myoelectric control caused by electrode shift using a high density electrode array. In Proceedings of the EMBC, San Diego, CA, USA, 28 August–1 September 2012; pp. 4324–4327. [Google Scholar]
- Lv, B.; Sheng, X.; Zhu, X. Improving myoelectric pattern recognition robustness to electrode shift by autoencoder. In Proceedings of the EMBC, Honolulu, HI, USA, 18–21 July 2018; pp. 5652–5655. [Google Scholar]
- Fan, Z.; Wang, Z.; Li, G.; Wang, R. A canonical correlation analysis based emg classification algorithm for eliminating electrode shift effect. In Proceedings of the EMBC, Orlando, FL, USA, 16–20 August 2016; pp. 867–870. [Google Scholar]
- Yang, W.; Yang, D.; Li, J.; Liu, Y.; Liu, H. Emg dataset augmentation approaches for improving the multi-dof wrist movement regression accuracy and robustness. In Proceedings of the IEEE ROBIO, Kuala Lumpur, Malaysia, 12–15 December 2018; pp. 1268–1273. [Google Scholar]
Electrode Position | sEMG Rearrangement |
---|---|
2 → 4 | I(2) = I’(4) |
3 →5 | I(3) = I’(5) |
4 → 6 | I(4) = I’(6) |
5 → 7 | I(5) = I’(7) |
6 → 8 | I(6) = I’(8) |
7 → 1 | I(7) = I’(1) |
8 → 2 | I(8) = I’(2) |
1 → 3 | I(1) = I’(3) |
M1 | M2 | M3 | C1-3 | C2-3 | |
---|---|---|---|---|---|
SUB #01 | 73.7 | 55.3 | 94.0 | 20.3 | 38.7 |
SUB #02 | 69.8 | 78.3 | 94.0 | 24.2 | 15.7 |
SUB #03 | 79.3 | 72.0 | 91.3 | 12.0 | 19.3 |
SUB #04 | 21.3 | 60.0 | 93.0 | 71.7 | 33.0 |
SUB #05 | 60.8 | 59.5 | 96.2 | 35.4 | 36.7 |
SUB #06 | 28.2 | 89.0 | 97.5 | 69.3 | 8.5 |
SUB #07 | 48.5 | 63.2 | 90.7 | 42.2 | 27.5 |
SUB #08 | 34.2 | 81.3 | 95.7 | 61.5 | 14.4 |
SUB #09 | 36.5 | 78.2 | 96.8 | 60.3 | 18.6 |
SUB #10 | 61.5 | 83.5 | 98.2 | 36.7 | 14.7 |
OVERALL | 51.4 | 72.0 | 94.7 | 43.3 | 22.7 |
S1 | S2 | C1-3 | |
---|---|---|---|
SUB #01 | 42.7 | 94.0 | 51.3 |
SUB #02 | 61.7 | 94.0 | 32.3 |
SUB #03 | 80.5 | 91.3 | 10.8 |
SUB #04 | 29.7 | 93.0 | 63.3 |
SUB #05 | 33.0 | 96.2 | 63.2 |
SUB #06 | 29.8 | 97.5 | 67.7 |
SUB #07 | 71.3 | 90.7 | 19.4 |
SUB #08 | 84.7 | 95.7 | 11.0 |
SUB #09 | 72.5 | 96.8 | 24.3 |
SUB #10 | 98.7 | 98.2 | −0.5 |
OVERALL | 60.5 | 94.7 | 34.2 |
Task | Work | Electrode | Channel | Classifier | Gesture | Response Time | Resolution of Shift Correction | Accuracy |
---|---|---|---|---|---|---|---|---|
Advanced prediction | OUR | dry | 8 | ANN | 6* | 338ms (< GD) | arbitrary angle | 94.7% |
Traditional recognition | Li et al. [15] | dry | 8 | SVM | 8* | GD+ | fixed (45°) | 78.4% |
Vimos et al. [10] | dry | 8 | SVM | 6* | GD+ | fixed (45°) | 92.4% | |
Steinhardt et al. [16] | dry | 8 | SRC | 6* | GD+ | fixed (22.5°) | 95.7% | |
Zhang et al. [14] | dry | 8 | RF | 15 | GD+ | fixed (45°) | 91.5% | |
Lv et al. [33] | SA | 192 | SAE | 10* | GD+ | fixed (1-cm shift) | 85.0% | |
Fan et al. [34] | SA | 30 | LDA | 11* | GD+ | fixed (1-cm shift) | 88.2% | |
Yang et al. [35] | dry | 8 | CNN | 10* | GD+ | fixed (45°) | 63.2% |
© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Xu, Z.; Shen, L.; Qian, J.; Zhang, Z. Advanced Hand Gesture Prediction Robust to Electrode Shift with an Arbitrary Angle. Sensors 2020, 20, 1113. https://doi.org/10.3390/s20041113
Xu Z, Shen L, Qian J, Zhang Z. Advanced Hand Gesture Prediction Robust to Electrode Shift with an Arbitrary Angle. Sensors. 2020; 20(4):1113. https://doi.org/10.3390/s20041113
Chicago/Turabian StyleXu, Zhenjin, Linyong Shen, Jinwu Qian, and Zhen Zhang. 2020. "Advanced Hand Gesture Prediction Robust to Electrode Shift with an Arbitrary Angle" Sensors 20, no. 4: 1113. https://doi.org/10.3390/s20041113
APA StyleXu, Z., Shen, L., Qian, J., & Zhang, Z. (2020). Advanced Hand Gesture Prediction Robust to Electrode Shift with an Arbitrary Angle. Sensors, 20(4), 1113. https://doi.org/10.3390/s20041113