sEMG-Based Gesture Recognition with Convolution Neural Networks
Abstract
:1. Introduction
2. The Proposed sEMG-Based Gesture Recognition
2.1. Database
2.2. Data Analysis and Processing
2.2.1. Windowing
2.2.2. Feature Extraction and Classification
3. Experiments and Results
4. Conclusions
Author Contributions
Conflicts of Interest
References
- Graupe, D.; Salahi, J.; Kohn, K.H. Multifunctional prosthesis and orthosis control via microcomputer identification of temporal pattern differences in single-site myoelectric signals. J. Biomed. Eng. 1982, 4, 17–22. [Google Scholar] [CrossRef]
- Park, G.; Kim, H. Low-Cost Implementation of a Named Entity Recognition System for Voice-Activated Human-Appliance Interfaces in a Smart Home. Sustainability 2018, 10, 488. [Google Scholar] [CrossRef]
- Rho, S.; Yeo, S.-S. Bridging the semantic gap in multimedia emotion/mood recognition for ubiquitous computing environment. J. Supercomput. 2013, 65, 274–286. [Google Scholar] [CrossRef]
- Electromyogram Pattern Recognition for Control of Powered Upper-Limb Prostheses: State of the Art and Challenges for Clinical Use–ProQuest. Available online: https://search.proquest.com/openview/c52c612e950984f56fb0d21d8aa23b11/1?pq-origsite=gscholar&cbl=48772 (accessed on 12 January 2018).
- Rho, S.; Chen, Y. Social Internet of Things: Applications, architectures and protocols. Future Gener. Comput. Syst. 2018, 82, 667–668. [Google Scholar] [CrossRef]
- Chen, B.W.; Ji, W. Intelligent Marketing in Smart Cities: Crowdsourced Data for Geo-Conquesting. IT Prof. 2016, 18, 18–24. [Google Scholar] [CrossRef]
- Cyber Physical Systems Technologies and Applications–Science Direct. Available online: https://www.sciencedirect.com/science/article/pii/S0167739X15003325 (accessed on 19 March 2018).
- Clustering Algorithm for Internet of Vehicles (IoV) Based on Dragonfly Optimizer (CAVDO)|Springer Link. Available online: https://link.springer.com/article/10.1007/s11227-018-2305-x (accessed on 19 March 2018).
- Naik, G.R.; Nguyen, H.T. Nonnegative Matrix Factorization for the Identification of EMG Finger Movements: Evaluation Using Matrix Analysis. IEEE J. Biomed. Health Inform. 2015, 19, 478–485. [Google Scholar] [CrossRef] [PubMed]
- Naik, G.R.; Kumar, D.K.; Palaniswami, M. Signal processing evaluation of myoelectric sensor placement in low-level gestures: Sensitivity analysis using independent component analysis. Expert Syst. 2014, 31, 91–99. [Google Scholar] [CrossRef]
- Naik, G.R.; Kumar, D.K.; Weghorn, H.; Palaniswami, M. Subtle Hand Gesture Identification for HCI Using Temporal Decorrelation Source Separation BSS of Surface EMG. In Proceedings of the 9th Biennial Conference of the Australian Pattern Recognition Society on Digital Image Computing Techniques and Applications (DICTA 2007), Glenelg, Australia, 3–5 December 2007; pp. 30–37. [Google Scholar]
- Naik, G.R.; Acharyya, A.; Nguyen, H.T. Classification of finger extension and flexion of EMG and Cyberglove data with modified ICA weight matrix. In Proceedings of the 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Chicago, IL, USA, 26–30 August 2014; pp. 3829–3832. [Google Scholar]
- Englehart, K.; Hudgins, B. A robust, real-time control scheme for multifunction myoelectric control. IEEE Trans. Biomed. Eng. 2003, 50, 848–854. [Google Scholar] [CrossRef] [PubMed]
- Hudgins, B.; Parker, P.; Scott, R.N. A new strategy for multifunction myoelectric control. IEEE Trans. Biomed. Eng. 1993, 40, 82–94. [Google Scholar] [CrossRef] [PubMed]
- Zardoshti-Kermani, M.; Wheeler, B.C.; Badie, K.; Hashemi, R.M. EMG feature evaluation for movement control of upper extremity prostheses. IEEE Trans. Rehabil. Eng. 1995, 3, 324–333. [Google Scholar] [CrossRef]
- Gu, Y.; Yang, D.; Huang, Q.; Yang, W.; Liu, H. Robust EMG pattern recognition in the presence of confounding factors: Features, classifiers and adaptive learning. Expert Syst. Appl. 2018, 96, 208–217. [Google Scholar] [CrossRef]
- Englehart, K.; Hudgin, B.; Parker, P.A. A wavelet-based continuous classification scheme for multifunction myoelectric control. IEEE Trans. Biomed. Eng. 2001, 48, 302–311. [Google Scholar] [CrossRef] [PubMed]
- Lucas, M.-F.; Gaufriau, A.; Pascual, S.; Doncarli, C.; Farina, D. Multi-channel surface EMG classification using support vector machines and signal-based wavelet optimization. Biomed. Signal Process. Control 2008, 3, 169–174. [Google Scholar] [CrossRef]
- Kakoty, N.M.; Hazarika, S.M.; Gan, J.Q. EMG Feature Set Selection through Linear Relationship for Grasp Recognition. J. Med. Biol. Eng. 2016, 36, 883–890. [Google Scholar] [CrossRef]
- Chowdhury, R.; Reaz, M.; Ali, M.; Bakar, A.; Chellappan, K.; Chang, T. Surface Electromyography Signal Processing and Classification Techniques. Sensors 2013, 13, 12431–12466. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Shin, S.; Tafreshi, R.; Langari, R. A performance comparison of hand motion EMG classification. In Proceedings of the 2nd Middle East Conference on Biomedical Engineering, Doha, Qatar, 17–20 February 2014; pp. 353–356. [Google Scholar]
- Geethanjali, P.; Ray, K.K.; Shanmuganathan, P.V. Actuation of prosthetic drive using EMG signal. In Proceedings of the TENCON 2009 IEEE Region 10 Conference, Singapore, 23–26 November 2009; pp. 1–5. [Google Scholar]
- Zhang, Y.; Na, S.; Niu, J.; Jiang, B. The Influencing Factors, Regional Difference and Temporal Variation of Industrial Technology Innovation: Evidence with the FOA-GRNN Model. Sustainability 2018, 10, 187. [Google Scholar] [CrossRef]
- Chen, B.-W.; Abdullah, N.N.B.; Park, S.; Gu, Y. Efficient multiple incremental computation for Kernel Ridge Regression with Bayesian uncertainty modeling. Future Gener. Comput. Syst. 2018, 82, 679–688. [Google Scholar] [CrossRef] [Green Version]
- Naik, G.R.; Al-Timemy, A.H.; Nguyen, H.T. Transradial Amputee Gesture Classification Using an Optimal Number of sEMG Sensors: An Approach Using ICA Clustering. IEEE Trans. Neural Syst. Rehabil. Eng. 2016, 24, 837–846. [Google Scholar] [CrossRef] [PubMed]
- Liu, Y.-H.; Huang, H.-P.; Weng, C.-H. Recognition of electromyographic signals using cascaded kernel learning machine. IEEE ASME Trans. Mechatron. 2007, 12, 253–264. [Google Scholar] [CrossRef]
- Chen, B.W.; Chen, C.Y.; Wang, J.F. Smart Homecare Surveillance System: Behavior Identification Based on State-Transition Support Vector Machines and Sound Directivity Pattern Analysis. IEEE Trans. Syst. Man Cybern. Syst. 2013, 43, 1279–1289. [Google Scholar] [CrossRef]
- Liarokapis, M.V.; Artemiadis, P.K.; Katsiaris, P.T.; Kyriakopoulos, K.J.; Manolakos, E.S. Learning human reach-to-grasp strategies: Towards EMG-based control of robotic arm-hand systems. In Proceedings of the 2012 IEEE International Conference on Robotics and Automation, Saint Paul, MN, USA, 14–18 May 2012; pp. 2287–2292. [Google Scholar]
- Yao, D.; Yang, J.; Zhan, X. A Novel Method for Disease Prediction: Hybrid of Random Forest and Multivariate Adaptive Regression Splines. J. Comput. 2013, 8. [Google Scholar] [CrossRef]
- Gokgoz, E.; Subasi, A. Comparison of decision tree algorithms for EMG signal classification using DWT. Biomed. Signal Process. Control 2015, 18, 138–144. [Google Scholar] [CrossRef]
- Robinson, C.P.; Li, B.; Meng, Q.; Pain, M.T.G. Pattern Classification of Hand Movements Using Time Domain Features of Electromyography; ACM Press: New York, NY, USA, 2017; pp. 1–6. [Google Scholar]
- Al-Ani, A.; Koprinska, I.; Naik, G.R.; Khushaba, R.N. A dynamic channel selection algorithm for the classification of EEG and EMG data. In Proceedings of the 2016 International Joint Conference on Neural Networks (IJCNN), Vancouver, BC, Canada, 24–29 July 2016; pp. 4076–4081. [Google Scholar]
- Guo, Y.; Naik, G.R.; Huang, S.; Abraham, A.; Nguyen, H.T. Nonlinear multiscale Maximal Lyapunov Exponent for accurate myoelectric signal classification. Appl. Soft Comput. 2015, 36, 633–640. [Google Scholar] [CrossRef]
- Chen, B.W.; Wang, J.C.; Wang, J.F. A Novel Video Summarization Based on Mining the Story-Structure and Semantic Relations among Concept Entities. IEEE Trans. Multimedia 2009, 11, 295–312. [Google Scholar] [CrossRef]
- Atzori, M.; Cognolato, M.; Müller, H. Deep Learning with Convolutional Neural Networks Applied to Electromyography Data: A Resource for the Classification of Movements for Prosthetic Hands. Front. Neurorobot. 2016, 10. [Google Scholar] [CrossRef] [PubMed]
- Geng, W.; Du, Y.; Jin, W.; Wei, W.; Hu, Y.; Li, J. Gesture recognition by instantaneous surface EMG images. Sci. Rep. 2016, 6. [Google Scholar] [CrossRef] [PubMed]
- Du, Y.; Jin, W.; Wei, W.; Hu, Y.; Geng, W. Surface EMG-Based Inter-Session Gesture Recognition Enhanced by Deep Domain Adaptation. Sensors 2017, 17, 458. [Google Scholar] [CrossRef] [PubMed]
- Côté-Allard, U.; Fall, C.L.; Campeau-Lecours, A.; Gosselin, C.; Laviolette, F.; Gosselin, B. Transfer Learning for sEMG Hand Gestures Recognition Using Convolutional Neural Networks. In Proceedings of the 2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC), Banff, AB, Canada, 5–8 October 2017. [Google Scholar]
- Zhai, X.; Jelfs, B.; Chan, R.H.M.; Tin, C. Self-Recalibrating Surface EMG Pattern Recognition for Neuroprosthesis Control Based on Convolutional Neural Network. Front. Neurosci. 2017, 11. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Atzori, M.; Gijsberts, A.; Castellini, C.; Caputo, B.; Hager, A.-G.M.; Elsig, S.; Giatsidis, G.; Bassetto, F.; Müller, H. Electromyography data for non-invasive naturally-controlled robotic hand prostheses. Sci. Data 2014, 1, 140053. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Gijsberts, A.; Atzori, M.; Castellini, C.; Muller, H.; Caputo, B. Movement Error Rate for Evaluation of Machine Learning Methods for sEMG-Based Hand Movement Classification. IEEE Trans. Neural Syst. Rehabil. Eng. 2014, 22, 735–744. [Google Scholar] [CrossRef] [PubMed]
- Smith, L.H.; Hargrove, L.J.; Lock, B.A.; Kuiken, T.A. Determining the Optimal Window Length for Pattern Recognition-Based Myoelectric Control: Balancing the Competing Effects of Classification Error and Controller Delay. IEEE Trans. Neural Syst. Rehabil. Eng. 2011, 19, 186–192. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Kuzborskij, I.; Gijsberts, A.; Caputo, B. On the challenge of classifying 52 hand movements from surface electromyography. In Proceedings of the 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), San Diego, CA, USA, 28 August–1 September 2012; pp. 4931–4937. [Google Scholar]
- Menon, R.; Caterina, G.D.; Lakany, H.; Petropoulakis, L.; Conway, B.A.; Soraghan, J.J. Study on Interaction Between Temporal and Spatial Information in Classification of EMG Signals for Myoelectric Prostheses. IEEE Trans. Neural Syst. Rehabil. Eng. 2017, 25, 1832–1842. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Fougner, A.; Scheme, E.; Chan, A.D.; Englehart, K.; Stavdahl, Ø. A multi-modal approach for hand motion classification using surface EMG and accelerometers. In Proceedings of the 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC, Boston, MA, USA, 30 August–3 September 2011; pp. 4247–4250. [Google Scholar]
- Gijsberts, A.; Caputo, B. Exploiting accelerometers to improve movement classification for prosthetics. In Proceedings of the 2013 IEEE International Conference on Rehabilitation Robotics (ICORR), Seattle, WA, USA, 24–26 June 2013; pp. 1–5. [Google Scholar]
- Xia, P.; Hu, J.; Peng, Y. EMG-Based Estimation of Limb Movement Using Deep Learning with Recurrent Convolutional Neural Networks: Emg-Based Estimation of Limb Movement. Artif. Organs 2017. [Google Scholar] [CrossRef] [PubMed]
Feature | Parameter |
---|---|
Histogram (HIST) | 10 bins along with threshold |
marginal Discrete Wavelet Transform (mDWT) | db7 wavelet, 3 level |
Method | Classification Accuracy |
---|---|
C-B1PB2 | |
CC | |
C-2B1 | |
C-2B2 | |
C-DK | |
C-SK | |
C-SK2 | |
C-B1PB2 (all gestures of DB2) |
© 2018 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Ding, Z.; Yang, C.; Tian, Z.; Yi, C.; Fu, Y.; Jiang, F. sEMG-Based Gesture Recognition with Convolution Neural Networks. Sustainability 2018, 10, 1865. https://doi.org/10.3390/su10061865
Ding Z, Yang C, Tian Z, Yi C, Fu Y, Jiang F. sEMG-Based Gesture Recognition with Convolution Neural Networks. Sustainability. 2018; 10(6):1865. https://doi.org/10.3390/su10061865
Chicago/Turabian StyleDing, Zhen, Chifu Yang, Zhihong Tian, Chunzhi Yi, Yunsheng Fu, and Feng Jiang. 2018. "sEMG-Based Gesture Recognition with Convolution Neural Networks" Sustainability 10, no. 6: 1865. https://doi.org/10.3390/su10061865