A Hand-Modeled Feature Extraction-Based Learning Network to Detect Grasps Using sEMG Signal
Abstract
:1. Introduction
- Shapes can be used to propose new local textural feature generators. Therefore, the frustum shape is used to present a new textural feature creation function, named the “frustum pattern”. By using the frustum pattern, a shape-related, graph-based local feature extraction methodology is investigated in this work.
- A new learning network called Frustum154 is presented in this paper. Frustum154 is a self-organized learning feature extraction method which uses two types of feature selection. In the feature generation/creation phase, the best features are chosen using a loss function. By using this function, Frustum154 automatically selects the best subbands for the problem.
- sEMG signal classification is an important signal processing topic for machine learning; deep learning models have been widely used to classify sEMG signals, achieving excellent accuracy. However, deep models are highly complex. Frustum154 is a hand-crafted, feature-based learning method which can choose the most appropriate model for signal classification problems.
- In order to demonstrate the universal classification ability of the suggested model, three sEMG signal datasets were used; the proposed model achieved over 94% classification for these datasets.
2. Material
2.1. Material
2.1.1. First sEMG Dataset
2.1.2. Second sEMG Dataset
2.1.3. Third sEMG Dataset
3. Frustum Pattern
4. The Proposed Learning Model: Frustum154
Algorithm 1: Pseudocode of the introduced Frustum154. |
Input: sEMG dataset |
Output: Results. |
00: Read each sEMG signal from datasets. |
01: Merge the channels |
02: Apply for TQWT decomposition to calculate 153 subbands. |
03: Extract statistical and texture features from sEMG signal and subbands. |
04: Obtain a 154-feature vector with a length of 414 (=384 + 30). |
05: Calculate the misclassification rate of each feature vector. |
06: Choose the best 20 feature vectors using the calculated misclassification rates. |
07: Merge the 20 feature vectors selected to create final feature vector. |
08: Choose the most informative feature by applying INCA. |
09: Classify the chosen features using kNN or SVM with 10-fold cross-validation. |
4.1. Feature Generation
4.2. Feature Selection
4.3. Classification
5. Experimental Protocol
5.1. Experimental Setup
5.2. Validation
5.3. Results
5.4. Time Complexity Analysis
6. Discussion
- The feature generation capabilities of the frustum pattern/graph were investigated and the sEMG classification was found to be highly successful;
- To maximize the effectiveness of TQWT, multiple parameter-based TQWT was used and 153 subbands were generated;
- An improved feature selector (INCA) was used;
- A novel hand-modeled learning method is proposed;
- The proposed Frustum154 achieved a higher classification rate than deep learning models (see Table 8);
- The present model showed good general classification success.
7. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Hargrove, L.J.; Englehart, K.; Hudgins, B. A comparison of surface and intramuscular myoelectric signal classification. IEEE Trans. Biomed. Eng. 2007, 54, 847–853. [Google Scholar] [CrossRef]
- Kamavuako, E.N.; Rosenvang, J.C.; Horup, R.; Jensen, W.; Farina, D.; Englehart, K.B. Surface versus untargeted intramuscular EMG based classification of simultaneous and dynamically changing movements. IEEE Trans. Neural Syst. Rehabil. Eng. 2013, 21, 992–998. [Google Scholar] [CrossRef] [PubMed]
- Kamavuako, E.N.; Scheme, E.J.; Englehart, K.B. Combined surface and intramuscular EMG for improved real-time myoelectric control performance. Biomed. Signal. Process. Control 2014, 10, 102–107. [Google Scholar] [CrossRef]
- Tuncer, T.; Dogan, S.; Subasi, A. Surface EMG signal classification using ternary pattern and discrete wavelet transform based feature extraction for hand movement recognition. Biomed. Signal. Process. Control 2020, 58, 101872. [Google Scholar] [CrossRef]
- Yavuz, E.; Eyupoglu, C. A cepstrum analysis-based classification method for hand movement surface EMG signals. Med. Biol. Eng. Comput. 2019, 57, 2179–2201. [Google Scholar] [CrossRef] [PubMed]
- Phinyomark, A.; Phukpattaranont, P.; Limsakul, C. Feature reduction and selection for EMG signal classification. Expert Syst. Appl. 2012, 39, 7420–7431. [Google Scholar] [CrossRef]
- Gokgoz, E.; Subasi, A. Comparison of decision tree algorithms for EMG signal classification using DWT. Biomed. Signal. Process. Control 2015, 18, 138–144. [Google Scholar] [CrossRef]
- Fang, Y.; Zhou, D.; Li, K.; Ju, Z.; Liu, H. Attribute-driven granular model for EMG-based pinch and fingertip force grand recognition. IEEE Trans. Cybern. 2019, 51, 789–800. [Google Scholar] [CrossRef] [Green Version]
- Chen, H.; Zhang, Y.; Li, G.; Fang, Y.; Liu, H. Surface electromyography feature extraction via convolutional neural network. Int. J. Mach. Learn. Cybern. 2020, 11, 185–196. [Google Scholar] [CrossRef]
- Fang, Y.; Zhang, X.; Zhou, D.; Liu, H. Improve inter-day hand gesture recognition via convolutional neural network based feature fusion. Int. J. Hum. Robot. 2020, 18, 2050025. [Google Scholar] [CrossRef]
- Mukhopadhyay, A.K.; Samui, S. An experimental study on upper limb position invariant EMG signal classification based on deep neural network. Biomed. Signal. Process. Control 2020, 55, 101669. [Google Scholar] [CrossRef]
- Karabulut, D.; Ortes, F.; Arslan, Y.Z.; Adli, M.A. Comparative evaluation of EMG signal features for myoelectric controlled human arm prosthetics. Biocybern. Biomed. Eng. 2017, 37, 326–335. [Google Scholar] [CrossRef]
- Arunraj, M.; Srinivasan, A.; Arjunan, S. A real-time capable linear time classifier scheme for anticipated hand movements recognition from Amputee subjects using surface EMG signals. IRBM 2020, 42, 277–293. [Google Scholar] [CrossRef]
- Hooda, N.; Das, R.; Kumar, N. Fusion of EEG and EMG signals for classification of unilateral foot movements. Biomed. Signal. Process. Control 2020, 60, 101990. [Google Scholar] [CrossRef]
- Bi, L.; Guan, C. A review on EMG-based motor intention prediction of continuous human upper limb motion for human-robot collaboration. Biomed. Signal. Process. Control 2019, 51, 113–127. [Google Scholar] [CrossRef]
- Rabin, N.; Kahlon, M.; Malayev, S.; Ratnovsky, A. Classification of human hand movements based on EMG signals using nonlinear dimensionality reduction and data fusion techniques. Expert Syst. Appl. 2020, 149, 113281. [Google Scholar] [CrossRef]
- Nishad, A.; Upadhyay, A.; Pachori, R.B.; Acharya, U.R. Automated classification of hand movements using tunable-Q wavelet transform based filter-bank with surface electromyogram signals. Future Gener. Comput. Syst. 2019, 93, 96–110. [Google Scholar] [CrossRef]
- Zhou, D.; Fang, Y.; Botzheim, J.; Kubota, N.; Liu, H. Bacterial memetic algorithm based feature selection for surface EMG based hand motion recognition in long-term use. In Proceedings of the 2016 IEEE Symposium Series on Computational Intelligence (SSCI), Athens, Greece, 6–9 December 2016; pp. 1–7. [Google Scholar]
- Coskun, M.; Yildirim, O.; Demir, Y.; Acharya, U.R. Efficient deep neural network model for classification of grasp types using sEMG signals. J. Ambient. Intell. Humaniz. Comput. 2021, 1–14. [Google Scholar] [CrossRef]
- Ouyang, G.; Zhu, X.; Ju, Z.; Liu, H. Dynamical characteristics of surface EMG signals of hand grasps via recurrence plot. IEEE J. Biomed. Health Inform. 2013, 18, 257–265. [Google Scholar] [CrossRef] [Green Version]
- Ma, R.; Zhang, L.; Li, G.; Jiang, D.; Xu, S.; Chen, D. Grasping force prediction based on sEMG signals. Alex. Eng. J. 2020, 59, 1135–1147. [Google Scholar] [CrossRef]
- Menon, R.; Di Caterina, G.; Lakany, H.; Petropoulakis, L.; Conway, B.A.; Soraghan, J.J. Study on interaction between temporal and spatial information in classification of EMG signals for myoelectric prostheses. IEEE Trans. Neural Syst. Rehabil. Eng. 2017, 25, 1832–1842. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Khushaba, R.N.; Takruri, M.; Miro, J.V.; Kodagoda, S. Towards limb position invariant myoelectric pattern recognition using time-dependent spectral features. Neural Netw. 2014, 55, 42–58. [Google Scholar] [CrossRef] [PubMed]
- Waris, A.; Niazi, I.K.; Jamil, M.; Englehart, K.; Jensen, W.; Kamavuako, E.N. Multiday evaluation of techniques for EMG-based classification of hand motions. IEEE J. Biomed. Health Inform. 2018, 23, 1526–1534. [Google Scholar] [CrossRef]
- Chada, S.; Taran, S.; Bajaj, V. An efficient approach for physical actions classification using surface EMG signals. Health Inf. Sci. Syst. 2020, 8, 3. [Google Scholar] [CrossRef]
- Wang, Y.; Wu, Q.; Dey, N.; Fong, S.; Ashour, A.S. Deep back propagation–long short-term memory network based upper-limb sEMG signal classification for automated rehabilitation. Biocybern. Biomed. Eng. 2020, 40, 987–1001. [Google Scholar] [CrossRef]
- Arteaga, M.V.; Castiblanco, J.C.; Mondragon, I.F.; Colorado, J.D.; Alvarado-Rojas, C. EMG-driven hand model based on the classification of individual finger movements. Biomed. Signal. Process. Control 2020, 58, 101834. [Google Scholar] [CrossRef]
- Pancholi, S.; Joshi, A.M. Electromyography-based hand gesture recognition system for upper limb amputees. IEEE Sens. Lett. 2019, 3, 1–4. [Google Scholar] [CrossRef]
- Jia, G.; Lam, H.-K.; Liao, J.; Wang, R. Classification of Electromyographic Hand Gesture Signals using Machine Learning Techniques. Neurocomputing 2020, 401, 236–248. [Google Scholar] [CrossRef]
- Simão, M.; Neto, P.; Gibaru, O. EMG-based online classification of gestures with recurrent neural networks. Pattern Recognit. Lett. 2019, 128, 45–51. [Google Scholar] [CrossRef]
- Simão, M.; Neto, P.; Gibaru, O. Uc2018 Dualmyo Hand Gesture Dataset. 2018. Available online: https://zenodo.org/record/1320922#.YhR8_ZYRVEY (accessed on 7 December 2021).
- Pizzolato, S.; Tagliapietra, L.; Cognolato, M.; Reggiani, M.; Müller, H.; Atzori, M. Comparison of six electromyography acquisition setups on hand movement classification tasks. PLoS ONE 2017, 12, e0186132. [Google Scholar] [CrossRef] [Green Version]
- Selesnick, I.W. Wavelet transform with tunable Q-factor. IEEE Trans. Signal Process. 2011, 59, 3560–3575. [Google Scholar] [CrossRef]
- Tuncer, T.; Dogan, S.; Özyurt, F.; Belhaouari, S.B.; Bensmail, H. Novel Multi Center and Threshold Ternary Pattern Based Method for Disease Detection Method Using Voice. IEEE Access 2020, 8, 84532–84540. [Google Scholar] [CrossRef]
- Vapnik, V. The support vector method of function estimation. In Nonlinear Modeling; Springer: Boston, MA, USA, 1998; pp. 55–85. [Google Scholar]
- Vapnik, V. The Nature of Statistical Learning Theory; Springer Science & Business Media: Berlin/Heidelberg, Germany, 2013. [Google Scholar]
- Maillo, J.; Ramírez, S.; Triguero, I.; Herrera, F. kNN-IS: An Iterative Spark-based design of the k-Nearest Neighbors classifier for big data. Knowl.-Based Syst. 2017, 117, 3–15. [Google Scholar] [CrossRef] [Green Version]
- Sapsanis, C.; Tzes, A.; Georgoulas, G. sEMG for Basic Hand Movements Data Set. UCI Machine Learning Repository 2014. Available online: https://archive.ics.uci.edu/ml/datasets/sEMG+for+Basic+Hand+movements (accessed on 7 December 2021).
- Sapsanis, C.; Georgoulas, G.; Tzes, A. EMG based classification of basic hand movements based on time-frequency features. In Proceedings of the 21st Mediterranean Conference on Control and Automation, Chania, Greece, 25–28 June 2013; pp. 716–722. [Google Scholar]
- Nishimura, Y.; Murakami, H. Initial shape-finding and modal analyses of cyclic frustum tensegrity modules. Comput. Methods Appl. Mech. Eng. 2001, 190, 5795–5818. [Google Scholar] [CrossRef]
- Subasi, A.; Qaisar, S.M. Surface EMG signal classification using TQWT, Bagging and Boosting for hand movement recognition. J. Ambient. Intell. Humaniz. Comput. 2020, 1–16. [Google Scholar] [CrossRef]
- Iqbal, O.; Fattah, S.A.; Zahin, S. Hand movement recognition based on singular value decomposition of surface EMG signal. In Proceedings of the Humanitarian Technology Conference (R10-HTC), 2017 IEEE Region 10, Dhaka, Bangladesh, 21–23 December 2017; pp. 837–842. [Google Scholar]
- Sapsanis, C.; Georgoulas, G.; Tzes, A.; Lymberopoulos, D. Improving EMG based classification of basic hand movements using EMD. In Proceedings of the 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Osaka, Japan, 3–7 July 2013; pp. 5754–5757. [Google Scholar]
- Tsinganos, P.; Cornelis, B.; Cornelis, J.; Jansen, B.; Skodras, A. Deep Learning in EMG-based Gesture Recognition. In Proceedings of the 5th International Conference on Physiological Computing System (PhyCS 2018), Seville, Spain, 19–20 September 2018; pp. 107–114. [Google Scholar]
Operation | Parameter | Output |
---|---|---|
Channel merging | Two channels | The used datasets comprise two-channeled sEMG signals. These channels are concatenated to use both channels. |
TQWT | Q = 1, 2, 3, 4 r = 2, 4, 6, 8 J = 6, 24, 46, 73 | 153 subbands |
Frustum pattern | Forty-nine overlapping blocks are used. The kernel function is ternary and the threshold value is chosen as half of the standard deviation of the signal | 154 feature vectors with a length of 384 |
Statistical feature extraction | We applied well-known statistical moments | 154 feature vectors with a length of 30 |
Feature merging | 154 feature vectors with a length of 414 | |
Normalization | Min-max normalization | 154 feature vectors are normalized |
Loss value generation | Cubic SVM and kNN (1NN with L1-norm) with 10-fold cross-validation. Herein, greedy model has been used. For DB1 (first database), kNN is the best classifier. For others (DB2 and DB3), the best loss value generator is Cubic SVM. | 154 loss values |
Top 20 feature vectors selection | Loss array | 20 feature vectors |
Feature merging | Concatenation function | Final feature vector with a length of 8280 |
INCA selector | Iteration range: [100, 512] Classifier: SVM or kNN (greedy search-based) | Length of the chosen feature vectors DB1: 279 DB2: 277 DB3: 295 |
Classification | kNN: k is 1, distance is Manhattan and voting is none. Validation is 10-fold CV. SVM: Kernel scale is auto, kernel is 3rd degree polynomial, C value is 1 and coding is one-vs-one. Validation is 10-fold CV. | Predicted values |
No. | Statistics | No. | Statistics | No. | Statistics |
---|---|---|---|---|---|
1 | Mean | 6 | Maximum | 11 | Kurtosis |
2 | Median | 7 | Minimum | 12 | Skewness |
3 | Variance | 8 | Standard deviation | 13 | Higuchi |
4 | Shannon entropy | 9 | Range | 14 | Energy |
5 | Log entropy | 10 | Sure entropy | 15 | Root mean square error |
True Label | Predicted Label | |||||
---|---|---|---|---|---|---|
C | H | L | P | S | T | |
C | 150 | 0 | 0 | 0 | 0 | 0 |
H | 0 | 149 | 0 | 1 | 0 | 0 |
L | 0 | 0 | 147 | 1 | 0 | 2 |
P | 0 | 0 | 1 | 149 | 0 | 0 |
S | 0 | 0 | 0 | 0 | 150 | 0 |
T | 1 | 1 | 1 | 2 | 0 | 145 |
Recall (%) | 100 | 99.33 | 98 | 99.33 | 100 | 96.67 |
Precision (%) | 99.34 | 99.33 | 98.66 | 97.39 | 100 | 98.64 |
F1 (%) | 99.67 | 99.33 | 98.33 | 98.35 | 100 | 97.64 |
True Label | Predicted Label | |||||
---|---|---|---|---|---|---|
C | H | L | P | S | T | |
C | 297 | 1 | 0 | 0 | 2 | 0 |
H | 3 | 291 | 0 | 2 | 0 | 4 |
L | 1 | 1 | 273 | 18 | 0 | 7 |
P | 0 | 2 | 19 | 273 | 0 | 6 |
S | 1 | 0 | 0 | 0 | 299 | 0 |
T | 0 | 1 | 11 | 12 | 0 | 276 |
Recall (%) | 99 | 97 | 91 | 91 | 99.67 | 92 |
Precision (%) | 98.34 | 98.31 | 90.10 | 89.51 | 99.34 | 94.20 |
F1 (%) | 98.67 | 97.65 | 90.55 | 90.25 | 99.50 | 93.09 |
True Label | Predicted Label | |||||
---|---|---|---|---|---|---|
C | H | L | P | S | T | |
C | 442 | 5 | 0 | 0 | 3 | 0 |
H | 4 | 431 | 0 | 7 | 2 | 6 |
L | 0 | 0 | 421 | 19 | 0 | 10 |
P | 0 | 2 | 14 | 425 | 0 | 9 |
S | 5 | 1 | 0 | 0 | 444 | 0 |
T | 2 | 6 | 18 | 14 | 0 | 410 |
Recall (%) | 98.22 | 95.78 | 93.56 | 94.44 | 98.67 | 91.11 |
Precision (%) | 97.57 | 96.85 | 92.94 | 91.40 | 98.89 | 94.25 |
F1 (%) | 97.90 | 96.31 | 93.24 | 92.90 | 98.78 | 92.66 |
Performance Metrics | DB1 | DB2 | DB3 |
---|---|---|---|
Accuracy | 98.89 | 94.94 | 95.30 |
Precision | 98.89 | 94.97 | 95.32 |
F1 | 98.89 | 94.95 | 95.30 |
Phase | Step | Training | Test |
---|---|---|---|
Feature extraction | Feature vectors creation using TQWT and frustum pattern | ||
Feature vector selection using loss values | |||
Feature concatenation | |||
Feature selection | INCA | ||
Classification | kNN/SVM | ||
Total |
Study | Method | Dataset | Accuracy (%) |
---|---|---|---|
Subasi and Qaisar [41] | Statistical feature extraction | DB1 | 94.11 |
Nishad et al. [17] | Statistical (entropy) feature extraction with TQWT decomposition | DB1 | 98.55 |
Iqbal et al. [42] | Singular value decomposition and principal component analysis (SVD+PCA) and kNN classifier | DB1 | 86.71 |
Sapsanis et al. [43] | Statistics and emprical mode decomposition (EMD) transformation | DB1 | 86.64 |
Coskun et al. [19] | One dimensional convulotional neural network (1D-CNN) | DB1 | 94.94 |
Tsinganos et al. [44] | Convolutional neural network | DB1 | 72.06 |
Rabin et al. [16] method | Short time Fourier transform-based feature generation and principle component analysis/diffusion map-based feature reduction + kNN | DB1 | 76.4 |
Frustum154 | DB1 | 98.89 | |
DB2 | 94.94 | ||
DB3 | 95.30 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Baygin, M.; Barua, P.D.; Dogan, S.; Tuncer, T.; Key, S.; Acharya, U.R.; Cheong, K.H. A Hand-Modeled Feature Extraction-Based Learning Network to Detect Grasps Using sEMG Signal. Sensors 2022, 22, 2007. https://doi.org/10.3390/s22052007
Baygin M, Barua PD, Dogan S, Tuncer T, Key S, Acharya UR, Cheong KH. A Hand-Modeled Feature Extraction-Based Learning Network to Detect Grasps Using sEMG Signal. Sensors. 2022; 22(5):2007. https://doi.org/10.3390/s22052007
Chicago/Turabian StyleBaygin, Mehmet, Prabal Datta Barua, Sengul Dogan, Turker Tuncer, Sefa Key, U. Rajendra Acharya, and Kang Hao Cheong. 2022. "A Hand-Modeled Feature Extraction-Based Learning Network to Detect Grasps Using sEMG Signal" Sensors 22, no. 5: 2007. https://doi.org/10.3390/s22052007
APA StyleBaygin, M., Barua, P. D., Dogan, S., Tuncer, T., Key, S., Acharya, U. R., & Cheong, K. H. (2022). A Hand-Modeled Feature Extraction-Based Learning Network to Detect Grasps Using sEMG Signal. Sensors, 22(5), 2007. https://doi.org/10.3390/s22052007