Next Article in Journal
Scented Solutions: Examining the Efficacy of Scent Interventions in Mitigating Driving Fatigue
Previous Article in Journal
The Prediction of the Remaining Useful Life of Rotating Machinery Based on an Adaptive Maximum Second-Order Cyclostationarity Blind Deconvolution and a Convolutional LSTM Autoencoder
Previous Article in Special Issue
Design, Fabrication and Evaluation of a Stretchable High-Density Electromyography Array
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Evaluation of Hand Action Classification Performance Using Machine Learning Based on Signals from Two sEMG Electrodes

School of Engineering, Faculty of Engineering and Physical Sciences, University of Southampton, Southampton SO17 1BJ, UK
*
Authors to whom correspondence should be addressed.
Sensors 2024, 24(8), 2383; https://doi.org/10.3390/s24082383
Submission received: 25 January 2024 / Revised: 21 March 2024 / Accepted: 30 March 2024 / Published: 9 April 2024
(This article belongs to the Special Issue EMG Sensors and Signal Processing Technologies)

Abstract

:
Classification-based myoelectric control has attracted significant interest in recent years, leading to prosthetic hands with advanced functionality, such as multi-grip hands. Thus far, high classification accuracies have been achieved by increasing the number of surface electromyography (sEMG) electrodes or adding other sensing mechanisms. While many prescribed myoelectric hands still adopt two-electrode sEMG systems, detailed studies on signal processing and classification performance are still lacking. In this study, nine able-bodied participants were recruited to perform six typical hand actions, from which sEMG signals from two electrodes were acquired using a Delsys Trigno Research+ acquisition system. Signal processing and machine learning algorithms, specifically, linear discriminant analysis (LDA), k-nearest neighbors (KNN), and support vector machines (SVM), were used to study classification accuracies. Overall classification accuracy of 93 ± 2%, action-specific accuracy of 97 ± 2%, and F1-score of 87 ± 7% were achieved, which are comparable with those reported from multi-electrode systems. The highest accuracies were achieved using SVM algorithm compared to LDA and KNN algorithms. A logarithmic relationship between classification accuracy and number of features was revealed, which plateaued at five features. These comprehensive findings may potentially contribute to signal processing and machine learning strategies for commonly prescribed myoelectric hand systems with two sEMG electrodes to further improve functionality.

1. Introduction

Myoelectric prosthetic hands, controlled by surface electromyography (sEMG) signals generated from contractions of muscles (for example, flexor and extensor carpi radialis) within a residual limb are commonly used to restore hand functions for those with upper limb deficiencies, such as upper limb amputees. To date, many prescribed myoelectric hands still adopt two-electrode sEMG designs [1,2,3]. Comprehensive training is often provided to upper limb amputees in clinics to help amputees adapt to myoelectric prosthetic hand use when conducting activities of daily living. Training often involves use of clinically friendly tools such as the MyoBoy system (Ottobock, Duderstadt, Germany) [4], which is also a two-electrode sEMG system. For a myoelectric prosthetic hand system with two sEMG electrodes, the two electrodes are usually located on the lateral and medial side of the residual limb. Corresponding sEMG signals generated during muscle contractions are intrinsically processed to control prosthetic hand actions, such as open and close functions for single-grip hands [5].
Although single-grip hands are widely prescribed, high rejection rates (of up to 44%) are still common [6,7] owing to non-intuitive and unstable control caused by signal artifacts [8,9]. Multi-grip prosthetic hands have been recently made clinically available through the UK National Health Service [10] with the aim to improve function and overall user adherence. To facilitate this, advanced prosthetic hands comprising a greater number of sEMG electrodes have emerged. For instance, MyoPlus (Ottobock, Duderstadt, Germany) [11] and the COAPT system (COAPT LLC., Chicago, IL, USA) [12] utilize up to 17 sEMG electrodes to achieve 14 grip patterns. However, compared with commonly prescribed two-electrode myoelectric hands, multi-electrode hands are generally heavier and could also present increased risks associated with skin irritation [13]. Moreover, analysis of large-scale datasets has been reported [14] to limit efficiency of data processing and real-time feedback for prosthetic control. For the foreseeable future, global healthcare systems, particularly those in low-resource environments, may still prefer conventional low-cost myoelectric prosthetic hands, such as those based on two sEMG electrodes.
Machine learning has been identified [15] as a promising means to classify hand actions for more intuitive and accurate control of prosthetic hands. This typically entails feature extraction from raw sEMG signals, such as maximum, minimum, and standard deviation (SD) values obtained during muscle contraction. Extracted features are used to train machine learning models as a means to establish correspondence between sEMG signal features and hand actions, known as classification. Trained classification models can subsequently be used to classify hand actions based on new sEMG datasets, whereby classification accuracies can be calculated by dividing the number of correct predictions by the total number of predictions. Li et al. [16] reported classification accuracy of approximately 93% using linear discriminant analysis (LDA), artificial neural network, and fuzzy logic algorithms to categorize six hand actions with ten electrodes. This accuracy decreased to approximately 92% and 89% when reducing electrode count to six and four, respectively [16]. Using support vector machines (SVM) and k-nearest neighbors (KNN) algorithms, Mukhopadhyay et al. [17] reported accuracies of approximately 96% when categorizing eight hand actions using seven electrodes. An accuracy range of 79–82% was also reported [18] using LDA for ten hand actions with four electrodes. For multi-electrode systems, various feature types and different approaches to extract sEMG signal features have also been reported. For example, sEMG feature evaluation has been conducted [19] to improve pattern recognition robustness, which suggested that, among the fifty features extracted in the study, only four features were major contributors to achieve 99% accuracy when classifying four hand actions with four electrodes on a single subject. Moreover, a two-dimensional matrix image-based approach was reported [20] for classification of sEMG signals, which was compared using common machine learning algorithms, including KNN and SVM.
Despite their high accuracies, the aforementioned models are based on multi-electrode systems. For two-sEMG-electrode systems, machine learning algorithms and feature extraction have been reported [21] with a view to achieving similar classification accuracies. For example, robust hand gesture recognition based on two-electrode sEMG systems and SVM algorithms has been reported, where overall classification accuracies of over 90% [22] and 97% [23] were achieved. However, a recent review [24] reporting the use of sEMG signals to classify hand gestures highlighted the lack of focus on two-sEMG-electrode systems in the field. Additionally, classification accuracy based on two-sEMG-electrode systems was reported [25] to vary by up to 20%, depending on machine learning algorithms utilized. Thus, it is important to compare different classification algorithms and feature types to facilitate comprehensive machine learning evaluation. To the best of our knowledge, there is currently a lack of such comparative studies on machine learning algorithms based on signals produced by two-sEMG-electrode systems, which is the focus and main contribution of this work.

2. Materials and Method

Figure 1 shows an overview of this study. In particular, during the initial sEMG data acquisition stage, two sEMG electrodes from a Delsys Trigno Research+ acquisition system were attached to the belly of the flexor and extensor carpi radialis muscles located in the forearm of an able-bodied participant. Each participant was instructed to conduct six designated hand actions while real-time sEMG signals were collected. Raw sEMG data collated from the nine participants were subsequently processed, including data segmentation, rectification, filtering, and signal feature extraction. The extracted features were then used with KNN, SVM, and LDA machine learning algorithms, respectively, to train classification models. Hand action classification accuracies were evaluated by varying the algorithm and number of features. Further details of each stage are included in the following sections, as also indicated in Figure 1.

2.1. sEMG Data Acquisition

Nine able-bodied participants without upper limb loss or injuries were recruited for this study (6 female, 3 male; 24 ± 6 years). This study utilized a Delsys Trigno Research+ acquisition system which included a base station, two wireless differential sEMG electrodes with a sampling frequency of 1111 Hz, and EMGWorks software (version 4.7.9, Delsys Inc., Natick, MA, USA), illustrated in Figure 2 [26]. Each participant was instructed to perform wrist flexion and extension, enabling the researcher to feel for the approximate locations of the flexor carpi radialis and extensor carpi radialis. Subsequently, two sEMG electrodes were attached to the right forearm at the belly of the flexor carpi radialis (medial electrode) and the extensor carpi radialis (lateral electrode) using double-sided adhesive, as shown in Figure 3a. Electrode placement protocol was based on Surface ElectroMyoGraphy for the Non-Invasive Assessment of Muscles (SENIAM) recommendations [27], which describes optimal sEMG sensor application procedures. Electrode locations were selected in line with previous studies [28,29,30].
Following the electrode setup, each participant was instructed to sit and rest their forearms on a desk at an angle of approximately 45° to create a 90° elbow angle (Figure 3b). Subsequently, each participant was asked to conduct six hand actions, including hand open (HO), hand close (HC), wrist extension (WE), wrist flexion (WF), wrist pronation (WP), and wrist supernation (WS), as shown in Figure 3c. These hand actions represent agonist and antagonist movements that trigger flexor and extensor muscle groups within the forearm, and thus are commonly exploited for sEMG signal processing in myoelectric control research [31,32,33].
A session was defined as one data recording consisting of 10 repetitions of the same hand action, as shown in Figure 3d. Each session commenced with the participant resting their hand for approximately eight seconds. A hand action was initiated and held for one second, which was then repeated ten times. The process was cued by a visual prompt, with a hand rest period lasting for four seconds in between. A time sequence of a typical session is shown in Figure 3d. Each hand action session was repeated five times. In total, each hand action was performed 50 times for each participant.

2.2. sEMG Data Processing

Figure 4 outlines the data processing protocol. Figure 5a shows typical sEMG signals generated during WE actions, and Figure 5b shows the corresponding segmented signals. Signal segments were defined based on the time sequence, as shown in Figure 3d. The duration of each signal segment was two seconds, ensuring the signal associated with a one-second hand action was fully encased. The start and end points of each segment were positioned so that the location of peak signal activity across both electrodes occurred at the center of that segment. Full-wave rectification [34] was then performed on the segmented signals (Figure 5c), converting negative signal components into positive counterparts. Subsequently, a third-order low-pass Butterworth Filter with a cut-off frequency of 5 Hz was applied to the rectified signals (Figure 5d), producing envelope-like profiles. The filtered signals were subsequently used for feature extraction. Pearson’s correlation coefficient was calculated using filtered sEMG signals from both electrodes to study the correlation between the sEMG signals produced by the medial and lateral electrodes. Low correlation was defined as a correlation coefficient of 0.5 or below [35].

2.3. Feature Extraction

Six features were extracted from full-wave rectified and filtered sEMG signals for each electrode. This resulted in a total of 12 features per data segment, including mean voltage (Vmean), maximum voltage (Vmax), minimum voltage (Vmin), standard deviation of voltage (VSD), skewness (skew) of voltage, and kurtosis (kurt) of voltage. These features have been previously adopted for pattern recognition using sEMG signal classification [36]. Table 1 summarizes equations that were used to determine feature values of Vmax, Vmin, Vmean, VSD, skewness, and kurtosis. These features were subsequently input into machine learning models without calibration or standardization to simplify the process.

2.4. Machine Learning

Features obtained from each participant were used to train KNN, LDA, and SVM algorithms, respectively. The flowchart in Figure 6 outlines the underlying principles for KNN (Figure 6a) [37], SVM (Figure 6b) [38], and LDA (Figure 6c) [39] algorithms. Overall classification accuracy, along with action-specific accuracy, precision, sensitivity, and F1-score [40] were used to define a model’s performance metrics. Model performance metrics were subsequently evaluated with respect to the number of features used (up to 12) for training. All mean performance metrics were calculated using all possible combinations of the 12 features across the two electrodes.
A 5-fold validation process [41] was used to quantify classification performance for each machine learning model. Confusion matrices were subsequently generated using feature values, enabling calculation of four parameters for each hand action. These are true positive (TP), true negative (TN), false positive (FP), and false negative (FN).
Action-specific performance metrics, displayed in Equations (7)–(10), were calculated.
Action   Precision = TP TP + FP
  Action   Sensitivity   =   TP   TP   +   FN
Action-Specific   Accuracy   =   TP   +   TN TP + FP + TN + FN
Action   F1-Score   =   2   ×   Precision   ×   Sensitivity Precision + Sensitivity
Overall classification accuracy across all hand actions can be defined as the percentage of total TP, shown in Equation (11).
Overall   Classification   Accuracy   =   T P Total   Data   Samples  

3. Results

3.1. Processed sEMG Signals

Figure 7 shows mean and ±1 SD of processed sEMG signals obtained from all participants, plotted on a range from 0 to 250 μV. Considering all hand actions, HO produced the highest correlation coefficient of 0.99 (Figure 7a), while HC resulted in the lowest correlation coefficient of 0.05 (Figure 7d). WE, WF, WS, and WP produced correlations of 0.25, −0.48, −0.34, and −0.07, respectively.
Table 2 summarizes the mean of feature values across all participants, which ranged within 18.6–53.8 µV (Vmean), 62.5–204.5 µV (Vmax), 9.0–23.2 µV (Vmin), and 10.6–52.1 µV (VSD). Skewness and kurtosis fell in the ranges of 1.5–2.4 and 5.1–10.2, respectively. There is a distinct difference between Vmax values from the medial and lateral electrodes for WE and WF. For instance, WE led to Vmax of 81.8 µV and 204.5 µV across the medial and lateral electrodes, respectively, while WF led to Vmax of 171.0 µV and 102.3 µV across the medial and lateral electrodes. This may be due to the fact that electrodes were placed directly on the corresponding activation muscles, namely extensor carpi radialis and flexor carpi radialis during WE and WF. On the other hand, Vmax values obtained from medial and lateral electrodes for HC were 155.5 µV and 153.2 µV, respectively, while Vmax values obtained from medial and lateral electrodes during HO were 101.4 µV and 148.4 µV, respectively. The difference in Vmax between the two electrodes during HC and HO was minimal. This may be because HO and HC actions require activations from similar muscle groups and thus the Vmax feature may not be best suited to distinguish these types of hand actions. However, it was noted that the medial electrode resulted in VSD of 22.5 for HO and 34.9 for HC, which could be used as dominant features in the classification algorithm. Nonetheless, this study included all features to train machine learning models with a view to focus on comparing the machine learning algorithms.

3.2. Action-Specific Performance Metrics

Figure 8 displays typical examples of confusion matrices generated from one participant using SVM. It is worth noting that features used for classification were randomly selected for the three-feature and six-feature models.
For the three-feature model, WE action was correctly classified six times (third row and third column in Figure 8a) and incorrectly classified 44 times, resulting in a classification accuracy of 12%. This increased to 42 times (Figure 8b) and 50 times (Figure 8c) when the number of features increased to six and twelve, respectively. TP, TN, FP, and FN were calculated using confusion matrices generated from the machine learning models using all possible feature combinations. Subsequently, performance metrics (using Equations (7)–(11)) were calculated using all possible feature combinations.
Figure 9 displays mean TP, TN, FP, and FN values as a function of the number of features for the six different hand actions. WF action resulted in the highest number of correct classifications of all hand actions (maximum TP of 47.8), whereas WP and WS show the lowest number of correct classifications (maximum TP of 40.9 and 41.1, respectively). HO and WF were shown to have the lowest false classification, namely FP (Figure 9c) and FN (Figure 9d), dropping below 5 (10% of total data samples). TN (Figure 9b) has a mean maximum value of 243.7, further demonstrating a high level of correct classification. Performance metrics, as shown in Figure 10, were subsequently calculated using TN, TP, FP, and FN values based on Equations (7)–(11).
Figure 10 displays mean values of action-specific performance metrics as a function of number of features. It is evident that all performance metrics increased with number of features and eventually plateaued. We defined the onset of the plateau as the point at which relative change between consecutive feature accuracies was within 5%. For precision (Figure 10a) and sensitivity (Figure 10b), profiles began plateauing at approximately five features, with mean corresponding accuracies of 79 ± 8% and 79 ± 8%, respectively, with both reaching a maximum accuracy of 88 ± 6%. Action-specific accuracy (Figure 10c) began to plateau at two features (corresponding to a mean of 88 ± 2%), reaching a maximum of 97 ± 2%. F1-score (Figure 10d) plateaued at five features, corresponding to a mean value of 79 ± 8%, and reached a maximum value of 87 ± 7%.

3.3. Overall Accuracy

Figure 11 shows overall classification accuracy for each algorithm in relation to number of features using all possible feature combinations. Overall accuracy in the range of 30–95% was achieved for the selected numbers of features, with a maximum overall classification accuracy of 93 ± 2%. Akin to Figure 10, a logarithmic relationship was also revealed, which started to plateau at five features (83%). Overall classification accuracy achieved using SVM algorithm was slightly higher than those from KNN and LDA algorithms. It should be noted that the minimum–maximum range is relatively low (13 ± 5%) for all data points, representing low variation across participants.
Figure 12 compares overall classification accuracy across KNN, LDA, and SVM classification algorithms. It is evident that SVM resulted in higher accuracy (up to 93 ± 3%) than that of LDA (up to 92 ± 4%) and KNN (up to 90 ± 5%). This trend is consistent regardless of the number of features used for classification.

4. Discussion

This study utilized sEMG signals from two electrodes to classify typical hand actions. A high correlation coefficient from HO actions (0.99) is shown in Figure 7a. This may be attributable to activation of dorsal interossei muscles, located between the fingers during HO action. However, corresponding sEMG signals from medial and lateral forearm muscles likely represent subsequent movement associated with wrist stabilization, which requires concurrent contractions, and thus can be considered complementary. In contrast, HC actions (Figure 7d) resulted in the lowest correlation coefficient (0.05). This may potentially be due to contractions of the palmaris longus (medial area of the forearm) required during HC action, whereas the lateral muscles play a secondary and assistive role to ensure wrist stabilization for different actions. Low correlation coefficients (−0.5 to 0.5) were obtained for all other hand actions, which indicates limited crosstalk between medial and lateral sEMG signals. This was expected because the two electrodes were placed at opposite sides of the forearm on two distinct muscle groups. Crosstalk reduction among electrodes is desirable for signal processing since multi-collinearity caused by crosstalk may lead to unstable numerical solutions and model bias [42], which are known factors affecting classification accuracy. It is plausible to note that high-density sEMG multi-electrode systems (i.e., >2 electrodes) may incur greater signal crosstalk than those of two-sEMG-electrode systems. Thus, a tradeoff between advanced functions and accuracies may need to be considered in future multi-electrode systems.
Precision (88 ± 6%, in Figure 10a) and sensitivity (88 ± 6%, Figure 10b) were lower than previously reported values, i.e., 98% [43,44]. Considering the function-driven nature of a prosthetic hand, precision is usually considered more important than sensitivity for classification models [45]. Imprecise classification models may result in a greater number of false positives, which may subsequently lead to unstable control of a prosthesis. Real-world consequences include unprovoked prosthetic hand activations, resulting in items being dropped or gripped harder than intended, both of which have been frequently reported in this field [46] as one of the main contributing factors for high rejection rates [9]. On the other hand, low sensitivity could be associated with higher rates of false negatives, for instance, when a prosthesis user intends to perform a hand action, but the activation is not realized. Such devices could require users to overexert themselves when producing muscle activations, potentially leading to muscle fatigue or damage over time [47]. Consequently, such prosthetic hands are often deemed less responsive, which has also been reported to potentially be associated with device rejection [48].
Figure 10 and Figure 11 indicate logarithmic relationships between the number of features used and corresponding performance metrics. Classification accuracy plateaued at five features, indicating a small difference in classification performance (i.e., relative difference of less than 5%) when further increasing the number of features beyond five. However, this logarithmic relationship may be dependent on the type and number of features selected. While we only used time-domain features in this study, features in the frequency or time-frequency domain, for instance, may further increase classification accuracy. Additionally, multimodal sensor fusion via addition of other sensing features, such as inertial measurement units for motion detection [49] and/or force sensors for fingertip haptic feedback [50], could also be employed to create a more data-rich environment in order to improve overall accuracy. In those cases, it would be prudent to consider the potential constraints on data transfer and processing for large-scale datasets. Nonetheless, these results suggest there may be a limit to the number of features required to achieve high performance metrics when using two-sEMG-electrode prosthetic systems.
Figure 10 shows high action-specific classification accuracies (>90%) and F1-scores (>80%) were achieved for each hand action. High F1-scores obtained in this study suggest that in such a two-sEMG-electrode setup, features and machine learning algorithms adopted may be suitable for future prosthetic hand controls and actuation. Figure 11 shows high overall classification accuracies (93 ± 2%) can be achieved using sEMG signals from only two electrodes, which falls in the range of those reported in previous studies (79–99%) [16,17,18,19,20,22,23]. In addition, a relatively small variation in overall classification accuracy (13 ± 5%) indicates accuracy is somewhat not dependent on participants’ capabilities. We thus believe there could still be research potential in this field, particularly with a view of unravelling and optimizing the performance of these two-electrode systems. This could be performed in parallel with the development of multi-electrode myoelectric prosthetic hands, thus presenting alternative options to meet users’ multi-faceted needs.
In this study, highest accuracy was achieved by SVM (up to 93 ± 3%) when classifying six hand actions, as compared with LDA (up to 92 ± 4%) and KNN (up to 90 ± 5%), as shown in Figure 12. This may be due to the fact that SVM algorithm training protocol is based on maximizing the separation of classification boundaries for different hand actions. Separation of classification boundaries is primarily sensitive to sEMG features generated by distinctive hand actions, such as WF and WE. Feature values located at a greater distance from the classification boundary would not notably affect classification performance. In contrast, should a hand action, such as HO and HC, involve sEMG signals generated by similar muscle groups, the separation of classification boundaries would reduce, which may affect overall accuracy. For those types of hand actions, using an alternative algorithm or hybrid algorithms may be favorable compared to SVM. It is worth noting that detailed comparisons across different machine learning algorithms for hand action classification have not been performed. Comprehensive machine learning evaluation may be advisable to inform appropriate selection of classification algorithms best suited for desired hand actions. This may further inform potential design of upper limb prostheses incorporating machine learning.
Results produced in this study suggest SVM may be well-suited for classifying hand actions with distinct muscle contractions, such as WE and WF actions. However, use of hybrid machine learning models may potentially affect the separation of classification boundaries for different hand actions with similar muscle activation patterns, which could be explored in future work. In particular, SVM algorithm could be explored when used in conjunction with sine cosine and cuckoo search algorithms [51], which may facilitate studies involving a wider variety of hand actions. Although use of advanced algorithms may notably increase computational demand, they may potentially enable improved control of myoelectric prosthetic hands.
Future work could incorporate feature calibration to include scaling and normalization, which may further improve classification accuracy. Additionally, this study focused on the mean performance metrics at a set number of features. Further investigation could be conducted to better understand the impact of specific features, which may help to achieve optimized classification models. While this study involved comparisons of three commonly used machine learning algorithms, other algorithms such as logistic regression, Naive Bayes, and decision tree, as well as hybrid algorithms could also be explored with a view to further improve overall classification accuracies. State-of-the-art hybrid machine learning algorithms could be adapted to decipher and classify real-time sEMG signals with additional hand actions. Examples of these include two-phase cuckoo search-based deep learning [51] or deep learning models combining auto-encoders and neural networks [43].

5. Conclusions

This study reports the acquisition, processing, and machine learning analysis of two- sEMG-electrode signals obtained using a Delsys Trigno Research+ acquisition system. sEMG signals for six common hand actions were attained from nine able-bodied participants. Twelve sEMG features were extracted for machine learning studies to facilitate comparison of KNN, SVM, and LDA algorithms. Hand action classification accuracies were evaluated and compared across these algorithms. Classification accuracy was also studied as a function of the number of extracted features. Results revealed that overall classification accuracy of 93 ± 2% can be achieved based on two-electrode sEMG signals, which is in line with values reported in the literature. Individual hand action classification accuracy was found to reach 97 ± 2%, while action precision and sensitivity of up to 88 ± 6% were obtained. A logarithmic relationship between number of selected features and classification accuracy was found, which also indicated five features could be sufficient to achieve a reasonably high overall accuracy of 83%. Moreover, the effect of machine learning algorithms on overall classification accuracy was demonstrated, with the highest overall accuracy achieved using SVM (93 ± 3%), followed by LDA (92 ± 4%) and KNN (90 ± 5%). Our analysis suggests that it may be advisable to evaluate suitable classification algorithms based on different hand action selections. Future work could include further studies involving other machine learning classification algorithms, such as hybrid machine learning models, which may potentially reduce the separation of classification boundaries for hand actions with similar muscle activation patterns. Inclusion of participants with upper limb loss could also enable study of physiological differences in muscle actions.

Author Contributions

Conceptualization, H.O.S. and L.J.; methodology, H.O.S. and L.J.; software, H.O.S.; validation, H.O.S. and L.J.; formal analysis, H.O.S. and L.J.; investigation, H.O.S.; resources, H.O.S. and J.T.; data curation, H.O.S.; writing—original draft preparation, H.O.S.; writing—review and editing, H.O.S., K.M.D., J.T. and L.J.; visualization, H.O.S., K.M.D., J.T. and L.J.; supervision, L.J. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by the UK Engineering and Physical Sciences Research Council (EPSRC) Grant EP/S02249X/1 for the Center for Doctoral Training in Prosthetics and Orthotics.

Institutional Review Board Statement

The study was approved by the University Research Ethics Committee of the University of Southampton (ERGO ID: 90726, Date of Approval: 12 January 2024).

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

All data supporting this study are openly available from the University of Southampton repository at https://doi.org/10.5258/SOTON/D2941 (accessed on 24 January 2024).

Conflicts of Interest

The authors declare no conflicts of interest.

References

  1. Cipriani, C.; Zaccone, F.; Micera, S.; Carrozza, M.C. On the shared control of an EMG-controlled prosthetic hand: Analysis of user-prosthesis interaction. IEEE Trans. Robot. 2008, 24, 170–184. [Google Scholar] [CrossRef]
  2. Tam, S.; Boukadoum, M.; Campeau-Lecours, A.; Gosselin, B. Intuitive real-time control strategy for high-density myoelectric hand prosthesis using deep and transfer learning. Sci. Rep. 2021, 11, 11275. [Google Scholar] [CrossRef] [PubMed]
  3. Kumar, D.K.; Jelfs, B.; Sui, X.H.; Arjunan, S.P. Prosthetic hand control: A multidisciplinary review to identify strengths, shortcomings, and the future. Biomed. Signal Process. Control 2019, 53, 101588. [Google Scholar] [CrossRef]
  4. MyoBoy. Available online: https://shop.ottobock.us/Prosthetics/Upper-Limb-Prosthetics/Myo-Hands-and-Components/Myo-Software/MyoBoy/p/757M11~5X-CHANGE (accessed on 22 January 2024).
  5. Johnson, S.S.; Mansfield, E. Prosthetic training: Upper limb. Phys. Med. Rehabil. Clin. N. Am. 2014, 25, 133–151. [Google Scholar] [CrossRef] [PubMed]
  6. Salminger, S.; Stino, H.; Pichler, L.H.; Gstoettner, C.; Sturma, A.; Mayer, J.A.; Szivak, M.; Aszmann, O.C. Current rates of prosthetic usage in upper-limb amputees—Have innovations had an impact on device acceptance? Disabil. Rehabil. 2022, 44, 3708–3713. [Google Scholar] [CrossRef] [PubMed]
  7. Resnik, L.; Borgia, M.; Clark, M. Function and Quality of Life of Unilateral Major Upper Limb Amputees: Effect of Prosthesis Use and Type. Arch. Phys. Med. Rehabil. 2020, 101, 1396–1406. [Google Scholar] [CrossRef] [PubMed]
  8. Smail, L.C.; Neal, C.; Wilkins, C.; Packham, T.L. Comfort and function remain key factors in upper limb prosthetic abandonment: Findings of a scoping review. Disabil. Rehabil. Assist. Technol. 2021, 16, 821–830. [Google Scholar] [CrossRef] [PubMed]
  9. Chadwell, A.; Kenney, L.; Thies, S.; Head, J.; Galpin, A.; Baker, R. Addressing unpredictability may be the key to improving performance with current clinically prescribed myoelectric prostheses. Sci. Rep. 2021, 11, 3300. [Google Scholar] [CrossRef] [PubMed]
  10. NHS England (Ed.) Clinical Commissioning Policy Multi-Grip Prosthetic Hand (All Ages) (2009) [220801P]; NHS England: Leeds, UK, 2022. [Google Scholar]
  11. Ottobock Myoelectric Myoplus. Available online: https://www.ottobock.com/en-au/Prosthetics/UpperLimb_MyoPlus (accessed on 22 January 2024).
  12. COAPT. Available online: https://coaptengineering.com/ (accessed on 22 January 2024).
  13. Alizadeh-Meghrazi, M.; Sidhu, G.; Jain, S.; Stone, M.; Eskandarian, L.; Toossi, A.; Popovic, M. A Mass-Producible Washable Smart Garment with Embedded Textile EMG Electrodes for Control of Myoelectric Prostheses: A Pilot Study. Sensors 2022, 22, 666. [Google Scholar] [CrossRef]
  14. Farrell, T.R. Determining delay created by multifunctional prosthesis controllers. J. Rehabil. Res. Dev. 2011, 48, xxi–xxxviii. [Google Scholar] [CrossRef]
  15. Franzke, A.; Kristofferson, M.; Bongers, R.; Murgia, A.; Pobatschnig, B.; Unglaube, F.; van der Sluis, C. Users’ and therapists’ perceptions of myoelectric multi-function upper limb prostheses with conventional and pattern recognition control. PLoS ONE 2019, 14, e0220899. [Google Scholar] [CrossRef]
  16. Li, G.; Schultz, A.E.; Kuiken, T.A. Quantifying pattern recognition-based myoelectric control of multifunctional transradial prostheses. IEEE Trans. Neural Syst. Rehabil. Eng. 2010, 18, 185–192. [Google Scholar] [CrossRef]
  17. Mukhopadhyay, A.K.; Samui, S. An experimental study on upper limb position invariant EMG signal classification based on deep neural network. Biomed. Signal Process. Control 2020, 55, 101669. [Google Scholar] [CrossRef]
  18. Kaufmann, P.; Englehart, K.; Platzner, M. Fluctuating emg signals: Investigating long-term effects of pattern matching algorithms. Annu. Int. Conf. IEEE Eng. Med. Biol. Soc. 2010, 2010, 6357–6360. [Google Scholar] [PubMed]
  19. Phinyomark, A.; Quaine, F.; Charbonnier, S.; Serviere, C.; Tarpin-Bernard, F.; Laurillau, Y. EMG feature evaluation for improving myoelectric pattern recognition robustness. Expert Syst. Appl. 2013, 40, 4832–4840. [Google Scholar] [CrossRef]
  20. Wen, T.; Zhang, Z.; Qiu, M.; Zeng, M.; Luo, W. A two-dimensional matrix image based feature extraction method for classification of sEMG: A comparative analysis based on SVM, KNN and RBF-NN. J. X-ray Sci. Technol. 2017, 25, 287–300. [Google Scholar] [CrossRef] [PubMed]
  21. Li, W.; Shi, P.; Yu, H. Gesture Recognition Using Surface Electromyography and Deep Learning for Prostheses Hand: State-of-the-Art, Challenges, and Future. Front. Neurosci. 2021, 15, 621885. [Google Scholar] [CrossRef]
  22. Tavakoli, M.; Benussi, C.; Alhais Lopes, P.; Osorio, L.B.; de Almeida, A.T. Robust hand gesture recognition with a double channel surface EMG wearable armband and SVM classifier. Biomed. Signal Process. Control 2018, 46, 121–130. [Google Scholar] [CrossRef]
  23. Phukan, N.; Kakoty, N.M.; Shivam, P.; Gan, J.Q. Finger movements recognition using minimally redundant features of wavelet denoised EMG. Health Technol. 2019, 9, 579–593. [Google Scholar] [CrossRef]
  24. Tosin, M.C.; Machado, J.C.; Balbinot, A. SEMG-based upper limb movement classifier: Current scenario and upcoming challenges. J. Artif. Intell. Res. 2022, 75, 83–127. [Google Scholar] [CrossRef]
  25. Ekinci, E.; Garip, Z.; Serbest, K. Electromyography based hand movement classification and feature extraction using machine learning algorithms. Politek. Derg. 2023, 26, 1621–1633. [Google Scholar] [CrossRef]
  26. DELSYS Trigno Research+. Available online: https://delsys.com/trigno/research/ (accessed on 22 January 2024).
  27. Hermens, H.J.; Freriks, B.; Disselhorst-Klug, C.; Rau, G. Development of recommendations for SEMG sensors and sensor placement procedures. J. Electromyogr. Kinesiol. 2000, 10, 361–374. [Google Scholar] [CrossRef] [PubMed]
  28. Kim, K.T.; Park, S.; Lim, T.H.; Lee, S.J. Upper-Limb Electromyogram Classification of Reaching-to-Grasping Tasks Based on Convolutional Neural Networks for Control of a Prosthetic Hand. Front. Neurosci. 2021, 15, 733359. [Google Scholar] [CrossRef] [PubMed]
  29. Ghapanchizadeh, H.; Ahmad, S.A.; Ishak, A.J. Recommended surface EMG electrode position for wrist extension and flexion. In Proceedings of the 2015 IEEE Student Symposium in Biomedical Engineering & Sciences (ISSBES), Shah Alam, Malaysia, 4 November 2015; pp. 108–112. [Google Scholar]
  30. Corbett, E.A.; Perreault, E.J.; Kuiken, T.A. Comparison of electromyography and force as interfaces for prosthetic control. J. Rehabil. Res. Dev. 2011, 48, 629–641. [Google Scholar] [CrossRef] [PubMed]
  31. Zhu, Z.; Li, J.; Boyd, W.J.; Martinez-Luna, C.; Dai, C.; Wang, H.; Wang, H.; Huang, X.; Farrell, T.R.; Clancy, E.A. Myoelectric Control Performance of Two Degree of Freedom Hand-Wrist Prosthesis by Able-Bodied and Limb-Absent Subjects. IEEE Trans. Neural Syst. Rehabil. Eng. 2022, 30, 893–904. [Google Scholar] [CrossRef] [PubMed]
  32. Heerschop, A.; van der Sluis, C.K.; Otten, E.; Bongers, R.M. Looking beyond proportional control: The relevance of mode switching in learning to operate multi-articulating myoelectric upper-limb prostheses. Biomed. Signal Process. Control 2020, 55, 101647. [Google Scholar] [CrossRef]
  33. Kapelner, T.; Negro, F.; Aszmann, O.C.; Farina, D. Decoding Motor Unit Activity from Forearm Muscles: Perspectives for Myoelectric Control. IEEE Trans. Neural Syst. Rehabil. Eng. 2017, 26, 244–251. [Google Scholar] [CrossRef] [PubMed]
  34. Rodríguez-Tapia, B.; Soto, I.; Martínez, D.M.; Arballo, N.C. Myoelectric Interfaces and Related Applications: Current State of EMG Signal Processing–A Systematic Review. IEEE Access 2020, 8, 7792–7805. [Google Scholar] [CrossRef]
  35. Mukaka, M.M. Statistics corner: A guide to appropriate use of correlation coefficient in medical research. Malawi Med. J. 2012, 24, 69–71. [Google Scholar]
  36. Nazmi, N.; Abdul Rahman, M.A.; Yamamoto, S.; Ahmad, S.A.; Zamzuri, H.; Mazlan, S.A. A Review of Classification Techniques of EMG Signals during Isotonic and Isometric Contractions. Sensors 2016, 16, 1304. [Google Scholar] [CrossRef]
  37. Rebala, G.; Ravi, A.; Churiwala, S. Clustering. In An Introduction to Machine Learning; Rebala, G., Ravi, A., Churiwala, S., Eds.; Springer International Publishing: Cham, Switzerland, 2019; pp. 67–76. [Google Scholar]
  38. Rebala, G.; Ravi, A.; Churiwala, S. Classification. In An Introduction to Machine Learning; Rebala, G., Ravi, A., Churiwala, S., Eds.; Springer International Publishing: Cham, Switzerland, 2019; pp. 57–66. [Google Scholar]
  39. Zhao, H.; Lai, Z.; Leung, H.; Zhang, X. Linear Discriminant Analysis. In Feature Learning and Understanding: Algorithms and Applications; Zhao, H., Lai, Z., Leung, H., Zhang, X., Eds.; Springer International Publishing: Cham, Switzerland, 2020; pp. 71–85. [Google Scholar]
  40. Yacouby, R.; Axman, D. Probabilistic Extension of Precision, Recall, and F1 Score for More Thorough Evaluation of Classification Models. In Proceedings of the First Workshop on Evaluation and Comparison of NLP Systems, Online, 8–12 November 2020; Association for Computational Linguistics: St. Stroudsburg, PA, USA, 2020; pp. 79–91. [Google Scholar]
  41. Fushiki, T. Estimation of prediction error by using K-fold cross-validation. Stat. Comput. 2009, 21, 137–146. [Google Scholar] [CrossRef]
  42. Dormann, C.F.; Elith, J.; Bacher, S.; Buchmann, C.; Carl, G.; Carré, G.; Marquéz, J.R.G.; Gruber, B.; Lafourcade, B.; Leitao, P.J.; et al. Collinearity: A review of methods to deal with it and a simulation study evaluating their performance. Ecography 2013, 36, 27–46. [Google Scholar] [CrossRef]
  43. Jia, G.Y.; Lam, H.K.; Liao, J.K.; Wang, R. Classification of electromyographic hand gesture signals using machine learning techniques. Neurocomputing 2020, 401, 236–248. [Google Scholar] [CrossRef]
  44. Khan, M.U.; Khan, H.; Muneeb, M.; Abbasi, Z.; Abbasi, U.B.; Baloch, N.K. Supervised Machine Learning based Fast Hand Gesture Recognition and Classification Using Electromyography (EMG) Signals. In Proceedings of the 2021 International Conference on Applied and Engineering Mathematics (ICAEM), Taxila, Pakistan, 30–31 August 2021; pp. 81–86. [Google Scholar]
  45. Hicks, S.A.; Strümke, I.; Thambawita, V.; Hammou, M.; Riegler, M.A.; Halvorsen, P.; Parasa, S. On evaluation metrics for medical applications of artificial intelligence. Sci. Rep. 2022, 12, 5979. [Google Scholar] [CrossRef] [PubMed]
  46. Chadwell, A.; Kenney, L.; Thies, S.; Galpin, A.; Head, J. The Reality of Myoelectric Prostheses: Understanding What Makes These Devices Difficult for Some Users to Control. Front. Neurorobotics 2016, 10, 7. [Google Scholar] [CrossRef]
  47. Merletti, R.; Roy, S. Myoelectric and mechanical manifestations of muscle fatigue in voluntary contractions. J. Orthop. Sports Phys. Ther. 1996, 24, 342–353. [Google Scholar] [CrossRef]
  48. Stephens-Fripp, B.; Alici, G.; Mutlu, R. A Review of Non-Invasive Sensory Feedback Methods for Transradial Prosthetic Hands. IEEE Access 2018, 6, 6878–6899. [Google Scholar] [CrossRef]
  49. Krasoulis, A.; Vijayakumar, S.; Nazarpour, K. Multi-Grip Classification-Based Prosthesis Control with Two EMG-IMU Sensors. IEEE Trans. Neural Syst. Rehabil. Eng. 2020, 28, 508–518. [Google Scholar] [CrossRef]
  50. Treadway, E.; Gillespie, B.; Bolger, D.; Blank, A.; O’Malley, M.; Davis, A. The role of auxiliary and referred haptic feedback in myoelectric control. In Proceedings of the 2015 IEEE World Haptics Conference (WHC), Evanston, IL, USA, 22–26 June 2015; IEEE: Piscataway, NJ, USA, 2015; pp. 13–18. [Google Scholar]
  51. Joshi, A.A.; Aziz, R.M. A two-phase cuckoo search based approach for gene selection and deep learning classification of cancer disease using gene expression data with a novel fitness function. Multimed. Tools Appl. 2024. [Google Scholar] [CrossRef]
Figure 1. Study overview.
Figure 1. Study overview.
Sensors 24 02383 g001
Figure 2. Block diagram of the measurement system illustrating steps involved in sEMG signal acquisition.
Figure 2. Block diagram of the measurement system illustrating steps involved in sEMG signal acquisition.
Sensors 24 02383 g002
Figure 3. (a) sEMG electrodes placed on lateral and medial sides of a participant’s forearm. (b) A photo demonstrating hand and arm positions on a desk prior to data collection. (c) Illustrations of the six hand actions. (d) Time sequence of a hand action session.
Figure 3. (a) sEMG electrodes placed on lateral and medial sides of a participant’s forearm. (b) A photo demonstrating hand and arm positions on a desk prior to data collection. (c) Illustrations of the six hand actions. (d) Time sequence of a hand action session.
Sensors 24 02383 g003
Figure 4. Signal processing block diagram.
Figure 4. Signal processing block diagram.
Sensors 24 02383 g004
Figure 5. (a) Typical sEMG signals from WE action as a function of time, (b) segmented signals, (c) full-wave rectified signals, and (d) filtered signals, including exemplar identification of features of minimum and maximum voltages (Vmin and Vmax).
Figure 5. (a) Typical sEMG signals from WE action as a function of time, (b) segmented signals, (c) full-wave rectified signals, and (d) filtered signals, including exemplar identification of features of minimum and maximum voltages (Vmin and Vmax).
Sensors 24 02383 g005
Figure 6. Flowchart outlining machine learning algorithms for (a) KNN, (b) SVM, and (c) LDA.
Figure 6. Flowchart outlining machine learning algorithms for (a) KNN, (b) SVM, and (c) LDA.
Sensors 24 02383 g006
Figure 7. Mean and ±1 SD of sEMG signals for (a) HO, (b) WE, (c) WP, (d) HC, (e) WF, and (f) WS actions.
Figure 7. Mean and ±1 SD of sEMG signals for (a) HO, (b) WE, (c) WP, (d) HC, (e) WF, and (f) WS actions.
Sensors 24 02383 g007
Figure 8. Example of confusion matrices obtained with SVM algorithm based on signals obtained from one participant using (ac).
Figure 8. Example of confusion matrices obtained with SVM algorithm based on signals obtained from one participant using (ac).
Sensors 24 02383 g008
Figure 9. Mean profiles of (a) true positive (TP), (b) true negative (TN), (c) false positive (FP), and (d) false negative (FN) across all participants.
Figure 9. Mean profiles of (a) true positive (TP), (b) true negative (TN), (c) false positive (FP), and (d) false negative (FN) across all participants.
Sensors 24 02383 g009
Figure 10. Mean profiles of (a) precision, (b) sensitivity, (c) action-specific accuracy, and (d) F1-score across all participants. Vertical lines indicate plateau onset values for number of features and horizontal lines indicate corresponding mean performance metrics.
Figure 10. Mean profiles of (a) precision, (b) sensitivity, (c) action-specific accuracy, and (d) F1-score across all participants. Vertical lines indicate plateau onset values for number of features and horizontal lines indicate corresponding mean performance metrics.
Sensors 24 02383 g010
Figure 11. Overall classification accuracy as a function of number of features averaged across all participants, where the corresponding values in the literature are indicated by the shaded band and the dashed line indicates accuracy at which plateauing occurs.
Figure 11. Overall classification accuracy as a function of number of features averaged across all participants, where the corresponding values in the literature are indicated by the shaded band and the dashed line indicates accuracy at which plateauing occurs.
Sensors 24 02383 g011
Figure 12. Comparison of overall classification accuracy across KNN, LDA, and SVM algorithms for 3, 6, 9, and 12 features.
Figure 12. Comparison of overall classification accuracy across KNN, LDA, and SVM algorithms for 3, 6, 9, and 12 features.
Sensors 24 02383 g012
Table 1. List of the features with corresponding equations and definitions.
Table 1. List of the features with corresponding equations and definitions.
FeatureEquation Definition
Vmean 1 n i = 1 n V i (1)The mean absolute voltage.
Vmax m a x ( V i ) (2)The maximum absolute voltage value.
Vmin m i n ( V i ) (3)The minimum absolute voltage value.
VSD 1 n 1 i = 1 n | V i V mean | 2 (4)The standard deviation of the absolute voltage.
Skew i n ( V i V mean ) 3 1 n i = 1 n V i V mean 2 3 (5)A measure of asymmetry.
Kurt   1 n i = 1 n V i V mean 4 1 n i + 1 n ( V i V mean ) 2 2 (6)A measure of tailedness.
n: total number of sEMG signal samples; Vi: ith sEMG signal sample after rectification and filtering
Table 2. Mean of feature values obtained from all participants.
Table 2. Mean of feature values obtained from all participants.
Medial
ActionVmean (µV)Vmax (µV)Vmin (µV)VSD (µV)SkewKurt
HO28.5101.410.622.51.86.2
HC38.7155.511.734.91.86.5
WF33.8171.09.736.82.17.0
WE19.081.89.012.72.410.2
WS18.662.510.510.62.18.3
WP23.8121.69.221.72.07.9
Lateral
ActionVmean (µV)Vmax (µV)Vmin (µV)VSD (µV)SkewKurt
HO45.8148.416.133.11.55.2
HC51.9153.223.228.61.65.7
WF33.6102.314.618.21.97.7
WE53.8204.513.852.11.65.1
WS33.483.017.115.01.55.6
WP37.2109.417.219.61.66.3
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Shaw, H.O.; Devin, K.M.; Tang, J.; Jiang, L. Evaluation of Hand Action Classification Performance Using Machine Learning Based on Signals from Two sEMG Electrodes. Sensors 2024, 24, 2383. https://doi.org/10.3390/s24082383

AMA Style

Shaw HO, Devin KM, Tang J, Jiang L. Evaluation of Hand Action Classification Performance Using Machine Learning Based on Signals from Two sEMG Electrodes. Sensors. 2024; 24(8):2383. https://doi.org/10.3390/s24082383

Chicago/Turabian Style

Shaw, Hope O., Kirstie M. Devin, Jinghua Tang, and Liudi Jiang. 2024. "Evaluation of Hand Action Classification Performance Using Machine Learning Based on Signals from Two sEMG Electrodes" Sensors 24, no. 8: 2383. https://doi.org/10.3390/s24082383

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop