Next Article in Journal
FW-PSO Algorithm to Enhance the Invulnerability of Industrial Wireless Sensor Networks Topology
Previous Article in Journal
Using Vehicle Interior Noise Classification for Monitoring Urban Rail Transit Infrastructure
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Advanced Hand Gesture Prediction Robust to Electrode Shift with an Arbitrary Angle

School of Mechatronic Engineering and Automation, Shanghai University, Shanghai 200444, China
*
Author to whom correspondence should be addressed.
Sensors 2020, 20(4), 1113; https://doi.org/10.3390/s20041113
Submission received: 6 January 2020 / Revised: 15 February 2020 / Accepted: 17 February 2020 / Published: 18 February 2020
(This article belongs to the Section Electronic Sensors)

Abstract

:
Recent advances in myoelectric controlled techniques have made the surface electromyogram (sEMG)-based sensing armband a promising candidate for acquiring bioelectric signals in a simple and convenient way. However, inevitable electrode shift as a non-negligible defect commonly causes a trained classifier requiring continuous recalibrations. In this study, a novel hand gesture prediction is firstly proposed; it is robust to electrode shift with arbitrary angle. Unlike real-time recognition which outputs target gestures only after the termination of hand motions, our proposed advanced prediction can provide the same results, even before the completion of signal collection. Moreover, by combining interpolated peak location and preset synchronous gesture, the developed simplified rapid electrode shift detection and correction at random rather than previous fixed angles are realized. Experimental results demonstrate that it is possible to achieve both electrode shift detection with high precision and gesture prediction with high accuracy. This study provides a new insight into electrode shift robustness which brings gesture prediction a step closer to practical applications.

1. Introduction

Hand gesture serves not only as an auxiliary enhancer of reinforcing information delivery in human conversations, but also as a primary method for transferring instructions with human–computer interaction devices [1]. The capability of machines to recognize distinctive gesture characteristics can be harnessed in a wide variety of applications [2,3]. The fast growing characteristic of surface electromyogram (sEMG) has made it a promising candidate for hand motion detection, recognition or even prediction (as bio-signal sensing technique is). While the sEMG signal cannot be solely utilized in discriminating dynamic or spatial hand gestures [4], it has particular advantages of non-invasive sensing and decoding fine muscular activity simply and directly. Additionally, compared with vision-based approaches which could be greatly affected by background illumination and complexity or other external factors [5], the sEMG signal can also provide a responsive way which is capable of eliminating interference by ambient light and noise to take full control of a prosthesis [6].
By attaching certain sets of electrodes to the skin, sEMG signals are typically recorded via electrical activities such as muscle contraction. Different hand gestures activate specific muscular regions; thus, hand or fingers movement intentions can be identified while performing multiple ones [7]. However, there are usually classification errors due to an electrode shift and when the wearing position may deviate from that of previous use [8]. Recent advances in wearable sEMG sensors (e.g., sEMG armband) have facilitated the process of bioelectrical signal acquisition, whereas electrode position should be consistent in case of reducing the classification accuracy [9]. The position-dependent properties of such bioelectrical signals are the critical challenge to make recognition performance less interference with subtle or larger position change during practical use.
The conventional solutions to address the above classification issues can be categorized into two basic strategies: training the classifier for general or individual [10]. General recognition classifiers are trained with a standard and open-access but sample-limited dataset which requires implementing the same electrode configuration to allow data collection regularization. To further train and test the general recognition systems, the sensing armband ought to be placed on the exactly unaltered position which seems unlikely to achieve in actual applications [11]. In comparison to general classification models, although training an individual classifier does not need to have a pre-defined consistent wearing style, its model has to be retrained and retested on every new session [12]. Take into account user experience; a previous case study implies that amputees who have to refresh prothesis up 3.2 times per day are willing to recalibrate the classifier no more than every 2.4 h [13]. Therefore, selecting an optimal training strategy and minimizing calibration times are the key concerns which should be addressed in priority.
However, there has been little systematic literature that focus on exploring the effective solutions to ring-armband position-changed issues. A previous study published by Zhang et al. [14] indicates that sensor rotation strongly deteriorates the accuracy of classification model training by signals acquired from one position. Specifically, the proposed model can only make rotating correction and remapping of every 45°. Li et al. [15] initially established a polar coordinate to measure shifting angles, whereas predicted rotating resolution limits to ± 45° as well. Steinhardt et al. [16] take advantage of particular time-domain features (e.g., mean absolute value) to repeatedly characterize orientation shift between initial and new wearing position. Vimos et al. [10] adopted one training set that was acquired from the same suggested position; the involved model can only present higher accuracies on π/4-based positions.
Furthermore, to our knowledge, many state-of-the-art research regarding real-time improvement of hand gesture recognition still only pay attention to investigate the critical parameters which may cause human perceivable delay [17]; for example, feature [18,19] and classifier selection [20,21,22,23,24]. However, their common trait is that the following data processing procedures have to wait until sEMG recording is finished (i.e., motions ended). Although recognition technologies present good real-time performance [25,26], there is an inherent flaw–procedure separation of signal acquisition and processing as indicated above. So, the method of conducting data preprocessing and extracting features while bioelectrical signals are being collected is a novel point to gain a decent real-time response, endowing the model with the capacity of gesture prediction.
In this paper, we firstly report a hand gesture prediction model with an emphasis on estimating electrode shift effects to provide a new insight into wearing-independence based on an sEMG armband. The main contributions of our work are as follows:
  • the proposed model enables advanced predictive capability via an improved artificial neural network (ANN) substantially, which could output predicted results in 338 ms from hand gestures start, with above 94% accuracy;
  • the developed method allows electrode displacement detection at random angles rather than conventional fixed coarse resolution, capable of satisfying actual requirements;
  • this system can make simplified rapid correction according to electrode shift.
As a demonstration of our proposed solution’s utility, we randomly conduct eight independent experiment sessions over time in which the subjects could perform various wearing styles. The method of effectiveness and robustness to ring–electrode rotation, together with a predictive classifier, show promising potential in making hand gesture recognition or even prediction more applicable in practice.
We describe the materials and methods in Section 2; in Section 3, the experimental results demonstrate the decent performance both of electrode shift detection and improved ANN-based hand gesture prediction. Finally, we will discuss the novelty of our proposed prediction model and shift correction strategy, then give a brief conclusion.

2. Materials and Methods

There are two primary objectives of this work: 1. To investigate the optimal strategy for correcting electrode rotation shift with an arbitrary angle; 2. To realize hand gesture prediction. By means of a wearable armband mounted around the forearm, we are able to access raw sEMG data from either initial suggested or new shifting positions. Then, the raw acquired signal data can be preprocessed, including data normalization; low-pass filtering for removing sEMG noise is used to make the collected information more representative of target gestures.
Meanwhile, for reducing electrode shift, the proposed method underlying robustness to the position-shifted model contains four basic procedures, including electrode shift identification by interpolated peak location (IPL), signal matrix rearrangement, three-standard shifting training and multi-session prediction testing, as shown in Figure 1. It should be noted that the trained model combining majority voting needs to output target gestures before motion is finished. The involved sEMG sensing device, developed rotating correction and hand motion prediction method will be described in detail as follows.

2.1. Electrode Registration and Data Acquisition

All experiments are conducted with ten able-bodied subjects (two females included, 24 ± 1.5 years) by Myo armband, a commercially available sEMG sensor, which consists of eight circular-distributed bipolar dry electrodes. This sensor is capable of normally measuring forearm muscular electrical activity with a sampling frequency of 200 Hz. Additionally, the collected sEMG data can be transmitted to host computer (OS: Windows 10; CPU: i7-9750H; RAM: 16 GB) via built-in Bluetooth in real time; then, with a front panel graphical user interface for data display and via MATLAB for next data processing and analysis.
For data preprocessing and segmentation, we implement an initial data process on the smoothed signals with marking the region of muscular activity. This practice is achieved by calculating the energy spectrum using short-time Fourier transform (STFT), which can eliminate inactive intervals and keep effective information. Then, we employ sliding windows to convert signal data into feature vectors. The window length in this study is 200 ms with a step of 5 ms, and every sliding window can automatically align with the onset of the muscular activity region and slide along the processing direction.

2.1.1. Standard Configuration and Initial Position Designation

The armband manufacturer recommends a specific wearing style as standard configuration; the logo-printed channel (channel 4 or CH4) should be approximately aligned with subject’s middle finger, as illustrated in Figure 2a. To avoid more variables while acquiring signals and make contrast with previous studies, we designate the above-suggested configuration as the initial position for the next rotation shift analysis.

2.1.2. Employed and Synchronous Gesture Definition

Six typical hand gestures (Figure 2b) are employed with the Myo armband, including hand closed (HC), hand opened (HO), wrist extension (WE), wrist flexion (WF), double tap (DT) and no movement (NM). Subjects were asked to finish each gesture in 2 s. Specially, we also define a synchronous gesture (WE) to predict the rotation angle at onset of each independent session, and the above synchronous operation is merely required to be performed once.

2.1.3. Training and Testing Dataset Organization

To realize correcting rotation shift with arbitrary angle and build a user-specific model using minimum training data, we took three representative positions to perform these selected gestures. It should be noted that every single gesture (gesture NM included) only comprises five repetitions. To evaluate the trained ANN classifier, subjects were required to repeat each gesture (gesture NM excluded) 15 times in one independent session regardless of wearing style; the testing set consists of eight sessions over time. For every repetition, the subjects started with his/her arm relaxed, and then performed designated gestures, then returned to the initial relax position. Notably, before performance evaluation, sEMG data acquirement from three training wearing positions was conducted independently.

2.2. Electrode Shift Detection Based on IPL and Synchronous Gesture

Although sEMG signals can be recorded in a simple-operated way, the properties of non-linearity and non-stationarity are considered to be barriers to the achievement of high classification accuracy [27]. Inspired by feature-based methods, we attempted to take advantage of optimal feature to quantitatively measure electrode shift during session switching. That is, the shifted electrode position can be corrected to standard configuration only when the shift angles are confirmed, then feed the proofed data to a trained ANN classifier to make gesture predictions. To address this problem, we propose a specific gesture based on a signal synchronization method [28] that can activate minimum muscular regions as muscle contraction. This work uses IPL to detect the maximum interpolated amplitude via the sum of preprocessed data in each sensing channel, and then the interpolated curve can be plotted using a cubic spline interpolation as shown in Figure 3a.
Concerning the proposed rotation shifting prediction strategy with high accuracy of arbitrary angle, there is a need to increase the position-correction tolerance of the present model. Therefore, Standard Space is defined to regularize, and then we make a rearrangement of the raw signals allowing the position detection model to be robust to sensor shift. After electrode shifting status is assured, the raw channels will be transformed to Standard Space, which are ordered by the highest sum channel using IPL. Moreover, we choose WE to serve as the synchronous gesture since both maximum signal value and data sum consistently concentrate in one specific channel. Experimentally, the tolerance of Standard Space is increased to ± CH4 (Figure 3b). To further identify the specific offset orientation with arbitrary angle, we apply the maximum data value of interpolated amplitude to represent electrode shift.
We also provide a schematic view (Figure 3a) to illustrate that the synchronous process can be activated by gesture WE, as shown in the blue-dotted zone. Meanwhile, the shift-corrected angle at a new position can be expressed precisely as 130° rather than the initial CH4 angle (135°). Specifically, raw sEMG signals acquired from the sEMG sensor are represented as [29]:
I ( n ) = ( I 1 ( n ) , , I 8 ( n ) ) T
Each employed gesture takes 2 s with the Myo armband which normally works with a sampling rate of 200 Hz, that is, it contains 400 items of minimum measurement unit I(n). It is defined as:
S n = ( I ( n 399 ) , , I ( n ) ) T 400 × 8
Moreover, signal preprocessing can be expressed as:
F = Ψ [ abs [ S n ] ] 400 × 8
where Sn is raw sEMG data acquired in 2 s, Ψ represents a 4th order low-pass Butterworth filter with a cutoff frequency of 5 Hz. We can add the items in each column to seek the highest sum of signal channel data (NCH4) which is defined as:
N C H 4 = max (   F ( 1 ) , ,   F ( 8 ) ) 1 × 8

2.3. Data Rearrangement

By means of the above electrode shift detection using IPL, the armband rotating position can be confirmed and transformed to Standard Space. Therefore, the shifted degree can be expressed via the offset angle whose channel possesses maximum interpolated amplitude relative to initial electrode configuration. To correct the interfered sEMG samples, signal data have to be rearranged to Standard Space. The correction procedure is only done once at the onset of each experiment session by performing synchronous gestures. The rearrangement method can be briefly described by establishing a polar coordinate as illustrated in Figure 4, and all the sampling points which are collected in one gesture duration (2 s) are present in eight polar axes in steps of π/4, respectively. Then, the identified peak signal polar can be rotated counterclockwise to 3π/4, and the shifted channel satisfies the definition of Standard Space.
Applying the above rearrangement strategy, shifted position P’ can be transformed to Standard Space by raw position P as calculated in the following:
P ( m o d ( k + n 4 ,     8 ) ) P ,   n = 1 , 2 , , 8
Similarly, shifted sEMG data I(n) of each electrode channel can be expressed as:
I ( m o d ( k + n 4 ,   8 ) ) = I ( n ) ,   n = 1 , 2 , , 8
where k is the channel of maximum interpolated amplitude, mod means the remainder after division. Supposing k is 2 (instance as Figure 4), the relation between raw and shifted channel is listed in Table 1.

2.4. Training Set Selection

The above shift detection and correction allow the raw shifted data to transform to the unified sEMG data arrangement on Standard Space. By means of these data, we can train one classifier without taking random shift conditions into consideration. To evaluate the shift-corrected performance at various sessions over time, we employ the above strategy of shift detection and data rearrangement, and set the criterion of whether the corrected position presents under the tolerance of Standard Space (i.e., ± CH4 or 135° ± 22.5°). As shown in Figure 5a, the corrected rotation offsets among all testing sessions are confined to Standard Space. Meanwhile, we also select three representative wearing positions (close to 135° and 135 ± 22.5°) to organize the training set (Figure 5b).

2.5. ANN-Based Hand Gesture Prediction

Five time-domain features are selected for further extracting characteristic information from preprocessed data, including mean absolute value, root mean square, slope sign change, wave length, and Hjorth parameter. For outputting predicted gestures before motions are finished and decreasing spurious misclassification errors, we add postprocessing to the last link of prediction system. By combining the majority voting method [30], every sliding window (200 ms or 40 sampling points) generates one specific code in accordance with predefined gestures via the prediction function of educated classifier. Then, the current window shifts into the next with the step of 5 ms. If the gestural labels accumulate to the threshold (55), the postprocessing terminates.
Our forward-propagation ANN model is implemented with three layers and trained by using full batch gradient descent with cross-entropy cost function, and applied regularization using weight decay (λ = 150). The number of input layer nodes is 376, that is, the product of one sliding window length and total channel number (40 × 8 = 320), plus features extraction functions creating a vector of 56 features (7 × 8 = 56), corresponding to the length of feature vectors. Meanwhile, we take the size of a hidden layer to be half of the input one using the tanh transfer function [12]. Moreover, we set six nodes on the output layer in order to correspond to the six pre-defined gestures (including NM). The specific prediction procedure will be detailed at the Algorithm 1.
Algorithm 1. ANN-based prediction model.
Input: F ( n ) ,   T ( n ) c o r r e c t e d   s E M G   &   t e s t i n g   s e t
Output: 1 ,   2 ,   3 ,   4 ,   5   g e s t u r e   c o d e
1:initialize s; // where s denotes stride length
2:initialize ξ ( X ) ; // where   ξ ( X ) denotes time-domain features bag function;
3:training process: extract features from F ( n ) using   ξ ( X ) to form ( F i ,   L i ) ξ ( F ( n ) )
// where Fi and Li denote feature vector and corresponding label vector, respectively
4:compute weight decay && employ regularization to F ( F i μ ) σ
5:apply ( F ,   Υ ) to classifier, and form classifier.predict
6:for each windowed T ( n ) , feed ξ ( T ( n ) ) to classifier for predicting gesture code
7:count number of different generated code to form n
8:if n == preset threshold of outputting gesture, then
  get the predicted gesture code, else return None

3. Experimental Results

3.1. Prediction Accuracy with Electrode Shift Correction

Applying the above electrode shift correction strategy, Figure 6a shows the overall classification accuracy for all gestures with various sessions. For each group of presented hand gestures, the corresponding prediction results are much better than the control groups in Section 3.2. The highest and lowest precisions for hand motions are WF (98.6%) and HO (88.8%), respectively. With respect to sensitivity, the gesture HO has the highest rate (96.2%) and WF has the lowest (93.1%). Furthermore, Figure 6b highlights the developed prediction model for the responsive superiority with an averaged response time 338 ms. It should be noted that this prediction procedure starts at the onset of hand motion, and terminates at the corresponding response time. Therefore, the prediction results can be presented before gestures are completed, eliminating external device delay as perceived by users [31].

3.2. Accuracy Improvement on Electrode Shift

In this study, we also explore the governing parameters that may significantly influence the prediction accuracy, including the number of wearing positions during training data acquisition. The gesture prediction accuracy comparison between electrode shift corrected and non-corrected situations is summarized as shown in Table 2. Specifically, as for non-correction of electrode shift, the lowest and highest accuracy improvements are SUB #3 (12.0%) and SUB #04 (71.7%), respectively. This is probably because of distinctive wearing styles and forearm girths among individuals.
Moreover, the OVERALL row of Table 2 indicates that the system performance will decrease dramatically (fall from 94.7% to 51.4%) without proposed correction strategy. The averaged prediction results with one specific wearing style also shows an accuracy of 72.0% (also a drop from 94.7%), which is not up to the standard of real applications.

3.3. Synchronous Gesture Selection Varies Accuracies

A comparison is also made in this work among different gestures to emphasize the significance of selecting the appropriate synchronous gesture. From Table 3, a certain prediction accuracy of lifting varies among subjects and could be easily realized while shifting from gesture WF to WE; the overall accuracy is improved by 34.2%.

4. Discussion

With the considerable advancements and a broad range of applications of sEMG-based technologies, myoelectric-controlled terminals (e.g., prostheses) have pointed out more greater requirements for real-time system responses, simplified rapid error-correction and high recognition or even prediction accuracy. Specifically, in clinical scenarios, it is inevitable that the electrode shift happens during the sEMG sensors being taken on and off, which will definitely cause degradation of classification performance [32]. Thus, this study reveals that the electrode displacement correction of an arbitrary angle is a crucial necessity to enhance robustness on hand gesture prediction.

4.1. Governing Parameters Varies Accuracies

In this research, we analyze the governing parameters that may cause degradation of prediction performance without correcting electrode shift and selecting enough training sets or appropriate synchronous gestures. From the analysis of the given clues, it is more obvious that the classification accuracy for all gestures among different experimental sessions are all improved to varying degrees.
Meanwhile, as for selecting an appropriate number of training wearing positions, solely taking advantage of one preset electrode arrangement whose electrode position is almost or exactly the same as the standard configuration presents lower accuracy than choosing three positions. That is, there is an accuracy lifting that appropriately chooses more training positions which can enable prediction model to better cope with the electrode shift with arbitrary angles.
Obviously, as Figure 7 illustrates, there are two activation muscular regions while performing WF, whereas WE only presents one activation region which could minimize interference in shift detection. Notably, the signals from non-boxed channels can be filtered at preprocessing; however, other hand motions involved in this paper always possess rather complex sEMG characteristics which is hard to be filtered effectively.

4.2. Performance of Electrode Shift Correction

There is a large volume of literature published regarding non-shift occurrences, and various classification methods and data processing strategies have been applied on gesture recognition with decent performance. However, electrode shift should be considered together with practical use, and this factor influencing accuracies have been shown by less studies as listed in Table 4. As Table 4 shows, the sEMG armband and high-density electrodes are the primary sources to obtain raw acquired bioelectrical signals. Note that resolutions of electrode displacement correction in previous studies range roughly between 22.5° to 45°, or 1 cm or 2 cm shift, thereby showing much resistance towards clinical or practical deployments.
Meanwhile, the response time (i.e., real-time performance evaluation) of recognition systems has not been investigated via related algorithms, reducing the introduction of gesture prediction techniques. The proposed strategy for correcting electrode displacement endows the model with the ability of shift detection with any arbitrary angle. Various experiments also demonstrate that the preset hand motions could be accurately identified, no matter how electrode rotation is offset away from the standard configuration. The main breakthrough also presents gesture prediction which evidently achieves high real-time classification accuracy.

4.3. Performance with Gesture Prediction

The final experimental result is related to the evaluation of predictive accomplishment via our improved ANN-based gesture prediction model. The postprocessing using majority voting compensates for the device intrinsic delay, which highlights the superiority of the prediction strategy. In the specific classification case of postprocessing, the trained classifier will generate one gesture code or label corresponding to one certain target hand motion in the same step of sliding window processing, then the output labels are added up to calculate the sum of each emerged gesture code. Eventually, the present model determines the final predicted gesture depending on which cumulative result reaches the threshold first. The scheme implementation superiority of majority voting for predicting rather than recognizing is not needed to obtain the entire part of the sEMG signals, which provides proof of the high real-time performance based on our improved data processing method.
Besides, compared with the strategy of conventional training and testing dataset segmentation, the current work focuses on implementing the proposed prediction model plus electrode shift detection in real scenarios. We believe that this novel strategy could provide a proof-of-concept for an innovation in organizing acquired bioelectrical data, to better stimulating practical application and evaluating classification performance.

5. Conclusions

In this research, we firstly propose a novel strategy for enhancing robustness to electrode displacement with any arbitrary angle on hand gesture prediction. Compared with traditional shift-correction methods with fixed coarse resolutions, the developed method allows electrode displacement prediction to be performed at random angles via IPL and synchronous gestures. In addition, our improved ANN-based model has dramatically improved hand gesture prediction by combining majority voting. This study also provides a new insight into a simplified rapid correction strategy on electrode shift with the promising potential in practical applications.

Author Contributions

Experiments, analysis, writing—original draft preparation, Z.X.; conceptualization, methodology, Z.Z.; project administration, L.S. & J.Q. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by the National Natural Science Foundation of China under Grant No. 91748119, and Shanghai Science and Technology Commission under Grant No.18JC1410402.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Li, K.; Cheng, J.; Zhang, Q.; Liu, J. Hand gesture tracking and recognition based human-computer interaction system and its applications. In Proceedings of the IEEE ICIA, Wuyishan, China, 11–13 August 2018; pp. 667–672. [Google Scholar]
  2. Kaur, H.; Rani, J. A review: Study of various techniques of hand gesture recognition. In Proceedings of the IEEE ICPEICES, Delhi, India, 4–6 July 2016. [Google Scholar]
  3. Sayin, F.S.; Ozen, S.; Baspinar, U. Hand gesture recognition by using semg signals for human machine interaction applications. In Proceedings of the SPA, Poznan, Poland, 19–21 September 2018; pp. 27–30. [Google Scholar]
  4. Shin, S.; Tafreshi, R.; Langari, R. Emg and imu based real-time hci using dynamic hand gestures for a multiple-dof robot arm. J. Intell. Fuzzy Syst. 2018, 35, 861–876. [Google Scholar] [CrossRef]
  5. Al-Shamayleh, A.S.; Ahmad, R.; Abushariah, M.A.M.; Alam, K.A.; Jomhari, N. A systematic literature review on vision based gesture recognition techniques. Multimed. Tools Appl. 2018, 77, 28121–28184. [Google Scholar] [CrossRef]
  6. Cheok, M.J.; Omar, Z.; Jaward, M.H. A review of hand gesture and sign language recognition techniques. Int. J. Mach. Learn. Cybern. 2019, 10, 131–153. [Google Scholar] [CrossRef]
  7. Leone, F.; Gentile, C.; Ciancio, A.L.; Gruppioni, E.; Davalli, A.; Sacchetti, R.; Guglielmelli, E.; Zollo, L. Simultaneous semg classification of hand/wrist gestures and forces. Front. Neurorobotics 2019, 13, 42. [Google Scholar] [CrossRef] [PubMed]
  8. Li, Q.X.; Chan, P.P.K.; Zhou, D.; Fang, Y.; Liu, H.; Yeung, D.S. Improving robustness against electrode shift of semg based hand gesture recognition using online semi-supervised learning. In Proceedings of the ICMLC, Jeju, Korea, 10–13 July 2016; pp. 344–349. [Google Scholar]
  9. Motoche, C.; Benalcázar, M.E. In Real-time hand gesture recognition based on electromyographic signals and artificial neural networks. In Proceedings of the ICANN, Rhodes, Greece, 4–7 October 2018; pp. 352–361. [Google Scholar]
  10. Vimos, V.H.; Benalcázar, M.; Oña, A.F.; Cruz, P.J. A novel technique for improving the robustness to sensor rotation in hand gesture recognition using semg. In Proceedings of the CSEI, Ambato, Ecuador, 24 October 2019; pp. 226–243. [Google Scholar]
  11. Kim, J.; Mastnik, S.; André, E. Emg-based hand gesture recognition for realtime biosignal interfacing. In Proceedings of the IUI, Gran Canaria, Spain, 13–16 January 2008; pp. 30–39. [Google Scholar]
  12. Zhang, Z.; Yang, K.; Qian, J.; Zhang, L. Real-time surface emg pattern recognition for hand gestures based on an artificial neural network. Sensors 2019, 19, 3170. [Google Scholar] [CrossRef] [Green Version]
  13. Young, A.J.; Hargrove, L.J.; Kuiken, T.A. Improving myoelectric pattern recognition robustness to electrode shift by changing interelectrode distance and electrode configuration. IEEE Trans. Biomed. Eng. 2012, 59, 645–652. [Google Scholar] [CrossRef] [Green Version]
  14. Zhang, Y.; Chen, Y.; Yu, H.; Yang, X.; Lu, W.; Liu, H. Wearing-independent hand gesture recognition method based on emg armband. Pers. Ubiquitous Comput. 2018, 22, 511–524. [Google Scholar] [CrossRef]
  15. Li, Z.; Wang, F.; Zhao, X.; Ding, Q.; Zhang, D.; Han, J. The method for gestures recognition based on myo rotation shifts estimation and adaptive correction. Acta Automatica Sinica 2019, 1–11. [Google Scholar]
  16. Steinhardt, C.R.; Bettthauser, J.; Hunt, C.; Thakor, N. Registration of emg electrodes to reduce classification errors due to electrode shift. In Proceedings of the IEEE BioCAS, Cleveland, OH, USA, 17–19 October 2018; pp. 1–4. [Google Scholar]
  17. Farina, D.; Sartori, M. Surface electromyography for man-machine interfacing in rehabilitation technologies. In Surface Electromyography: Physiology, Engineering, and Applications; IEEE Press: Piscataway, NJ, USA, 2016; pp. 540–560. [Google Scholar]
  18. Pancholi, S.; Joshi, A.M. Electromyography-based hand gesture recognition system for upper limb amputees. IEEE Sens. Lett. 2019, 3, 1–4. [Google Scholar] [CrossRef]
  19. Javaid, H.A.; Rashid, N.; Tiwana, M.I.; Anwar, M.W. Comparative analysis of emg signal features in time-domain and frequency-domain using myo gesture control. In Proceedings of the ICMRE, Valenciennes, France, 7–11 February 2018; pp. 157–162. [Google Scholar]
  20. Chen, H.; Tong, R.; Chen, M.; Fang, Y.; Liu, H. A hybrid cnn-svm classifier for hand gesture recognition with surface emg signals. In Proceedings of the ICMLC, Chengdu, China, 15–18 July 2018; pp. 619–624. [Google Scholar]
  21. Qi, J.; Jiang, G.; Li, G.; Sun, Y.; Tao, B. Surface emg hand gesture recognition system based on pca and grnn. Neural Comput. Appl. 2019. [Google Scholar] [CrossRef]
  22. Qi, J.; Jiang, G.; Li, G.; Sun, Y.; Tao, B. Intelligent human-computer interaction based on surface emg gesture recognition. IEEE Access 2019, 7, 61378–61387. [Google Scholar] [CrossRef]
  23. Pinzón-Arenas, J.O.; Jiménez-Moreno, R.; Herrera-Benavides, J.E. Convolutional neural network for hand gesture recognition using 8 different emg signals. In Proceedings of the STSIVA, Bucaramanga, Colombia, 24–26 April 2019; pp. 1–5. [Google Scholar]
  24. Neacsu, A.A.; Cioroiu, G.; Radoi, A.; Burileanu, C. Automatic emg-based hand gesture recognition system using time-domain descriptors and fully-connected neural networks. In Proceedings of the TSP, Budapest, Hungary, 1–3 July 2019; pp. 232–235. [Google Scholar]
  25. Lu, Z.; Chen, X.; Zhang, X.; Tong, K.-Y.; Zhou, P. Real-time control of an exoskeleton hand robot with myoelectric pattern recognition. Int. J. Neural Syst. 2017, 27, 1750009. [Google Scholar] [CrossRef] [PubMed]
  26. Crepin, R.; Fall, C.L.; Mascret, Q.; Gosselin, C.; Campeau-Lecours, A.; Gosselin, B. Real-time hand motion recognition using semg patterns classification. In Proceedings of the IEEE EMBC, Honolulu, HI, USA, 18–21 July 2018; pp. 2655–2658. [Google Scholar]
  27. Farina, D.; Merletti, R. Comparison of algorithms for estimation of emg variables during voluntary isometric contractions. J. Electromyogr. Kinesiol. 2000, 10, 337–349. [Google Scholar] [CrossRef]
  28. Han, Y.; Zhao, J. Accurate substrate analysis based on a novel finite difference method via synchronization method on layered and adaptive meshing. IEEE Trans. Comput-Aided Des. Integr. Circuits Syst. 2013, 32, 1520–1532. [Google Scholar] [CrossRef]
  29. Benalcázar, M.E.; Anchundia, C.E.; Zea, J.A.; Zambrano, P.; Jaramillo, A.G.; Segura, M. Real-time hand gesture recognition based on artificial feed-forward neural networks and emg. In Proceedings of the EUSIPCO, Rome, Italy, 3–7 September 2018; pp. 1492–1496. [Google Scholar]
  30. Ruta, D.; Gabrys, B. Classifier selection for majority voting. Inf. Fusion 2005, 6, 63–81. [Google Scholar] [CrossRef]
  31. Mizuno, H.; Tsujiuchi, N.; Koizumi, T. Forearm motion discrimination technique using real-time emg signals. In Proceedings of the EMBC, Boston, MA, USA, 30 August–3 September 2011; pp. 4435–4438. [Google Scholar]
  32. Boschmann, A.; Platzner, M. Reducing classification accuracy degradation of pattern recognition based myoelectric control caused by electrode shift using a high density electrode array. In Proceedings of the EMBC, San Diego, CA, USA, 28 August–1 September 2012; pp. 4324–4327. [Google Scholar]
  33. Lv, B.; Sheng, X.; Zhu, X. Improving myoelectric pattern recognition robustness to electrode shift by autoencoder. In Proceedings of the EMBC, Honolulu, HI, USA, 18–21 July 2018; pp. 5652–5655. [Google Scholar]
  34. Fan, Z.; Wang, Z.; Li, G.; Wang, R. A canonical correlation analysis based emg classification algorithm for eliminating electrode shift effect. In Proceedings of the EMBC, Orlando, FL, USA, 16–20 August 2016; pp. 867–870. [Google Scholar]
  35. Yang, W.; Yang, D.; Li, J.; Liu, Y.; Liu, H. Emg dataset augmentation approaches for improving the multi-dof wrist movement regression accuracy and robustness. In Proceedings of the IEEE ROBIO, Kuala Lumpur, Malaysia, 12–15 December 2018; pp. 1268–1273. [Google Scholar]
Figure 1. Scheme of adaptive channel shift strategy using preprocessed data plus improved ANN classifier with majority voting method. The electrode shifting position can be identified only using the predefined synchronous gesture, and once the changed configuration is established during acquiring training data from three representative wearing positions, the subject does not require further recalibrate the algorithm on one session.
Figure 1. Scheme of adaptive channel shift strategy using preprocessed data plus improved ANN classifier with majority voting method. The electrode shifting position can be identified only using the predefined synchronous gesture, and once the changed configuration is established during acquiring training data from three representative wearing positions, the subject does not require further recalibrate the algorithm on one session.
Sensors 20 01113 g001
Figure 2. Scheme of standard electrode configuration and applied hand gestures. (a) Logo channel, also defined as CH4 is suggested to be aligned with middle finger. (b) Red-marked synchronous gesture, WE is used to identify electrode shift. Here hand closed (HC), hand opened (HO), wrist extension (WE), wrist flexion (WF), double tap (DT) and no movement (NM).
Figure 2. Scheme of standard electrode configuration and applied hand gestures. (a) Logo channel, also defined as CH4 is suggested to be aligned with middle finger. (b) Red-marked synchronous gesture, WE is used to identify electrode shift. Here hand closed (HC), hand opened (HO), wrist extension (WE), wrist flexion (WF), double tap (DT) and no movement (NM).
Sensors 20 01113 g002
Figure 3. Scheme of initial position on right forearm and IPL by WE. (a) Based on sum of each channel signal, the interpolated curve can be plotted using a cubic spline interpolation, and the shifting angle can be represented as red-marked point (130°). (b) One full electrode position range is divided into 45°, and the tolerance of Standard Space is increased to ± CH4 as red-dotted zone.
Figure 3. Scheme of initial position on right forearm and IPL by WE. (a) Based on sum of each channel signal, the interpolated curve can be plotted using a cubic spline interpolation, and the shifting angle can be represented as red-marked point (130°). (b) One full electrode position range is divided into 45°, and the tolerance of Standard Space is increased to ± CH4 as red-dotted zone.
Sensors 20 01113 g003
Figure 4. Polar scatter of every sampling point in each sensing channel on 8 polar axes. Raw channel is transformed to shifted channel according to maximum channel amplitude.
Figure 4. Polar scatter of every sampling point in each sensing channel on 8 polar axes. Raw channel is transformed to shifted channel according to maximum channel amplitude.
Sensors 20 01113 g004
Figure 5. (a) Electrode shift on various experiments. (b) 3 representative wearing position for acquiring training set.
Figure 5. (a) Electrode shift on various experiments. (b) 3 representative wearing position for acquiring training set.
Sensors 20 01113 g005
Figure 6. (a) Confusion matrix for the proposed hand gesture prediction model with electrode shift correction. (b) Comparison of averaged prediction response and activity time on each target hand gesture.
Figure 6. (a) Confusion matrix for the proposed hand gesture prediction model with electrode shift correction. (b) Comparison of averaged prediction response and activity time on each target hand gesture.
Sensors 20 01113 g006
Figure 7. Acquired raw sEMG signals of two pre-defined gestures from Myo armband mounted on forearm. (a) WF. (b) WE. Red dashed box represents anterior forearm muscular region, green dashed box represents posterior forearm muscular region.
Figure 7. Acquired raw sEMG signals of two pre-defined gestures from Myo armband mounted on forearm. (a) WF. (b) WE. Red dashed box represents anterior forearm muscular region, green dashed box represents posterior forearm muscular region.
Sensors 20 01113 g007
Table 1. The relation between raw and shifted channel.
Table 1. The relation between raw and shifted channel.
Electrode PositionsEMG Rearrangement
2 → 4I(2) = I’(4)
3 →5I(3) = I’(5)
4 → 6I(4) = I’(6)
5 → 7I(5) = I’(7)
6 → 8I(6) = I’(8)
7 → 1I(7) = I’(1)
8 → 2I(8) = I’(2)
1 → 3I(1) = I’(3)
Table 2. The classification accuracy for all gestures with different experimental sessions.
Table 2. The classification accuracy for all gestures with different experimental sessions.
M1M2M3C1-3C2-3
SUB #0173.755.394.020.338.7
SUB #0269.878.394.024.215.7
SUB #0379.372.091.312.019.3
SUB #0421.360.093.071.733.0
SUB #0560.859.596.235.436.7
SUB #0628.289.097.569.38.5
SUB #0748.563.290.742.227.5
SUB #0834.281.395.761.514.4
SUB #0936.578.296.860.318.6
SUB #1061.583.598.236.714.7
OVERALL51.472.094.743.322.7
M1 represents the method without using proposed electrode shift correction strategy; M2 stands for the method merely using 1 wearing position to acquire training dataset; M3 represents our proposed shift-correction strategy based on improved ANN; C1-3 stands for prediction accuracy lifting compared M3 with M1; C2-3 stands for prediction accuracy lifting compared M3 with M2. Unit: %.
Table 3. The classification accuracy for all gestures with different synchronous gestures.
Table 3. The classification accuracy for all gestures with different synchronous gestures.
S1S2C1-3
SUB #0142.794.051.3
SUB #0261.794.032.3
SUB #0380.591.310.8
SUB #0429.793.063.3
SUB #0533.096.263.2
SUB #0629.897.567.7
SUB #0771.390.719.4
SUB #0884.795.711.0
SUB #0972.596.824.3
SUB #1098.798.2−0.5
OVERALL60.594.734.2
S1 represents the shift correction method by WF; S2 stands for the method by WE; C1-3 stands for prediction accuracy lifting compared S2 with S1. Unit: %.
Table 4. The results of different hand gesture recognitions.
Table 4. The results of different hand gesture recognitions.
TaskWorkElectrodeChannelClassifierGestureResponse TimeResolution of Shift CorrectionAccuracy
Advanced predictionOURdry8ANN6*338ms (< GD)arbitrary angle94.7%
Traditional recognitionLi et al. [15]dry8SVM8*GD+fixed (45°)78.4%
Vimos et al. [10]dry8SVM6*GD+fixed (45°)92.4%
Steinhardt et al. [16]dry8SRC6*GD+fixed (22.5°)95.7%
Zhang et al. [14]dry8RF15GD+fixed (45°)91.5%
Lv et al. [33]SA192SAE10*GD+fixed (1-cm shift)85.0%
Fan et al. [34]SA30LDA11*GD+fixed (1-cm shift)88.2%
Yang et al. [35]dry8CNN10*GD+fixed (45°)63.2%
* Note: SA (self-adhesive electrode); GD (gesture duration); n* represents total gesture number, including no-movement (NM).

Share and Cite

MDPI and ACS Style

Xu, Z.; Shen, L.; Qian, J.; Zhang, Z. Advanced Hand Gesture Prediction Robust to Electrode Shift with an Arbitrary Angle. Sensors 2020, 20, 1113. https://doi.org/10.3390/s20041113

AMA Style

Xu Z, Shen L, Qian J, Zhang Z. Advanced Hand Gesture Prediction Robust to Electrode Shift with an Arbitrary Angle. Sensors. 2020; 20(4):1113. https://doi.org/10.3390/s20041113

Chicago/Turabian Style

Xu, Zhenjin, Linyong Shen, Jinwu Qian, and Zhen Zhang. 2020. "Advanced Hand Gesture Prediction Robust to Electrode Shift with an Arbitrary Angle" Sensors 20, no. 4: 1113. https://doi.org/10.3390/s20041113

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop