Next Article in Journal
Sub-Frame Evaluation of Frame Synchronization for Camera Network Using Linearly Oscillating Light Spot
Previous Article in Journal
Evaluation of Multivariable Modeling Methods for Monitoring the Health of Guyed Towers in Overhead Power Lines
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Action Recognition of Lower Limbs Based on Surface Electromyography Weighted Feature Method

1
School of Engineering, Qufu Normal University, Rizhao 276826, China
2
School of Instrument Science and Engineering, Southeast University, Nanjing 210096, China
*
Author to whom correspondence should be addressed.
Sensors 2021, 21(18), 6147; https://doi.org/10.3390/s21186147
Submission received: 25 July 2021 / Revised: 3 September 2021 / Accepted: 8 September 2021 / Published: 13 September 2021
(This article belongs to the Section Intelligent Sensors)

Abstract

:
To improve the recognition rate of lower limb actions based on surface electromyography (sEMG), an effective weighted feature method is proposed, and an improved genetic algorithm support vector machine (IGA-SVM) is designed in this paper. First, for the problem of high feature redundancy and low discrimination in the surface electromyography feature extraction process, the weighted feature method is proposed based on the correlation between muscles and actions. Second, to solve the problem of the genetic algorithm selection operator easily falling into a local optimum solution, the improved genetic algorithm-support vector machine is designed by championship with sorting method. Finally, the proposed method is used to recognize six types of lower limb actions designed, and the average recognition rate reaches 94.75%. Experimental results indicate that the proposed method has definite potentiality in lower limb action recognition.

1. Introduction

Muscles are an important part of the human body and provide power for locomotion [1]. Surface electromyography (sEMG) is derived from the surface of human skeletal muscle, is closely related to neuromuscular activity and contains much information related to limbs [2]. sEMG signals are acquired by placing sEMG sensors on the muscle surface and recording electrical signals [3]. sEMG has been widely used in the field of action recognition and has achieved compelling results [4,5,6,7,8]. Since sEMG involves nonstationary bioelectric signals, wearers will experience muscle fatigue, unstable sEMG signals and other interference during the signal acquisition process. Therefore, as the training time increases, the SNR will decrease and the noise will increase [9]. This requires the extraction of more representative signal features to represent the activity of muscle [10]. Many researchers use time-domain and frequency-domain analysis methods to obtain feature vectors [11,12,13]. However, pure time-domain and frequency-domain features contain less useful information to characterize the activity of muscle. Recent studies have shown that features in the time-frequency domain can effectively characterize the activity information of sEMG signals [14]. After the researchers extract the features, they usually perform dimensionality reduction processing on the features [15,16] to reduce the dimensionality of features while retaining some effective features as much as possible; this can reduce the redundancy of the feature vector, but will still discard some effective features. Therefore, the discrimination of actions and the recognition rate will be reduced. In response to the above problems, the weighted feature method based on the correlation between muscles and actions was proposed. The extracted original features of different channels are weighted differently. Therefore, the weighted features can be obtained. In the same action and different channels, the redundancy of features can be reduced. In different actions and the same channel, the discrimination of features can be increased.
At present, classification algorithms that are commonly used for the construction of sEMG signal classification models include linear discriminant analysis (LDA), the nonparameter estimation classifier K-nearest neighbors (KNN), SVM based on discriminant function analysis, convolutional neural networks (CNNs), etc. [17,18,19]. The LDA classifier finds the most suitable linear combination of data variables. Its calculation is easier and suitable for identifying the difference between linear samples, but it is not ideal for nonlinear classification [20]. KNN is easier to implement and has efficient computing power, but the classification recognition rate is greatly reduced when new samples of imbalanced data sets are input [21]. A CNN can automatically extract signal features and is suitable for high-dimensional data processing, but it easily falls into a local optimum. Moreover, the encapsulation of the feature extraction process increases the difficulty of improving the network [22]. SVM can solve the problems of high dimensionality and nonlinearity, and the processing process is transparent, and one of the links can be optimized and improved [23]. In view of the advantages and disadvantages of different classifiers, this paper selects SVM as the classifier and optimizes it by improving the genetic algorithm to improve the action recognition rate. The framework of the overall action recognition in this paper is shown in Figure 1, where the MAV is mean absolute value, RMS is root mean square, WA is Willision amplitude, EC1 is wavelet packet coefficient energy value, IGA-SVM is improved genetic algorithm–support vector machine classifier.

2. Methods

2.1. The Framing Energy Method Is Used to Extract the Main Feature Signals

The time for an ordinary person to complete a lower limb action is approximately 3 s. Therefore, each signal acquisition time should be as long as possible during the experiment to include all the effective signals of actions, but this will produce the disadvantages of a large amount of data and a useless resting signal. These disadvantages will increase the difficulty of signal processing. We extract the main feature segment from a complete signal to solve the problem. The main feature segment contains most of the effective information of actions and filters out of resting signals, so the timeliness of action recognition is improved [24].
During the experiment, the signal is divided into three segments: the initial segment, the main feature segment, and the ending segment. The initial segment is defined as the data segment from the beginning of the action to 0.5 s, the main feature segment is from the end point of the initial segment to the end of the following 2 s, and the ending segment follows the main feature segment. The ending segment is ignored and not processed. The initial point is detected by using the signal frame energy method [25,26]. We use 64 sampling points of the signal as the sliding window frame and 32 sampling points as the increment and calculate the adaptive threshold t h through the resting signal. After that, we calculate the energy value of each frame of the signal and record it as E n . Comparing E n with t h , when E n is greater than t h three times in a row, this frame is judged as the starting point of the initial segment. Suppose the signal is X ( n ) = x ( 1 ) , x ( 2 ) , . . . , x ( N ) , where N is the sum of the data length, and the calculation method of the signal framing energy method is as follows:
( 1 ) Select the appropriate frame length and frame shift to divide the normalized signal into frames:
( M 1 ) × I + L = N
where M is the total number of frames of the signal, I is the incremental frame step, L is the sampling point of each sliding window, namely, the frame length of each frame, and N is the total length of the signal. The signal after obtaining the split frame is X ( n ) = x ( 1 ) , x ( 2 ) , . . . , x ( N ) , where x ( 1 ) = x ( 1 ) , . . . , x ( L ) , x ( 2 ) = x ( 1 + I ) , . . . , x ( L + I ) , . . . .
( 2 ) Calculate the total energy of each frame signal:
E n ( i ) = i = 1 n + L a m p i n 2
where E n ( i ) is the total energy of the signal per frame and a m p i n is the amplitude of the n t h sampling point in frame i, with n < L .
( 3 ) Calculate the adaptive threshold t h based on the signal energy when standing steadily:
t h = i = 1 M E n ( i ) M
If E n is greater than t h in a certain frame and is greater than t h in the next three frames, then this frame is the starting frame of the signal action segment.
( 4 ) Obtain the main feature segment by extracting 2 s of data 0.5 s after the start point of the action:
S N = ( F S 1 ) × F s M S N = S N + 0.5 × F s M E N = M S N + 2 × F s
Among them, S N is the detected sampling point at the starting point, F S is the number of frames at the starting point, F s is the sampling frequency, M S N is the sampling point at 0.5 s after the starting point, and M E N is the sampling point at 2.5 s after the starting point. The signal from M S N to M E N is the signal of the main feature segment.

2.2. Feature Extraction from the Main Feature Segment Signal

sEMG signal analysis methods mainly include time domain analysis, frequency domain analysis, parameter modeling, time-frequency analysis, nonlinear dynamic analysis and so on. Through the analysis of the differences between the physiological responses of sEMG signals [27,28], this paper selects the features in Table 1 to recognize the lower limb actions. The overall trends of the selected features under different actions are similar, but the local responses are different. Therefore, we consider extracting, optimizing and fusing the features in Table 1 as the input of the lower limb action classifier.
The calculation methods of the four features in Table 1 are as follows:
s E M G M A V = 1 M j = 1 M ( 1 n 1 i = 1 n | x i | )
s E M G R M S = 1 M j = 1 M 1 n 1 i = 1 n x i 2
s E M G W A = 1 M j = 1 M ( i = 1 n 1 f ( | x i x i + 1 | ) )
s E M G E C 1 = log 10 ( 1 M j = 1 M | S j | 2 )
From (5)–(8), x i is the amplitude of the i-th sampling point of the sEMG signal, M is the total number of windows after signal framing, n is the length of the window, f ( x ) = 1 when x t h and f ( x ) = 0 otherwise, and S j is the wavelet packet coefficient of the j-th window.

2.3. Weighted Feature Method

sEMG signals are complex and nonstationary [31], so the difference in the extracted features is small. From the perspective of feature differences, first, we calculate the signal energy value s u m a b s X i of each muscle under different lower limb actions and record and establish the energy table. The energy value represents the contribution degree of each muscle to different actions. Second, according to the contribution degree, we calculate the correlation degree between muscles and actions and establish the correlation degree table. Finally, according to the correlation coefficient in the correlation table, the features of each channel are given different weights to complete the purpose of feature optimization. The correlation between muscles and actions indicates that each muscle plays a different role in different actions. As mentioned in Section 1, the feature dimensionality reduction method may cause the loss of some effective information. However, using feature fusion directly can cause feature redundancy. Therefore, the weighted feature method based on the correlation between muscles and actions is proposed. The extracted original features of different channels are weighted differently, and the weighted features can be obtained. In the same action and different channels, the redundancy of features is reduced. In different actions and the same channel, the discrimination of features is increased. The specific calculation method is as follows:
( 1 ) Remove the noise and baseline drift of the signal X to obtain X .
( 2 ) Take the absolute value of signal X to obtain a b s X .
( 3 ) Calculate the energy value of each channel, that is, the sum of amplitudes, and obtain s u m a b s X i , where i is 1 , 2 , . . . , n , and n is the total number of channels. The number of channels selected in this experiment is 8, so n = 8 .
( 4 ) Add up the energies of all channels to obtain s u m a b s X , where the weight of each channel can be calculated as follows:
C i = n × s u m a b s X i s u m a b s X , i = 1 , 2 , . . . , n
( 5 ) Therefore, F e a t u r e = i = 1 n f e a t u r e i × C i , where F e a t u r e is the fusion feature vector, and f e a t u r e i is the feature vector of the corresponding channel.

2.4. Improved Genetic Algorithm-Support Vector Machine Classifier

To achieve classification, SVM establishes a classification hyperplane as a decision surface to maximize the isolation edge between different classes [32]. This is the core idea of SVM. Therefore, SVM uses the Lagrange multiplier method to solve the classification process and introduces a penalty factor c, which is responsible for controlling the intensity of punishment for incorrect demarcation points. By controlling the distance between the wrong demarcation point and its correct position, we can keep it within a reasonable range. When the value of c is high, the occurrence of incorrect demarcation points will be greatly reduced, but overfitting will occur. When the value of c is very low, this will lead to a large number of incorrect demarcation points, which will cause the training model to be unreasonable. The performance of the kernel function selected by SVM also determines the accuracy of classification, the radial basis kernel function k ( x i , x j ) = e g | | x i x j | | 2 exhibits better performance [33], and the value of the kernel parameter g directly affects the prediction accuracy of the model. This paper combines a genetic algorithm to optimize the parameters c and g, constructs the population fitness based on the action recognition rate, and uses the ranking method on the basis of the championship method. For the sorting based on the fitness, the population is divided into four grades of good, well, medium, and bad, and the offspring are selected according to a certain proportion to carry out cross mutation and adaptively evolve the population to optimize the population and fitness. The specific implementation process is as follows:
( 1 ) Fusion feature vector:
E i = [ e i 1 , e i 2 , . . . , e i a ] , i = 1 , 2 , . . . , n is used to represent the feature sample vector of the sEMG signal, a is the dimension of the feature vector, and n is the number of samples. The fusion feature vector is then:
X = [ E 1 , E 2 , . . . , E m ] T
where m is the number of features, X is divided into training set X p and test set X T , X p is used to train the classifier, and X T is used to detect the classification effect of the classifier. Suppose there are m groups of samples in the training set X p , and the corresponding categories are Y p , i.e., X p = [ x 1 , x 2 , . . . , x m ] T and Y p = [ y 1 , y 2 , . . . , y m ] T , where y 1 means raise right leg, y 2 means lower right leg, y 3 means raise left leg, y 4 means lower left leg, y 5 means sitting to standing, and y 6 means standing to sitting.
( 2 ) Construct an improved genetic algorithm-support vector machine: GA-SVM uses the ability of the genetic algorithm to find the optimal solution to optimize the penalty factor c and the kernel parameter g, which will improve the classifier’s performance to a certain extent. However, the genetic algorithm will fall into a local optimal solution, resulting in limited improvement in the classification recognition rate. In this paper, on the basis of the characteristic of selecting the optimal solution via championship, the genetic algorithm is combined with the sorting method to improve GA-SVM, namely, IGA-SVM (improved genetic algorithm-support vector machine).
The championship execution process is to select a certain number of individuals from the parent population each time, then select the best one to enter the offspring population, and to repeat this operation until the new population reaches the size of the original population. Therefore, the main goal of the tournament selection operator is to find the optimal individual in the population. This has certain advantages, but it also presents the disadvantages of destroying the diversity of the population and reducing the ability to search for the population, thus causing the system to fall into a local optimal solution. Through integration with the sorting method, we establish a hierarchical elimination system, in which the individuals in the population are sorted according to their fitness, and the population is divided into four levels: bad, medium, well, and good. When the next generation is selected, the four levels of populations are selected according to a certain proportion. This not only ensures that the proportion of outstanding individuals in the population is large, but also maintains the diversity of the population and obtains the global optimal solution. The improvement steps are as follows:
( 1 ) The initial population number is determined, the fusion feature vector X p is used as the training data of the genetic algorithm, the fitness of the individuals in the population is calculated, and the matrix that records the parameters and fitness of the population is denoted as o l d p o p .
( 2 ) The individuals in the population are sorted from small to large according to their fitness values and recorded as matrix s o r t p o p .
( 3 ) The sorted individuals are divided into four grades: bad, medium, well, and good, expressed as C b a d , C m i d , C w e l l , C g o o d with C = C b a d + C m i d + C w e l l + C g o o d , where C is the total population number.
( 4 ) According to the principle that the good is chosen more and the bad chosen less, the diversity of the population should be considered as much as possible, and it should be ensured that all outstanding individuals exist. The four levels of bad, medium, well, and good are selected according to the probabilities of P , P + σ , P + 2 σ , P + 3 σ ( 0 < P < 1 , 0 < σ < 1 ) .
( 5 ) The individuals selected are recombined to obtain a new population n e w p o p . At this time, the individuals contained in the population are recorded as C n e w = [ C b a d × P ] + [ C m i d × ( P + σ ) ] + [ C w e l l × ( P + 2 σ ) ] + [ C g o o d × ( P + 3 σ ) ] .
( 6 ) Step (5) will discard a part of the population, resulting in an incomplete population matrix, so it is necessary to insert individuals into n e w p o p . The insertion principle adopts the principle of survival of the fittest. If C C n e w C g o o d , then C C n e w individuals are randomly selected from C g o o d to join the new population n e w p o p ; if C C n e w C g o o d , then all the individuals in C g o o d are input into the new population, and C C n e w C g o o d random individuals in C w e l l are selected to be input into n e w p o p so that the population number of n e w p o p is C, and cross-mutation is performed in n e w p o p to obtain better offspring.
( 7 ) The 5-fold cross-validation method is used to calculate the fitness of the population, and the fitness is the IGA population fitness accuracy rate. If the fitness accuracy rate does not reach the target value, steps (2)–(6) are repeated until the fitness reaches the target value, and the value of the penalty factor c and the kernel parameter g are recorded at this time to obtain b e s t c , b e s t g .
( 8 ) b e s t c , b e s t g are applied to the SVM to obtain the IGA-SVM training network, the best classification model C l a s s i f y m o d e l is trained through the training set, and the test set X T is input into C l a s s i f y m o d e l to obtain the best classification accuracy.

3. Results and Analysis

3.1. Experiment Procedure

At present, the known open source database does not contain the data of the six lower limb actions studied in this paper. Therefore, this paper designs an sEMG acquisition experiment. The DELSYS TrignoTM wireless sEMG signal acquisition instrument is used to acquire sEMG signals. The sampling frequency of this instrument can reach 2000 Hz. As shown in Figure 2, the hardware linked is as follows: the sEMG signals are acquired and transmitted by the sEMG sensors worn on the subjects. The sEMG signals are received by the signal receiving base station with wireless. The base station transmits the sEMG signals to PC with USB, and the PC is equipped with EMGworks Acquisition. According to the literature [34], the main sEMG features are distributed over 10–500 Hz. According to the body’s physiological structure and the distribution of leg muscles [35], we selected CH1-right leg rectus femoris, CH2-right biceps femoris, CH3-right tibial anterior muscle, CH4-right gastrocnemius, CH5-left leg rectus femoris, CH6-left biceps femoris, CH7-left tibialis anterior muscle, and CH8-left gastrocnemius as the signal acquisition sources, as shown in Figure 3. Because sEMG is susceptible to interference, volunteers needed to wipe the abovementioned muscles with alcohol before data collection and remove dander on the skin surface to reduce interference. In addition, volunteers should not exercise vigorously within 24 h before signal collection.
This paper studies six lower limb actions: raising the right leg, lowering the right leg, raising the left leg, lowering the left leg, sitting to standing, and standing to sitting, as shown in Figure 4. The subjects were three healthy (24–26)-year-old males (75 ± 5 kg, 175 ± 5 cm) and one healthy 25-year-old female (50 kg, 165 cm). None of them suffered from any neuromuscular system or joint diseases. All subjects volunteered to participate in this experiment and were informed of the experimental content before the experiment. To standardize the actions in the collection process, it was stipulated that when the signal was collected, subjects wearing the Fourier X2 and 8 sEMG sensors perform the above six types of lower limb actions. In the experiment process, each action is performed 500 times and each action lasted 10 s. Considering the sampling frequency of sEMG sensors is 2000 Hz, therefore, each time 8 channels acquire 8 × 20,000 data due to per channel acquires 20,000 data. These data are stored as a csv file, which is called a set of sEMG signals. By the end of experiment, 12,000 sets of sEMG signals can be obtained. Considering the fatigue of subjects during experiment, each time the subjects acquire signals within 10 min to ensure the quality of signals.

3.2. Analysis of Suraface Electromyography Physiological Characteristics in Action Recognition

The sEMG signal is a nonstationary, nonlinear, weak electrical signal that is very susceptible to interference from environmental noise and power frequency noise. Before feature extraction and subsequent processing, it is necessary to denoise the sEMG signal and use the improved wavelet denoising method [36] to filter out the signal noise, power frequency interference and baseline drift and obtain a smooth sEMG signal with less noise. The literature [36] has already introduced improved wavelet denoising in detail, so this paper will not go into details. Here, we consider subject No. 1’s raised right leg as an example to demonstrate the processing process. The comparison results before and after denoising are shown in Figure 5.
Due to the long signal length, this paper applies the signal framing energy method to obtain the main feature segment signal with a shorter data length. First, the energy threshold of the signal is calculated, and the sum of the energy in the resting state and the sum of the energy exerted in raising the right leg are obtained, as shown in Figure 6. Second, according to the adaptive threshold, by judging that the values in three consecutive frames are greater than the adaptive threshold, this point is selected as the starting point of the action, and the data 2.5 s after the starting point are extracted. In addition, the data 0.5 s after the starting point are selected as the main feature signal segment, as shown in Figure 7.

3.3. Analysis of the Weighted Feature Process

The time domain features and wavelet packet coefficient energy features of the main feature signals are extracted and normalized to the interval of (−1,1), and then the correlation coefficients between muscles and actions are calculated. First, extraction of the signal of the main feature segment, calculation of the energy value of each muscle during the six kinds of actions, and establishment of an energy table were performed. As an example, for subject No. 1, the energy value is calculated with steps (1)–(3) in Section 2.3, as shown in Table 2, where 1–4 represent the signal acquisition channels of the right rectus femoris, biceps femoris, tibialis anterior muscle, and gastrocnemius muscle, and 5–8 represent the signal acquisition channels of the left leg rectus femoris, biceps femoris, tibial anterior muscle, and gastrocnemius muscle. Second, according to Table 2, steps (4)–(5) in Section 2.3 are used to calculate the correlation coefficient between muscles and actions and to establish the table of correlations, as shown in Table 3. The greater the correlation coefficient is, the greater the correlation among the muscles and the actions.
For simplicity, we only select 50 sets of data for each of the six actions to show the energy features of the normalized wavelet packet coefficients, as shown in Figure 8. EC1 represents the energy features of wavelet packet coefficients, and WFM-EC1 represents the energy features of wavelet packet coefficients after optimization. The x-axis coordinates indicate A—raise right leg, B—lower right leg, C—raise left leg, D—lower left leg, E—sitting to standing, and F—standing to sitting, and it can be seen from the figure that EC1 has high feature redundancy and low discrimination under 6 actions with 8 channels. After WFM, WFM-EC1 has low redundancy and increased discrimination under different actions, which helps improve the recognition rate.

3.4. Analysis of the Optimization Process of the Improved Genetic Algorithm

The genetic algorithm is optimized through the combination of the championship method and sorting method. To expand the diversity and universality of the population, the population size is selected as C = 2000 , the crossover probability is 0.7, the mutation probability is 0.1, and the population selection probability parameter is set to P = 0.4 , σ = 0.2 . A total of 12,000 sets of data samples (2000 sets for each action) are selected from the collected data for preprocessing and feature extraction. Each action takes 1/2 of the sets of data as the training set, and the 5-fold cross-validation method is used to calculate the fitness accuracy of the population to obtain the best population newpop. The convergence process of the population is shown in Figure 9. The population evolution generation stops at 50 iterations, and the average fitness of IGA is significantly higher than that of GA.

4. Discussion

Each subject produced 250 groups for each action, a total of 6000 groups (1500 groups for each subject) were used as the training set data, and the remaining data were used as the test set to verify the action recognition rates of different methods. First, we use the traditional lower-limb action recognition method to identify and classify the six kinds of lower-limb actions. Inputting the fusion feature vector without WFM into GA-SVM, the classification results are shown in Table 4.
Second, the traditional GA-SVM classifier is used to classify the fusion feature vectors after WFM, and the results are shown in Table 5. By comparison with Table 4, the recognition rates of the six lower limb actions increase by 6.7%, 9.9%, 7.7%, 3.4%, 3.3%, and 5.9%, and the average recognition rate increases by 6.15%. The experimental results show that the WFM can improve the recognition rate of the six types of lower limb actions effectively in the data set of this paper.
Third, we input the fusion feature vector without WFM into IGA-SVM for classification, and the classification results obtained are shown in Table 6. By comparison with Table 4, the recognition rates of the six lower limb actions increase by 1.8%, 2.3%, 1%, 1.3%, 0.8% and 4.6%. The results indicate that the IGA-SVM can improve the recognition rate of lower limb actions slightly in the data set of this paper.
Finally, the IGA-SVM classifier is used to classify the WFM fusion feature vector, and the results are shown in Table 7. The average recognition rates of the six actions of raising the right leg, lowering the right leg, raising the left leg, lowering the left leg, sitting to standing, and standing to sitting are 94.3%, 95.9%, 96.2%, 96.6%, 94.5% and 91%, respectively, and the average recognition rate of all actions is 94.75%. By comparison with Table 4, the method proposed in this paper increases the recognition rates of the six lower limb actions by 9.3%, 13.3%, 10.3%, 7.8%, 6.1%, and 9.8%, and the average recognition rate is increased by 9.44%. The result suggests that the WFM and IGA-SVM have a good performance in lower limb recognition, and the improvement of recognition rate is relatively obvious in the data set of this paper. Meanwhile, Table 8 shows the classification results of the test set, the number of six lower limb actions recognized are 980, 984, 981, 981, 1075 and 999. It can be seen that the difference between the results is small. From Table 8, we can also see that the actions of recognizing errors are in two similar actions, such as action5 and action6. The reason for this result may be that both actions involve the same muscles and the sEMG signals acquired are correlated. To further improve the recognition rate, the correlation can be eliminated by improving the algorithm or fusing other signals.
To further prove the effectiveness of the method proposed in this paper, we select the most commonly used classification methods in action recognition, the BP neural network, LIBSVM, and KNN [37,38], to classify the experimental data. According to the experiment, the average classification recognition rates of the three classification methods for the six lower extremity actions are 79.74%, 81.22% and 80.69%, respectively. As shown in Figure 10, the recognition rates of the method proposed in this paper for the six lower limb actions are higher than those of the above three classification methods, so the method of this paper is more suitable for the classification of lower limb actions.

5. Conclusions

From the perspective of action recognition in lower limb rehabilitation training, taking the six actions of raising the right leg, lowering the right leg, raising the left leg, lowering the left leg, sitting to standing, and standing to sitting as the recognition targets of lower limb actions. We propose a lower limb action recognition method based on weighted features. First, after optimizing the features of the extracted main feature segment signal, the redundancy of the feature vector is reduced, and a feature vector with greater discrimination is obtained. Second, we design an improved genetic algorithm-support vector machine classifier, obtain the global optimal solution, and establish a multi-sEMG feature fusion action recognition model. After the WFM and IGA-SVM mentioned in this paper, the recognition rates of the six lower limb actions are 94.3%, 95.9%, 96.2%, 96.6%, 94.5%, and 91%, and the average recognition rate of all actions is 94.75%. Compared with the unimproved classification method and the commonly used classification method in Figure 10, WFM and IGA-SVM obtain better recognition accuracy and performance. The experimental results show that the weighted feature method of features and improved genetic algorithm-support vector machine classification can improve the recognition rate of lower limb actions in the data set of this paper.
The method designed in this paper is mainly aimed at improving the recognition rate of lower limb movements. The next step is to take into account non-ideal factors such as muscle fatigue and electrode offset to improve the recognition rate of lower limb actions under non-ideal conditions, and to apply the method to wearable lower limb exoskeleton robots to contribute to the field of rehabilitation treatment. In addition, similar actions, such as sitting and standing, may have different EEG signals. We will explore whether the recognition rate is improved by fusing EEG signals.

Author Contributions

Conceptualization, J.W. (Jiashuai Wang), D.C. and C.L.; methodology, J.W. (Jiashuai Wang); software, J.W. (Jiashuai Wang), J.W. ( Jinqiang Wang); validation, D.C. and C.L.; resources, D.C.; data curation, J.W. (Jiashuai Wang), J.W. ( Jinqiang Wang); writing—original draft preparation, J.W. (Jiashuai Wang); writing—review and editing, J.W. (Jiashuai Wang), D.C., C.L.; visualization, J.W. (Jiashuai Wang); supervision, D.C.; project administration, D.C.; funding acquisition, D.C. All authors have read and agreed to the published version of the manuscript.

Funding

This research was supported in part by the National Key Research and Development Program of China under Grant 2018YFC2001704, in part by the National Natural Science Foundation of China under Grant 61703232 and Grant 62073187, in part by the Major Scientific and Technological Innovation Project in Shandong Province under Grant 2019JZZY011111, in part by the Science and Technology Innovation Special Plan Project of Rizhao under Grant 2019CXZX2212, and in part by the Jining City Key Research and Development Program Project under Grant 2020JKNS004.

Institutional Review Board Statement

All subjects gave their informed consent for inclusion before they participated in the study. The study was conducted in accordance with the Declaration of Helsinki, and the protocol was approved by the biomedical Ethics Committee of Qufu Normal University (2021-053).

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Wang, J.; Sun, Y.; Sun, S. Recognition of Muscle Fatigue Status Based on Improved Wavelet Threshold and CNN-SVM. IEEE Access 2020, 8, 207914–207922. [Google Scholar] [CrossRef]
  2. De Luca, C.J. Physiology and Mathematics of Myoelectric Signals. IEEE Trans. Biomed. Eng. 1979, BME-26, 313–325. [Google Scholar] [CrossRef] [PubMed]
  3. Devries, H.A.; Moritani, T.; Nagata, A.; Magnussen, K. The relation between critical power and neuromuscular fatigue as estimated from electromyographic data. Ergonomics 1982, 25, 783–791. [Google Scholar] [CrossRef] [PubMed]
  4. Nielsen, J.L.G.; Holmgaard, S.; Jiang, N.; Englehart, K.B.; Farina, D.; Parker, P.A. Simultaneous and Proportional Force Estimation for Multifunction Myoelectric Prostheses Using Mirrored Bilateral Training. IEEE Trans. Biomed. Eng. 2011, 58, 681–688. [Google Scholar] [CrossRef] [PubMed]
  5. Mora-Tola, E.; Loja-Duchi, J.; Ordoñez-Torres, A.; Vázquez-Rodas, A.; Astudillo-Salinas, F.; Minchala, L.I. Robotic Knee Exoskeleton Prototype to Assist Patients in Gait Rehabilitation. IEEE Lat. Am. Trans. 2020, 18, 1503–1510. [Google Scholar] [CrossRef]
  6. Palermo, F.; Cognolato, M.; Gijsberts, A.; Müller, H.; Caputo, B.; Atzori, M. Repeatability of grasp recognition for robotic hand prosthesis control based on sEMG data. In Proceedings of the 2017 International Conference on Rehabilitation Robotics (ICORR), London, UK, 17–20 July 2017; pp. 1154–1159. [Google Scholar]
  7. Atzori, M.; Gijsberts, A.; Castellini, C.; Caputo, B.; Hager, A.G.M.; Elsig, S.; Giatsidis, G.; Bassetto, F.; Muller, H. Electromyography data for non-invasive naturally-controlled robotic hand prostheses. Sci. Data 2014, 1, 140053. [Google Scholar] [CrossRef] [Green Version]
  8. Toledo-Perez, D.C.; Martinez-Prado, M.A.; Gomez-Loenzo, R.A.; Paredes-Garcia, W.J.; Rodriguez-Resendiz, J. A study of movement classification of the lower limb based on up to 4-EMG channels. Electronics 2019, 8, 259. [Google Scholar] [CrossRef] [Green Version]
  9. Khan, T.; Lundgren, L.E.; Järpe, E.; Olsson, M.C.; Viberg, P. A novel method for classification of running fatigue using change-point segmentation. Sensors 2019, 19, 4729. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  10. Al-Mulla, M.R.; Sepulveda, F.; Colley, M. A review of non-invasive techniques to detect and predict localised muscle fatigue. Sensors 2011, 11, 3545–3594. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  11. Ai, Q.; Zhang, Y.; Qi, W.; Liu, Q.; Chen, A.K. Research on lower limb motion recognition based on fusion of sEMG and accelerometer signals. Symmetry 2017, 9, 147. [Google Scholar] [CrossRef]
  12. Xi, X.; Tang, M.; Miran, S.M.; Luo, Z. Evaluation of Feature Extraction and Recognition for Activity Monitoring and Fall Detection Based on Wearable sEMG Sensors. Sensors 2017, 17, 1229. [Google Scholar] [CrossRef]
  13. Ebied, A.; Kinney-Lang, E.; Spyrou, L.; Escudero, J. Muscle Activity Analysis Using Higher-Order Tensor Decomposition: Application to Muscle Synergy Extraction. IEEE Access 2019, 7, 27257–27271. [Google Scholar] [CrossRef]
  14. Shi, X.; Qin, P.; Zhu, J.; Zhai, M.; Shi, W. Feature Extraction and Classification of Lower Limb Motion Based on sEMG Signals. IEEE Access 2020, 8, 132882–132892. [Google Scholar] [CrossRef]
  15. Hajian, G.; Etemad, A.; Morin, E. An Investigation of Dimensionality Reduction Techniques for EMG-based Force Estimation. In Proceedings of the 41st Annual International Conference of the IEEE engineering in Medicine and Biology Society (EMBC), Berlin, Germany, 23–27 July 2019; pp. 698–701. [Google Scholar]
  16. Zhonghua, L.; Zhihui, L.; Weihua, O.; Kaibing, Z.; Ruijuan, Z. Structured optimal graph based sparse feature extraction for semi-supervised learning. Signal Process. 2020, 170, 107456. [Google Scholar]
  17. Liang, S.; Chen, J.; Wu, Y.; Yan, S.; Huang, J. Recognition of Subtle Gestures by 2-Channel sEMG Using Parameter Estimation Classifiers Based on Probability Density. IEEE Access 2020, 8, 169835–169850. [Google Scholar] [CrossRef]
  18. Yang, C.; Xi, X.; Chen, S.; Miran, S.M.; Hua, X.; Luo, Z. SEMG-based multifeatures and predictive model for knee-joint-angle estimation. AIP Adv. 2019, 9, 065124. [Google Scholar] [CrossRef]
  19. Shi, W.T.; Lyu, Z.J.; Tang, S.T.; Chia, T.L.; Yang, C.Y. A bionic hand controlled by hand gesture recognition based on surface EMG signals: A preliminary study. Biocybern. Biomed. Eng. 2018, 38, 126–135. [Google Scholar] [CrossRef]
  20. Xing, K.; Yang, P.; Huang, J.; Wang, Y.; Zhu, Q. A real-time EMG pattern recognition method for virtual myoelectric hand control. Neurocomputing 2014, 136, 345–355. [Google Scholar] [CrossRef]
  21. Wang, S.; Zhang, L. Regularized Deep Transfer Learning: When CNN Meets kNN. IEEE Trans. Circuits Syst. II Express Briefs 2020, 67, 2269–2273. [Google Scholar] [CrossRef]
  22. Bao, T.; Zaidi, S.A.R.; Xie, S.; Yang, P.; Zhang, Z.-Q. Inter-Subject Domain Adaptation for CNN-Based Wrist Kinematics Estimation Using sEMG. IEEE Trans. Neural Syst. Rehabil. Eng. 2021, 29, 1068–1078. [Google Scholar] [CrossRef]
  23. Liu, X.; Cheng, X.; Lee, K. GA-SVM-Based Facial Emotion Recognition Using Facial Geometric Features. IEEE Sens. J. 2021, 21, 11532–11542. [Google Scholar]
  24. Turgunov, A.; Zohirov, K.; Muhtorov, B. A new dataset for the detection of hand movements based on the SEMG signal. In Proceedings of the IEEE 14th International Conference on Application of Information and Communication Technologies (AICT), Tashkent, Uzbekistan, 7–9 October 2020; pp. 1–4. [Google Scholar]
  25. Hu, Y.; Wong, Y.; Dai, Q.; Kankanhalli, M.; Geng, W.; Li, X. sEMG-Based Gesture Recognition with Embedded Virtual Hand Poses and Adversarial Learning. IEEE Access 2019, 7, 104108–104120. [Google Scholar] [CrossRef]
  26. Lam, Y.; Leong, P.H.; Mak, M. Frame-Based SEMG-to-Speech Conversion. In Proceedings of the 49th IEEE International Midwest Symposium on Circuits and Systems, San Juan, PR, USA, 6–9 August 2006; pp. 240–244. [Google Scholar]
  27. Wang, G.; Yan, Z.G.; Hu, X.; Xie, H.B.; Wang, Z.Z. Classification of surface EMG signals using harmonic wavelet packet transform. Physiol. Meas. 2006, 27, 1255–1267. [Google Scholar] [CrossRef]
  28. Zuo, L.; Sun, H.; Mao, Q.; Liu, X.; Jia, R. sNoise Suppression Method of Microseismic Signal Based on Complementary Ensemble Empirical Mode Decomposition and Wavelet Packet Threshold. IEEE Access 2019, 7, 176504–176513. [Google Scholar] [CrossRef]
  29. Zhao, M.; Kang, M.; Tang, B.; Pecht, M. Multiple Wavelet Coefficients Fusion in Deep Residual Networks for Fault Diagnosis. IEEE Trans. Ind. Electron. 2019, 66, 4696–4706. [Google Scholar] [CrossRef]
  30. Zheng, S. Research on Pattern Classification of Arm Motion Surface EMG Signal Based on Support Vector Machine. Master’s Thesis, Jilin University, Changchun, China, 2014. [Google Scholar]
  31. Moniri, A.; Terracina, D.; Rodriguez-Manzano, J.; Strutton, P.H.; Georgiou, P. Real-Time Forecasting of sEMG Features for Trunk Muscle Fatigue Using Machine Learning. IEEE Trans. Biomed. Eng. 2020, 68, 718–727. [Google Scholar] [CrossRef]
  32. Melgani, F.; Bazi, Y. Classification of electro-cardiogram signals with support vector machines and particle swarm optimization. IEEE Trans. Inf. Technol. Biomed. 2008, 12, 667–677. [Google Scholar] [CrossRef] [PubMed]
  33. Joarder, K.; Begg, R.K. Support vector machines and other pattern recognition approaches to the diagnosis of cerebral palsy gait. IEEE Trans. Biomed. Eng. 2007, 53, 2479–2490. [Google Scholar]
  34. Yang, S.; Chai, Y.; Ai, J.; Sun, S.; Liu, C. Hand Motion Recognition Based on GA Optimized SVM Using sEMG Signals. In Proceedings of the 11th International Symposium on Computational Intelligence and Design (ISCID), Hangzhou, China, 8–9 December 2018; pp. 146–149. [Google Scholar]
  35. Cui, C.; Bian, G.; Hou, Z.; Zhao, J.; Zhou, H. A multimodal framework based on integration of cortical and muscular activities for decoding human intentions about lower limb motions. IEEE Trans. Biomed. Circuits Syst. 2017, 11, 889–899. [Google Scholar] [CrossRef]
  36. Li, C.; Cao, D.; Yuan, Y. Research on Improved Wavelet Denoising Method for sEMG Signal. In Proceedings of the Chinese Automation Congress (CAC), Hangzhou, China, 22–24 November 2019; pp. 5221–5225. [Google Scholar]
  37. Su, B.-Y.; Wang, J.; Liu, S.-Q.; Sheng, M.; Jiang, J.; Xiang, K. A CNN-Based Method for Intent Recognition Using Inertial Measurement Units and Intelligent Lower Limb Prosthesis. IEEE Trans. Neural Syst. Rehabil. Eng. 2019, 27, 1032–1042. [Google Scholar] [CrossRef]
  38. Chen, L.; Fu, J.; Wu, Y.; Li, H.; Zheng, B. Hand gesture recognition using compact CNN via surface electromyography signals. Sensors 2020, 20, 672. [Google Scholar] [CrossRef] [PubMed] [Green Version]
Figure 1. Action recognition framework.
Figure 1. Action recognition framework.
Sensors 21 06147 g001
Figure 2. Hardware link.
Figure 2. Hardware link.
Sensors 21 06147 g002
Figure 3. Graph of muscle positions.
Figure 3. Graph of muscle positions.
Sensors 21 06147 g003
Figure 4. The diagram of 6 actions.
Figure 4. The diagram of 6 actions.
Sensors 21 06147 g004
Figure 5. Comparison results of the time domain and frequency domain before and after wavelet denoising.
Figure 5. Comparison results of the time domain and frequency domain before and after wavelet denoising.
Sensors 21 06147 g005
Figure 6. Acquisition of the data from the signal action starting point to the end point of the main feature segment.
Figure 6. Acquisition of the data from the signal action starting point to the end point of the main feature segment.
Sensors 21 06147 g006
Figure 7. Obtaining the main feature segment of the signal.
Figure 7. Obtaining the main feature segment of the signal.
Sensors 21 06147 g007
Figure 8. Comparison of energy features of wavelet packet coefficients of 8 channels before and after WFM.
Figure 8. Comparison of energy features of wavelet packet coefficients of 8 channels before and after WFM.
Sensors 21 06147 g008
Figure 9. Convergence curve comparison chart of the population average fitness.
Figure 9. Convergence curve comparison chart of the population average fitness.
Sensors 21 06147 g009
Figure 10. Comparison of recognition rates of different classification methods.
Figure 10. Comparison of recognition rates of different classification methods.
Sensors 21 06147 g010
Table 1. Action recognition-related physiological features of sEMG signals.
Table 1. Action recognition-related physiological features of sEMG signals.
FeaturesFeature Description
s E M G M A V Mean absolute value
s E M G R M S Root mean square
s E M G W A Willison amplitude
s E M G E C 1 Wavelet packet coefficient energy [29,30]
Table 2. Energy table.
Table 2. Energy table.
Energy ValueChannel of the Acquisition Signal
12345678
action14.40757.39953.38533.02902.504710.03376.07796.1198
action25.61254.250916.77692.20351.98995.00362.76656.7932
action32.77136.85334.11978.55123.745216.54965.36843.8048
action43.77452.88122.60028.20394.289425.128212.03142.5562
action531.336913.815023.48562.139818.227214.990524.09172.6756
action623.302412.594725.43072.079315.195819.699129.05333.3588
Table 3. Correlation between muscles and actions.
Table 3. Correlation between muscles and actions.
Correlation CoefficientChannel of the Acquisition Signal
12345678
action10.82081.37800.63040.56410.46641.86861.13191.1397
action20.98900.74912.95650.38830.35070.88170.48751.1971
action30.42831.05920.63671.32160.57882.55770.82970.5880
action40.49130.37500.33841.06780.55833.27061.56590.3327
action51.91720.84521.43680.13091.11510.91711.47390.1637
action61.42620.77081.55640.12730.93001.20561.77810.2056
Table 4. Recognition rate of the fusion feature vector without the WFM using GA-SVM classification.
Table 4. Recognition rate of the fusion feature vector without the WFM using GA-SVM classification.
ActionsSubject IDAverage Accuracy
1234
action186.885.275.692.485
action2848079.686.882.6
action39481.685.282.885.9
action494.893.687.679.288.8
action592.495.681.284.488.4
action666.476.891.69081.2
Average accuracy86.485.4683.4685.9385.31
Table 5. Recognition rate of the fusion feature vector with the WFM using GA-SVM classification.
Table 5. Recognition rate of the fusion feature vector with the WFM using GA-SVM classification.
ActionsSubject IDAverage Accuracy
1234
action195.293.29286.491.7
action294.89490.890.492.5
action396.895.293.289.293.6
action494.49295.287.292.2
action595.289.29092.491.7
action684.486.484.493.287.1
Average accuracy93.4691.6690.9389.891.46
Table 6. Recognition rate of the fusion feature vector without the WFM using IGA-SVM classification.
Table 6. Recognition rate of the fusion feature vector without the WFM using IGA-SVM classification.
ActionsSubject IDAverage Accuracy
1234
action190.486.476.49486.8
action288.485.2788884.9
action394.481.687.284.486.9
action495.29489.28290.1
action591.295.28486.489.2
action670.886.893.292.485.8
Average accuracy88.488.284.6687.8687.28
Table 7. Recognition rate of the fusion feature vector with the WFM using IGA-SVM classification.
Table 7. Recognition rate of the fusion feature vector with the WFM using IGA-SVM classification.
ActionsSubject IDAverage Accuracy
1234
action197.295.295.689.294.3
action297.696.494.894.895.9
action39897.69693.296.2
action498.496.898.892.496.6
action597.69293.295.294.5
action687.691.688.89691
Average accuracy96.0694.9394.5393.4694.75
Table 8. The results statistics of the fusion feature vector with the WFM using IGA-SVM classification.
Table 8. The results statistics of the fusion feature vector with the WFM using IGA-SVM classification.
Actual ActionsPredicted ActionsTotal Number of Actions
action1action2action3action4action5action6
action194326324221000
action218959311631000
action309962018111000
action45009667221000
action5140100945311000
action6014011659101000
Total number of actions98098498198110759996000
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Wang, J.; Cao, D.; Wang, J.; Liu, C. Action Recognition of Lower Limbs Based on Surface Electromyography Weighted Feature Method. Sensors 2021, 21, 6147. https://doi.org/10.3390/s21186147

AMA Style

Wang J, Cao D, Wang J, Liu C. Action Recognition of Lower Limbs Based on Surface Electromyography Weighted Feature Method. Sensors. 2021; 21(18):6147. https://doi.org/10.3390/s21186147

Chicago/Turabian Style

Wang, Jiashuai, Dianguo Cao, Jinqiang Wang, and Chengyu Liu. 2021. "Action Recognition of Lower Limbs Based on Surface Electromyography Weighted Feature Method" Sensors 21, no. 18: 6147. https://doi.org/10.3390/s21186147

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop