1. Introduction
The deployment of electric powered wheelchairs (EPWs) has been increased rapidly for better quality of life for handicapped and elderly people over the last 20 years [
1]. However, most of these EPWs are traditionally controlled by joysticks, which are not suitable for people with severe physical disabilities like spinal cord injury or hemiplegia. Therefore, alternative HMI methods for wheelchair operation have been studied, such as voice recognition [
2], eye moment [
3], head gesture [
4] and biopotential signals interface [
5,
6,
7,
8]. Bio-potential signals such as Electrooculargraphic (EOG), electromyographic (EMG) and electroencephalographic (EEG) are also deployed to control such kind of smart service robot [
9,
10,
11]. Compared to other signals, EMG signal is more easily obtained and has less signal interference, especially for the weak groups, such as patients with upper limb amputation and elderly people for which other methods are inconvenient for their daily life.
The EMG signal, which is sampled from the skin’s surface, can directly reflect the movement intention of human body and could be very suitable to serve as the controller. Englehart and Hudgins investigated a continuous segmentation or continuous classification by using different analysis window lengths [
12]. Moon used a preset threshold value method to distinct the scapulae muscle signal at fairly high recognition rates [
13]. Hiroki Tamura
et al. obtained s-EMG signals from facial movements and six patterns were recognized to control the wheelchair movements [
14]. Lai Wei combined forehead sEMG signals and color face image information to control the smart robot [
5]. However, it suffers from poor reliability and robustness in long-term operations, which were mainly caused by muscle fatigue. These may arise not only because of peripheral changes at the level of the muscle, but also because the central nervous system fails to drive the motoneurons adequately [
15]. Usually, the amplitude of the sEMG signals from muscles we used without fatigue is lower than 400 μV. However, after fatigue occurs in muscles, there are significant rises to 600 μV, which can be seen
Figure 1. Qin and Mitsuhiro investigate a torque estimation method for muscle fatigue tracking, using stimulus evoked electromyography in the context of a functional electrical stimulation rehabilitation system [
16]. However, how to choose the best signal evaluation parameter according to specific muscle condition and method of reducing the influence of muscle fatigue is still not very clear, and has been discussed during recent decades [
17,
18].
Figure 1.
The amplitude of surface electromyography (sEMG) signals.
Figure 1.
The amplitude of surface electromyography (sEMG) signals.
To investigate the real time and long time performance of the proposed system in this paper, forehead sEMG signals are first obtained and analyzed by using three dry electrodes sensor. Then, a four-order autoregressive model parameter algorithm is deployed to acquire data compression and extract four coefficients of the signals. An incremental online support vector machine (SVM) is employed to classify sEMG patterns that have the systematic parameters corresponding to the changes of the signals during the running time. Furthermore, the improved incremental training algorithm is presented and applied on the HMI system.
2. The Research Objects and Methods
The sEMG signal controlled intelligent wheelchair should provide a high degree of intuitive and dexterous control, and offer a convenient and high level of performance for disabled people. The human face has plenty of facial muscles to make expressions. The frontalis muscle, located on the front of the head, is one of the largest muscles which can provide continual and distinct, recognizable signals. The main components of the system include sEMG signals collected by electrodes placed on the skin over a user’s forehead muscle, electrodes accompanied by miniature pre-amplifiers to differentiate small signals of interest, signal features detected after amplified, filtered, digitized via standard EMG instruments, and finally transferred to a wheelchair controller. Five subjects voluntarily participated in the experiment and have independent consciousness without facial disease, deformity or trauma. In addition, there was no muscle fatigue before the experiments were performed.
The experimental system consists of an intelligent wheelchair with two independently driven motors, a laptop computer for human-machine interface, and a data acquisition device. A Cyberlink or NIA system is used to acquire the sEMG signals of the subjects. The interface unit has an amplifier with a gain of 50,000, and a filter system provides a band pass filter with a bandwidth of 0.5 to 45 Hz. The weak bioelectricity signals from forehead surface skin are adopted from the user by wearing a sensory headband. The interface box processes the signals and connects to a computer through a serial or USB port.
Figure 2 shows the headband devices we used in the experimental system.
Figure 2.
The headband EMG device in the human-machine interaction (HMI) system.
Figure 2.
The headband EMG device in the human-machine interaction (HMI) system.
The muscle movements are defined to be control signals for a HMI system, namely single jaw click (SJC) and double jaw click (DJC). They are generated by the subjects contracting the masseter muscle and buccinator muscle with a jaw clenching and chewing-like movement. Every subject uses a designed sEMG pattern based HMI system to control an intelligent wheelchair, and the control signals are recorded at the same time. There are five control states designed for the wheelchair, namely: “Forward”, “Backward”, “Left”, “Right”, and “Stop”. Note that the first four states are scanned in turn. Users can use a single jaw click movement to switch the direction to whereas they want. If the wheelchair is in the control state of “Forward”, “Backward”, “Left”, or “Right”, a double jaw click movement will be used to switch the wheelchair to the “Stop” state. The complete structure of the HMI system is depicted in
Figure 3.
Figure 3.
The diagram of the proposed HMI system.
Figure 3.
The diagram of the proposed HMI system.
4. Adaptive Incremental Online SVM
sEMG patterns gradually change during long durations of muscle activities due to muscle fatigue, so the threshold method will lose its effectiveness. As we know, a SVM classifier constructs an optimal separating hyperplane in a high dimension feature space of training data that is mapped using a nonlinear kernel function [
23,
24]. The key ideas of SVM are: getting a maximized classification margin by solving dual optimization problem, and then obtaining an optimal separating hyperplane.
If the data is linearly separable, the decision function is:
where {
xi, yi},
i = 1,…,
l, are the training samples,
yi is the class label of the training sample
xi,
l is the number of the training samples,
b is the offset.
When the data is nonlinearly separable, the original data is mapped into a high dimensional feature space by kernel function
K(
xi, x), in which the mapped data is linearly separable. Then, the decision function is as follows:
According to the Karush-Kuhn-Tucke (KKT) condition [
25], there is only one solution for the optimization problem based on the quadratic optimization theory, and
α = [
α1,
α2,…,
αl] is the optimal solution for the dual problem. When it satisfies the KKT condition, the problem is divided into the following three situations:
where
α is Lagrangian dual,
c is an upper bound for samples that lie on the wrong side of the hyperplane. Nonzero constant
αi is the sample corresponding to support vector. Samples are located on the classification interval
0 < αi < c. When
αi = c, samples lie in the classification interval.
Incremental support vector machines are instrumental in practical applications of online learning. The purpose of the incremental algorithm is to add the new sample data to the three sets and to delete the redundant data, while maintaining all other samples that still belong to the three sets. In the classical SVM, the hyperplane in m dimensions can be shown as wT∙x + b = 0. In our updating samples method, only the samples that are closest to the current boundary are chosen to train the classifier. There is a maximum distance of all update samples to boundary, which is defined as 2δ.
If the new added training sample violates the KKT conditions, the Lagrange multiplier α that is obtained from the previous sample will not be the optimum solution. The proposed improved online incremental training algorithm with the updating method is as follows. The standard SVM is used to initialize the classification and a training set is obtained. The next step is choosing the most informative patterns that have the best possibility of becoming the support vectors in the in-put sEMG sequence, and checking whether these samples meet the KKT conditions. If not, try the next data until KKT conditions are satisfied. Then construct samples including the update data, and train samples online to form a new classifier. The process repeats until there is no in-put sEMG data.
5. Experiment Results
Traditional SVM, such as LIBSVM, employed a radial basis function kernel to classify data, with the same parameter (
C = 1 and
γ = 0.5) [
26]. The data is normalized and limited to the range of −1 to 1. 800 samples are randomly selected from 2,000 pre-collected signals as a training set. After that, choose 800 samples in the rest samples randomly as a testing set. When the result is compared with the method given above, the classification accuracy is shown in
Table 2. The training time was generated at 300 samples.
Table 2.
Comparison real time performance of the LIBSVM and the Incremental online support vector machine (SVM).
Table 2.
Comparison real time performance of the LIBSVM and the Incremental online support vector machine (SVM).
| LIBSVM | Incremental online SVM | |
---|
Single jaw click | 73% | 90% | recognition rate |
92 ms | 58 ms | training time |
Double jaw click | 81% | 93% | recognition rate |
103 ms | 62 ms | training time |
Note that five subjects in the experiment are required to control the intelligent wheelchair running throughout a specified path in the lab. A map of the experiment environment with the designed path is given in
Figure 5. The size of the map is 8,000 × 6,000 mm and the actual size of the wheelchair is 1,000 × 700 mm. “Forward”, “Backward”, “Left”, “Right”, and “Stop”: we tried to generate an average for each control state over two experiment days. For each experiment, the five subjects are required to control the intelligent wheelchair for 90 min without a rest.
Figure 5.
The path designed for experiment.
Figure 5.
The path designed for experiment.
Figure 6 shows the average time consumptions of five subjects in repetitive experiments during deferent periods. The average consuming time is 103 s for one round at the beginning, but the average time increased to 154 s at the last period when muscle fatigue occurred. After using the new method, the average time at the last period could decrease to 110 s.
Table 3 shows the average recognition rate of five subjects controlling the wheelchair using the proposed HMI system with the two online SVM in different periods. The recognition of jaw click, or what is called “single jaw click” and “double jaw click”, is the most important factor of the experiment.
Figure 6.
The five subjects’ average time consumptions of repetitive experiments during different periods.
Figure 6.
The five subjects’ average time consumptions of repetitive experiments during different periods.
Table 3.
Comparison long time performance of the LIBSVM and the Incremental online SVM.
Table 3.
Comparison long time performance of the LIBSVM and the Incremental online SVM.
Subjects | Recognition rate for five state using LIBSVM | Recognition rate for five state using incremental online SVM in seconds |
---|
0–20 min | 20–40 min | 40–60 min | 0–20 min | 20–40 min | 40–60 min |
---|
A | 86.5 | 80.1 | 73.8 | 92.6 | 88.4 | 86.7 |
B | 88.7 | 81.6 | 73.3 | 92.8 | 89.5 | 87.4 |
C | 83.1 | 79.2 | 70.9 | 86.7 | 82.8 | 79.3 |
D | 90.4 | 85.5 | 79.2 | 93.6 | 90.1 | 88.3 |
E | 80.8 | 75.3 | 70.1 | 84.9 | 81.6 | 78.4 |
Average | 85.9 | 80.34 | 73.46 | 90.12 | 86.48 | 84.02 |
Experimental results indicate that the performance of the sEMG based HMI system with LIBSVM gradually declines as time increases with the occurrence of muscle fatigue. The subjects control the intelligent wheelchair by this system using more time in different periods, as shown in
Table 3. Also, one should note that, although the threshold EMG method has a pretty high recognition rate before muscle fatigue, the performance after that declines quickly. The incremental online SVM proposed showed good properties in both convergence and optimality.
6. Conclusions
Muscle fatigue can negatively influence the performance of human-machine interaction using sEMG signals. This paper presents a novel sEMG based HMI system for the hands-free control of an intelligent wheelchair both in real time and in long time operations. The system is used to control the simple movements of the intelligent wheelchair, i.e., forward, turn-left, turn-right, backward and stop. The experimental results show that the proposed HMI is easy to operate in real-time and has a high recognition rate. It could be deployed in many aspects of real world applications, such as helping the elderly and disabled, rehabilitation, and so on.
The speed and accuracy of pattern recognition of forehead sEMG signals are to be further improved in our next stage of research. By improving the incremental online SVM algorithm, we hope to reduce the false recognition rate, and obtain a fast response time. In addition, we will investigate how to effectively reduce the noisy signals from facial movements when a user is talking and looking around in the practical application. Moreover, Electrooculography (EOG) signals will be combined with EMG signals to generate the control signals for wheelchair motion control so that the control method is more flexible and robust.
Finally, it is worth remarking that Brain Computer Interface (BCI) based on the Electrooculography (EEG) signals have been used to control the electric-powered wheelchair during the last decade [
17,
18]. However, these EEG based HMIs are not performing well in the real world since they are non-stationary and low in signal to noise ratio. This was mainly caused by the subject’s brain conditions or dynamically changing environments. Therefore, it is necessary to carefully design signal processing algorithms for feature extraction, classification and adaptation. Alternatively, EEG signals could be fused with EMG and EOG signals for accurate human intension detection in real time.