Next Article in Journal
Transformation Method for Solving System of Boolean Algebraic Equations
Next Article in Special Issue
Generalization of Neural Networks on Second-Order Hypercomplex Numbers
Previous Article in Journal
Assessment of Complex System Dynamics via Harmonic Mapping in a Multifractal Paradigm
Previous Article in Special Issue
Learning Neural Representations and Local Embedding for Nonlinear Dimensionality Reduction Mapping
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Imaginary Finger Movements Decoding Using Empirical Mode Decomposition and a Stacked BiLSTM Architecture

by
Tat’y Mwata-Velu
1,
Juan Gabriel Avina-Cervantes
1,
Jorge Mario Cruz-Duarte
2,
Horacio Rostro-Gonzalez
1 and
Jose Ruiz-Pinales
1,*
1
Telematics and Digital Signal Processing Research Groups (CAs), Department of Electronics Engineering, University of Guanajuato, Salamanca 36885, Mexico
2
Tecnológico de Monterrey, Av. Eugenio Garza Sada 2501 Sur, Col. Tecnológico, Monterrey 64849, Mexico
*
Author to whom correspondence should be addressed.
Mathematics 2021, 9(24), 3297; https://doi.org/10.3390/math9243297
Submission received: 26 October 2021 / Revised: 7 December 2021 / Accepted: 16 December 2021 / Published: 18 December 2021

Abstract

:
Motor Imagery Electroencephalogram (MI-EEG) signals are widely used in Brain-Computer Interfaces (BCI). MI-EEG signals of large limbs movements have been explored in recent researches because they deliver relevant classification rates for BCI systems. However, smaller and noisy signals corresponding to hand-finger imagined movements are less frequently used because they are difficult to classify. This study proposes a method for decoding finger imagined movements of the right hand. For this purpose, MI-EEG signals from C3, Cz, P3, and Pz sensors were carefully selected to be processed in the proposed framework. Therefore, a method based on Empirical Mode Decomposition (EMD) is used to tackle the problem of noisy signals. At the same time, the sequence classification is performed by a stacked Bidirectional Long Short-Term Memory (BiLSTM) network. The proposed method was evaluated using k-fold cross-validation on a public dataset, obtaining an accuracy of 82.26%.

1. Introduction

Latterly, the applications of EEG-based BCIs have been the subject of increased research and development [1,2]. One of the most critical applications is that people with motor disabilities can control a wheelchair [3], artificial limbs, or mobile robots [4]. Other significant applications are in medicine. For example, Brain-Computer Interface (BCI) spellers using code-modulated Visual Evoked Potentials (cVEP) help patients with Amyotrophic Lateral Sclerosis (ALS) to cope with their cognitive impairments [5], and a BCI to detect sleep apnea based on EEG analysis [6].
Specifically, Motor Imagery (MI) is widely used in the current BCI systems where improvements to MI-BCI systems are constantly proposed as a finger rehabilitation system decoded the movement of the right-hand index finger [7], and the Capsule Network (CapsNet) [8]. These applications highlighted the effectiveness of pattern recognition methods based on MI-EEG signals [9]. MI-EEG signals are generated when a subject imagines a body limb movement. Such signals are especially captured in the sensorimotor cortex [10,11], located in the posterior area in the frontal lobe, involved in imagined movements and muscle control [12].
The sensorimotor cortex produces similar activation patterns during the imagined and physical movement according to the synchronization and desynchronization of the mu-rhythm [13]. Li Feng et al. implemented a left- and right-hand MI-EEG signal classifier for a BCI using a Continuous Wavelet Transform (CWT) and a simplified Convolutional Neural Network (CNN) [14]. Tyagi and Nehra analyzed and extracted relevant features from MI-EEG signals using Principal Component Analysis (PCA) and Linear Discriminant Analysis (LDA) [15]. Afterward, the features were classified using Support Vector Machines (SVM), k-Nearest Neighbors (k-NN), and an Artificial Neural Network (ANN).
EEG signals measured by non-invasive systems are difficult to classify because of noise added in the sensors, which leads to a low Signal-to-Noise Ratio (SNR). Besides, the electrodes placed physically over the scalp produce a loss of signal potential due to volume conduction effects [16]. In all, the processing and classification of MI-EEG signals generated during the imagined finger movements present other significant technical challenges. Large limb movements imply a substantial number of inter-neuronal connections because the involved muscles require a more significant amount of energy than the fingers [17]. Therefore, the classification of imagined finger movements becomes complex because the fingers move closely together, and specific individual moving features merge.
Kaya et al. developed a public MMI-EEG dataset with five BCI paradigms [18]; it includes a paradigm related to the MI of individual finger movement. The authors reported only 43% accuracy in finger movement decoding using an SVM classifier in such a study. Xiuling Liu et al. [19] proposed a parallel spatial–temporal self-attention-based convolutional neural network for MI EEG signal classification. This method uses a spatial-temporal representation of raw EEG signals that uses the self-attention mechanism to extract distinguishable spatial–temporal features. It outperformed state-of-the-art methods for intra-subject and inter-subject classification. Jiacan Xu et al. [20] proposed a deep multi-view feature learning method for the classification of MI EEG signals. First, a multi-view representation of the EEG signals is obtained by extracting time domain, frequency domain, time-frequency domain and spatial features. Afterward, a parametric t-SNE method is used to extract the deep features from the multi-view representation. Then, a support vector machine (SVM) classifier is used to classify those deep features. The proposed method was tested on the BCI competition IV 2a dataset obtaining excellent classification results. Vernon et al. [21] proposed a convolutional network architecture named EEGNet for EEG signal classification. The architecture includes a filter-bank structure based on two convolutional layers to adaptively extract common spatial patterns. The authors showed that EEGNet generalizes well across BCI paradigms, and achieves comparable performance to other methods, especially for the case of limited training data. Anam et al. [22] implemented an Autonomous Deep Learning (ADL) architecture for the classification of individual finger movements based on MI EEG signals. ADL is an architecture capable of constructing by itself its structure and adapts to the input changes. The authors showed that for the case of subject-dependent classification ADL achieves an accuracy of around 77%.
The present work aims to process and classify MI-EEG signals of individual finger movements from one hand by addressing the problem of noisy signals using a method based on the EMD and using more powerful sequence classification architectures, including BiLSTM Recurrent Neural Networks (RNN).
Hence, an Empirical Mode Decomposition based preprocessing stage followed by a stacked BiLSTM network classifier is proposed in this study. The main contributions of this paper are summarized as follows,
  • An approach to decode imagined individual finger movement from one hand based on a stacked BiLSTM architecture.
  • An approach for tackling noisy MI-EEG signals based on EMD.
  • An improved state-of-the-art result for the task of subject-dependent imaginary finger movement classification.

2. Materials and Methods

2.1. Dataset

The EEG BCI dataset was built by Kaya et al. [18], considering five interaction paradigms related to motor imagery. In particular, the interest of this paper relies on the MI of five right-hand fingers, corresponding to paradigm #3 (5F). The subset of the dataset corresponding to the finger movement imagery consisted of MI-EEG signals from eight subjects captured with the Nihon, Kohden-Japan EEG-1200 JE-921A equipment. Two women and six men aged between 20 and 35 produced 19 file sessions of 4600 MI-EEG samples per subject. This dataset provided 45 min of MI-EEG for all subjects divided into three interaction segments; each segment consisted of the presentation of about 300 MI symbols. The equipment uses 22 electrodes; 19 are active and distributed according to the international standard 10–20 for EEG electrode positioning, shown in Table 1.
In the creation protocols of the dataset, the developers assert that test subjects were declared good physical and mental health at the capture time [18].
A recliner chair, suited for all participants, was placed at 200 cm from the monitor and slightly above the reference eye. Later, an eGUI displays the five fingers of the right hand. When a number from one to five is displayed just above a finger as a signal for starting the task, the test subject executes the corresponding imagery movement for one second. Hence, digits 1, 2, 3, 4, and 5 correspond to the thumb, index finger, middle finger, ring finger, and pinkie finger, respectively. The task involves the imagination of the flexion from a finger up and down. This paradigm does not have a neutral state since signals related to this task were not considered in the original dataset.
MI-EEG signals were recorded at 200 and 1000 Hz, where the latter is referred to as a High Frequency (HFREQ). The 5F dataset contains thirteen HFREQ files and six files at 200 Hz collected between 2015 and 2016. Software Neurofax [24] served to bandpass filter the raw MI-EEG signals from 0.53 to 70 Hz for 200 Hz and from 0.53 to 100 Hz for HFREQ signals. Signals from each sensor are arranged into a matrix as follows
EEG ( t ) = x 1 , 1 x 1 , m x n , 1 x n , m ,
where n and m are the number of samples and the number of signals, respectively. All 19 files (13 HFREQ and six files at 200 Hz) from the 5F dataset containing the captured signals (lasting from 3582 to 4040 s) were retrieved to be directly utilized in the proposed signal processing framework. It is because a preliminary preprocessing was applied during and after capturing in the creation of the EEG BCI dataset [18].

2.2. Overall Flowchart

The channels C3, Cz, P3, and Pz were selected to decode the right-hand finger movement in MI-EEG signals. This choice considers channels focused on the primary motor cortex (M1) and the cerebellum, involved in the motor imagery signals generation [25]. The fact that right-hand finger imagery movements activate the left cerebral hemisphere and the left-hand finger imagery movements activate the right one, P3 and C3 electrode signals are processed, including those from Cz and Pz for the cortex activation maps during the predefined tasks [26,27]. In addition, the {C3, Cz, P3, P3} combination revealed a better decoding accuracy among the {C3, Cz}, {C3, P3}, {C3, P3, Cz}, {C3, P3, Cz}, and {T3, C3, Cz} combinations evaluated in the electrode preselecting step.
Figure 1 shows the head positioning configuration of the corresponding electrodes, which are the most representative and discriminant electrodes in the proposed tasks.
Furthermore, Figure 2 shows the flowchart of the proposed approach.

2.3. EEG Signal Preprocessing Based on EMD

One problem when dealing with EEG signals is that they are inherently non-stationary. This phenomenon is because brain processes change due to brain state changes, e.g., mental fatigue [28]. This non-stationarity has severe implications for the generalization ability of deep neural network architectures [29].
Current approaches for dealing with non-stationarity are mainly based on trend removal [30]. The main trend removal approaches are high-pass filtering, moving average removal, polynomial fitting, and empirical mode decomposition. Among high-pass filtering approaches, empirical mode decomposition is one of the most reliable methods in terms of efficiency and simplicity [30,31].
EMD method decomposes a signal into an Intrinsic Mode Functions (IMFs) sum. The IMFs obtained from natural EEG signals provide analytical features (amplitudes, frequency, and phase), which improve the BiLSTM learning algorithm. It is a specific benefit of the EMD approach targeted in this study.
Empirical Mode Decomposition (EMD) is a signal processing tool proposed by Huang et al. to analyze nonlinear and non-stationary signals [32]. IMFs must fulfill the following constraints:
  • The number of local extrema and zero crossings must be equal or differ by at most one on the entire signal.
  • The mean value between the lower and upper envelope must be zero.
EMD can be used to denoise 1-D EEG signals because of the frequency-decreasing property of IMFs [33]. The IMFs represent the oscillation modes in the signal, so the first IMF contains the highest frequency, and the last IMF contains the lowest frequency. Algorithm 1 shows the steps performed by the EMD algorithm. Once completed the sifting process, the original MI-EEG signal can be recovered as follows
EEG = IMF 1 + IMF 2 + + IMF N + R N ,
where N is the number of computed IMFs from the original EEG signal, and R N ( t ) is the final residue. EMD operates similarly to a filter bank of bandpass filters for modes with indexes greater than 1 and a high-pass filter for mode 1 [34]. Therefore, Equation (3) describes the signals relations obtained when the first EMD step is applied to the EEG signal,
EEG ( t ) = C 1 ( t ) + IMF 1 ( t ) ,
where C 1 ( t ) represents the low-frequency components of the signal.
Algorithm 1: MI-EEG Signal Decomposition using EMD.
1: Let X be the matrix form of signals denoted by EEG(t).
2: Find the max { X } and min { X } .
3: Use the Cubic Spline interpolation to construct the upper EEGU(t) and lower EEGL(t) envelope connecting all max { X } and min { X } points.
4: Calculate the local Mean EEGC(t) = 1 2 (EEGU(t) + EEGL(t)).
5: Obtain Ri(t) = EEG(t) − EEGC(t).
6: Conclude the ith IMF order if Ri(t) satisfies the given IMF conditions with IMFi(t) = Ri(t), otherwise repeat steps 1 to 5.
7: Find the remaining IMFk(t) component by subtracting EEGU(t) − IMFi(t) and repeating the sifting process until to obtain a constant residue (no more oscillations).
Huang et al. developed a method that allows determining the number of iterations to stop the sifting process [35]. This method is based on two criteria:
  • All local maxima are strictly positive, while all local minima are strictly negative; and
  • The number of extrema points remains unchanged.
Moreover, the sifting process also stops when the standard deviation of the difference between two successive sifting steps is smaller than a threshold [32]. This last sifting stoppage criterion is given by
i = 1 N C i 1 ( t ) C i ( t ) 2 i = 0 N C i ( t ) 2 S S C ,
where S S C is a predefined threshold.
On the other hand, if IMFs do have different frequencies at the analyzed time, their analytic form (AIMF) can be expressed as
AIMF i ( t ) i = 1 N A i ( t ) cos θ i ( t ) ,
where A i ( t ) and θ i ( t ) are the instantaneous amplitude and phase of each IMFi, respectively. These parameters can be estimated using the Hilbert transform [35] as follows
A i ( t ) = R 2 ( t ) + R ^ 2 ( t ) , θ i ( t ) = tan 1 R ^ ( t ) R ( t ) ,
where R ( t ) represents both the corresponding IMF component and the real term of AIMF,
AIMF ( t ) = R ( t ) + j R ^ ( t ) ,
and R ^ ( t ) is the Hilbert Transform (HT) of R ( t ) given by
R ^ ( t ) = HT { R ( t ) } = 1 π R ( t ) * 1 t ,
where * is the convolution operator. Consequently, the analytic form for the i-th IMF becomes
AIMF ( t ) = A i ( t ) e j θ i ( t ) .
Zhang et al. found that the first four IMFs account for most of the cumulative variance contribution rate [36]. Thus, in our approach, the sum of only the first four IMFs will be used as the preprocessed EEG signal. Figure 3 illustrates the empirical mode decomposition of an EEG signal. So, the last IMFs and the residue capture the signal trend.
Figure 4 shows an example of the preprocessed EEG signal using the sum of the first four IMFs. Notice how the residue captures the trend of the signal.

2.4. Bidirectional LSTM (BiLSTM)

Long-Short Term Memory (LSTM) networks are a type of recurrent neural network initially designed to solve the vanishing gradient problem of recurrent neural networks when dealing with long sequences [37]. An LSTM network’s architecture consists of a layer of LSTM units followed by a standard feedforward network. Figure 5 shows a single functional block of an LSTM unit.
In a general perspective, an LSTM unit operates as follows: let x t be the current input at time t, the output of the input gate is as follows,
i t = σ ( W i x x t + W i h h t 1 + b i ) ,
where W i x and W i h are weight matrices, h t 1 is the previous hidden state of the unit, and b i is the bias vector. The function σ ( x ) ( 0 , 1 ) is a sigmoid function used for gating.
Similarly, the output of the forget gate f t is computed as
f t = σ ( W f x x t + W f h h t 1 + b f ) .
Finally, the outputs of the output gate o t and cell state c t are as follows,
c t = i t tanh ( W c x x t + W c h h t 1 + b c ) + c t 1 ,
o t = σ ( W o x x t + W o h h t 1 + b o ) ,
h t = o t tanh ( c t ) ,
where ⊙ is the Hadamard product.
A BiLSTM consists of two parallel LSTM layers: one for the forward direction and one for the backward direction [37,38]. Because the input is processed twice, BiLSTMs extract more information from the input. Thus, improving contextual information to make better predictions than LSTMs. Therefore, BiLSTMs present faster convergence and accuracy than LSTMs [39].
Figure 6 presents the BiLSTM architecture consisting of two LSTM layers, keeping past and future context at any time of the sequence.
The outputs of each LSTM are combined according to the following equation:
y t = W h y h t + W h y h t + b y ,
where h t and h t are the outputs of the forward and backward LSTMs.

2.5. Proposed Architecture

A feature matrix R W × 4 constituted by each preprocessed EEG signal is applied to the input of the first BiLSTM layer. A stacked architecture has been chosen to learn the complexity of the features extracted by the BiLSTM network. Several experiments were completed for 2, 3, and 4 stacked layers to determine the number of BiLSTM layers to implement, and the configuration with 3 stacked BiLSTM layers provided the highest accurate classification.
Each BiLSTM layer consists of 12 memory units, as illustrated in Figure 7. The output of the stacked BiLSTMs is a matrix R W × 12 . This matrix is then converted into a vector of size W × 12 , i.e., R 12 W × 1 . The value of W is 170 for 200 Hz signals, whereas for 1000 Hz signals, it is 850. This vector is the input to a dense layer.
The dense layer uses the SoftMax activation function to classify the representative features into the class labels.
The batch size for all network training was set to 330. The model was implemented in Python 3.6 using Keras and TensorFlow. The loss function was defined as the Categorical Cross-Entropy, the learning algorithm was defined as the Nadam optimizer, and the additional metrics to be computed during training was the Accuracy metric.
Moreover, the Cyclical Learning Rate (CLR) [40] method was used to accelerate the convergence of the training algorithm. Another reason for using CLR is that it can help the training algorithm escape from local minima. So, the minimum and maximum learning rates were set to 10 9 and 10 3 , respectively. The step size was 8 times the number of iterations per epoch.
The complete model was trained for 300 epochs on a Windows 10 desktop equipped with an NVIDIA GTX 1080 Ti GPU; each training was repeated at least twice.

3. Results and Discussion

This study implements the test subject-dependent approach where signals from a single subject were classified to decode right-hand finger imagery movements. The subject signals were coded using two particular sampling frequencies, 1000 Hz or 200 Hz. EEG signals used in the experiments, corresponding to the electrodes C3, Cz, P3, and Pz, are presumably involved in the movements from the right-hand fingers [26].
Hence, k-fold CV was used to assess the training performance of the model. Considering that the dataset is relatively small, the value of k was set to 200. So, the dataset was split into 200 disjoint subsets of equal size. Then, one different subset is taken as the test data for each training. At the same time, the remaining subsets 199 are used as learning data. The performance is taken as the average of the 200 testing accuracies. The number of samples in the datasets were between 940 and 1917.
Table 2 shows the model accuracy for different subjects and sampling frequencies.
As a result, the highest testing accuracy (76.13%) for signals at 1000 Hz corresponds to subject F, while the lowest accuracy (66.0%) corresponds to subject I. The highest testing accuracy (82.26%) for signals at 200 Hz corresponds to subject C, while the lowest accuracy (75.2%) corresponds to subject B.
Table 3 shows the impact of the number of BiLSTM layers on the accuracy of the proposed method. These results show that accuracy reaches a maximum at three BiLSTM layers. Therefore, the configuration of three BiLSTM layers was used in all tests for this study.
Table 4 shows the number of model parameters for the case of 1000 Hz and 200 Hz signals. Besides, the last BiLSTM layer outputs the whole processed sequence. Therefore, the proposed architecture is different from other architectures because they use only the last state of the last BiLSTM layer for classification.
The results obtained by the presented approach outperformed those reported in [18], where an average accuracy of 43% was achieved in the decoding of five fingers movements. Besides, the Support Vector Machines (SVM) method was used to classify MI-EEG signals from every single subject using only the C3 channel.
The network model was also trained and tested on the samples of all subjects (A, B, C and F) to determine what was the behavior of the accuracy for samples of different subjects For that purpose, the network model was trained each time using data from four subjects (A, B, C, and F) and for one of two available sampling frequencies: 1000 Hz and 200 Hz.
As a result, the proposed model achieved 80.04% and 82.26% accuracy for 1000 Hz and 200 Hz signals, respectively. Those results outperform the results obtained by Kaya et al. for five subjects, achieving an accuracy between 40% and 60% [18]. They achieved an accuracy between 20% and 40% for three subjects, considering all the 13 subjects.
Table 5 shows a comparison with other results from the literature for subjects A, B, C, and F, with 200 Hz and 1000 Hz signals.
The choice of subjects is mainly because, for those subjects, the dataset contains both 200 Hz and 1000 Hz signals. Thus, the results obtained by Anam et al. with the subject-dependent classification are slightly smaller than the results obtained by our approach for the case of 200 Hz signals [41]. However, in the case of 1000 Hz signals, the method of Anam et al. performs better than the proposed framework, except for subject C. This issue can be explained by the number of parameters used for the 1 kHz case, which was about five times the number of parameters for 200 Hz, as shown in Table 4. Hence, the proposed model presented overfitting issues for this case. EEGNet [21] was also trained using the proposed preprocessing method for comparison purposes. As a result, the accuracy was very close to the accuracy of the proposed approach. Finally, the EMD-based preprocessing method resulted in about 32.6% faster training convergence for all tests.

4. Conclusions

The data initially collected was intended to be used in BCI systems; unfortunately, the accurate decoding of the signals in real applications is still a challenge. This study aimed to decode individual imagined movements of the five right-hand fingers using a public EEG BCI dataset [18]. The Empirical Mode Decomposition (EMD) method was used as a preprocessing step of MI-EEG signals from the C3, Cz, P3, and Pz sensors. Next, the obtained IMF signals were classified using a recurrent neural network, described as BiLSTM. This work contributes to contrast the achieved high accuracy compared to the related state-of-the-art references, using specifically a Deep Learning approach. Numerical results showed that the decoding accuracy depends heavily on the availability of training data. Maximum accuracy of 82.9% was obtained by using stratified 200-fold CV.

Author Contributions

Conceptualization, T.M.-V.; data curation, H.R.-G. and J.M.C.-D.; formal analysis, H.R.-G. and J.G.A.-C.; funding acquisition, J.G.A.-C.; investigation, T.M.-V., J.R.-P., J.M.C.-D., and J.G.A.-C.; methodology, J.R.-P. and J.G.A.-C.; software, T.M.-V., H.R.-G., and J.G.A.-C.; validation, J.R.-P. and J.G.A.-C.; writing—original draft, T.M.-V.; writing—review and editing, J.R.-P., J.M.C.-D., and J.G.A.-C. All authors read and agreed to the published version of the manuscript.

Funding

This project was funded by the Mexican National Council of Science and Technology (CONACyT), under Grant Number 763527/600853 and the University of Guanajuato Grant 145790.

Institutional Review Board Statement

Ethical review and approval are waived for this kind of study.

Informed Consent Statement

No Formal written consent was required for this study.

Acknowledgments

This project was fully supported by the Electrical and Electronics Departments of the Universidad de Guanajuato under the Program POA 2021, and The Mexican National Council of Science and Technology (CONACYT) through the grant 763527/600853.

Conflicts of Interest

The authors declare that they have no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.

References

  1. Geng, X.; Li, D.; Chen, H.; Yu, P.; Yan, H.; Yue, M. An improved feature extraction algorithms of EEG signals based on motor imagery brain-computer interface. Alex. Eng. J. 2021. [Google Scholar] [CrossRef]
  2. Acampora, G.; Trinchese, P.; Vitiello, A. A dataset of EEG signals from a single-channel SSVEP-based brain computer interface. Data Brief 2021, 35, 106826. [Google Scholar] [CrossRef]
  3. Na, R.; Hu, C.; Sun, Y.; Wang, S.; Zhang, S.; Han, M.; Yin, W.; Zhang, J.; Chen, X.; Zheng, D. An embedded lightweight SSVEP-BCI electric wheelchair with hybrid stimulator. Digit. Signal Process. 2021, 116, 103101. [Google Scholar] [CrossRef]
  4. Zhang, J.; Wang, M. A survey on robots controlled by motor imagery brain-computer interfaces. Cogn. Robot. 2021, 1, 12–24. [Google Scholar] [CrossRef]
  5. Verbaarschot, C.; Tump, D.; Lutu, A.; Borhanazad, M.; Thielen, J.; van den Broek, P.; Farquhar, J.; Weikamp, J.; Raaphorst, J.; Groothuis, J.T.; et al. A visual brain-computer interface as communication aid for patients with amyotrophic lateral sclerosis. Clin. Neurophysiol. 2021, 132, 2404–2415. [Google Scholar] [CrossRef] [PubMed]
  6. Vimala, V.; Ramar, K.; Ettappan, M. An Intelligent Sleep Apnea Classification System Based on EEG Signals. J. Med. Syst. 2019, 43, 36. [Google Scholar] [CrossRef] [PubMed]
  7. Miao, M.; Zeng, H.; Wang, A.; Zhao, F.; Liu, F. Index finger motor imagery EEG pattern recognition in BCI applications using dictionary cleaned sparse representation-based classification for healthy people. Rev. Sci. Instrum. 2017, 88, 094305. [Google Scholar] [CrossRef]
  8. Ha, K.W.; Jeong, J.W. Motor Imagery EEG Classification Using Capsule Networks. Sensors 2019, 19, 2854. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  9. Chaudhary, S.; Taran, S.; Bajaj, V.; Siuly, S. A flexible analytic wavelet transform based approach for motor-imagery tasks classification in BCI applications. Comput. Methods Programs Biomed. 2020, 187, 105325. [Google Scholar] [CrossRef]
  10. Abdalsalam M, E.; Yusoff, M.Z.; Mahmoud, D.; Malik, A.S.; Bahloul, M.R. Discrimination of four class simple limb motor imagery movements for brain-computer interface. Biomed. Signal Process. Control 2018, 44, 181–190. [Google Scholar] [CrossRef]
  11. Argiris, G.; Budai, R.; Maieron, M.; ius tamara, I.; Skrap, M.; Tomasino, B. Neurosurgical lesions to sensorimotor cortex do not impair action verb processing. Sci. Rep. 2020, 10, 523. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  12. Oostra, K.M.; Van Bladel, A.; Vanhoonacker, A.C.L.; Vingerhoets, G. Damage to Fronto-Parietal Networks Impairs Motor Imagery Ability after Stroke: A Voxel-Based Lesion Symptom Mapping Study. Front. Behav. Neurosci. 2016, 10, 5. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  13. Salankar, N.; Mishra, A.; Mishra, P. Functional Connectivity and Classification of Actual and Imaginary Motor Movement. Int. J. Eng. Adv. Technol. 2019, 9, 529–535. [Google Scholar] [CrossRef]
  14. Li, F.; He, F.; Wang, F.; Zhang, D.; Xia, Y.; Li, X. A novel simplified convolutional neural network classification algorithm of motor imagery EEG signals based on deep learning. Appl. Sci. 2020, 10, 1605. [Google Scholar] [CrossRef]
  15. Tyagi, A.; Nehra, V. Classification of motor imagery EEG signals using SVM, k-NN and ANN. CSI Trans. ICT 2016, 4, 135–139. [Google Scholar] [CrossRef]
  16. Alzahrani, S.; Anderson, C. A Comparison of Conventional and Tri-Polar EEG Electrodes for Decoding Real and Imaginary Finger Movements from One Hand. Int. J. Neural Syst. 2021, 31, 2150036. [Google Scholar] [CrossRef]
  17. Tanaka, Y.; Saga, N.; Hashimoto, Y. Event-related synchronization and desynchronization of EEG in association with electromyography of human limb movement. Proc. JSME Annu. Conf. Robot. Mechatron. 2017. [Google Scholar] [CrossRef]
  18. Kaya, M.; Binli, M.; Ozbay, E.; Yanar, H.; Mishchenko, Y. A large electroencephalographic motor imagery dataset for electroencephalographic brain computer interfaces. Sci. Data 2018, 5, 180211. [Google Scholar] [CrossRef] [Green Version]
  19. Liu, X.; Shen, Y.; Liu, J.; Yang, J.; Xiong, P.; Lin, F. Parallel Spatial-Temporal Self-Attention CNN-Based Motor Imagery Classification for BCI. Front. Neurosci. 2020, 14, 587520. [Google Scholar] [CrossRef] [PubMed]
  20. Xu, J.; Zheng, H.; Wang, J.; Li, D.; Fang, X. Recognition of EEG Signal Motor Imagery Intention Based on Deep Multi-View Feature Learning. Sensors 2020, 20, 3496. [Google Scholar] [CrossRef]
  21. Lawhern, V.; Solon, A.; Waytowich, N.; Gordon, S.; Hung, C.; Lance, B. EEGNet: A Compact Convolutional Network for EEG-based Brain-Computer Interfaces. J. Neural Eng. 2016, 15, 056013. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  22. Anam, K.; Bukhori, S.; Hanggara, F.; Pratama, M. Subject-independent Classification on Brain-Computer Interface using Autonomous Deep Learning for finger movement recognition. In Proceedings of the 42nd Annual International Conference of the IEEE Engineering in Medicine Biology Society (EMBC), Montreal, QC, Canada, 20–24 July 2020; pp. 447–450. [Google Scholar] [CrossRef]
  23. Syakiylla, S.; Syakiylla Sayed Daud, S.; Sudirman, R. Decomposition Level Comparison of Stationary Wavelet Transform Filter for Visual Task Electroencephalogram. J. Teknol. 2015, 74, 7–13. [Google Scholar] [CrossRef] [Green Version]
  24. Mishchenko, Y.; Kaya, M.; Ozbay, E.; Yanar, H. Developing a 3- to 6-state EEG-based brain-computer interface for a robotic manipulator control. IEEE Trans. Biomed. Eng. 2018, 66, 977–987. [Google Scholar] [CrossRef] [PubMed]
  25. Lotze, M.; Montoya, P.; Erb, M.; Hũlsmann, E.; Flor, H.; Klose, U.; Birbaumer, N.; Grodd, W. Activation of cortical and cerebellar motor areas during executed and imagined hand movements: An fMRI study. J. Cogn. Neurosci. 1999, 11, 491–501. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  26. Banissy, M.; Annett, L.; Asiedu-Offei, P.; Rosch, R.; Gurd, J. Left, right, hand ’n space. In Handedness: Theories, Genetics and Psychology, 1st ed.; Nova Science Publishers: Hauppauge, NY, USA, 2012; pp. 109–122. [Google Scholar]
  27. Alomari, M.; Awada, E.; Younis, O. Subject-Independent EEG-Based Discrimination Between Imagined and Executed, Right and Left Fists Movements. Eur. J. Sci. Res. 2014, 118, 364–373. [Google Scholar]
  28. Von Bünau, P.; Meinecke, F.C.; Scholler, S.; Müller, K.R. Finding stationary brain sources in EEG data. In Proceedings of the 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology, Buenos Aires, Argentina, 31 August–4 September 2010; Volume 1, pp. 2810–2813. [Google Scholar] [CrossRef]
  29. Igl, M.; Farquhar, G.; Luketina, J.; Boehmer, W.; Whiteson, S. Transient Non-stationarity and Generalisation in Deep Reinforcement Learning. In Proceedings of the International Conference on Learning Representations (ICLR 2021), Vienna, Austria, 4 May 2021; pp. 1–16. [Google Scholar]
  30. Lentka, Ł.; Smulko, J. Analysis of effectiveness and computational complexity of trend removal methods. Zesz. Nauk. Wydziału Elektrotech. Autom. Politech. Gdańskiej 2016, 51, 111–114. [Google Scholar]
  31. Hyndman, R.J.; Athanasopoulos, G. Forecasting: Principles and Practice, 2nd ed.; OTexts: Melbourne, Australia, 2018. [Google Scholar]
  32. Huang, N.E.; Shen, Z.; Long, S.R.; Wu, M.C.; Shih, H.H.; Zheng, Q.; Yen, N.C.; Tung, C.C.; Liu, H.H. The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis. Proc. R. Soc. Lond. Ser. A Math. Phys. Eng. Sci. 1998, 454, 903–995. [Google Scholar] [CrossRef]
  33. Chen, Y.; Zhang, G.; Gan, S.; Zhang, C. Enhancing seismic reflections using empirical mode decomposition in the flattened domain. J. Appl. Geophys. 2015, 119, 99–105. [Google Scholar] [CrossRef]
  34. Flandrin, P.; Rilling, G.; Goncalves, P. Empirical mode decomposition as a filter bank. IEEE Signal Process. Lett. 2004, 11, 112–114. [Google Scholar] [CrossRef] [Green Version]
  35. Huang, N.; Wu, M.L.; Long, S.; Shen, S.; Qu, W.; Gloersen, P.; Fan, K. A confidence limit for the empirical mode decomposition and Hilbert spectral analysis. Proc. R. Soc. Lond. Ser. A Math. Phys. Eng. Sci. 2003, 459, 2317–2345. [Google Scholar] [CrossRef]
  36. Zhang, Y.; Ji, X.; Zhang, S. An approach to EEG-based emotion recognition using combined feature extraction method. Neurosci. Lett. 2016, 633, 152–157. [Google Scholar] [CrossRef]
  37. Yu, Y.; Si, X.; Hu, C.; Zhang, J. A Review of Recurrent Neural Networks: LSTM Cells and Network Architectures. Neural Comput. 2019, 31, 1235–1270. [Google Scholar] [CrossRef]
  38. Schuster, M.; Paliwal, K. Bidirectional recurrent neural networks. IEEE Trans. Signal Process. 1997, 45, 2673–2681. [Google Scholar] [CrossRef] [Green Version]
  39. Graves, A.; Schmidhuber, J. Framewise phoneme classification with bidirectional LSTM and other neural network architectures. Neural Netw. Off. J. Int. Neural Netw. Soc. 2005, 18, 602–610. [Google Scholar] [CrossRef]
  40. Smith, L.N. Cyclical Learning Rates for Training Neural Networks. In Proceedings of the 2017 IEEE Winter Conference on Applications of Computer Vision (WACV), Santa Rosa, CA, USA, 24–31 March 2017; pp. 464–472. [Google Scholar]
  41. Anam, K.; Nuh, M.; Al-Jumaily, A. Comparison of EEG pattern recognition of motor imagery for finger movement classification. In Proceedings of the International Conference on Electrical Engineering, Computer Science and Informatics (EECSI), Bandung, Indonesia, 18–20 September 2019; pp. 24–27. [Google Scholar] [CrossRef]
Figure 1. Sensors selection from a Nihon Kohden-Japan EEG-1200 JE-921A equipment. MI-EEG signals from four sensors on the left sensorimotor and cognitive cortex were selected (see Table 1).
Figure 1. Sensors selection from a Nihon Kohden-Japan EEG-1200 JE-921A equipment. MI-EEG signals from four sensors on the left sensorimotor and cognitive cortex were selected (see Table 1).
Mathematics 09 03297 g001
Figure 2. MI-EEG signal processing flowchart. The stimulus appears on the experimental Graphical User Interface (eGUI) represented by a number from 1 to 5 directly above the finger. During this period, the MI flexion-extension of the corresponding finger is generated. Next, captured MI-EEG signals from C3, Cz, P3, and Pz electrodes are processed for each individual.
Figure 2. MI-EEG signal processing flowchart. The stimulus appears on the experimental Graphical User Interface (eGUI) represented by a number from 1 to 5 directly above the finger. During this period, the MI flexion-extension of the corresponding finger is generated. Next, captured MI-EEG signals from C3, Cz, P3, and Pz electrodes are processed for each individual.
Mathematics 09 03297 g002
Figure 3. Empirical mode decomposition for an EEG signal. Top to bottom: Original signal, IMF1–IMF6, residue. The amplitudes are measured in μV and the horizontal axis in Samples Per Second (SPS).
Figure 3. Empirical mode decomposition for an EEG signal. Top to bottom: Original signal, IMF1–IMF6, residue. The amplitudes are measured in μV and the horizontal axis in Samples Per Second (SPS).
Mathematics 09 03297 g003
Figure 4. Example of the preprocessing of an EEG signal based on EMD. Top to bottom: an EEG signal, the resulting preprocessed signal, and the residue. The amplitudes are measured in μV and the horizontal axis in Samples Per Second (SPS).
Figure 4. Example of the preprocessing of an EEG signal based on EMD. Top to bottom: an EEG signal, the resulting preprocessed signal, and the residue. The amplitudes are measured in μV and the horizontal axis in Samples Per Second (SPS).
Mathematics 09 03297 g004
Figure 5. Architecture of a Long Short-Term Memory (LSTM) unit.
Figure 5. Architecture of a Long Short-Term Memory (LSTM) unit.
Mathematics 09 03297 g005
Figure 6. A Bidirectional LSTM (BiLSTM) Network unfolded in time. It combines the outputs of two LSTM layers in parallel, processing the input sequence in opposite directions.
Figure 6. A Bidirectional LSTM (BiLSTM) Network unfolded in time. It combines the outputs of two LSTM layers in parallel, processing the input sequence in opposite directions.
Mathematics 09 03297 g006
Figure 7. Proposed BiLSTM architecture. It consists of three BiLSTM layers, one flatten and one dense layers.
Figure 7. Proposed BiLSTM architecture. It consists of three BiLSTM layers, one flatten and one dense layers.
Mathematics 09 03297 g007
Table 1. Nomenclature and functions for the electrodes in the international 10/20 system [23].
Table 1. Nomenclature and functions for the electrodes in the international 10/20 system [23].
Brain RegionElectrodeFunction
Frontal F p 1 Attention
F p 2 Judgment restrains impulses
F 7 Verbal expression
F 3 Motor planning
F 4 Motor planning of left-upper extremity
F 8 Emotional expression
Temporal T 3 Verbal memory
T 4 Emotional memory
T 5 Verbal understanding
T 6 Emotional understanding and motivation
Central C 3 Sensorimotor integration (right)
C z Sensorimotor integration (midline)
C 4 Sensorimotor integration (left)
Parietal P 3 Cognitive processing special temporal
P z Cognitive processing
P 4 “Math word problems”, “Non-verbal reasoning”
Occipital O 1 Visual processing
O z Incontinence
O 2 Visual processing
Table 2. Results for subject-dependent finger movement decoding.
Table 2. Results for subject-dependent finger movement decoding.
SubjectTraining (%)Testing (%)Training (%)Testing (%)
1000 Hz200 Hz
A90.7672.4385.3581.84
B73.4774.0779.9475.2
C98.6980.0488.9282.26
E80.3371.93--
F88.8576.1388.7379.70
G73.5968.0--
H91.2273.33--
I93.566.0--
Table 3. Accuracy depending on BiLSTM layers number.
Table 3. Accuracy depending on BiLSTM layers number.
Subjects
BiLSTMABCFABCF
# LayersAccuracy (%) at 1000 HzAccuracy (%) at 200 Hz
266.2567.1375.854.9569.1567.5679.871.17
372.4374.0780.0476.1381.8475.282.2679.70
463.3571.8374.9267.2575.5574.4178.670.9
Table 4. Parameters of the model for training with signals at 1000 Hz and 200 Hz.
Table 4. Parameters of the model for training with signals at 1000 Hz and 200 Hz.
LayerCellsOutput ShapeParametersOutput ShapeParameters
Type 1000 Hz1000 Hz200 Hz200 Hz
Bidirectional12(None, 850,24)1632(None, 170,24)1632
Bidirectional_112(None, 850,24)3552(None, 170,24)3552
Bidirectional_212(None, 850,24)3552(None, 170,24) 3552
Flatten(None, 20,400)0(None, 2638)0
Dense_1(None, 5)102,005(None, 5)13,418
Activation(None, 5)0(None, 5)0
Total 110,771 22,154
Table 5. Comparison with other state-of-the-art approaches.
Table 5. Comparison with other state-of-the-art approaches.
SubjectSamplingMethods
FrequencyAnam et al. [41]Proposed MethodVernon et al. [21]
(ADL Network)(EMD+BiLSTM)(EMD+EEGNet)
A200 Hz77.46%81.84%80.16%
1000 Hz77.75%72.43%75.45%
B200 Hz77.82%75.2%80.02%
1000 Hz74.61%74.07%76.07%
C200 Hz81.62%82.26%80.17%
1000 Hz77.49%80.04%78.90%
F200 Hz78.13%79.70%80.01%
1000 Hz77.20%76.13%75.20%
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Mwata-Velu, T.; Avina-Cervantes, J.G.; Cruz-Duarte, J.M.; Rostro-Gonzalez, H.; Ruiz-Pinales, J. Imaginary Finger Movements Decoding Using Empirical Mode Decomposition and a Stacked BiLSTM Architecture. Mathematics 2021, 9, 3297. https://doi.org/10.3390/math9243297

AMA Style

Mwata-Velu T, Avina-Cervantes JG, Cruz-Duarte JM, Rostro-Gonzalez H, Ruiz-Pinales J. Imaginary Finger Movements Decoding Using Empirical Mode Decomposition and a Stacked BiLSTM Architecture. Mathematics. 2021; 9(24):3297. https://doi.org/10.3390/math9243297

Chicago/Turabian Style

Mwata-Velu, Tat’y, Juan Gabriel Avina-Cervantes, Jorge Mario Cruz-Duarte, Horacio Rostro-Gonzalez, and Jose Ruiz-Pinales. 2021. "Imaginary Finger Movements Decoding Using Empirical Mode Decomposition and a Stacked BiLSTM Architecture" Mathematics 9, no. 24: 3297. https://doi.org/10.3390/math9243297

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop