Non-Linear Adapted Spatio-Temporal Filter for Single-Trial Identification of Movement-Related Cortical Potential

Abstract

The execution or imagination of a movement is reflected by a cortical potential that can be recorded by electroencephalography (EEG) as Movement-Related Cortical Potentials (MRCPs). The identification of MRCP from a single trial is a challenging possibility to get a natural control of a Brain–Computer Interface (BCI). We propose a novel method for MRCP detection based on optimal non-linear filters, processing different channels of EEG including delayed samples (getting a spatio-temporal filter). Different outputs can be obtained by changing the order of the temporal filter and of the non-linear processing of the input data. The classification performances of these filters are assessed by cross-validation on a training set, selecting the best ones (adapted to the user) and performing a majority voting from the best three to get an output using test data. The method is compared to another state-of-the-art filter recently introduced by our group when applied to EEG data recorded from 16 healthy subjects either executing or imagining 50 self-paced upper-limb palmar grasps. The new approach has a median accuracy on the overall dataset of 80%, which is significantly better than that of the previous filter (i.e., 63%). It is feasible for online BCI system design with asynchronous, self-paced applications.

1. Introduction

Neurological conditions such as locked-in syndrome [1], amyotrophic lateral sclerosis [2], stroke [3] and cerebral palsy [4] affect how the brain communicates with other organs in the body by disrupting neurological networks; they primarily affect the motor control of the muscles [5]. These conditions limit patients from communicating with the outside world, affecting their social life [6]. Brain–Computer Interfaces (BCIs) facilitate such patients’ ability to regain their social lives through non-muscular communication techniques that use brain signals to transmit information [7]. Such communications via BCI have helped researchers develop several interventions focused on rehabilitating motor control after neurological disorders [8,9,10]. Despite the potential for BCIs to be used in clinical rehabilitation, traditional average-based algorithms limit their practicality [11]. Therefore, recently, researchers started working on updating BCI algorithms from average-based to single-trial [12,13,14] for better intervention development for neurorehabilitation.

The BCI paradigm for neurorehabilitation is based on the study of specific variations of brain signals recorded via electroencephalography (EEG) to design interventions that help patients [5]. One of the most widely used signals within EEG is known as movement-related cortical potential (MRCP) [15,16,17,18]. It is a low-frequency negative shift in the EEG signal that occurs around 2 s before a voluntary movement is planned or executed [19,20]. MRCPs have been extensively studied to understand the neural basis of movement and have been found to be related to a variety of factors, including the complexity of the movement, the size of the muscle group being activated, and the direction of the movement [17,21,22]. Therefore, MRCPs have been used extensively to develop BCIs that allow patients who are otherwise incapable of communicating to interact with surroundings, as well as to assist patients with motor impairments in the process of neurorehabilitation [23]. One of the major issues is to detect MRCPs in a noisy EEG signal, which is usually done by using spatial filters.

Spatial filtering is one of the most used EEG signal processing approaches for artifact removal and improving the detection accuracy of cortical potentials [24]. MRCPs have a well-defined spatial distribution, being positioned directly over the primary motor cortex area on the scalp [25]. An ankle dorsiflexion task, for example, produces the highest MRCP over the apex (Cz of 10–20 montage) [24]. The most common spatial filters used in EEG-based BCI systems are the Laplacian [26], the Common Spatial Pattern (CSP) [27], and different Independent Component Analysis (ICA) variants [24,28]. Several studies have demonstrated the effectiveness of spatial filtering techniques for enhancing the detection of MRCP signals in a variety of contexts, including motor imagery [29], robot-assisted rehabilitation [30], and BCI applications [31]. Although these filters and ICA algorithms increased SNR and detection accuracy, there are some limitations to these methods. For example, prior information about the spatial aspects of the signal is needed to set the parameters. Furthermore, cases where the EEG signals are contaminated with artifacts, such as eye blinks or muscle activity, can result in the extraction of non-brain related components, which can lead to incorrect or unreliable results [32]. In the literature, other techniques such as matched filtering and optimized spatial filtering have also been used extensively to extract MRCPs from EEG signals [33]. Although these methods can be effective in certain scenarios, they have several disadvantages, such as complexity, sensitivity to noise, and less adaptability to changing morphology of MRCPs across individuals [34,35,36].

Therefore, an innovative non-linear EEG filter was developed for identification of MRCP during motor execution or imagination [37]. The results were promising, showing better performances than a previous state-of-the-art matched and optimized spatial filter used in the literature. Our previous study involved sixteen healthy adults seated on a chair and performing 50 motor executions and 50 motor imaginations of palmer grasps as their EEGs were recorded (for task and recording details see [37]). The proposed filter mapped the input EEG channels into a prototype waveform resembling an MRCP during movements intentions and forced the output to zero otherwise. The filter was optimized in the least mean squared sense on a training set and then tested on external data. In the proposed filter, however, there were some limitations, such as the random selection of values of frequencies of sampling cosine functions; this random choice affects the filter’s performance, which is always slightly different when it’s applied to the same data. Additionally, the proposed filter had some pre-processing issues, impeding its application in real-time systems.

Specifically, some pre-processing was performed in optimal conditions, i.e., using the test data:

1.

Whitening of the data was computed by singular value decomposition (SVD), which requires the test signal to be available; thus, the filter could not be applied in real time with the proposed implementation, but an alternative whitening is required, based only on training data;

2.

Artifact removal was based on the estimation of independent components (ICs) from which the one representing the blinks was identified and removed; all data were included to estimate the ICs, whereas, in order to keep low the computational cost and allow for real time application, the filters extracting the artifacts should be defined only using training data and kept fixed when applied on new testing signals.

Our current study is based on the same dataset as the previous one, but we modified two aspects: (1) whitening and (2) artifact removal algorithms, which were modified to make the filter suitable for real-time implementation. Moreover, a new filter is developed, called Non-Linear Spatio-Temporal Filter (NLSTF), and it is compared to the previous approach when applied to the same signals (but using only the training set for data cleaning and filter definition). It is based on the following innovations with respect to the previous method:

1.

Delayed data are also used by the filter, so that filtering is applied both in time and space;

2.

Simple non-linear functions (i.e., low order polynomials) are used;

3.

The filters are ranked in terms of their accuracy in cross-validation on training data with respect to the orders of the temporal filter and of the polynomial non-linearity;

4.

A majority voting on the best filters is finally used to process the test set.

Our hypothesis was that this new approach would improve the accuracy of the previously proposed non-linear filter and make it more appropriate for real-time use.

2. Methods

The acquisition of experimental data and the innovative processing algorithm are described below.

2.1. Experimental Data

A short introduction is provided to the experimental dataset, which is the same as in [37], where more details can be found.

Sixteen healthy subjects participated in the experiment (four men; mean age ± std: 28 ± 12 years), after signing a written informed consent and under the approval of the local ethical committee (number 20130081).

The subjects performed a grasp of the right hand using a handle with a force transducer (Noraxon USA, Scottsdale, AZ; sample rate of 2000 Hz). Maximum voluntary contraction (MVC) was measured three times, keeping the highest value. The participants performed 50 grasps at 60% MVC (with a visual feedback given through a PC) during the motor execution (ME) task. After that, the subjects performed 50 trials of motor imagination (MI) of palmer grasp. For each movement type, there was a 2–3-min break after the 25th trial. For the ME tasks, the force was used to determine the movement onset, defined as the instant where all values in a 200-ms-wide moving time window were above the baseline (assessed during the rest phase). All onsets were visually inspected.

EEG was recorded by a continuous nine-channel monopolar amplifier (Nuamps Express, Neuroscan), using Ag/AgCl ring electrodes and sampling at 500 Hz with an A/D converter with 32 bits. The following channels were considered: F3, Fz, F4, C3, Cz, C4, P3, Pz and P4. The reference was placed on the right ear lobe and the ground at the nasion. The electrode impedance was below 5 kΩ. Subjects were asked to minimize eye blinks and facial and body movements. At the beginning of each trial, a digital trigger was sent from a visual cueing program to the EEG amplifier.

Some details of the experimental acquisition procedure (i.e., visual cue, subject positioning and considered EEG channels) are provided in Figure 1.

2.2. Signal Processing

An innovative non-linear spatio-temporal filter was developed to emphasize the MRCPs. It is compared to a previous filter developed by our group [37].

2.2.1. Pre-Processing

Data were divided in training (i.e., first 70% of the trace) and testing dataset (remaining 30%).

Possible jumps in the signals were automatically identified and compensated. Low frequency drifts were removed by high-pass filtering at 0.04 Hz (Butterworth filter with 40 dB per decade of attenuation outside of the pass band) [19].

Blink artifacts were then removed, as the component extracted by the Second Order Blind-source Identification (SOBI) algorithm [38] with lowest fractal dimension (computed by Sevcik’s method [39]) [40]. The weights of the artifactual component were estimated on the training set and then kept fixed when processing the test set.

Finally, data were resampled to 50 Hz after an anti-alias low-pass filter with cut-off 20 Hz (Butterworth filter with roll-off 40 dB/decade).

An example of experimental data and pre-processing is provided in Figure 2.

2.2.2. Non-Linear Spatio-Temporal Filter

The innovative filter is referred to as a non-linear spatio-temporal filter (NLSTF). A linear least squared problem was defined with respect to the weights of the filter. As input features, the present and previous samples (up to N lags) of the EEG channels were included. Thus, both a spatial and a temporal filter were defined. Moreover, the powers of the EEG signals (up to an order Q) were also included. Cross-terms were not considered, for simplicity. The sign of the signal was preserved, e.g., $-x^2\left(t\right)$ was used for samples for which $x\left(t\right)<0$. This way, a non-linear processing of the input data was included (but still defining a linear problem for the estimation of the weights of the filter).

Consider a target desired output $y\left(t\right)$, which is equal to a prototype of the MRCP corresponding to each movement intention of the subject in the training set and zero otherwise. The same prototype as in [37] was used, i.e., a 1-s-long wave starting from 0 and linearly increasing until the instant of a movement intention; then, it was defined as 0 from the following time sample.

The desired output was fit by a linear model

$$y=XW+r$$

(1)

where W is the vector of filter weights, r the residual error and the matrix X includes the predictors, i.e., the signals of all the $N_{ch}$ channels ($N_{ch}=9$ in our case), possibly delayed (with N delayed repetitions, with $N=0$ if only the present data is used) and elevated to a power (ranging from 1 to Q). A general expression for the matrix X is

$$X=\left[\begin{array}{cccccccc}1& x_{11}& \cdots & x_{M1}& \cdots & sign\left(x_{11}\right){\left|x_{11}\right|}^Q& \cdots & sign\left(x_{M1}\right){\left|x_{M1}\right|}^Q\\ 1& x_{12}& \cdots & x_{M2}& \cdots & sign\left(x_{12}\right){\left|x_{12}\right|}^Q& \cdots & sign\left(x_{M2}\right){\left|x_{M2}\right|}^Q\\ ⋮& ⋮& ⋮& ⋮& ⋮& ⋮& ⋮& ⋮\\ 1& x_{1S}& \cdots & x_{MS}& \cdots & sign\left(x_{1S}\right){\left|x_{1S}\right|}^Q& \cdots & sign\left(x_{MS}\right){\left|x_{MS}\right|}^Q\end{array}\right]$$

(2)

where S is the number of samples in the training set, $M=N_{ch}·(N+1)$ is the number of signals (also including delayed data, if $N>0$) and the functions $sign$ and absolute value are used to keep the sign of the signals when elevated to an even power. As a simple representative example, we show this matrix in a trivial case: assume that the order of the filter is 1, the order of the non-linearity is $Q=2$ and consider only one EEG channel ($N_{ch}=1$); in such a case, the X matrix is

$$X=\left[\begin{array}{ccccc}\mathbf{1}& {\mathbf{x}}_1\left(t\right)& {\mathbf{x}}_1(t-1)& sign\left({\mathbf{x}}_1\left(t\right)\right){\mathbf{x}}_1^2\left(t\right)& sign\left({\mathbf{x}}_1(t-1)\right){\mathbf{x}}_1^2(t-1)\end{array}\right]$$

(3)

where $t=1,\cdots ,S$ is the sampled time and bold font indicates vectors of S sampled values (a circular shift is used for ${\mathbf{x}}_1(t-1)$ in the training set; edge effects are negligible).

In order to minimize the energy of the residual, the problem was pseudo-inverted after whitening the matrix X (which is needed to satisfy the sphericity of residuals [37]). SVD was applied in [37], but it is estimated on the signals to be processed, hindering real-time application in practice. To remove this issue, we computed the whitening matrix by solving an eigenvalue problem for the covariance of the predictors defined on training data. Eigenvectors associated to eigenvalues close to zero (i.e., smaller than ${10}^{-10}$) were removed (they could be due to the removal of an artefactual component). Moreover, a constant equal to one thousandth of the maximal eigenvalue was added to all eigenvalues, in order to regularize the computation of their reciprocal (needed for whitening). Test data were then approximately whitened by using such a whitening matrix (which was fixed and only based on training data, so that it could be applied as a pre-processing to new test data as soon as they are acquired).

In order to smooth the output of the filter, we applied a low-pass exponential filter of order 2. Then, a match filter with template extracted from the training set was applied. The template was defined as the average of aligned MRCPs on 1-s epochs preceding movement intentions and zero in the previous and following 1-s epochs. Examples of estimated templates and outputs of the corresponding match filters are shown in Figure 3.

The output of a match filter was then compared to a threshold computed based on the Receiver Operating Characteristics curve applied on the training data. A balanced classification problem was considered (both in the training and test sets) selecting 1-s long epochs either immediately before a movement intention (MRCP) or at the middle times between two consecutive movement intentions (rest; an additional epoch was taken 5 s before the first movement intention to get the same number of MRCPs and rest epochs).

The maximal order of the temporal filter was $N=5$ and the maximum power of the non-linear terms was $Q=3$. Many different classifiers were then obtained, i.e., $(N+1)\times Q=6\times 3=18$ (notice that we considered also the cases $N=0$, i.e., only spatial filter, and $Q=1$, i.e., linear processing of the signals). In addition, the non-linear spatial filter considered in our previous work [37] was included in the set of classifiers. They were all tested in five-fold cross-validation on the training set to select the three classifiers with best accuracy. Finally, a majority voting on those three best classifiers was used to get the classification of the test data. In fact, we noted improvements with respect to using a single classifier (Wilcoxon sign rank test indicated a statistically significant improvement on the default tests described below, from an overall median accuracy of 77% to 80%) and not much variation when using majority voting among five instead of three classifiers (overall median accuracy of 82%, but no statistically significant variation with respect to using only three classifiers).

2.2.3. Performance Evaluation

The two algorithms (NL-SF and NLSTF) have been compared in terms of accuracy, true positive rate (TPR) and false positive rate (FPR), in motor execution (ME) or imagination (MI) conditions. The default condition, using 70% of MRCPs for training and all the nine EEG channels, was compared with the following cases (also considered in [37]):

1.

The training data was reduced to the 40% of the MRCPs;

2.

Only six channels were used, removing the electrodes F3, P4 and Fz (central channels were kept as they are the most important for MRCP detection; of the remaining channels, we removed one on the left hemisphere, another on the right hemisphere, and the last on the midline; the same choice was considered in [37]).

Furthermore, as additional test, both NL-SF and NLSTF were compared to the optimal spatial filter, OSF [19] (using the default conditions) to confirm the superior performances already found in [37] when comparing NL-SF versus OSF.

Possible statistical differences between conditions of interest were tested in paired comparisons, using the Wilcoxon signed rank test.

3. Results

Figure 2 shows an example of signals. Raw data and their counterpart after removing jumps are shown in Figure 2A; the data obtained after filtering and cleaning from artifacts are given in Figure 2B.

Examples of outputs of the filters NL-SF and NLSTF are shown in Figure 3. The same data as in Figure 2 are considered. In the case of NLSTF, the 3 optimal filters have been used to process the data (the orders of the temporal filters were 3, 0 and 1, respectively; the orders of the polynomial non-linearity were 1, 3 and 3, respectively). Templates (shown in Figure 3C) are obtained by averaging and then used by match filters, whose outputs are in Figure 3A,B.

Figure 4 considers the processing of the entire dataset (default tests). It shows the number of delayed data and the non-linearity used in the best filters selected for MRCP identification. As a majority voting among the outputs of three filters was performed for each of the 16 subjects, 48 cases are displayed for ME and for MI. Notice that there are many cases in which a temporal filter is important: a simple spatial filter is selected 12 times in ME and eight times in MI tests, so that there are four cases for ME and eight for MI in which it was not among the three best filters. Moreover, the NL-SF was selected among the three best filters only in few cases (five times in ME and never in MI conditions).

Figure 5 compares the performances of NL-SF and NLSTF in all tests, considering both default and cases with reduced number of EEG channels or training data. Notice that NLSTF has a larger median accuracy in all tested conditions. As data were split into ME and MI, sometimes, the size of the sample was not sufficient to get statistical significance. However, when pooling together the ME and MI cases, the accuracy was always statistically larger for the NLSTF (p < 0.001, p = 0.007 and p < 0.001, for the default, reduced number of channels and reduced training set cases, respectively); moreover, the TPR was always statistically larger for the NLSTF (p < 0.001, p = 0.032 and p < 0.001, respectively); on the other hand, the FPR was not statistically different.

The comparison of the non-linear filters with OSF confirmed the lower performances of the latter. Tested on default conditions, the median accuracies showed statistically significant improvements when using the non-linear filters. Specifically, the improvements of median accuracy with respect to OSF were 19.3% (p < 0.001) and 28.9% (p < 0.001) in ME and 15.4% (p = 0.0017) and 20.0% (p < 0.001) in MI, for NL-SF and NLSTF, respectively.

4. Discussion

The MRCP reflects the execution or imagination of a movement. Its automatic identification is finding many applications, e.g., in commercial systems [41], neuro-motor prosthesis [42], neurofeedback [43], identification of different movements [44]. It has been also investigated in combination with the sensory motor rhythm [45], in order to increase identification performances. Movements of different parts of the body have been investigated, e.g., hand [46], leg [19], tongue [47]. Many properties of MRCPs have been studied, finding details reflecting the speed and force of the movement [21], its trajectory [22], and the variations in case of pathology [48]. Its identification from single trials could allow for a natural control of a BCI [46] and support patients affected by neurological conditions such as locked-in syndrome [1], amyotrophic lateral sclerosis [2], stroke [3] and cerebral palsy [4].

Different methods have been proposed in the literature to identify MRCPs. Some of them have shown good performances, but they need to process epochs of EEG, making the application difficult in self-paced applications: for example, very good performances have been obtained by the Linearity Preserving Projections with Linear Discriminant Analysis [49] and by the Adaptive Riemann Kernel associated to a Support Vector Machine [50]. In this work, we focused on a filter, which has the advantage of processing the data in real time, allowing for an asynchronous control of the BCI and self-paced training.

Different filters have been proposed to focus selectively on the cortical response of the motor area (e.g., the Laplacian filter, found to be the best choice among simple filters [51]) or to emphasize the energy of the EEG during movement intention and reduce it otherwise (e.g., CSP and OSF [19]). However, those approaches do not impose the MRCPs to be similar in different trials, so that its identification after filtering could be difficult. Moreover, forcing the energy to be large during motor intention (regardless of the output waveform) makes CSP and OSF prone to outliers. In order to get similar MRCPs and avoid being biased by outliers, we introduced a non-linear spatial filter (NL-SF) that maps the EEG of a training dataset to a prototype which is the same during motor intentions and zero otherwise [37]. This method outperformed the OSF (which was the state-of-the-art alternative) when using a match filter to identify the MRCP [37].

However, the tests of our previous work were performed in optimal conditions, i.e., after cleaning the entire data [37] (indeed, the cleaning procedure was applied on the whole dataset, including also the test data). Moreover, the EEGs were whitened by SVD, which is a data driven procedure applied on the entire data. Finally, our approach included a stochastic variability, which made its output a bit different even when applied on the same data.

In the present work, we introduce a new approach which is built upon the previous NL-SF. Again, we consider a filter that maps the training EEG data to the same prototype of MRCPs, in order to facilitate its identification based on a simple match filter. The innovations with respect to the NL-SF are the introduction of a temporal filter (in addition to the spatial one) and the use of simple polynomial non-linear processing, obtaining a non-linear spatio-temporal filter (NLSTF). In this way, the new filter has two degrees of freedom: the order of the temporal filter and the order of the polynomial processing. Thus, more filters are defined and can be used to make a more stable identification of MRCPs, by majority voting on more classifications. Specifically, the best three filters were identified based on cross-validation on training data and then used to discriminate between MRCP and rest epochs. Different alternatives could also be explored in the future, e.g., an optimal weighted integration of the outputs of the best filters, considering their performances in cross-validation.

A simple pre-processing of the data was performed, removing jumps and frequency components out of the range of interest. The components affected by artifacts were also identified and removed (Figure 2). The filter was fit only on training data. Thus, the spatial filter identifying blink artifacts was defined only on training data and then kept fixed.

Then, linear problems were defined for the estimation of the weights of the spatial and temporal filters (the processing of the data was non-linear, and powers of the EEG signals were included into the predictors; however, the problems for the estimation of the weights of the filters were still linear). Whitening of the data was useful to optimize the solution of those problems: it was defined by solving an eigenvalue problem on training data and then eigenvectors and eigenvalues were kept fixed for processing test signals. This approach was applied both to NL-SF and to NLSTF, to prepare the data before filter optimization and application to test data. Examples of their outputs after template matching are shown in Figure 3. The templates were obtained by averaging on training data. The three best NLSTFs show similar templates, but small differences can affect the classification of borderline epochs. Thus, majority voting was useful to get more stable estimations (statistically significant improvements were found considering majority voting on three filters instead of using a single one; a not significant improvement in performance was obtained by majority voting with five filters).

The method is fully adapted to the specific data of the subject: not only the weights, but also the types of the filters (i.e., the orders of the temporal filter and of the non-linear processing) were chosen to improve the performances on training data. By looking at the filters most selected for the classification (Figure 4), we realize that they are very different: the order of the temporal filter, when pooling together the tests on ME and MI, is almost uniformly distributed between 0 (i.e., simple spatial filter) and 5 (the maximal order considered); concerning the non-linear processing, the least selected approach was NL-SF (which was also included among the possible choices), followed by the simple linear method. Thus, both temporal filtering and non-linear processing emerged as useful approaches to improve classification performances.

The performances of the two filters are given in terms of accuracy, TPR and FPR (Figure 5). Tests with a reduced number of EEG channels (six instead of nine) and a smaller training set (20 instead of 35 MRCPs) were also included. Both conditions (which are the same as those considered in [37]) challenge the filters, as less information is provided. However, they offer important opportunities. In fact, a BCI system with fewer EEG channels is cheaper (in terms of hardware needed, storage memory, and power consumption for recording/transmission/processing), less cumbersome, more convenient, and more stable to possible connection problems; a shorter training session is less troublesome for the user, who sometimes needs to recalibrate the system. Our tests show that the NLSTF has an accuracy that is larger than that of NL-SF in all conditions. This is due to a better sensitivity, i.e., the ability to identify the true MRCPs. On the other hand, the identification of false positives is equivalent for the two methods. The reduction of information provided to the filter had a negative effect on the performances, slightly reducing the median accuracy; however, significant effects were identified only in a few cases (i.e., lower accuracy for reduced EEG channels in MI tests and for reduced training in ME).

The tested filters outperformed OSF, which in turn was found superior to the classical filters Laplacian and CSP in detecting MRCPs [19]. These results confirm, based on the actual data (cleaned using only the training set), the importance of the non-linear processing, already found in [37] (where OSF and NL-SF were compared).

In summary, our innovative NLSTF allows to identify the MRCP (by template matching) with better performances than our previous state-of-the-art NL-SF. The improvements are also found with a low number of channels and in the case of a reduced training set. Even if the accuracy is still limited and there is some inter-subject variability, performances of our method can be improved if the patient trains by self-paced sessions [52].

5. Conclusions

We discuss an innovative non-linear filter for MRCP identification in BCI applications. A previous approach had state-of-the-art performances, but included the following problems: whitening was implemented by SVD, requiring the test signal to be available; artifact identification and removal was applied to the entire data; a stochastic choice affected the non-linear transformation, making the performance not stable. Our new method solves these problems, being completely designed on training data and using simple polynomial non-linearities. It is a non-linear spatio-temporal filter, feasible for self-paced applications. By choosing between different input data (EEGs with different delays and elevated to different powers), multiple filters are designed; thus, our approach has some degrees of freedom to better adapt to the user. The filters with best cross-validated performance are selected and used in test data by majority voting. This innovative method outperforms the previous approach when they are applied on the same data with same pre-processing. It could be of interest in online BCI system design.