Next Article in Journal
Design and Optimization of Laser Processing Control System and Process Parameters for Micro Crystal Resonator Frequency Modulation
Next Article in Special Issue
Retrospective Analysis of Functional Pain among Professional Climbers
Previous Article in Journal
The Impact of Personality and Demographic Variables in Collaborative Filtering of User Interest on Social Media
Previous Article in Special Issue
MATLAB Analysis of SP Test Results—An Unusual Parasympathetic Nervous System Activity in Low Back Leg Pain: A Case Report
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

The Application of Integration of EEG Signals for Authorial Classification Algorithms in Implementation for a Mobile Robot Control Using Movement Imagery—Pilot Study

Faculty of Electrical Engineering, Automatic Control and Informatics, Opole University of Technology, Prószkowska 76 Street, 45-758 Opole, Poland
*
Author to whom correspondence should be addressed.
Appl. Sci. 2022, 12(4), 2161; https://doi.org/10.3390/app12042161
Submission received: 2 February 2022 / Revised: 11 February 2022 / Accepted: 15 February 2022 / Published: 18 February 2022
(This article belongs to the Special Issue Advances in Technology of Brain-Computer Interface)

Abstract

:
This paper presents a new approach to the issue of recognition and classification of electroencephalographic signals (EEG). A small number of investigations using the Emotiv Epoc Flex sensor set was the reason for searching for original solutions including control of elements of robotics with mental orders given by a user. The signal, measured and archived with a 32-electrode device, was prepared for classification using a new solution consisting of EEG signal integration. The new waveforms modified in this way could be subjected to recognition both by a classic authorial software and an artificial neural network. The properly classified signals made it possible to use them as the signals controlling the LEGO EV3 Mindstorms robot.

1. Introduction

An electroencephalogram (EEG) is an image of electric activity pattern coming from the cerebral cortex. Since the electric activity of the cortex is measured in microvolts, it must be increased many times—about a million times—to be observed on a computer screen. Presumably, most registered signals come from the neurons in which a large number of bioelectric phenomena, such as action potentials, post-synaptic potentials (PSP) and the phenomenon of a long-term neuron depolarization take place. Action potentials are connected with a short-term (10 ms or shorter) local current flow in an axon and a very limited potential field. Hence, it is not likely that they are the basis of electroencephalographic phenomena. PSP last much longer (50–200 ms) and they have a significantly larger electric field. Therefore, it is more likely that they become main EEG generators. A long-term depolarization of neurons or even glia may also cause changes in EEG. In a healthy brain, neuron activation potential is propagated along the axon to the nerve ending, where neurotransmitters are released. However, it is the synaptic potential that is the most important source of an electroencephalogram [1,2,3,4].
The brain-computer interface is the technology that makes a direct contact between the brain and an external device possible. The analysis of electroencephalogram signals has its contribution in the field of BCI technology investigations [5,6]. Principles of quantitative electroencephalography (EEG) related to neurotherapy are discussed in [7]. The general properties of the human EEG as well as the issues and obstacles associated with quantitative methods are described in this work [7]. The basic principles of event-related EEG/MEG synchronization (ERS) and desynchronization (ERD) are described in [8].
BCI technology dates back to the 1990s and though it is relatively new, it may significantly revolutionize in the future the solutions with which people communicate with computers and other devices. All currently commonly used methods that may be used by a user for giving commands to a computer are based on muscle movements [9]. Brain-computer interfaces which use electroencephalography (EEG) have been investigated for many years as a communication and control means for physically disabled persons. Through trainings, people may learn to use imaginary motor movements as computer input data or control assistive technology.
Article [10] provides review classification algorithms used to design brain–computer interface (BCI) systems based on electroencephalography (EEG). It compares the classification algorithm for a specific BCI in terms of performance. Authors found that the classification algorithms for EEG-based BCIs can be divided into four main categories: adaptive classifiers, matrix and tensor classifiers, transfer learning and deep learning, as well as other classifiers. The approach involving the use of a neural network, which was used in this pilot study, was also discussed in these publications [10,11].
The article [12] illustrates the functions of the OpenViBE platform with two VR applications based on BCI. In the first one, users can move a virtual ball by imagining hand movements. The second one allows control of a virtual spaceship using real or imagined foot movements. The online tests of the application, taking into account the evaluation of the platform computational performances, showed that it can be useful for designing VR applications controlled by BCI.
The paper [13] presents an introductory overview of BCI platforms. The main research focus was on the OpenViBE platform. Importantly, the results were contrasted with other well-known BCI platforms such as BCI2000, BCILAB, Biosig and Fieldtrip. The work considers BCI platforms as development and experimentation toolkits for researchers, clinicians and others wish to develop BCI and related aspects. The authors described the BCI processing steps, which consist of: signal acquisition (standardization), processing and classification of the obtained signals, and the final feedback.
The recurrent neural network (RNN) scheme to perform the trajectory control of redundant robot manipulators using remote centers of motion (RCM) constraints was proposed in article [14]. This is an artificial intelligence approach that can be used, for example, in surgical operations. Demonstration experiments are conducted in a laboratory setup environment using KUKA LWR4+. Their purpose was to verify the correct functioning of the project.
The literature review also includes a novel, multi-sensor, guided hand gesture recognition system for a surgical robot teleoperation. In this case, the multi-sensor data fusion model was used. The authors used an approach consisting of a multi-layer Recurrent Neural Network (RNN). The network included a a long-term memory module (LSTM) and a dropout layer (LSTM-RNN) is proposed for the classification of multiple hand gestures. Then, the recognized hand gestures are used to perform a set of human-robot collaboration tasks on a surgical robot platform. Tests have shown that classification performance and prediction time are similar between the LSTM-RNN model and traditional machine learning (ML) algorithms [15].
A novel multimodal fusion architecture is proposed in [16]. The goal is to calculate the respiratory and exercise parameters and simultaneously identify human actions. The research aims to understand breathing patterns during daily activities. It is possible thanks to the development of a wearable respiratory and activity monitoring (WRAM) system. A hybrid hierarchical classification (HHC) algorithm combining deep learning and threshold-based methods is designed.
Emotiv EPOC Flex Gel is a relatively cheap, light, wireless BCI headset which ensures control and effective measurement systems [17]. In the case of these investigations, an extended version of the headset with 32 electrodes, i.e., Emotiv EPOC Flex Gel, was used. Each sensor captures four different brain waves (delta, theta, alpha, and beta) in real time. The investigations using this device (version Flex Gel) are still very rare in spite of the fact that it is one of the most accurate and the most complex devices of the manufacturer [18]. Emotiv EPOC Flex Gel was used, among others, in the works pertaining to the comparison of the influence of smoothing filters on the quality of data registered with the Emotiv EPOC Flex Brain-Computer Interface headset during sound stimulation [19]. It was also used in the work investigating the changes in the activity brain waves during audio stimulation using non-invasive brain-computer interfaces [20,21]. Other investigations using the set discussed in the paper were presented in the paper on the training system based on brain-computer technology in the field of motor imaging [22] and the publication on recognition of eyeball movements based on EEG [23,24,25,26]. The use of the waves marked with Greek letters: Alpha (8–10 Hz), Beta (13–30 Hz), Delta (<4 Hz), Gamma (>30 Hz), Theta (4–7 Hz) and Mu (8–12 Hz) is currently applied for classification of EEG frequency range. It should be noted that the last ones strictly depend on brain activity and performance. They are also slightly similar to Alpha waves [19,27,28,29,30].
LEGO Mindstorms EV3 set is an extensive environment for learning mobile robot control. It offers many possibilities for each advancement level in operation of this type of devices. Its popularity made that this system, based on the Linux operating system, can communicate with many popular programs such as, for example, Matlab/Simulink [31]. The investigations carried out with LEGO set cover various aspects. However, there are few investigations pertaining to full control of the vehicle built based on EV3 cube (move forward and backward, to the right and to the left), although there are many investigations referring to the application of EEG signals and BCI technology in physiotherapy, rehabilitation and physiotherapeutic robotics [32,33,34].
The aim of this research was to design a system for classifying the EEG signal in the control of a mobile robot. When analyzing the solutions available in the literature, an original approach was chosen. A method with filtering the raw signal followed by periodic signal integration was designed and tested. The data obtained in this way was classified by an artificial neural network and a proprietary expert algorithm. The resulting data allowed to control the constructed robot. The conducted research can find many applications, because their use is possible in the design of rehabilitation apparatus for disabled people or supporting the movement of wheelchairs.

2. Presentation of Components, Data Acquisition Methodology and the System

The structure of the analysis and EEG signal classification system presented in this paper consists of many components. A raw EEG signal (raw data) was registered with Emotiv EPOC Flex Gel. The acquisition of measurements was performed with dedicated EmotivPRO software. Figure 1 provides more information on how the classification system works, it helps to understand its features and structure. Figure 2 shows the electrodes configuration in the set. It also shows the research environment. Here can be seen a participant in the study, a program (EmotivPRO) running on the computer, and the vehicle constructed by authors based on LEGO EV3 cube.
The electrodes in Emotiv EPOC Flex Gel were named in compliance with the following channels: Cz, Fz, Fp1, F7, F3, FC1, C3, FC5, FT9, T7, TP9, CP5, CP1, P3, P7, O1, Pz, Oz, O2, P8, P4, CP2, CP6, TP10, FC6, C4, FC2, F4, F8 and Fp2. Table 1 shows the names of the particular channels, which are presented on visualization from Figure 2.
Two persons took part in the investigation, who were subjected to a few hundred tests. The signals were registered at rest and giving order to move forward, backward, to the right and to the left. Being concentrated is a significant element determining the test results. A person subjected to the test should be in a state of deep concentration, otherwise the course of the tests will surely be disturbed and thus their final result will be incorrect. After collecting a large number of measurements, their further processing and classification using various methods was possible, which is described in detail in Section 2.1.

2.1. Presentation of the Complete System

A raw EEG signal archived by EmotivPRO software may be exported and investigated in any software environment. Figure 3 shows a signal analysis dashboard in the set manufacturer’s software
In the case of the investigations carried out, all analyses and classification methods were performed with Matlab, a platform for programming and numerical calculations. The system operation diagram is shown in Figure 1. This program makes proper signal filtering for their further proper analysis and classification possible. In the next stage, the signal is integrated after time for each of 32 electrodes separately. It takes place within one second for the whole period of the signal duration. In this way, one-second integrated samples are obtained. They will make it possible to determine and clearly classify the type of a signal based on the evoked potential subjected to integration (Details in Section 2.2).
The obtained samples containing the signal integrated after time can be subjected to proper classifying algorithm. Two algorithms classifying integrated samples of the EEG signal collected from the electrodes were suggested. Figure 1 effectively shows the discussed issue. The first of them is a classic algorithm comparing the values of the particular electrodes. The other one is based on a multilayer neural network. A user may choose between the two classifying systems. The recognized signal evokes a move order forward, backward, to the right or to the left. It can be passed to the robot to perform the requested orders (see Figure 1).

2.2. Signal Filtration with a High-Pass Filter

The EEG signal coming from 32 electrodes should be subjected to proper filtering. A high-pass filter with a sampling frequency of f s = 1000 Hz and bandwidth of f p = 200 Hz were used. A signal processed in this way may be subjected to further investigations. It is necessary in order to apply proper algorithms classifying given signals of bioelectric brain activity. Figure 4 shows the effects of high-pass filtration applied on the electroencephalographic signals coming from 32 electrodes of the headset. The plots presented in this way effectively facilitate the analysis of various states or mental orders. The example from Figure 4 shows the potentials evoked when a tested person was at rest.
Literature contains many investigations concerning a widely understood classification of EEG signals. Numerous investigations concern, among others, the use of the analysis of time and spectral features, the analysis of brain waves (delta, theta, alpha and beta) or the investigations of a hierarchic structure based on quantum mechanics to implement feature extraction and the EEG signal classification [35,36]. Not a very large number of the investigations using Emotiv Epoc Flex Gels offers the possibility of suggesting new approaches to the issues connected with the analysis and effective classification of electroencephalographic signals. A large number of electrodes reading the measurements, which ensure an accurate potential reading, can offer an original solution. Details are presented in the next stages of work.

2.3. Preparing the Signal for Classification

The investigation was based on the analysis of the archived signal samples provided by dedicated software. Signal measurements are taken with frequency of f e = 128 Hz. Hence, each second of the measurement gives 128 signal samples. The application of rolling integration is a significant and innovative idea. Its task is signal integration from each electrode separately in the time of 1 s.
An example situation may be assumed that a registered 10 s of a signal will consist of V = 1280 samples and a 4-s signal will consist of V = 512 samples. Variable V denotes the number of samples in an archived signal. It is necessary for calculating variable k, which shows the number of full one-second periods in the signal measured. The dependence is shown in Equation (1).
k = V 128 ,
Variable S is defined for each electrode: S 1 S 32 (Equation (2)). Each of them is a table containing signal samples of a given electrode integrated for one second. Hence, the particular values of the integrated waveform are given under consecutive values of table S ( 1 k ) . For example, taking into account Equations (1) and (2), for a 5-s signal, the integrated samples coming from the first electrode will be contained in table S 1 ( 1 k ) , where k = 5 .
S 1 32 ( 1 k ) = ( ( ( 1 k ) 1 ) · 128 ) + 1 ( 1 k ) · 128 | E 1 32 | .
The operation on the signal takes place in a continuous way, where the absolute value of the 32 electrodes marked E 1 32 is integrated by the period of 128 samples, after which the sample value is written to value S. Thus the consecutive data come from the signal range 1 ÷ 128 , 129 ÷ 256 , which is shown in Equation (2).
Such an original approach is not pointless. This way the chances for erroneous treatment of a signal in the case of interference or other negative artefacts are minimized. It could happen in the case of EEG signal integration for a shorter period. One-second integration periods make it possible to obtain uniqueness in further signal classification. Section 3 contains the results of the signal processing carried out. It presents detailed results of the performed investigations.

3. Results of the Signal Analysis before Classification

The purpose of this section is to present the results of the performed procedures from Section 2.2 and Section 2.3. The tests, in which the measurements were taken for their further classification were carried out on two persons. The persons subjected to tests were aged 23 and 50. They were asked to evoke orders of various types of a mobile robot movement.

3.1. Order to Move Forward and to the Right

Since it is a pilot study, EEG signal measurements presenting evoking several dozen attempts of each type of robot movements, i.e., move forward, backward, to the right and to the left based on motor imagery were collected from each of the two study members. The measurements at rest were also collected. The persons tested did not perform any additional movements which might cause the appearance of biological artefacts in the archived signal, i.e., eye blinking, eyelid raising, lip movement. This paper was written based on over a dozen examples of collected EEG signal measurements and their further classification together with the resulting mobile robot movement.
Filtered EEG signals with the values obtained by integrating one-second measurement duration periods can be read from the characteristics shown in Figure 5 and Figure 6. Figure 5 contains a set of measurements pertaining to evoking potential at rest and also imitating a move forward and to the right. These are the measurements taken with the participation of the first person tested. The investigations concerning the second person are presented in the same way. The results are shown in Figure 6. In this case the person tested was asked again to perform the sequence of movements: at rest, move forward, at rest, move to the right. Exactly the same course of orders took place in the case of the first person tested (See Figure 6).
Analyzing the characteristics in Figure 5 and Figure 6, a full repeatability may be observed. For the first person tested (Figure 5), two filtered signal measurements together with the integrated values obtained based on them were presented. To present the situation more clearly, only more significant channels taking part in the investigation were presented in the legend. They were extrapolated mainly by using the expert method, based on numerous tests. The same procedure was applied for testing the second person (Figure 6). At this stage of testing, the biggest participation of the following channels was observed: F3, F, F8, T8, FT10, FC6, C4, FC2, F4. The integrated values obtained marked as I n t e g r a t e d s i g n a l s will be used for signal classification (Section 4 and Section 5).

3.2. Order to Move Backward and to the Left

At this stage of investigation, there was continuation, but the orders of movement imagery were altered. For a full control of the movement of the mobile robot set for this stage, the sequence: at rest – move backward – at rest – move to the left were studied. A bigger number of measurements with the participation of two persons was taken again, but in the papers, only four examples, two for each person, were shown. It was the same in case of Section 3.1. Figure 7 and Figure 8 show the results of filtration and integration of the archived EEG signal. A full repeatability and advantage of one-second integration period, owing to which uniqueness is possible, can be observed. For this stage of testing, the biggest participation of the channels: PO10, PO9, F7, F3, FC1, C3, FC5, FT9, T7 is manifested. Their participation is crucial for move backward and to the left orders. They will be also vital for classification purposes because based on the Integrated signals waveforms, two suggested classifying systems based on expert approach and neural networks will operate (Section 4).

4. Proposition of Signal Classification Algorithms

Integrated EEG signals of the particular channels from Section 3 may be used by designed classification systems. Based on the literature review and provisions from Section 1 and Section 2, the following elements will be used for this purpose: a classic expert system and algorithms in the field of artificial intelligence, i.e., the feed-forward neural network.
Both systems have their advantages and disadvantages, of which a designer should be aware. A classic algorithm may not be able to foresee well enough all types of interference to properly classify a given type of a signal. A neural network, on the other hand, may be impervious to an unclear signal due to, for example, incorrect contact of electrodes with the head of the test participant.

4.1. The Classification System Based on the Classic Expert System

In order to design a classic classifying algorithm based on the expert approach, a good knowledge of the properties of the object, signal or phenomenon under study is required. It is no different in the case of recognition of the type of the EEG signal. In order to determine a proper structure of a classic algorithm, it is necessary to get to know well the properties and the range of values of the EEG signal at rest, They should be determined for each electrode separately, as the voltage value for all channels is different. Thus, a few signals corresponding to the restful state were collected and the range of changeability of their values was determined. Next, the deviations from standard for the particular channels when evoking moves forward, backward and to the left were to be investigated.
Based on the gathered knowledge on the activity of the particular channels during various types of evoking movement, an algorithm which will classify future signals containing information on evoking an order of a given movement may be designed. This way it is known that an exemplary move to the right will be characteristic of deviation from standard (not always simultaneous and the same) of channels: T8, FT10, FC6, C4, FC2, F8 and F4. The move to the left, however, will be characteristic of the activity (not necessarily simultaneous) of channels: F7, F3, FC1, C3, FC5, FT9 or T7. The details pertaining to the activity of the channels during various types of movements can be seen on the figure legends in Section 3.
Detection of the particular types of channel activities makes it possible to expose the right signal which will make it possible to control a mobile robot. In the following subsection, an algorithm basing on the use of artificial intelligence for the EEG signal classification will be suggested.

4.2. Presentation of the Classification System Based on the Neural Network

In this part, an advanced approach containing a neural network algorithm will be analyzed. Two neural networks of the same number of nonlinear hidden layers were suggested. The first one was a Jordan recurrent neural network, and the other one was a feed-forward neural network. The choice of the number of layers, activation functions and the number of neurons in each layer was selected based on trials and errors using the expert approach. In order to carry out the procedure of the neural network training, a training data set consisting of input and output data had to be prepared. In the process of training input training set P was used, which used a matrix of patterns p of the number of rows i = 32. The input values were integrated one-second EEG signal periods. The number of rows depends on the number of electrodes in the set. Output set T was also necessary in the process of the neural network training. It consisted of the matrix of input patterns t of the number of rows j = 5. Value j results from five types of recognized signals, i.e., at rest, move forward, backward, to the right and to the left. Input and output training patterns included the same number of elements n = 250. The form of the data presented may be supported by the formulae [37]:
P = p ( 1 ) p ( i ) = p 1 ( 1 ) p 2 ( 1 ) p n ( 1 ) p 1 ( 2 ) p 2 ( 2 ) p n ( 2 ) p 1 ( 31 ) p 2 ( 31 ) p n ( 31 ) p 1 ( 32 ) p 2 ( 32 ) p n ( 32 ) .
The matrix of output patterns used looks as follows:
T = t ( 1 ) t ( j ) = t 1 ( 1 ) t 2 ( 1 ) t n ( 1 ) t 1 ( 2 ) t 2 ( 2 ) t n ( 2 ) t 1 ( 3 ) t 2 ( 3 ) t n ( 3 ) t 1 ( 4 ) t 2 ( 4 ) t n ( 4 ) t 1 ( 5 ) t 2 ( 5 ) t n ( 5 ) .
It was decided to choose a network consisting of four hidden layers of the neural network. In the first of them, tangensoidal function ( t a n s i g ( . ) ) with 35 neurons was used. The second layer is the activation function l o g s i g ( . ) consisting of 30 neurons. Radial basis function ( r a d b a s ( . ) ) with 20 neurons constitutes the third layer. The last hidden layer is again the function t a n s i g ( . ) with 15 neurons. The last part, the feed-forward neural network consists of a single layer containing five neurons with linear activation function p u r e l i n ( . ) . In such a structure of the designed neural network, the linear part gives the effect of adding up nonlinear neuron activation functions. They can be found in the front layers. An output signal of the designed neural network classifying a signal will be according to formulae [37,38,39]:
y = W ( 5 ) f ( 4 ) [ ( W ( 4 ) f ( 3 ) + b ( 4 ) ) ( W ( 3 ) f ( 2 ) + b ( 3 ) ) ( W ( 2 ) f ( 1 ) + b ( 2 ) ) ( W ( 1 ) u + b ( 1 ) ) ] + b ( 5 ) .
Figure 9 shows a reference diagram of the designed feed-forward neural network. Graphic interpretation of Equation (5) is shown in the same figure.
According to Formula (5), on the reference graphics of network Figure 9, the vector of output data is marked with y, and the network inputs, which are integrated values of the EEG signal from 32 electrodes, are marked with u.
The suggested neural network structure from diagram Figure 9 proved to be better than the other Jordan neural network tested. This fact is confirmed by the learning procedure results shown on characteristics Figure 10.
The following section will present the classification results of integrated EEG signal periods as well as the results of robot control.

5. Observation of Signal Classification Results by Algorithms

Verification of operation of the designed classifying system was the last stage of the investigation. For this purpose, the functioning of the classic algorithm and the solutions from the scope of artificial intelligence were tested. The tests were performed using archived EEG signals subjected to initial processing in Section 3. Figure 11, Figure 12, Figure 13 and Figure 14 show the results of signal recognition pertaining to the investigation participants by both algorithms presented in Section 4.
The parts of characteristics marked as Classification results show integrated signals and corresponding classification results obtained by the authorial classic algorithm and the artificial neural network. It can be observed that both suggested systems show a full recognition of the registered signal. The results are discussed in detail in Section 6.
The figures showing the control string implemented into LEGO EV3 cube are tagged as Control and rotation of the robot’s motors. Verification of the robot movements as instructed can be done easily by observation of rotation of the drives of the right and left wheel drives. Figure 11, Figure 12, Figure 13 and Figure 14 show that for the move forward both wheels are moving, for the move to the left only the right motor is moving, move to the right is possible due to rotations of the left motor. Move backward is evoked by rotation of both motors in opposite direction.

6. Discussion

In discussing the results, it should be noted that this paper presented a few original methods and approaches to the issue of EEG signal classification that have not been used so far.
Within the research work, two authorial algorithms for data classification were designed. The first of the algorithms studied was based on a classic expert approach to the task of the recognition of data subjected to filtration and integration. The examples from Section 3 and Section 5 prove that it did very well. Its operation made an absolutely correct classification of the measurements taken possible. The results of this procedure together with control signals and the measured robot drive rotations are shown in Figure 11, Figure 12, Figure 13 and Figure 14.
Designing the second algorithm, it was decided to use artificial intelligence as a classifier of processed EEG signals. Based on the obtained learning results of the two neural networks under study for signal classification, it is clear that the feed-forward neural network obtained better parameters than the Jordan recurring neural network (see Figure 10). Therefore, it was decided that the system of correct EEG signal recognition will be based on it. The tests and checking the system capabilities required the performance of a few learning trials with various numbers of hidden layers and also with different approach to the selection of the activation function of the layers of the neural network used. It also turned out that a set of 250 training data with 50 integrated signal samples each give satisfying results. The neural network has good generalizing properties. It is confirmed by the results shown in Figure 11, Figure 12, Figure 13 and Figure 14, where the classification results with algorithms based on artificial intelligence and classic approach are presented. Such research methodology made it possible to present clear effects of the mobile robot control. We can observe robot move forward, where rotations increase for both drives, and for the move to the right, when only the left wheel moves (see Figure 11 and Figure 13). Similarly, for the reverse control signal we can see reverse drive rotations, and the move to the left generates rotations only of the right vehicle motor (see Figure 12 and Figure 14).

7. Conclusions and Future Work

Summarizing the results obtained in the experiment it can be claimed that the tasks set out for the algorithms classifying integrated EEG signals have been fulfilled. Summing up the obtained results (Section 6), it should be noted that the investigations were carried out on an Emotiv EPOC Flex headset. Its use is quite rare in comparison with other sets of this manufacturer. Taking into account the possibility of its application in robotics components control, it can be observed that the methods of EEG signal classification presented in this paper provide new possibilities for the development of investigations in this direction.
Taking into account the expert approach, it can be observed that carrying out a bigger number of investigations and an improved signal analysis may even further improve algorithm in the context of resistance to various types of interference and artefacts.
The paper presented new approaches which surely require further research and tests. Thus it generates an open issue worth further experiments. At the same time, it will not be possible to assess the authorial approach consisting of integrating the EEG signal from the particular electrodes. In the future, it will also be worth investigating the efficiency of algorithms for a bigger group of people subjected to tests, which will allow for a deeper testing of their functioning. The main limitation of the approach is the low number of study participants. The reason is that it is only a pilot study. However, it has undoubted opportunities for further development. Certainly, tests should be carried out on a larger number of participants, which will make the algorithms resistant to fluctuations in the signal value ranges. There is also a great opportunity to expand the system in the field of deep learning. Further research may result in new approaches in this matter, as well as the improvement of existing solutions. Artificial intelligence and other methods can also help in its application in various solutions, such as the movement of disabled people or neurological rehabilitation. For this purpose, it may be necessary to use the solutions from the literature presented in the introduction to the article.
Comparing the obtained results, it should be stated that this article also proposes an original approach to the subject of EEG signal analysis. One of the more important reasons is to use the Emotiv Epoc Flex Gel headset. The use of this set in this type of research is sporadic. Therefore, it shows the originality of the research carried out. However, the most important aspect that allows us to conclude the similarity of the results with the literature review is the solution consisting of periodical integration of the signal. This influences the necessity of a new approach to designing a system based on deep learning, or a classic expert algorithm.

Author Contributions

Conceptualization, D.P. and S.P.; methodology, D.P.; software, D.P.; validation, S.P. and D.P.; formal analysis, S.P.; investigation, D.P.; resources, D.P. and S.P.; data curation, D.P.; writing—original draft preparation, D.P.; writing—review and editing, S.P.; visualization, D.P.; supervision, D.P. and S.P.; project administration, S.P. and D.P.; funding acquisition, S.P. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by National Science Center, Grant number: 2021/05/X/ST7/34, MINIATURA 5.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Not applicable.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Marcuse, L.V.; Fields, M.C.; Yoo, J.Y.J. Rowan’s Primer of EEG E-Book; Elsevier Health Sciences: Amsterdam, The Netherlands, 2015. [Google Scholar]
  2. Thakor, N.V.; Sherman, D.L. EEG signal processing: Theory and applications. In Neural Engineering; Springer: Berlin/Heidelberg, Germany, 2013; pp. 259–303. [Google Scholar]
  3. He, B.; Astolfi, L.; Valdés-Sosa, P.A.; Marinazzo, D.; Palva, S.O.; Bénar, C.G.; Michel, C.M.; Koenig, T. Electrophysiological brain connectivity: Theory and implementation. IEEE Trans. Biomed. Eng. 2019, 66, 2115–2137. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  4. Baravalle, R.; Guisande, N.; Granado, M.; Rosso, O.A.; Montani, F. Characterization of visuomotor/imaginary movements in EEG: An information theory and complex network approach. Front. Phys. 2019, 7, 115. [Google Scholar] [CrossRef] [Green Version]
  5. Tudor, M.; Tudor, L.; Tudor, K.I. Hans Berger (1873-1941)—The history of electroencephalography. Acta Medica Croat. Cas. Hravatske Akad. Med. Znan. 2005, 59, 307–313. [Google Scholar]
  6. Paszkiel, S.; Rojek, R.; Lei, N.; Castro, M.A. A Pilot Study of Game Design in the Unity Environment as an Example of the Use of Neurogaming on the Basis of Brain–Computer Interface Technology to Improve Concentration. NeuroSci 2021, 2, 109–119. [Google Scholar] [CrossRef]
  7. Kaiser, D.A. Basic principles of quantitative EEG. J. Adult Dev. 2005, 12, 99–104. [Google Scholar] [CrossRef]
  8. Pfurtscheller, G.; Da Silva, F.L. Event-related EEG/MEG synchronization and desynchronization: Basic principles. Clin. Neurophysiol. 1999, 110, 1842–1857. [Google Scholar] [CrossRef]
  9. Paszkiel, S. Brain–Computer Interface Technology. In Analysis and Classification of EEG Signals for Brain–Computer Interfaces; Springer: Berlin/Heidelberg, Germany, 2020; pp. 11–17. [Google Scholar]
  10. Lotte, F.; Congedo, M.; Lécuyer, A.; Lamarche, F.; Arnaldi, B. A review of classification algorithms for EEG-based brain–computer interfaces. J. Neural Eng. 2007, 4, R1. [Google Scholar] [CrossRef]
  11. Lotte, F.; Bougrain, L.; Cichocki, A.; Clerc, M.; Congedo, M.; Rakotomamonjy, A.; Yger, F. A review of classification algorithms for EEG-based brain–computer interfaces: A 10 year update. J. Neural Eng. 2018, 15, 031005. [Google Scholar] [CrossRef] [Green Version]
  12. Renard, Y.; Lotte, F.; Gibert, G.; Congedo, M.; Maby, E.; Delannoy, V.; Bertrand, O.; Lécuyer, A. Openvibe: An open-source software platform to design, test, and use brain–computer interfaces in real and virtual environments. Presence 2010, 19, 35–53. [Google Scholar] [CrossRef] [Green Version]
  13. Lindgren, J.; Lecuyer, A. OpenViBE and other BCI software platforms. In Brain–Computer Interfaces 2: Technology and Applications; Wiley: Hoboken, NJ, USA, 2016; pp. 179–198. [Google Scholar]
  14. Su, H.; Hu, Y.; Karimi, H.R.; Knoll, A.; Ferrigno, G.; De Momi, E. Improved recurrent neural network-based manipulator control with remote center of motion constraints: Experimental results. Neural Netw. 2020, 131, 291–299. [Google Scholar] [CrossRef]
  15. Qi, W.; Ovur, S.E.; Li, Z.; Marzullo, A.; Song, R. Multi-Sensor Guided Hand Gesture Recognition for a Teleoperated Robot Using a Recurrent Neural Network. IEEE Robot. Autom. Lett. 2021, 6, 6039–6045. [Google Scholar] [CrossRef]
  16. Qi, W.; Aliverti, A. A multimodal wearable system for continuous and real-time breathing pattern monitoring during daily activity. IEEE J. Biomed. Health Inform. 2019, 24, 2199–2207. [Google Scholar] [CrossRef] [PubMed]
  17. Lang, M. Investigating the Emotiv EPOC for Cognitive Control in Limited Training Time; University of Canterbury: Christchurch, New Zealand, 2012. [Google Scholar]
  18. Emotiv. Epoc Flex User Manual; Emotiv: San Francisco, CA, USA, 2018. [Google Scholar]
  19. Browarska, N.; Kawala-Sterniuk, A.; Zygarlicki, J.; Podpora, M.; Pelc, M.; Martinek, R.; Gorzelańczyk, E.J. Comparison of Smoothing Filters’ Influence on Quality of Data Recorded with the Emotiv EPOC Flex Brain–Computer Interface Headset during Audio Stimulation. Brain Sci. 2021, 11, 98. [Google Scholar] [CrossRef]
  20. Browarska, N.; Kawala-Sterniuk, A.; Zygarlicki, J. Initial study on changes in activity of brain waves during audio stimulation using noninvasive brain–computer interfaces: Choosing the appropriate filtering method. Bio-Algorithms Med.-Syst. 2021, 17, 79–93. [Google Scholar] [CrossRef]
  21. Shih, J.J.; Krusienski, D.J.; Wolpaw, J.R. Brain-computer interfaces in medicine. In Mayo Clinic Proceedings; Elsevier: Berlin/Heidelberg, Germany, 2012; Volume 87, pp. 268–279. [Google Scholar]
  22. Paszkiel, S.; Dobrakowski, P. Brain–computer technology-based training system in the field of motor imagery. IET Sci. Meas. Technol. 2021, 14, 1014–1018. [Google Scholar] [CrossRef]
  23. Antoniou, E.; Bozios, P.; Christou, V.; Tzimourta, K.D.; Kalafatakis, K.; Tsipouras, G.M.; Giannakeas, N.; Tzallas, A.T. EEG-Based Eye Movement Recognition Using the Brain–Computer Interface and Random Forests. Sensors 2021, 21, 2339. [Google Scholar] [CrossRef]
  24. Jafar, F.; Fatima, S.F.; Mushtaq, H.R.; Khan, S.; Rasheed, A.; Sadaf, M. Eye controlled wheelchair using transfer learning. In Proceedings of the IEEE 2019 International Symposium on Recent Advances in Electrical Engineering (RAEE), Islamabad, Pakistan, 28–29 August 2019; Volume 4, pp. 1–5. [Google Scholar]
  25. Yu, X.; Qi, W. A user study of wearable EEG headset products for emotion analysis. In Proceedings of the 2018 International Conference on Algorithms, Computing and Artificial Intelligence, Sanya, China, 21–23 December 2018; pp. 1–7. [Google Scholar]
  26. Zhang, X.; Pan, J.; Shen, J.; Din, Z.U.; Li, J.; Lu, D.; Wu, M.; Hu, B. Fusing of Electroencephalogram and Eye Movement with Group Sparse Canonical Correlation Analysis for Anxiety Detection. IEEE Trans. Affect. Comput. 2020, 1. [Google Scholar] [CrossRef]
  27. Sawangjai, P.; Hompoonsup, S.; Leelaarporn, P.; Kongwudhikunakorn, S.; Wilaiprasitporn, T. Consumer grade EEG measuring sensors as research tools: A review. IEEE Sens. J. 2019, 20, 3996–4024. [Google Scholar] [CrossRef]
  28. Kaur, B.; Singh, D.; Roy, P.P. EEG based emotion classification mechanism in BCI. Procedia Comput. Sci. 2018, 132, 752–758. [Google Scholar] [CrossRef]
  29. Cimtay, Y.; Ekmekcioglu, E. Investigating the use of pretrained convolutional neural network on cross-subject and cross-dataset EEG emotion recognition. Sensors 2020, 20, 2034. [Google Scholar] [CrossRef] [Green Version]
  30. Sasaki, M.; Iversen, J.; Callan, D.E. Music improvisation is characterized by increase EEG spectral power in prefrontal and perceptual motor cortical sources and can be reliably classified from non-improvisatory performance. Front. Hum. Neurosci. 2019, 13, 435. [Google Scholar] [CrossRef] [PubMed]
  31. LEGO. User Guide LEGO MINDSTORMS EV3; LEGO: Billund, Denmark, 2013. [Google Scholar]
  32. Koudelkova, Z.; Dankova, S.; Filip, M.; Dabrovska, M. The Possibility of Using BCI Applications in Physiotherapy. In MATEC Web of Conferences; EDP Sciences: Les Ulis, France, 2019; Volume 292, p. 01033. [Google Scholar]
  33. Lange, G.; Low, C.Y.; Johar, K.; Hanapiah, F.A.; Kamaruzaman, F. Classification of electroencephalogram data from hand grasp and release movements for BCI controlled prosthesis. Procedia Technol. 2016, 26, 374–381. [Google Scholar] [CrossRef] [Green Version]
  34. Lins, A.A.; de Oliveira, J.M.; Rodrigues, J.J.; de Albuquerque, V.H.C. Robot-assisted therapy for rehabilitation of children with cerebral palsy-a complementary and alternative approach. Comput. Hum. Behav. 2019, 100, 152–167. [Google Scholar] [CrossRef]
  35. Aliakbaryhosseinabadi, S.; Kamavuako, E.N.; Jiang, N.; Farina, D.; Mrachacz-Kersting, N. Classification of movement preparation between attended and distracted self-paced motor tasks. IEEE Trans. Biomed. Eng. 2019, 66, 3060–3071. [Google Scholar] [CrossRef]
  36. Li, Y.; Zhou, R.; Xu, R.; Luo, J.; Jiang, S.X. A quantum mechanics-based framework for EEG signal feature extraction and classification. IEEE Trans. Emerg. Top. Comput. 2020, 1. [Google Scholar] [CrossRef]
  37. Majewski, P.; Pawuś, D.; Szurpicki, K.; Hunek, W.P. Toward Optimal Control of a Multivariable Magnetic Levitation System. Appl. Sci. 2022, 12, 674. [Google Scholar] [CrossRef]
  38. Kriegeskorte, N.; Golan, T. Neural network models and deep learning. Curr. Biol. 2019, 29, R231–R236. [Google Scholar] [CrossRef]
  39. Suzuki, K. Artificial Neural Networks: Architectures and Applications; BoD–Books on Demand: Norderstedt, Germany, 2013. [Google Scholar]
Figure 1. Data analysis and robot control systems.
Figure 1. Data analysis and robot control systems.
Applsci 12 02161 g001
Figure 2. Presentation of electrodes configuration and place of research.
Figure 2. Presentation of electrodes configuration and place of research.
Applsci 12 02161 g002
Figure 3. Signal analysis dashboard in EmotivPRO software.
Figure 3. Signal analysis dashboard in EmotivPRO software.
Applsci 12 02161 g003
Figure 4. The observation of raw and filtered EEG signals.
Figure 4. The observation of raw and filtered EEG signals.
Applsci 12 02161 g004
Figure 5. Comparison of filtered and integrated signals- move forward and to the right (person 1).
Figure 5. Comparison of filtered and integrated signals- move forward and to the right (person 1).
Applsci 12 02161 g005
Figure 6. Comparison of filtered and integrated signals- move forward and to the right (person 2).
Figure 6. Comparison of filtered and integrated signals- move forward and to the right (person 2).
Applsci 12 02161 g006
Figure 7. Comparison of filtered and integrated signals-move backward and to the left (person 1).
Figure 7. Comparison of filtered and integrated signals-move backward and to the left (person 1).
Applsci 12 02161 g007
Figure 8. Comparison of filtered and integrated signals-move backward and to the left (person 2).
Figure 8. Comparison of filtered and integrated signals-move backward and to the left (person 2).
Applsci 12 02161 g008
Figure 9. The structure of the feed-forward neural network.
Figure 9. The structure of the feed-forward neural network.
Applsci 12 02161 g009
Figure 10. The observation of learning procedure results.
Figure 10. The observation of learning procedure results.
Applsci 12 02161 g010
Figure 11. Observation of the signal classification results and the robot control effect (person 1).
Figure 11. Observation of the signal classification results and the robot control effect (person 1).
Applsci 12 02161 g011
Figure 12. Observation of the signal classification results and the robot control effect (person 1).
Figure 12. Observation of the signal classification results and the robot control effect (person 1).
Applsci 12 02161 g012
Figure 13. Observation of the signal classification results and the robot control effect (person 2).
Figure 13. Observation of the signal classification results and the robot control effect (person 2).
Applsci 12 02161 g013
Figure 14. Observation of the signal classification results and the robot control effect (person 2).
Figure 14. Observation of the signal classification results and the robot control effect (person 2).
Applsci 12 02161 g014
Table 1. Names of electrodes.
Table 1. Names of electrodes.
L H F p 1 L D F C 5 R A C 4 L J P z
R H F p 2 L C F C 1 R B T 8 R N P 4
L G F 7 R C F C 2 L Q C P 5 R O P 8
L F F 3 R D F C 6 L P C P 1 L M P O 9
R K F z R E F T 10 R P C P 2 L L O 1
R F F 4 L B T 7 R Q C P 6 R J O z
R G F 8 L A C 3 L O P 7 R L O 2
L E F T 9 L K C z L N P 3   R M P O 10  
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Pawuś, D.; Paszkiel, S. The Application of Integration of EEG Signals for Authorial Classification Algorithms in Implementation for a Mobile Robot Control Using Movement Imagery—Pilot Study. Appl. Sci. 2022, 12, 2161. https://doi.org/10.3390/app12042161

AMA Style

Pawuś D, Paszkiel S. The Application of Integration of EEG Signals for Authorial Classification Algorithms in Implementation for a Mobile Robot Control Using Movement Imagery—Pilot Study. Applied Sciences. 2022; 12(4):2161. https://doi.org/10.3390/app12042161

Chicago/Turabian Style

Pawuś, Dawid, and Szczepan Paszkiel. 2022. "The Application of Integration of EEG Signals for Authorial Classification Algorithms in Implementation for a Mobile Robot Control Using Movement Imagery—Pilot Study" Applied Sciences 12, no. 4: 2161. https://doi.org/10.3390/app12042161

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop