Mind the Move: Developing a Brain-Computer Interface Game with Left-Right Motor Imagery ^†

Abstract

Brain-computer interfaces (BCIs) are becoming an increasingly popular technology, used in a variety of fields such as medical, gaming, and lifestyle. This paper describes a 3D non-invasive BCI game that uses a Muse 2 EEG headband to acquire electroencephalogram (EEG) data and OpenViBE platform for processing the signals and classifying them into three different mental states: left and right motor imagery and eye blink. The game is developed to assess user adjustment and improvement in BCI environment after training. The classification algorithm used is Multi-Layer Perceptron (MLP), with 96.94% accuracy. A total of 33 subjects participated in the experiment and successfully controlled an avatar using mental commands to collect coins. The online metrics employed for this BCI system are the average game score, the average number of clusters and average user improvement.

1. Introduction

Brain-Computer Interface (BCI) is a direct link between a computer and the human brain [1]. It is the most recent advancement in Human-Computer Interface technology (HCI). In contrast with the typical input devices (mouse, keyboard, etc.), BCI receives the waves generated by the brain at various locations in the human head and converts these signals into commands and actions that can control the computer. BCI applications are traditionally directed toward the medical field, mainly targeting patients with chronic neurological diseases [2].

BCI systems are classified as invasive or non-invasive according to where the sensors are placed. A system is considered an invasive BCI [3] if the array of electrodes used to collect brain signals is implanted inside the brain. On the other hand, when the electrodes are placed on top of the head, the system is non-invasive [4]. The first category has the benefit of having a higher-quality signal with fewer artifacts and noise. Patients with serious brain disabilities, such as blindness or paralysis, are the primary targets of invasive BCI since neurosurgery is a risky and costly operation. The non-invasive category [5] has many benefits, including portability, affordability, and safety of the method.

Electroencephalography (EEG) is the most frequently used non-invasive method [6]. EEG, records the electrical activity of the brain as it occurs on the scalp’s surface [7]. EEG equipment can be either clinical or commercial. Clinical devices are utilized in medical applications and typically have 64 or 32 electrodes. Commercial devices are more affordable and widely available mainly for lifestyle applications, and they have anywhere between 1 and 64 electrodes.

BCI has the potential to be integrated with 3D and Extended Reality (XR) software [8,9] to help people with neuro and motor rehabilitation. The BCI could be used to measure the patient’s brain activity, while the 3D/XR software could be used to provide the patient with immersive, interactive, and engaging experiences. For example, the patient could be guided through a series of exercises designed to help them improve their motor skills. The BCI could be used to measure the patient’s progress and provide feedback, allowing the patient to adjust the exercises accordingly. BCI has made a great advancement in the medical area with numerous applications, for example, controlling prosthetic limbs [10] or smart home systems [11]. As a result of this advancement, we are now seeing actions to deploy BCI outside of the medical arena. In the last decade, literature has shown that two out of five BCI application scenarios primarily focused on non-medical fields [12]. One of these fields is the gaming industry [13,14]. BCI in gaming can be used to create a more immersive gaming experience. It allows gamers to use their thoughts and feelings to control the game, instead of using a mouse, keyboard, or gamepad. For example, a gamer could use their thoughts to move objects in the game or control the speed of their character. BCI-enabled gaming could also allow people with disabilities to enjoy gaming in a more interactive way. There are several potential benefits of using BCI in gaming [15], including:

• Increased immersion: one of the key advantages of using BCI in gaming is that it has the potential to increase immersion by allowing players to directly control game characters with their thoughts. This could lead to a more immersive and realistic gaming experience.
• Increased interactivity: another benefit of using BCI in gaming is that it could allow for increased interactivity between players and game characters/environments. For example, if a player’s EEG signals indicated they are feeling scared, this could trigger a change in the game environment (e.g., an enemy appearing), which would then require the player to react accordingly (e.g., by fighting back).
• Accessibility: another potential advantage of using BCI in gaming is that it could make games more accessible for people with disabilities who may not be able to use traditional input devices such as keyboards or controllers.

Despite the potential benefits, there are also challenges associated with using BCI in gaming [16]. One of the main challenges is that BCI technology is still in its infancy and significant technical challenges need to be overcome before it can be widely used. For example, current BCI systems often have high error rates, making them impractical for gaming use. In addition, they require users to undergo a period of training to learn how to control the device. This can take a considerable amount of time, making it difficult for users to become proficient in using the device. These systems are designed to work with specific games, making them difficult to use with other types of games. There is also a need to develop more user-friendly and affordable BCI systems [17].

This study presents an approach to create a BCI game that is controlled by mental commands as expressed by EEG signals. The Muse 2 headband is utilized to measure EEG signals from multiple players, and OpenViBE is employed to record EEG data, conduct offline signal processing and classification, and perform real-time data acquisition and online EEG analysis and classification. Finally, the 3D game is developed in the Unity software, in which players are required to collect coins through motor imagery actions, such as left and right movements. The aim of this work is to determine whether users can adjust and improve their performance in manipulating the BCI system through training.

2. Related Work

In the past years, several papers were published that connected BCIs with games. The goal of these works was to develop games that can be controlled with mental commands. Six papers from the literature are presented in this section. Three different commercial EEG devices are used, the Emotiv EPOC+ [18,19,20], Neurosky Mindwave [20,21,22] and the Muse 2 headband [23]. The Emotiv EPOC+ has the better spatial resolution since it has 14 electrodes followed by the Muse 2 headband which has four EEG channels and the Neurosky Mindwave with only one electrode. Although the Emotiv EPOC+ has the best spatial resolution, it is heavier and more difficult to wear. The Muse 2 headband is the easiest to use due to its design and light weight.

Hjørungdal et al. [18] designed a game-based learning framework for controlling brain-actuated wheelchairs. The framework consists of a computer game, brain-computer interface (BCI) hardware, and a wheelchair control system. To acquire the raw EEG data, the Emotiv EPOC+ was used. The game provides real-time feedback to the users, allowing them to learn to control the wheelchair through their brain activity. The BCI hardware measures brain activity to generate commands for the wheelchair. The wheelchair control system translates the BCI commands into actual wheelchair movement. The framework is evaluated through a user study on a group of people with severe motor disabilities. Five game levels were developed for more extensive user training. Three male subjects tested the framework and the results indicate that users can learn to control the wheelchair with a high degree of accuracy and speed.

Fiałek and Liarokapis [20] presented the results of a comparison between two commercial EEG devices for serious games and virtual environments. The first EEG device is the Emotiv EPOC+ with fourteen channels and the second is the NeuroSky MindWave Mobile with a single electrode. Both systems were evaluated in terms of their accuracy and usability. The usability was assessed through an online survey and the accuracy was evaluated through a series of experiments. Thirty-one participants tested the prototypes and the results showed that both BCIs had a good level of accuracy, with the Emotiv EPOC+ Mobile being slightly more accurate. The survey results showed that the NeuroSky MindWave Mobile was slightly easier to use than the Emotiv EPOC+. In conclusion, both BCIs are suitable for use in serious games and virtual environments, but the Emotiv EPOC+ is more accurate and the NeuroSky MindWave Mobile is slightly easier to use.

Alchalabi et al. [19] designed an EEG-based serious game for focus improvement. The game is designed to help individuals with Attention Deficit Hyperactivity Disorder (ADHD) improve their focus and attention skills. The game is based on EEG signals, which are collected via the EMOTIV EPOC+ headset, which has 14 EEG channels. The designed scenario consists of two different commands, neutral state, and push. The signals are then analyzed to detect the user’s attention levels. The game provides an interactive, engaging and entertaining environment to motivate users to practice focus exercises that are tailored to their individual needs. The results of an initial study with four individuals with ADHD are presented and discussed. The users tested the game both with a keyboard and an EEG device. The results show that the game was able to detect attention levels and improve focus in the participants when playing with mental commands.

Djamal et al. [21] presented a brain-computer interface (BCI) game controlling system using Fast Fourier Transform (FFT) and Learning Vector Quantization (LVQ) algorithms. The objective of the proposed system is to enable users to control a 3D game by interpreting brain signals. EEG signals are collected from users with NeuroSky headset and then pre-processed to eliminate artifacts. The FFT algorithm is applied to extract the frequency components from the signals. The extracted components are then used to construct the feature vector for the LVQ algorithm. The LVQ algorithm is used to classify the user’s intention to control the game. The system is evaluated using real EEG data collected from ten users. The results obtained show that the proposed system is able to accurately detect the user’s intention and control the game with minimum user input. Furthermore, the proposed system is able to achieve a classification accuracy of up to 88%.

Joselli et al. [22] developed MindNinja, which is an action game based on EEG signals that allow players to interact with the game using their own brain activity. The game is designed to help players develop cognitive skills such as concentration and focus. The paper presents the concept and development of the game, as well as an evaluation of its effectiveness. It includes a discussion of the design principles behind the game, the use of EEG signals as the basis for game control, and the implementation of a reinforcement learning algorithm to evaluate the player’s performance. For acquiring the raw EEG signals, the Neurosky Mindwave mobile headset was used. To evaluate their game, eleven subjects participated in the experiment and the following metrics were considered: Player Score, Missed Cuts, Attention Level, Stress Level, Engagement Level and Evolution Attention. Results from a study of the game’s effectiveness are presented, showing that players improved their concentration and focus when playing the game.

Glavas et al. [23] developed Zombie Jumper, which is a 2D BCI game with two different levels of difficulty. To control the game, two mental commands were used imagining moving forward and blinking. For recording EEG data they used the Muse 2 headband. The goal of the game was to avoid zombies by jumping over them. For signal processing OpenViBE software was employed and Linear Discriminant Analysis was the classification algorithm. A total of 37 subjects tested level 1 and the 18 best subjects tested level 2. The aim of this study was to evaluate the user adaptation in a BCI game environment. The evaluation metrics of this work were Average Scores and Improvement.

The literature has identified a major limitation in the form of a small sample size and a limited number of repetitions per subject, which can compromise the effectiveness of BCI improvement. This study aims to address this issue by expanding the sample size and increasing the number of repetitions per subject to assess the potential for improved BCI control and better adjustment in BCI environments.

3. Materials

The hardware and software utilized are described in the next sections. The following components are either installed or connected via Bluetooth to a PC that serves as an acquisition server.

3.1. Muse 2 Headband

The Muse 2 headband [24] (Figure 1) is a Bluetooth-enabled headband that is designed to monitor brain activity and provide feedback in real-time. It is used for various types of experiments [23,25,26,27,28], including brain-computer interface (BCI) experiments. It measures electroencephalogram (EEG) signals from the scalp, providing information related to attention, relaxation, and meditation. The main reason for choosing the Muse 2 headband is its portability and affordability. In addition, it provides real-time feedback and is easily accessible. On the other hand, the Muse 2 headband suffers from some limitations. The major one is the limited electrode coverage since the device has four electrodes (TP9, AF7, AF8 and TP10). Noise interference is also a limitation because the Muse 2 uses dry electrodes, which can be more vulnerable to interference from external noise sources, such as movement or electrical devices.

3.2. BlueMuse

BlueMuse [29] is an open-source EEG brainwave monitoring software. It is designed to be used with EEG headsets, such as the Muse 2 headband, to stream data via Bluetooth connection and provide real-time brainwave data visualization and analysis. BlueMuse supports a variety of features such as real-time data streaming, frequency analysis, mental state monitoring, and integration with virtual reality applications. It can also automatically detect available EEGs within Bluetooth range and allows the simultaneous streaming of data from multiple Muse devices.

3.3. Lab Streaming Layer

Lab Streaming Layer (LSL) [30] is an open-source framework for streaming data between applications and devices in real-time. It provides a set of tools and libraries that enable applications to easily send and receive data streams over a local network or the Internet. It supports the capture of data from a variety of sources, including EEG and eye-tracking systems, motion tracking systems, and audio and video devices. LSL is designed to facilitate the integration of data streams into research applications and software. In this study, BlueMuse utilizes the Muse 2 headband through Bluetooth, setting up an LSL stream to transfer the EEG signals to OpenViBE’s acquisition server. This data is then recorded, processed, and classified by OpenViBE.

3.4. OpenViBE

OpenViBE is an open-source software platform for designing, testing, and using brain-computer interfaces (BCIs). It offers a complete range of signal processing algorithms, data visualization tools, and application programming interfaces (APIs) for developing custom BCI applications. OpenViBE is optimized for real-time processing and is suitable for a range of BCI applications such as motor imagery, event-related potentials, and steady-state visual evoked potentials.

4. Methods

In Figure 2, the flowcharts for both the offline and online processes of the proposed BCI system are provided. The methods used in the system are outlined in the following section.

OpenViBE’s acquisition server is employed to capture raw EEG data with a sampling frequency of 256 Hz. A scenario is then created to process the data and extract features for classification. After importing the raw data, a Chebyshev bandpass filter is applied to each EEG channel in the range of 8–40 Hz to minimize noise and artifacts. Then the continuous EEG recordings are divided into shorter, consecutive time periods or epochs. The purpose of EEG time epoching is to analyze the EEG signals in order to identify patterns or features that are related to specific behaviors or states. In this case, the epoching process involved dividing the EEG signals into non-overlapping windows of 3 s each, resulting in the classifier producing a decision every 3 s. The duration of the epoch was determined through a series of iterative short experiments using trial and error methodology to identify the optimal window length. The signal is then segmented and divided into five distinct frequency bands: Alpha waves 8–12 Hz, Beta 1 waves 12–20 Hz, Beta 2 waves 20–30 Hz, Gamma 1 waves 30–35 Hz and Gamma 2 waves 35–40 Hz.

The energy of each band is calculated and the obtained features are used as the basis for the classifier. Figure 3 presents the offline scenario.

4.1. Classification

A multi-layer perceptron (MLP) [31,32] is a type of artificial neural network that has multiple layers of neurons between the input and output layers. MLPs are typically composed of an input layer, one or more hidden layers, and an output layer. Each layer contains a set of neurons which are connected to the neurons in the previous layer. The input layer receives input data and the output layer produces output data. The hidden layers are used to extract features from the data and the weights and biases of the neurons are adjusted during the training process. The MLP is used for supervised learning tasks such as pattern recognition and classification. The main advantages of MLP are:

• It is a powerful and flexible algorithm that can be used to classify EEG data with high accuracy.
• It can be used to classify EEG data in a non-linear manner, allowing for more accurate classifications than linear methods.
• It is also capable of dealing with large datasets and can be used to classify EEG data from multiple subjects.
• It can also be used to perform unsupervised learning, which can help reduce the cost of labeling EEG data.
• It is able to generalize well to new data and can be used to classify EEG data from different individuals.
• It is computationally efficient and can be used to classify EEG data in real-time.

In this study, a two-layer MLP is employed to classify the three classes, with hyperbolic tangent as the activation function in the hidden layer and softmax function in the output layer; OpenViBE default configuration settings have been employed. Additionally, 10-fold cross-validation is used to obtain the classification results.

4.2. Online Processing

After the offline processing and training of the classification algorithm have been completed, a new scenario is set up for online testing as it is presented in Figure 4. The parameters remain the same as before, with the exception of time segmentation, which is no longer necessary. The Acquisition client is employed to bring in the real-time signal produced by the Muse 2 headband. The classified instances are then sent through an LSL stream to be used in the BCI application.

4.3. Dataset

In order to assess the BCI experiment, 33 individuals (18 males and 15 females) between the ages of 21 and 45 were recruited. Every participant was in good health and had either normal or corrected to normal vision and agreed to have their recordings used for the research by signing a formal consent. They were all guided by an experienced researcher to sit in a chair and keep their movements to a minimum for three separate EEG recordings. For the first recording, the subjects were instructed to blink hard every 1 s for a period of 5 min. For the second recording, they were instructed to look left and imagine that their left hand was moving for 5 min. For the third recording, they were instructed to look right and imagine that their right hand was moving for 5 min. The recordings of the mental commands for MI were obtained in an asynchronous manner, where the subjects were asked to perform MI mental commands continuously for a period of 5 min without any instruction from the researchers during the recordings. During the recordings, they were placed in a quiet room to reduce noise from the outside. All of the 99 recordings were included in the offline processing, providing 97 feature vectors per class as input for the classifier for each of the participants (

Table 1. The three classes are fully balanced and the feature vectors fed to the classifier for each class per subject is 97. The three classes are left MI, right MI, and blink.

Feature Vectors for Class 1 per Subject	Feature Vectors for Class 2 per Subject	Feature Vectors for Class 3 per Subject
97	97	97

). The three classes (left and right MI and blink), are balanced since the three recordings last 5 min each, and the same processing is applied to these signals.

4.4. Game Design

Unity is a cross-platform game engine developed by Unity Technologies. It is used to create video games for web, mobile, consoles, and other platforms. Unity is primarily used for 3D applications, although it also supports 2D development. Unity features a visual editor and powerful scripting system, allowing users to quickly build and prototype games. It also has a wide range of APIs, plugins, and assets to facilitate development. Unity can be used to create applications for OpenViBE, allowing developers to combine OpenViBE’s signal processing and visualization tools with Unity’s game engine for BCI applications.

This research created a 3D video game (as displayed in Figure 5) with an avatar that moves continuously forward. The goal of the game is to accumulate coins that are placed in various positions on the platform, requiring the player to move left, right, and jump. The platform has three lanes, each with coins on the ground or in the air. In total, the game includes 50 coins which are divided into 17 clusters (16 clusters of 3 coins and 1 cluster of 2 coins).

This project replaces standard keyboard input with brain commands. The Lab Streaming Layer (LSL) library is used to enable the game to receive live data streams. The stream outlet component provided by the LSL library is used to send time series data on the lab network, which is then sampled and sent to the game. The samples received from the stream are divided into three categories based on the user’s command.

• If the user is glancing right and imagining moving their right hand, the sample is put in the first category and the in-game avatar slides right.
• If the user is glancing left and imagining moving their left hand, the sample is put in the first category and the in-game avatar slides left.
• If the user blinks, the sample goes to the third category and the in-game avatar jumps.

5. Results

Table 2. Classification Results for the three classes. The obtained classification results in terms of True Positive Rate and Precision for each Motor Imagery command and Blink are presented for each subject. The last two columns provide the classification accuracy and ROC metric, respectively, for each subject.

Subjects	Right Motor Imagery (TPR/Precision) (%)	Left Motor Imagery (TPR/Precision) (%)	Blink (TPR/Precision) (%)	Overall (Acc %)	ROC Area
1	76.6/93.4	94.5/79.7	100.0/100.0	90.3	0.946
2	97.2/100.0	100.0/96.7	100.0/100.0	99	0.997
3	99.3/100.0	100/98.7	100.0/100.0	99.7	1
4	95.2/89.8	89.7/94.6	100.0/100.0	94.9	0.993
5	99.3/98.7	98.6/99.4	100.0/100.0	99.3	0.997
6	84.1/85.0	85.5/84.6	100.0/100.0	89.8	0.968
7	84.8/98.5	99.3/84.4	99.3/99.0	94.4	0.993
8	100.0/99.0	99.3/98.7	100.0/100.0	99.7	1
9	95.2/93.5	94.6/95.0	99.3/100.0	96.5	0.997
10	99.3/97.7	97.5/98.8	100.0/100.0	99.1	0.998
11	78.6/94.8	96.2/81.3	100.0/100.0	91.1	0.982
12	96.6/98.4	97.9/100.0	100.0/100.0	98.1	0.999
13	91.7/77.5	73.8/90.4	99.3/99.7	88.2	0.935
14	97.9/91.3	91.0/97.5	100.0/100.0	96.3	0.997
15	99.0/100.0	100.0/99.0	100.0/100.0	99.6	1
16	100.0/98.0	97.9/100.0	100.0/100.0	99.3	0.997
17	91.7/100.0	100.0/92.4	100.0/100.0	97.2	0.997
18	93.8/92.9	92.8/93.8	100.0/100.0	95.5	0.994
19	100.0/100.0	100.0/100.0	100.0/100.0	100	1
20	97.9/96.9	96.6/97.9	100.0/100.0	98.1	0.994
21	95.2/98.5	99.3/93.8	99.3/100.0	97.9	0.997
22	100.0/95.2	95.2/100.0	100.0/100.0	98.3	0.994
23	97.2/94.6	95.2/96.4	100.0/100.0	97.4	0.998
24	99.3/98.7	100.0/99.0	98.6/100.0	99.3	1
25	96.6/100.0	100.0/96.3	99.3/100.0	98.5	0.994
26	99.3/99.3	100.0/99.0	99.3/100.0	99.5	1
27	99.0/100.0	100.0/99.0	100.0/100.0	99.6	1
28	100.0/83.6	80.4/100.0	100.0/100.0	93.4	0.982
29	99.0/96.0	96.0/98.8	100.0/100.0	98.3	1
30	99.0/95.0	94.8/99.8	100.0/100.0	97.9	0.993
31	95.9/96.9	96.9/95.9	100.0/100.0	97.6	0.998
32	91.8/93.7	93.8/91.9	100.0/100.0	95.2	0.993
33	97.9/99.0	99.0/98.0	100.0/100.0	98.9	0.996

presents the classification results of EEG signals into three classes: Right Motor Imagery, Left Motor Imagery, and Blink. The Right Motor Imagery, Left Motor Imagery and Blink columns present the True Positive Rate (TPR) and Precision results of the respective commands, while the Overall column is the classification accuracy (Acc) for each subject. The average classification accuracy for all 33 subjects is 96.9%. The highest TPR is obtained for Left Motor Imagery (95.6% average for all subjects) and the lowest for Right Motor Imagery (95.4% average for all subjects). The results indicate that the proposed method is able to effectively classify the EEG signals into the three categories, with a high degree of accuracy.

Most of the participants (26) had not previously taken part in a BCI game experiment, except seven of them (Subjects 1–5 and 32–33) that had participated in a previous BCI research [28]. To ensure that they were properly prepared, an experienced researcher briefed them on the protocol before the experiment started. After the recordings and the offline processing are completed, the subjects were instructed to perform the mental commands of left/right motor imagery in order to evaluate the online classification results for 90 s (45 s for left and 45 s for right movement). This process was repeated after the completion of the testing phase for every subject, examining if users had improved after the training and testing process. For the training process, they were given ten practice trials to become familiar with the game environment. After they had become familiar with the game, they played ten more times to evaluate the BCI system. The total number of coins collected (average from the ten repetitions per subject) and the total number of coin clusters (average from the ten repetitions per subject) were used as evaluation metrics. A coin cluster is assumed completed if at least one of its coins is collected.

Table 3. Game Results. The Average Game Score column represents the average number of coins collected during the ten repetitions and the respective percentage. The Average Coin Clusters column represents the average number of coin clusters in which the subject collected at least 1 coin.

Subjects	Average Game Score	Average Coin Clusters
1	28.7 (57.4%)	10.6 (62.3%)
2	36.8 (73.6%)	13.2 (77.6%)
3	30.2 (60.4%)	10.9 (64.1%)
4	28.6 (57.2%)	10.4 (61.1%)
5	39.4 (78.8%)	13.8 (81.1%)
6	29.6 (59.2%)	11.0 (64.7%)
7	31.4 (62.8%)	11.3 (66.4%)
8	21.0 (42.0%)	8.2 (48.2%)
9	25.1 (50.2%)	9.3 (54.7%)
10	27.6 (55.2%)	10.1 (59.4%)
11	24.3 (48.6%)	8.5 (50.0%)
12	25.5 (51.0%)	9.5 (55.8%)
13	32.0 (64.0%)	11.5 (67.6%)
14	35.0 (70.0%)	12.5 (73.5%)
15	21.3 (42.6%)	7.8 (45.8%)
16	26.4 (52.8%)	9.5 (55.8%)
17	27.9 (55.8%)	10.3 (60.5%)
18	25.8 (51.6%)	9.2 (54.1%)
19	31.4 (62.8%)	11.3 (66.4%)
20	27.6 (55.2%)	10.0 (58.8%)
21	23.3 (46.6%)	8.2 (48.2%)
22	27.9 (55.8%)	10.4 (61.1%)
23	31.5 (63.0%)	11.4 (67.0%)
24	25.3 (50.6%)	9.0 (52.9%)
25	24.8 (49.6%)	8.6 (50.5%)
26	24.2 (48.4%)	8.2 (48.2%)
27	28.6 (57.2%)	10.2 (60.0%)
28	22.3 (44.6%)	8.4 (49.4%)
29	31.5 (63.0%)	11.4 (67.0%)
30	22.6 (45.2%)	8.8 (51.7%)
31	23.2 (46.4%)	8.4 (49.4%)
32	28.4 (56.8%)	10.8 (63.5%)
33	23.8 (47.6%)	8.8 (51.7%)

presents the results of the online testing.

Table 3 displays the results of a BCI game for 33 subjects. The two metrics used to evaluate the game performance are Average Game Score and Average Coin Clusters. The Average Game Score for all subjects is 27.6 (55.3%). The best performing participant is subject 5 with a 39.4 (78.8%) average score and the worst is subject 8 with 21 (42%). During the testing phase, it is observed that users can be classified into three groups depending on their performance and their ability to synchronize their mental commands in the game as it is presented in Figure 6. The first group comprises ten individuals who experienced difficulty while playing the game and their score ranges from 40% to 49.9%. The primary challenge faced by players within this category is their inability to effectively manipulate the BCI. The second group consists of 14 subjects who could handle the game quite successfully. Their game score ranges from 50% to 59.9%. The last group includes nine users that could control the BCI with excellent precision. Their game score ranges from 60% to 69.9% (six subjects) and 70% to 78.8% (three subjects).

For the second online evaluation metric, Average Coin Clusters are employed. The average number of clusters is 10.04 (59%) ranging from 7.8 (45.8%) to 13.8 (81.1%). Subject 15 had the lowest average score and subject 5 had the best. These results are observed to be higher than the Average Game score results because most of the subjects could not collect all three coins from a cluster due to synchronization.

Another online metric used to evaluate the results is User Improvement (

Table 4. User Improvement. The table shows the number of correct Motor Imagery commands for 15 tries before and 15 tries after playing the game for each command. The results in the table present the True Positive Rate and Precision of each Motor Imagery command.

Subjects	Left MI (TPR/Precision) (%) (Before Training)	Left MI (TPR/Precision) (%) (After Training)	Right MI (TPR/Precision) (%) (Before Training)	Right MI (TPR/Precision) (%) (After Training)
1	73.3/66.8	80.0/80.0	66.7/71.4	80.0/80.0
2	86.7/81.3	93.3/87.5	80.0/88.5	86.7/92.9
3	80.0/70.6	86.7/72.2	66.7/76.9	66.7/83.3
4	73.3/73.3	73.3/78.6	73.3/73.3	80.0/75.0
5	93.3/82.4	100.0/88.2	80.0/92.3	86.7/100
6	80.0/75.0	80.0/80.0	73.3/78.6	80.0/80.0
7	86.7/81.3	93.3/93.3	80.0/85.7	93.3/93.3
8	46.7/50.0	60.0/56.3	53.3/50.0	53.3/57.1
9	66.7/71.4	73.3/78.6	73.3/68.8	80.0/75.0
10	73.3/68.8	80.0/80.0	66.7/71.4	80.0/80.0
11	66.7/62.5	73.3/66.8	60.0/64.3	66.7/71.4
12	66.7/66.7	80.0/75.0	66.7/66.7	73.3/78.6
13	80.0/80.0	93.3/87.5	80.0/80.0	86.7/92.9
14	86.7/92.9	93.3/100	93.3/87.5	100.0/93.8
15	60.0/52.9	53.3/57.1	46.7/53.8	60.0/56.3
16	66.7/62.5	73.3/64.7	60.0/64.3	60.0/69.2
17	80.0/75.0	80.0/80.0	73.3/78.6	80.0/80.0
18	73.3/68.8	80.0/75.0	66.7/71.4	73.3/78.6
19	86.7/92.9	93.3/87.5	93.3/87.5	86.7/92.9
20	80.0/75.0	80.0/80.0	73.3/78.6	80.0/80.0
21	53.3/57.1	60.0/64.3	60.0/56.3	66.7/62.5
22	66.7/62.5	73.3/73.3	60.0/64.3	73.3/73.3
23	86.7/72.2	100.0/78.9	66.7/83.3	73.3/100
24	100.0/75.0	100.0/88.2	66.7/100	86.7/100
25	93.3/87.5	93.3/87.5	86.7/92.9	86.7/92.9
26	66.7/90.9	80.0/70.6	53.3/73.7	66.7/76.9
27	100.0/78.9	100.0/100.0	73.3/100.0	100.0/100.0
28	40.0/100	53.3/88.9	100.0/62.5	93.3/66.7
29	100.0/100.0	100.0/100.0	100.0/100.0	100.0/100.0
30	0.0/-	20.0/100.0	100.0/50.0	100.0/55.6
31	100.0/50.0	100.0/60.0	0.0/-	33.3/100.0
32	66.7/90.9	86.7/100.0	93.3/73.7	100.0/88.2
33	53.3/72.7	80.0/100	80.0/63.2	100.0/83.3

). Every subject performed 15 motor imagery commands for each left and right movement before and after playing the game in order to monitor user adaptation to the BCI (60 trials for each subject). The results indicate that users have a 7.5% increase in the accuracy of both mental commands. The average correctly classified mental commands for left motor imagery before playing the game are 73.73% while after playing the game is 80.80%. The average improvement for all subjects in left motor imagery is 7.1%. For right motor imagery movement, the average correctly classified mental commands before playing the game are 71.66%, and after are 79.73%. For this movement the average improvement is 8.07%. These results are presented in Figure 7.

A Kolmogorov-Smirnov test was used to test for normality on the four data samples (Left MI before training, Left MI after training, Right MI before training and Right MI after training). The results of the first sample are D(33) = 0.18, p = 0.197. For the second sample are, D(33) = 0.18, p = 0.201. For the third sample are, D(33) = 0.15, p = 0.385. For the fourth sample are, D(33) = 0.14, p = 0.479. The results indicate that the data are normally distributed in all samples.

Since the data are normally distributed, a paired samples t-test is performed to assess the impact of user improvement before and after testing the game. The left motor imagery command results indicate a statistical difference before (M = 11.1, SD = 3.1) and after ((M = 12.1, SD = 2.6), [t(33) = 5.8, p < 0.001]) playing the game. The right motor imagery movement results also indicate a statistical difference before (M = 10.8, SD = 2.9) and after ((M = 12, SD = 2.3), [t(33) = 5.5, p < 0.001]) playing the game.

6. Discussion

This work article presents a 3D game that employs a non-invasive BCI. The game utilizes the Muse 2 EEG headband to gather EEG data, while the signals are processed and classified into three distinct classes—left and right motor imagery and blink—using the OpenViBE platform. The primary objective of the game is to evaluate the users’ adaptation and progress within the BCI environment after undergoing training. The classification algorithm employed is the MLP, which achieved an accuracy rate of 96.94%. The experiment involved 33 participants who effectively controlled an avatar by issuing mental commands in order to collect coins.

Our research aimed to explore the feasibility of commercial and low-cost wireless EEGs to create a reliable BCI system based on MI commands. The Muse 2 provides a user-friendly approach for studying motor imagery but it is important to acknowledge the limitations of the electrode positions employed in the system. The optimal electrode positions for capturing MI commands are over the motor cortex, specifically at the C3 and C4 locations in the 10–20 EEG system. These locations correspond to the primary motor cortex on the left and right sides of the brain, respectively. The sub-optimal electrode placement, lacking direct coverage of the motor cortex (such as C3 and C4 locations), may limit the spatial resolution and specificity of the recorded brain activity. As a result, the interpretation of neural signals related to motor imagery can be challenging, and the extracted features may not fully capture the intricacies of motor-related brain processes. To address this limitation, we employed a technique known as eye polarization, using left and right gaze. Through conducting a series of training sessions with multiple users, we aimed to evaluate the effectiveness and reliability of the resulting BCI system. Despite encountering certain limitations and challenges, our experimental results showed promising potential for the development of a reliable BCI game based on MI commands with a low-cost EEG headset.

While our BCI game showed promising results, it is essential to acknowledge its limitations. The game is designed around a three-second epoch, where the avatar continuously moves forward and receives commands for turning or jumping every 3 s based on the user’s mental command. The game maintains continuous movement without interruption or delay. However, in order to execute turning or jumping actions, the player must wait for a three-second interval. The avatar’s constant forward movement is implemented to ensure a captivating and engaging gameplay experience with no delays. Additionally, the placement of coins in our game is strategically designed to be collectible within the current analysis framework and time windows. Developing a BCI game requires careful adjustment of parameters to align with EEG processing. BCI game development necessitates the customization of parameters to suit the specific requirements and constraints of the chosen analysis approach. These considerations highlight the importance of adapting game designs to the capabilities and limitations of BCI systems, thus optimizing the user experience within the given constraints. Lastly, this BCI is not applicable to other games without adjustments.

Throughout the study, consistent stimuli are used during both the MI training phase and data collection for classification training. Participants engaged in asynchronous MI tasks, mentally simulating hand movement actions without physical execution. This approach ensured that all participants are exposed to identical mental tasks, eliminating potential confounding factors associated with varying stimuli. It is worth noting that Action Observation (AO), which involves observing motor actions performed by others (videos, games, real movement), activates similar neural pathways as MI. The main objective of AO is to visually perceive and process motor actions executed by someone else to activate relevant neural pathways and enhance motor cognition. While this study focused on MI training, future research could consider incorporating AO to explore potential synergistic effects on motor learning and performance. Combining both modalities may enhance neural activation and functional connectivity within the motor system, potentially leading to improved outcomes in motor skill acquisition and rehabilitation [33].

A comparison of this work to other papers from the literature is provided in

Table 5. Comparative study.

Authors	Subjects	EEG Device	Mental Commands	Reps per Subj	Experiment Duration (per Subj)	Evaluation Metrics
Hjørungdal et al. [18]	3	Emotiv EPOC+	4	-	-	Time to Complete the Task
Fiałek and Liarokapis [20]	31	NeuroSky MindWave Emotiv EPOC+	2	-	-	Avg Rating Values Learnability, Satisfaction Performance, Effort
Alchalabi et al. [19]	4	Emotiv EPOC+	2	1	-	Avg Focus (0.38), Avg Stress (0.49) Avg Relaxation (0.32) Avg Excitement (0.25) Avg Engagement (0.65)
Djamal et al. [21]	20	NeuroSky MindWave	2	4	-	Average Accuracy of Training and Testing Data
Joselli et al. [22]	11	NeuroSky MindWave	1	5	10 min	Avg Player Score (163.36) Avg Missed Cuts (8) Avg Attention Level (74) Avg Stress Level (41.27) Avg Engagement Level (75.73) Avg Evolution Attention (1.70)
Glavas et al. [23]	38	Muse 2 Headband	2	20	40 min	Classification Accuracy (98.75%) Avg Game Score 1 52.70% Avg Game Score 2 70.35% Improvement
This work	33	Muse 2 Headband	3	20	55 min	Classification Accuracy (96.94%) Avg Game Score 27.6 (55.3%) Avg Number of Clusters 10.04 (59%) Avg Improvement 7.5%

.

Table 5 presents a comparative study of various studies that have used EEG devices to measure brain activity and control mental commands. Although a direct comparison is not feasible, since each study has employed different subjects, devices, techniques and methods for evaluation, several remarks related can be drawn from the Table 5.

Hjørungdal et al. [18] used three subjects, the Emotiv Epoc EEG device, four mental commands and the time to complete the task as the evaluation metric. Fiałek and Liarokapis [20] used two different EEG devices, the Emotiv EPOC+ and NeuroSky MindWave, to compare them for two mental commands on 31 subjects. To evaluate their experiment they employed a combination of average rating values, learnability, satisfaction, performance, and effort.

Alchalabi et al. [19] used four subjects, the Emotiv EPOC+ EEG device, two mental commands and average focus, average stress, average relaxation, average excitement, and average engagement as evaluation metrics. Djamal et al. [21] used 20 subjects, the NeuroSky MindWave EEG device, two mental commands and the average accuracy of training and testing data as the evaluation metric. Joselli et al. [22] used 11 subjects, the NeuroSky MindWave Mobile EEG device, one mental command and a combination of average player score, average missed cuts, average attention level, average stress level, average engagement level, and average evolution attention as evaluation metrics. Glavas et al. [23] used 38 subjects and the Muse 2 headband to acquire EEG data for two mental commands. To evaluate their work they employed Classification Accuracy, Avg Game Score 1, Avg Game Score 2 and Improvement as evaluation metrics.

In this work, 33 subjects participated to evaluate the BCI game. In the presented literature, the minimum number of subjects is three and the maximum is 38 while the average number of subjects are 17.8 subjects, thus this study includes one of the largest datasets presented in the literature. Furthermore, the number of mental commands in this study is three, while the majority of the papers in the literature investigate BCI systems with two mental commands, with the exception of [18], that includes four mental commands however with a very limited number of participants. Furthermore, in this study multiple repetitions were performed, with the participants playing the game 20 times (ten for practise and the rest for evaluation). The combination of a large number of participants, with three mental commands and multiple recordings for each subject (train, practice and evaluation) compose a very complex experimental landscape, which surpasses any previous attempt presented in the literature. Moreover, the evaluation metrics in this work are extensive (including: Classification Accuracy, Game Score, Number of Clusters and User Improvement) in order to clearly asses the capabilities of the proposed system and capture its potentials.

7. Conclusions

In this study, a Brain-Computer Interface game featuring three Motor Imagery commands is developed using Unity Engine for the 3D game and OpenViBE platform for BCI implementation. The objective of this game is to assess user adjustment and improvement in BCI control. Raw EEG data is recorded using the Muse 2 headband. A total of 33 participants took part in the experiment and successfully controlled an in-game avatar through mental commands to collect coins. The results indicate a great degree of precision in controlling the avatar once participants became familiar with the BCI environment. However, some participants struggled with the left motor imagery command. Despite this, the results hold promise for further research in BCI game development.

The BCI game utilized MLP as its classification algorithm with a precision of 96.94%. The experiment is evaluated through the use of online evaluation metrics, including the average game score, the average number of clusters collected, and average user improvement. The results showed that there is a noticeable improvement in the users’ performance before and after training, demonstrating that subjects can learn to operate a BCI system.

In future research, the current BCI-based 3D MI game could be adapted as a training step in order to train users to control other types of robotic systems, such as robotic arms, wheelchairs, exoskeletons and drones. Furthermore, the comparison between different wearable EEG devices and their potential to effectively operate a BCI system should be addressed in future communications. The complexity of the BCI systems in terms of number of identified MI and mental state commands should also be clearly investigated, since up until now the majority of the related work includes just two mental commands. Building up the complexity of BCI systems must also be followed with the involvement of larger numbers of participants along with multiple recordings from each participant. Additionally, a training section will be incorporated into the game for the BCI system, ensuring that the stimuli used during both the training and online phases are consistent. As a result, evaluation analysis will be more reliable and it might be easier to notice significant effects and correlations. Finally, the current research could be extended to investigate the use of BCI technology in controlling augmented and virtual reality environments.