Evaluation of the User Adaptation in a BCI Game Environment ^†

Abstract

Brain-computer interface (BCI) technology is a developing field of study with numerous applications. The purpose of this paper is to discuss the use of brain signals as a direct communication pathway to an external device. In this work, Zombie Jumper is developed, which consists of 2 brain commands, imagining moving forward and blinking. The goal of the game is to jump over static or moving “zombie” characters in order to complete the level. To record the raw EEG data, a Muse 2 headband is used, and the OpenViBE platform is employed to process and classify the brain signals. The Unity engine is used to build the game, and the lab streaming layer (LSL) protocol is the connective link between Muse 2, OpenViBE and the Unity engine for this BCI-controlled game. A total of 37 subjects tested the game and played it at least 20 times. The average classification accuracy was 98.74%, ranging from 97.06% to 99.72%. Finally, playing the game for longer periods of time resulted in greater control.

1. Introduction

A brain-computer interface (BCI) [1,2] is a specialized communication system that enables people to interact with applications [3] by sending commands or messages without any muscle or nerve activity. There are many fields that can benefit from the development of BCIs, such as medical applications [4], games [5], and tourism [6]. In the medical field, BCIs can be used for detecting and diagnosing brain or sleep disorders; in gaming, many new applications have been developed for educational purposes or for expanding the gaming experience for people with movement disabilities. Lastly, tourism can benefit from controlling augmented or virtual touristic environments.

Depending on the placement of the sensors, BCI systems are divided into two categories: invasive and non-invasive [7,8]. If the electrodes used for acquiring the brain signals are placed as an array within the skull, then the system is categorized as an invasive BCI. Otherwise, if the electrodes are placed on the scalp, then the system is non-invasive. The advantage of the first category is that the signal has better quality, with reduced artifacts and noise. Because neurosurgery is a dangerous and costly procedure, patients with major neural disabilities such as blindness or paralysis are the primary targets of invasive BCI. There are several advantages of the non-invasive category, such as portability, low cost and the safety of the procedure. The most common non-invasive technique is electroencephalography (EEG). By definition, an EEG provides a recording of the electrical activity of the brain from the surface of the scalp. EEG devices can be either commercial or clinical. Clinical devices usually have 64 or 32 electrodes, and they are used in medical applications [9]. Commercial devices are cheaper and more accessible, and the number of electrodes on those devices varies from 1 to 64. The following

Table 1. Commercially available non-invasive EEG devices.

Device	EEG Electrodes	Sampling Rate	Price
Muse 2	4 (AF7, AF8, TP9, TP10)	256 Hz	250$
Neurosky MindWave	1 (FP1)	512 Hz	100€
Emotiv Insight	5 (AF3, AF4, T7, T8)	128 Hz	499$
Unicorn Hybrid Black	8 (Fz, C3, Cz, C4, Pz, PO7, Oz, PO8)	250 Hz	990$
Emotiv EPOC+	14 (AF3, F7, F3, FC5, T7, P7, O1, O2, P8, T8, FC6, F4, F8, AF4)	128 Hz	849$

presents a list of commercial EEG headsets from the literature.

The EEG devices that are presented in the table above have been employed by researchers to control mechatronic devices. Emotiv Epoc+ was used by Aguiar et al. [10] to control a robotic arm. Emotiv Insight was employed by Espiritu et al. [11] to command a wheelchair with brain commands. Neurosky MindWave was used by Katona et al. [12] to control a mobile robot. Lastly, Unicorn Hybrid Black was employed by Belkacem and Lakas [13] to achieve robot-drone interactions.

A BCI system consists of 5 major components [14], as is presented in Figure 1:

(1)

Signal acquisition;

(2)

Signal processing;

(3)

Feature extraction;

(4)

Feature translation;

(5)

Device output.

Brain activity [15] is measured and recorded by signal acquisition, and the raw data are de-noised by signal pre-processing. The next step is signal processing for filtering the signal in order to extract several features for the classification process. Then the classifier translates the brain signals into application commands. Many BCI applications provide feedback and re-training to the user.

In this work, a BCI-controlled game is developed. The name of the 2D game is Zombie Jumper, and the goal is to overcome obstacles using two mental commands. The Unity engine is used to build the game, and OpenViBE is used to process and classify the EEG signals. OpenViBE is also used for real-time classification, and through a lab streaming layer (LSL) stream, it sends data to the game. A Muse 2 headband is used to acquire EEG data from the 37 participants that tested the game.

2. Related Work

Several papers were published in the last few years that intend to connect BCIs with video games. The main target of these studies is to develop games that can be played by brain signal commands. In the following papers from the literature, the main technique that was used is attention and meditation levels to control the BCI and play the games. In the works of [16,17,18], the authors chose Neurosky devices with one sensor to acquire the EEG data, while [19,20,21] chose headsets with more EEG sensors, including the Emotiv Insight headset with 5 sensors, the EMOTIV EPOC+ headset with 14 sensors, and the Muse 2 headband with 4 sensors, respectively.

Wu et al. [16] developed an android racing BCI game controlled by Mindset, a commercially available EEG headset. NeuroSky Mindset is an EEG headband with only 1 sensor, and it is mostly used for attention-meditation values. The employed racing game is controlled by the user’s neural mind states. The user can alter the speed of the car and the map of the game with his attention and meditation level. To finish the game, the user must complete 2 map circles, and the map can dynamically change depending on the mean meditation level. For a mean meditation between 0 and 35, the most difficult map is loaded; for mean meditation values between 36 and 70, the intermediate one is loaded, and for 71 to 100, the easiest one is loaded. If the mean meditation level is high, then the user is nervous. The car’s speed is proportional to the attention level, so the higher the attention level of the player, the faster he can finish the game. For the testing phase, 5 subjects played the game; 1 of them managed to load the difficult map, 3 managed to load the intermediate map, and 1 loaded the easiest map. Their mean meditation scores were 27, 39, 47, 55, and 79.

Vasiljevic et al. [17] designed the Mental War game controlled by the NeuroSky MindWave headset, which has 1 electrode. The developed game is a tug-of-war game based on the user’s attention level. The game has 3 different modes: single player, competitive and collaborative. In single-player mode, the user competes against the computer, and there are 3 difficulty levels: easy, medium and hard. In competitive mode, which is multiplayer, users battle against each other, and the player with the greater attention level will pull the others to his side. In collaborative mode, users can team up and battle against the computer or a different team of players. The team’s attention level is the mean of the sum of each individual’s attention level. Therefore, if a team member has a low attention score, the others should score very high for their team to win. In the testing phase, 24 subjects participated in several scenarios to improve and evaluate the game. Attention levels and the duration of the game were the two evaluation metrics. The average attention score for the single-player mode was 53.49%, while for the multiplayer (competitive and collaborative) modes, the average score was 52.42%. The average duration of the single player mode was 90.99 s, and for the multiplayer mode, the average duration was 56.22 s.

Rosca et al. [19] developed a game called Mental Pool Game powered by the Unity engine and controlled by an Emotiv Insight headset with 5 EEG sensors. In this work, neutral state and attention level are used for playing the game. The user can control the speed and strength of a white ball with his attention level. Practically, these in-game features are proportional to the attention of the player. Moreover, they used the 3-axis gyroscope of the Emotiv headset to update the white ball’s position on the table in order to match the player’s position. To test the game, 3 healthy participants played the Mental Pool Game. Before the actual experiment, the subjects trained offline for the neutral and attention states. For the attention level, the subjects were instructed to visualize the movement of a cube. The participants played 1 round of each game until the exhaustion of the balls on the pool table.

Wang et al. [18] designed a parkour game based on EEG and electrooculograms (EOG). MindWave Mobile was employed for this BCI application. This headset consists of an earphone, an ear clip, and a sensor arm. The game has four levels, and the in-game avatar can run, jump to avoid obstacles, and collect coins. The avatar has three lives with which to complete one level, and the game rewards the user every time he jumps over an obstacle, collects coins, and finishes a level. More specifically, the attention level is associated with the avatar’s speed, meaning that an increase in attention level leads to an increase in the avatar’s speed. Additionally, blinking corresponds to in-game jumping. To test their application, 5 subjects played the game. To evaluate their experiment, they used two different algorithms and the following metrics: maximum meditation degree, maximum attention degree, and the time to pass the first stage. The average time to pass the first stage for the first algorithm was 34.6 s, and for the second one, it was 30.2 s.

Alchalabi et al. [20] developed a serious game to improve the focusing ability of people that suffer from ADHD. To acquire the EEG data, they used the EMOTIV EPOC+ headset, which has 14 EEG channels. The designed 3D game has two different commands, push and neutral state, and the objective of the game is to collect all the available cubes. The push command is related to the attention level, while the neutral command is related to the meditation level of the user. When the user is focused, the avatar is moving forward, and when the user relaxes, the avatar stops moving. Additionally, the user can rotate the avatar using the gyroscope from the Emotiv headset. Four healthy subjects participated in the experiment. In the beginning, they were trained for the 2 mental states, and after completing the training phase, they were introduced to the game. They played the game with 2 protocols: mental commands and a keyboard. It was observed that users had greater engagement and higher attention levels when playing with the mental commands.

3. Materials and Methods

3.1. Materials

The goal of this work is to develop a low-cost BCI-controlled game in order to train users to adapt to BCI environments. In the following subsection, the hardware and software used are presented.

3.1.1. EEG Headset

To acquire the raw EEG data, the Muse 2 [22] headband (Figure 2) is used, which is a 4-electrode commercial EEG headset. The 4 electrodes are placed in the frontal cortex, two on the left (TP9 and AF7) and two on the right (AF8 and TP10). Muse 2 is a portable, flexible and low-cost EEG headband, and it connects with the computer via Bluetooth.

3.1.2. BlueMuse

BlueMuse [23] is a streaming app for the Muse 2 headband. The connection between the computer and the EEG device is made via Bluetooth. It has many features that make it easier for the user, such as the automatic detection of available EEGs that are within a distance recognized by Bluetooth. Additionally, the application supports the simultaneous flow of data from several Muses, which facilitates the creation of various applications that need two or more devices at the same time. Data streaming is achieved via the lab streaming layer (LSL) protocol.

3.1.3. Lab Streaming Layer

LSL [24] is an open-source system used to stream, receive, synchronize, and record time series data streams acquired from numerous acquisition devices available in the network. The LSL protocol provides secure data transmission due to the inheritance of the TCP protocol. It reduces complexity over cross-platform application connectivity and minimizes data loss. For this work, BlueMuse connects with the Muse 2 headband via Bluetooth and automatically creates an LSL stream for transmitting the raw EEG data. The LSL stream is connected to OpenViBE’s acquisition server in order to record and then process and classify the brain signals.

3.1.4. OpenViBE

The BCI-controlled game is employed on the OpenViBE platform, which is an open-source software for recording, processing and classifying EEG data either offline or in real-time. OpenViBE has several integrated algorithms, and it also provides drivers for BCI headsets to connect with other applications and engines.

3.2. Methods

In the following subsection, the methods of the proposed BCI system are presented. In Figure 3, the flowchart diagram for the offline and the online process is shown.

3.2.1. Offline Processing

For recording raw EEG data, OpenViBE’s acquisition server was employed. A simple scenario was created that obtains the raw EEG data and stores them in a CSV file, excluding 2 channels from the Muse 2 headband (timestamp and AUX). When the recording phase was completed, a new scenario was designed in order to process the data and extract features to classify them. The brain signal was imported, and a pre-filter was applied. This filter was a Chebyshev bandpass filter between 8 and 30 Hz. This was used to cut noise, artifacts and low-frequency bands such as delta and theta waves. After the Chebyshev filter was applied, the signal was epoched in 1 s time windows with no overlap. The time window for this BCI application had to be short so the user could control the game avatar accurately. The final time window was decided through a trial-and-error process. Fast Fourier transform (FFT) was used for spectral analysis. Then, the EEG spectrum was split into 3 bands:

(1)

Alpha waves 8–12 Hz.

(2)

Beta low waves 12–20 Hz.

(3)

Beta high waves 20–30 Hz.

Finally, the average spectral amplitude per band was calculated and used as an input for the classifier. The offline processing scenario that is described is presented in Figure 4.

3.2.2. Classification

To classify the data, two algorithms were tested, linear discriminant analysis (LDA) and multi-layer perceptron (MLP). These algorithms are already integrated by OpenViBE. The obtained features from the processing phase were used as input to the classifiers.

LDA [25] is a common technique used for supervised classification problems. It can be used to reduce dimensions, visualize data and interpret the importance of the given features. LDA finds the projection hyperplane in order to minimize the interclass variance and maximize the distance of the projected means of the classes. Basically, the classifier finds a linear combination of features that can group data or separate them into two or more classes.

MLP [25,26] is a neural network. It has three different layers: the input layer, the hidden layer and the output layer. The input signal is sent to the input layer to be processed. The computational power of this algorithm derives from the hidden layers. They are located between the input and output layers, and their number varies depending on the problem. The neurons in this classification algorithm are trained with the back-propagation learning algorithm. The output layer is responsible for predicting and classifying data. Multi-layer perceptrons are more commonly used in pattern classification.

In

Table 2. Offline classification results for LDA and multi-layer perceptron.

Subjects	LDA—Blinking	LDA—Eyes Opened	LDA—Overall	Perceptron—Blinking	Perceptron—Eyes Opened	Perceptron—Overall
1	95.20%	100%	97.58%	97.20%	100%	98.62%
2	99%	0%	99.36%	98.6%	100%	99.14%
3	99%	100%	99.36%	97.6%	100%	98.79%
4	97.2%	100%	98.62%	98.3%	100%	99.13%
5	98.3%	99.3%	98.79%	96.9%	100%	98.44%
6	98.3%	100%	99.3%	97.6%	100%	98.79%
7	99%	100%	99.48%	97.9%	100%	98.96%
8	99%	100%	99.48%	97.2%	100%	98.62%
9	98.6%	100%	99.31%	97.2%	100%	98.62%
10	98.6%	100%	99.31%	97.6%	100%	98.79%
11	98.6%	100%	99.31%	97.2%	100%	98.62%
12	99%	100%	99.48%	96.9%	100%	98.44%
13	97.9%	100%	98.96%	97.6%	100%	98.79%
14	97.9%	100%	98.96%	96.9%	100%	98.44%
15	99%	100%	99.48%	99.7%	100%	99.72%
16	98.3%	100%	99.13%	96.6%	100%	98.27%
17	99%	100%	99.48%	97.2%	100%	98.62%
18	98.3%	100%	99.13%	97.2%	100%	98.62%
19	97.6%	99.7%	98.62%	96.9%	100%	98.44%
20	98.3%	99.3%	98.79%	97.2%	100%	98.62%
21	97.9%	100%	98.96%	97.2%	100%	98.62%
22	100%	100%	100%	97.6%	100%	98.79%
23	99%	100%	99.48%	95.2%	100%	97.06%
24	89.7%	100%	94.82%	97.6%	100%	98.79%
25	99%	100%	99.48%	99.3%	100%	99.65%
26	97.9%	100%	98.96%	99.3%	100%	99.65%
27	98.6%	100%	99.31%	97.6%	100%	98.79%
28	96.6%	100%	98.27%	97.6%	100%	98.79%
29	99%	100%	99.48%	97.6%	100%	98.79%
30	98.6%	100%	99.31%	97.6%	100%	98.79%
31	92.8%	100%	96.37%	97.2%	100%	98.62%
32	99%	100%	99.48%	97.2%	100%	98.62%
33	99%	100%	99.48%	97.9%	100%	98.96%
34	98.6%	100%	99.31%	97.9%	100%	98.96%
35	89.7%	100%	94.82%	97.6%	100%	98.79%
36	95.2%	100%	97.58%	97.2%	100%	98.62%
37	95.9%	98.6%	97.06%	97.9%	100%	98.96%

the classification accuracy and the confusion matrix of the offline classification process for the subjects that participated in the experiment are presented. The average classification accuracy for LDA is 98.75% while for Multi-layer Perceptron is 98.74%.

3.2.3. Online Scenario

After the completion of the offline processing and the training of the classification algorithm, a new scenario was created for the online testing. The parameters of this scenario were the same as in the previous scenario except for the time segmentation, which was not required. As an input, the acquisition client was used for importing the real-time signal obtained by the Muse 2 headband. The classified instances were transported through an LSL stream to be used for the BCI application. The real-time classification scenario is presented in Figure 5.

4. Game Design

The developed 2D game was designed in the Unity platform [27], which is a popular developing engine used for game development. Unity can produce BCI applications when the appropriate libraries and tools are imported. For this work, the liblsl library was used to connect Unity with the BCI. More specifically, LSL creates a live stream that allows the application to receive data from the EEG device. The data transmitted through this LSL stream are the classification results from OpenViBE’s real-time classification scenario. Basically, these data were translated into in-game commands, replacing the standard Unity input commands (keyboard input). Appropriate C-sharp scripts handled an LSL stream and read the transmitted data sample by sample.

In the designed game, the player must control the avatar accurately in order to overcome obstacles, monsters and ghosts and reach the end of the level. There are two levels: in the first one, the obstacles are static, and only the avatar is moving toward them, while in the second, both the avatar and the obstacles are moving. In order to make the game more competitive and fun, a scoreboard on the top left of the screen was employed. The score was updated based on the distance that the player moved and the number of obstacles that were successfully overcome. There were 2 available commands: move forward and jump. In order to move forward, the user has to imagine forward movement and look ahead. For jumping, the user has to perform a hard intentional blink. The game interface is presented in the following Figure 6.

5. Results and Discussion

5.1. Dataset

To evaluate the BCI experiment, 37 subjects, 20 males and 17 females with varying ages between 19 and 61, played the game. All participants were healthy with normal or corrected vision and signed a formal consent in order to use their recordings for this research. An experienced researcher instructed them to sit in a chair and minimize their movements to perform 2 separate EEG recordings. For the first recording, subjects had to perform intentional hard blinks every 2 s for 5 min. For the second recording, subjects were instructed to look ahead and imagine moving forward for 5 min. During the recording phase, participants were isolated in a quiet room in order to minimize external noise. All 74 recordings were included in the offline processing without excluding data with a total of 580 feature vectors as input to the classifier for every subject. The average classification accuracy was 98.74%, ranging from 97.06% to 99.72% (Table 2).

5.2. Game Testing

Moving on to online testing, players were introduced to the BCI game and Unity environment because they had zero experience in BCI experiments. With the aid of a researcher, subjects became familiar with the equipment and the whole procedure. After the training phase, they had to play level 1 of the game strictly 20 times in order to evaluate their performance. The game score was chosen as an evaluation metric, which increased proportionally to the distance covered and the obstacles that the player overcame. The 20 scores were split into 2 sets of 10; then, the average score of these 2 sets was calculated to examine how the user adapts to the game. To make the whole experiment more competitive and fun, subjects were informed that the 18 best players would advance to test level 2 of the game.

Table 3. Level 1 game scores.

Subjects	Average Score 1	Average Score 2	Improvement	Average Overall Score
Sub1	45.60	53.70	8.10%	49.65
Sub2	51.90	63.00	11.10%	57.45
Sub3	40.50	67.50	27.00%	54.00
Sub4	46.80	63.30	16.50%	55.05
Sub5	47.70	51.80	4.10%	49.75
Sub6	53.70	51.40	−2.30%	52.55
Sub7	41.90	56.60	14.70%	49.25
Sub8	43.60	46.60	3.00%	45.10
Sub9	35.80	63.30	27.50%	49.55
Sub10	48.90	47.10	−1.80%	48.00
Sub11	44.50	37.20	−7.30%	40.85
Sub12	32.60	48.00	15.40%	40.30
Sub13	46.75	48.90	2.15%	47.80
Sub14	35.60	46.40	10.80%	41.00
Sub15	66.20	60.90	−5.30%	63.55
Sub16	57.30	64.80	7.50%	61.05
Sub17	64.20	69.50	5.30%	66.85
Sub18	56.60	59.70	3.10%	58.15
Sub19	38.90	57.30	18.40%	48.10
Sub20	50.80	60.10	9.30%	55.45
Sub21	30.20	40.60	10.40%	35.40
Sub22	27.40	22.60	−4.80%	25.00
Sub23	66.80	50.80	−16.00%	58.80
Sub24	57.60	64.30	6.70%	60.95
Sub25	63.50	62.20	−1.30%	62.85
Sub26	34.00	41.70	7.70%	37.85
Sub27	41.00	53.10	12.10%	47.05
Sub28	51.20	54.10	2.90%	52.65
Sub29	49.00	65.00	16.00%	57.00
Sub30	62.90	60.70	−2.20%	61.80
Sub31	55.90	63.80	7.90%	59.85
Sub32	59.60	64.30	4.70%	61.95
Sub33	52.20	64.30	12.10%	58.25
Sub34	64.30	68.50	4.20%	66.40
Sub35	53.10	64.60	11.50%	58.85
Sub36	53.80	66.10	12.30%	59.85
Sub37	28.20	59.20	31.00%	43.7

presents the level 1 game scores from the 37 subjects. The maximum score for level 1 is 100. The average overall score for all subjects is 52.70. The best performance is 66.85 by Sub17, while the worst one is 25 by Sub22. 18 subjects were able to finish level 1 at least 1 time; 5 of them managed to finish it twice, and only Sub17, the player with the best overall score, finished it 3 times. It is observed that 78% of the players (29 out of 37) improved their performance in the second set of 10 tries. This observation indicates that most of the players adapted to the BCI-controlled game and became better after they familiarized themselves with it. The main difficulty for some players was synchronizing the jump command because of the 1-second delay of the commands due to the 1-second time windows chosen for the classifier.

Table 4. Level 2 game score results.

Subjects	Average Score 1	Average Score 2	Improvement	Average Overall Score
Sub2	65.20	77.40	8.13%	71.30
Sub4	61.60	78.40	11.20%	70.00
Sub15	62.60	83.30	13.8%	72.95
Sub16	96.60	71.20	−16.93%	83.90
Sub17	68.40	73.20	3.20%	70.80
Sub18	67.10	55.50	−7.73%	61.30
Sub20	43.20	58.90	10.46%	51.05
Sub23	61.30	65.20	2.60%	63.25
Sub24	82.10	84.30	1.46%	83.20
Sub25	84.45	78.55	−3.93%	81.50
Sub29	73.20	70.00	−2.13%	71.60
Sub30	77.90	58.10	−13.20%	68.00
Sub31	70.20	64.20	−4.00%	67.20
Sub32	72.00	103.80	21.20%	87.90
Sub33	51.40	61.10	6.46%	56.25
Sub34	61.20	74.30	8.73%	67.75
Sub35	69.40	67.80	−1.06%	68.60
Sub36	71.50	68.10	−2.26%	69.80

presents the level 2 game scores from the 18 best subjects qualified from level 1. The maximum score for level 2 is 150. The average overall score for all subjects is 70.35. The highest score achieved in this level is 87.9 by Sub32. The worst score is 51.05 by Sub20. A total of 8 subjects managed to finish the level at least 1 time, 4 subjects finished it twice, and 1 subject finished it 3 times. The results show that 55.55% of the players (10 out of 18) adapted to this level’s difficulty and improved their performance in the second set of tries. Level 2 is designed with increased difficulty compared to level 1 in order to test user adaptation in a more difficult BCI scenario. The results confirm that level 2 is indeed more challenging because the 18 players who advanced from level 1 have an average decrease of 13.33% in their level 2 scores.

A paired samples t-test was conducted to determine the effect of adaptation between the 2 sets of 10 tries for each level. The results for level 1 indicate a significant difference between average score 1 (M = 48.66, SD = 11.04) and average score 2 ((M = 56.29, SD = 10.15), (t(37) = 0.583, p < 0.01)). The results for level 2 indicate a not significant difference between average score 1 (M = 68.85; SD = 12.11) and average score 2 ((M = 71.85, SD = 11.65), (t(18) = 0.265, p = 0.287)).

A comparative study between this work and other papers from the literature is presented in the following

Table 5. Comparative study.

Authors	Subjects	EEG Device	Mental Commands	Reps per Subj	Evaluation Metrics
Wu et al. [16]	5	NeuroSky Mindset	2	1	Avg mean meditation (49.4)
Vasiljevic et al. [17]	24	NeuroSky MindWave	1	-	Avg attention single player (53.49); avg attention multiplayer (52.42)
Rosca et al. [19]	3	Emotiv Insight	2	1	Not presented
Wang et al. [18]	5	NeuroSky MindWave Mobile	2	1	Avg maximum attention 1 (73.6) Avg maximum attention 2 (76.4) Avg maximum meditation 1 (51) Avg maximum meditation 2 (47.4) Game duration 1 (34.6 s) Game duration 2 (30.2 s)
Alchalabi et al. [20]	4	Emotiv Epoc+	2	2	Avg focus (0.38), Avg stress (0.49) Avg relaxation (0.32) Avg excitement (0.25) Avg engagement (0.65)
This work	37	Muse 2 Headband	2	20	Classification accuracy (98.75%) Game score 1 (52.70/100) Game score 2 (70.35/150)

. The majority of the papers designed a BCI-controlled game with 2 mental commands using a commercially available EEG headset.

Wu et al. [16], Vasiljevic et al. [17], and Wang et al. [18] used the NeuroSky headset, which has 1 electrode, to acquire the EEG signals. Rosca et al. [19] and Alchalabi et al. [20] used Emotiv headsets with 5 and 14 electrodes, respectively. In this experiment, the Muse 2 headband was chosen, which is a 4-electrode EEG device rarely studied in the literature.

According to the majority of the papers in the literature, the subjects played the game only once, except Alchalabi et al. [20], where the subjects played the game two times, and Vasiljevic et al. [17], where repetitions per subject were not mentioned. In this work, user adaptation and improvement are the main focus; thus, subjects played the game 20 times (in 2 sets of 10) in order to measure their improvement.

Wu et al. [16] calculated the mean meditation of the players, and depending on the value, the difficulty of the game changes. Rosca et al. [19] did not present any evaluation metrics for the mental pool game they designed. The evaluation metrics from the works of Vasiljevic et al. [17] and Wang et al. [18] mainly focused on the game duration and on attention-meditation level. Alchalabi et al. [20] used several metrics for their game, such as focus, stress, relaxation, interest, excitement and engagement, in order to help ADHD patients. In this work, a more thorough approach is employed, as the game is tested on 37 subjects, while in other studies, the average number of subjects is 8.2. Additionally, user adaptation and improvement are taken into consideration as a user evaluation metric. Lastly, classification accuracy is presented as an offline metric, and the game score is presented as an online evaluation metric.

6. Conclusions

In this paper, Zombie Jumper was developed, which is a BCI-controlled game. The game consists of two commands, moving forward and jumping. To record the raw EEG data, a Muse 2 headband was used. The OpenViBE platform was employed to process and classify the brain signals, and the Unity engine was used to build the game. Lastly, in order to make the game BCI controlled, the LSL protocol was used as the connective link. To evaluate the experiment, 37 subjects tested the game. They played the first level 20 times, and the best 18 of them proceeded to the next level. It was observed that most of the participants adapted to the BCI game, significantly improving their performance while playing the game. All subjects were interviewed after their session and stated that they enjoyed Zombie Jumper.

For the classification process, two algorithms were tested, MLP and LDA. The accuracy of MLP was 98.74%, and this value was 98.75% for LDA. One of the goals of this work was to examine user adaptation and improvement. The results showed that users gradually improved their performance after each repetition. Level 1 average improvement was 7.63%, while for level 2, the average improvement was 2%. Lastly, the overall average level 1 score was 52.70, while the overall average level 2 score was 70.35.

In the future, the goal is to extend Zombie Jumper to be completely controlled by BCI. More difficult levels will be developed, and multiple characters will be added. Furthermore, the goal is to increase the number of commands allowing the character to perform extra moves. The BCI application in this study was based on a 2D game. Further research will be focused on the use of BCI in various research areas, such as the control of virtual or augmented environments, and commanding autonomous vehicles, such as electrical wheelchairs.