The Effects of VR and TP Visual Cues on Motor Imagery Subjects and Performance

Abstract

This study objectively evaluated the effects of Virtual Reality Visual Cues (VRVCs) and Traditional Plane Visual Cues (TPVCs) on motor imagery (MI) subjects and Brain-Computer Interface (BCI) performance when building a classification model for MI-BCIs. Four metrics, namely, imagery stability, brain activation and connectivity, classification accuracy, and fatigue level, were used to evaluate the effects of TPVCs and VRVCs on subjects and MI-BCI performance. Nine male subjects performed four types of MI (left/right-hand grip strength) under VRVCs and TPVCs while EEG and fNIRS signals were acquired. FBCSP and HFD were used to extract features, and KNN was used to evaluate MI-BCI accuracy. Rt-DTW was used to evaluate MI stability. PSD topography and the brain functional network were used to assess brain activation and connectivity. Cognitive load and fNIRS mean features were used to evaluate fatigue. The mean classification accuracies of the four types of MI under TPVCs and VRVCs were 50.83% and 51.32%, respectively. However, MI was more stable under TPVCs. VRVCs enhanced the connectivity of the brain functional network during MI and increased the subjects’ fatigue level. This study’s head-mounted VRVCs increased the subjects’ cognitive load and fatigue level. By comparing the performance of an MI-BCI under VRVCs and TPVCs using multiple metrics, this study provides insights for the future integration of MI-BCIs with VR.

1. Introduction

Brain-Computer Interfaces (BCIs) are a novel technology that utilizes signals derived from the central neural activity of the user [1,2]. The primary goal of BCIs is to enable communication and control between the brain and external devices, ultimately enhancing the quality of life and productivity of individuals, particularly those with severe motor dysfunction. Motor imagery (MI) BCIs constitute a critical category of BCIs. The research findings demonstrate that motor imagery (MI) can enhance the phenomenon of event-related synchronization (ERS) [3] in the alpha and beta frequency bands of the sensorimotor cortex, thereby increasing synchronized neural activity in the brain regions responsible for controlling movement during the response to external events or tasks [4]. MI-BCIs require users to mentally simulate the process of moving a specific body part, such as the hand or foot [5,6]. MI-BCIs have demonstrated notable efficacy in enhancing motor performance and ameliorating motor impairments among individuals with movement disorders, thereby facilitating their therapeutic interventions and rehabilitation [7]. However, this must be accomplished without actually executing the physical movement, thereby making it a cognitive activity that is infrequently performed in everyday life and that can be challenging to acquire and regulate.

Visual cues are commonly used to guide subjects in performing motor imagery (MI) in the development of MI-BCI classification models. Two primary forms of visual cues are employed, including Traditional Plane Visual Cues (TPVCs) and Virtual Reality Visual Cues (VRVCs). TPVCs are characterized by visual cues presented on a flat screen [8,9], such as those found on a computer or tablet. In contrast, VRVCs refer to cues presented within a virtual reality environment [10]. Research has shown that VRVCs can provide an immersive environment that makes viewers feel as though they are truly present in the scene. This immersive experience can increase their engagement and attention during the observation process, which can facilitate more efficient user MI training, reduce training cycles [11], and enhance the ease of use of MI-BCIs. However, the prolonged use of VRVCs may result in adverse effects such as eye fatigue and dizziness among users [12]. TPVCs are more commonly utilized, less expensive, and simpler [8,9]. Some users have grown accustomed to TPVCs and are more receptive to this approach. However, the visual information provided by TPVCs is relatively limited, which may result in reduced user immersion.

As mentioned earlier, the utilization of VRVCs and TPVCs in MI tasks has distinct effects on both subjects and MI-BCI performance. Previous research has investigated this impact by examining MI-BCI classification accuracy [13] and self-reported measures, such as satisfaction and cognitive load [14,15]. However, objective indicators have been lacking. This study aims to comprehensively evaluate the divergent effects of VRVC and TPVC MI tasks on subjects and MI-BCI performance. This study assesses imagery stability, brain activation and connection, and classification accuracy. EEG signals are utilized for the assessment, and functional near-infrared spectroscopy (fNIRS) signals are also acquired to gauge the subjects’ fatigue levels.

In this study, we acquired EEG and fNIRS signals during four types of MI of left-hand or right-hand grip strength under VRVCs and TPVCs. To evaluate the different effects of VRVC and TPVC MI tasks on subjects and MI-BCI performance, we used the Filter Bank Common Spatial Pattern (FBCSP) [16] and the Higuchi Fractal Dimension (HFD) [17] to extract EEG features, and K-Nearest Neighbor (KNN) was used to evaluate the classification accuracy of motion imagery EEG signals under VRVCs and TPVCs. We also used a random template-Dynamic Time Warping (rt-DTW) based method to assess the stability of subjects’ MI, Power Spectral Density (PSD) brain topography during MI and the functional brain network [18] in the resting state before and after the experiment to assess brain activation and connectivity, and a cognitive load scale with fNIRS signal mean characteristics [19] to assess the level of fatigue in the subjects. This study aimed to provide valuable insights for the future integration of MI-BCIs with VR.

2. Materials and Methods

2.1. Subjects

This study involved 9 male participants between the ages of 23 and 26 years who were all right-handed and free from any physical or mental illnesses. Before the experiment, all participants provided informed consent, and the study received approval from the Medical Ethics Committee of Kunming University of Science and Technology.

2.2. Research Approach

Figure 1 shows the study design for comparing the impact of VRVCs and TPVCs on MI tasks and performance. Figure 1a shows the VRVC and TPVC MI tasks for the left-hand and right-hand grip strengths. Figure 1b shows the analysis of the motor area EEG used to assess the classification accuracy, brain activation and connectivity, and imagery stability of the subjects. Figure 1c shows the analysis of the frontal area fNIRS and cognitive load scale scores used to evaluate the level of fatigue in the subjects.

2.3. Experimental Design and Procedure

(1)

One trial timing sequence

Considering the inherent delay in hemodynamic response, a block design [20] was used in this study in order to record EEG and fNIRS signals. Figure 2a shows a schematic of the timing for a single trial. The initial phase of 0–3 s entailed the subject attaining a state of wakeful relaxation to establish the baseline signal. Subsequently, during the 3–63 s interval, the subject was instructed to remain at rest to acquire the fNIRS baseline signal. Following this, at 63–66 s, the subjects were presented with either TPVCs or VRVCs to facilitate their execution of prompted MI tasks. These tasks included four types of MI tasks involving left-hand or right-hand grip strengths. During the 66–78 s interval, the subjects were asked to perform the same MI task 4 times. A flashing cross was presented before each MI task to prompt the subject to start it. The duration was 3 s. Finally, during the 78–88 s interval, the subjects were instructed to rest and prepare for the commencement of the subsequent trial.

(2)

MI tasks, training, and experimental design

This experiment involved four types of MI tasks for both left and right hands, namely, imagining left-hand lower grip strength (LLS, corresponding to the actual grip strength of 9–14 kg), left-hand greater grip strength (LGS, corresponding to the actual grip strength of 23–27 kg), right-hand lower grip strength (RLS), and right-hand greater grip strength (RGS).

Before the experiment, the subjects performed the four actual grip tasks and then underwent the corresponding MI training until they could imagine the corresponding grip strength clearly and controllably [21]. During the experiment, the subjects sat in a comfortable chair, relaxed their bodies, tried to avoid head and body movements, and wore an EEG-fNIRS cap. Subsequently, within the contexts of TPVCs and VRVCs, the subjects were individually prompted to mentally simulate the execution of grasping movements with their left or right hand, employing an optimal level of force. Each subject took part in two experimental phases, one under VRVCs and the other under TPVCs, using a VR head-mounted display (HTC vive-pro) and a computer screen as visual cues, respectively. Each experimental phase included 20 trials of LLS, LGS, RLS, and RGS MI tasks. After each task set, the participants completed the cognitive load scale [22] to assess their fatigue level, with at least one day between each experimental phase.

2.4. Data Acquisition

Figure 2b shows a schematic diagram of the EEG electrodes and fNIRS channels. A 32-channel EEG recording system (NeuSen W, neuracle, China) was used, with the reference electrode placed between Cz and Cpz and the ground electrode placed between Fz and Fpz. The sampling rate was 1000 Hz, and the electrode impedance was lowered to below 10 KΩ prior to data collection. fNIRS signals were acquired (Nirsmart, Danyang Huichuang, China) using 8 emitter and receiver probes (total of 10 fNIRS channels) placed around the frontal area electrodes (FP1, FPz, and FP2) with a sampling rate of 20 Hz. The data collected from both devices were simultaneously transmitted to MATLAB using the TCP/IP protocol for storage.

2.5. Data Processing Method

2.5.1. Preprocessing

(1)

EEG preprocessing

To prepare the EEG data for analysis, several preprocessing steps were performed. First, the raw EEG data were downsampled from 1000 Hz to 128 Hz. Next, a 50 Hz trap was applied, followed by bandpass filtering with a range of 0.1–40 Hz. An independent component analysis was then used to remove any artifacts, and a total of 9 channels (FC3, FC4, FCz, C3, C4, Cz, CP3, CP4, Pz) were selected for training the model.

(2)

fNIRS preprocessing

Physiological noise and baseline drift were removed from the raw fNIRS data using preprocessing steps. Specifically, the data were bandpass-filtered with a range of 0.01–0.2 Hz. Next, the optical density signal was converted into blood oxygen protein concentration using the Beer-Lambert law [23]. Finally, the mean values of the time series of the fNIRS channels around the forehead Fp1, Fpz, and Fp2 were calculated (Figure 2c).

2.5.2. Classification of Left-Hand and Right-Hand Grip Imagery Based on FBCSP and HFD

(1)

FBCSP

FBCSP is based on the Common Spatial Pattern (CSP) algorithm and uses a sliding frequency window to divide the data into multiple frequency bands. In this study, a window width of 4 Hz and a step size of 4 Hz were used to slide between 8 Hz and 32 Hz. CSP space domain filtering was then applied to the sub-bands, and the first three feature vectors were selected as the features of each sub-band. Finally, we used mutual information to select multiple sub-band features for classification.

(2)

HFD

HFD is a method for calculating the fractal dimension directly in the time domain without the need for phase space reconstruction [24]. The procedure of this method is as follows: for a channel of EEG data $X\left(t\right)$, a new time series is calculated using Equation (1):

$$X_k^m=\left[X\left(m\right),X(m+k),\dots ,X\left(m+\left[\frac{N-m}{k}\right]k\right)\right],m=1,2,\dots ,k$$

(1)

where $k=1,2,\dots ,k_{max},k_{max}$ is usually defined between 6 and 16, and N is the length of $X\left(t\right)$. The curve length $L_m\left(k\right)$ is calculated using Equation (2) for sequences $X_k^m$ with different k values:

$$L_m\left(k\right)=\frac{1}{k}\left(\sum _{i=1}^{\left[\frac{N-m}{k}\right]}\left|A(m,i,k)\right|\right)\left(\frac{N-1}{\left[\frac{N-m}{k}\right]k}\right)$$

(2)

where $A(m,i,k)=X(m+ik)-X(m+(i-1\left)k\right)$. The mean value $L\left(k\right)$ is calculated for each $L_m\left(k\right)$ using Equation (3):

$$L\left(k\right)=\frac{1}{k}\sum _{m=1}^k{L}_m\left(k\right)$$

(3)

Then, a straight line is obtained by fitting $ln\left(L\right(k\left)\right)$ with $ln(1/k)$ using the least squares method, and the slope of this line is the HFD characteristic, as in Equation (4):

$$\mathrm{H}\mathrm{F}\mathrm{D}=\frac{ln\left(L\right(k\left)\right)}{ln(1/k)}$$

(4)

Finally, the Principal Component Analysis is used to downscale the features composed of HFD and FBCSP, and K-Nearest Neighbor (KNN) is used to classify the four types of MI tasks (LLS, LGS, RLS, and RGS). The specific process is shown in Figure 3.

2.5.3. Stability Analysis of Grip Force Imagination Based on rt-DTW

The rt-DTW algorithm, proposed by Ding P. et al., was used in this study to calculate the speed fluctuations of the subjects performing the MI tasks and to analyze the stability of their performance. The algorithm consists of three main parts: random template sorting, correction template, and the calculation of speed fluctuations.

(1)

Random template sorting

In the first part, the dataset of each MI EEG trial for each subject is represented by $tmp$, and the $r$-th trial data $tm{p}_r$ is randomly selected from tmp as the initial template, where $1\le r\le 20$. The DTW distance $d_j$ between $tm{p}_j$ and $tm{p}_r$ is then calculated using Equation (5) for the $j$-th trial in $tmp$, and the trial data $tm{p}_j$ with the smallest distance from $tm{p}_k$ are obtained using Equation (6):

$$d_j=\frac{\mathrm{DTW}\left(tm{p}_j,tm{p}_r\right)}{\mathrm{length}\left(tm{p}_r\right)},1\le j\le 20,j\ne r$$

(5)

$$d_k=min\left(d_j\right),k\in j$$

(6)

where DTW is used to stretch or compress $tm{p}_j$ and $tm{p}_r$ along the time axis to align them, as well as to find the shortest distance between the two sequences [25]. The length function calculates the number of sample points in $tm{p}_r$.

(2)

Template calibration

To extract the data $tm{p}_k\left(w_{m,n}\right)$ from the k-th trail time window $w_{m,n}$, the window $w_{m,n}$ with a variable length is selected, where m denotes the left-end moment of the window and n denotes the right-end moment of the window, $0⩽m⩽0.5,1⩽n⩽3$. Initially $m=0$, in each cycle m remains unchanged, and at the end of each cycle m increases by 0.1, for a total of 6 cycles. Within each cycle, n is increased from 1 to 3 with a step size of 0.1. The data within the window $w(m,n)$ are extracted and used to calculate the minimum distance between $tm{p}_k\left(w_{m,n}\right)$ and $tm{p}_r\left(w_{m,n}\right)$ based on Equation (7). The corresponding window $w(m,n)$ with the minimum distance is selected, and $tm{p}_k\left(w_{m,n}\right)$ is used as the model.

$$d_k\left(w_{m,n}\right)=min\left(\frac{\mathrm{D}\mathrm{T}\mathrm{W}\left(tm{p}_k\left(w_{m,n}\right),tm{p}_r\left(w_{m,n}\right)\right)}{\mathrm{length}\left(tm{p}_k\left(w_{m,n}\right)\right)}\right)$$

(7)

(3)

Speed fluctuation calculation

For each trial in $tmp$, the data are extracted within the time window $w(m,n)$, the DTW distance is calculated with the model, and the time window $w_{x,y}$ with the smallest distance to the model is found. The parameters of $m,n$, and the number of cycles are the same as those in the template correction section. The ratio $s_i$ of the number of sampling points in the i-th trial data in $w(x,y)$ to the number of sampling points in the model length (model) is calculated using Equation (8). The closer the $s_i$ value is to 1, the closer $tm{p}_i\left(w_{x,y}\right)$ is to the model, and the execution time of the i-th trial is closer to that of the model, which is inversely proportional to the speed of execution. Finally, the standard deviation $\sigma$ of $s_i$ is calculated using Equation (9) to obtain the speed fluctuations of the subjects performing the MI tasks. The smaller the speed fluctuations, the higher the stability of the subjects performing the MI tasks.

$$s_i=length\left(tm{p}_i\left(w_{x,y}\right)\right)/\mathrm{length}\left(\mathrm{model}\right)$$

(8)

$$\sigma =\sqrt{\frac{1}{20}\sum _{i=1}^{20}{\left(s_i-\overline{s}\right)}^2}$$

(9)

where $\overline{s}$ is the mean of $s_i$.

2.5.4. Analysis of PSD Brain Topography and Brain Network in the Motor Area

(1)

Analysis of PSD Brain Topography in the Motor Area

The Power Spectral Density (PSD) describes the power distribution of a signal across different frequencies. In this study, the PSD features of each channel during the execution of the MI tasks by the subjects were plotted on a topography map to analyze the power information of the dominant frequency, reflecting brain activation in the motor area. In this study, a sliding time window with a length of 0.5 s and a step size of 0.5 s was utilized to extract the data, and the PSD features within the frequency band of 8–30 Hz were calculated to assess the subjects’ level of brain activation.

(2)

Analysis of Brain Network in the Motor Area

To analyze the brain network of the motor area, we calculated the Pearson correlation coefficient between the time series of each channel in the motor area, with values ranging from 1 to 9 (covering the 9 electrodes of the motor area). The calculation was performed using Equation (10):

$$A_{XY}=\frac{n{\sum }_{i=1,j=1}^n{x}_i{y}_j-\left({\sum }_{i=1}^n{x}_i\right)\left({\sum }_{j=1}^n{y}_j\right)}{\sqrt{n{\sum }_{i=1}^n{x}_i^2-{\left({\sum }_{i=1}^n{x}_i\right)}^2}\sqrt{n{\sum }_{j=1}^n{y}_j^2-{\left({\sum }_{j=1}^n{y}_j\right)}^2}}$$

(10)

The variable $A_{XY}$ represents the correlation degree between channels X and Y, where $x_i$ and $y_j$ are the i-th and j-th sample points of channels X and Y, respectively, and n is the number of sampling points. To avoid self-connection edges in the network, we set the diagonal of the resulting correlation matrix ($AM)$ to zero [26] and then took the absolute value of $A_{XY}$. The formula for $AM$ is shown in Equation (11):

$$AM=\left[\begin{array}{cccc}0& \left|A_{12}\right|& \cdots & \left|A_{19}\right|\\ \left|A_{21}\right|& 0& \cdots & \left|A_{29}\right|\\ \cdots & \cdots & \cdots & \cdots \\ \left|A_{91}\right|& \left|A_{92}\right|& \cdots & 0\end{array}\right]$$

(11)

Subsequently, we applied a threshold to $AM$ to obtain the binary correlation matrix $aM$, where $a_{XY}$ is either 0 or 1. The formula for $aM$ is demonstrated in Equation (12):

$$aM=\left[\begin{array}{cccc}0& a_{12}& \cdots & a_{19}\\ a_{21}& 0& \cdots & a_{29}\\ \cdots & \cdots & \cdots & \cdots \\ a_{91}& a_{92}& \cdots & 0\end{array}\right]$$

(12)

Finally, we calculated the average degree and average clustering coefficient to analyze the characteristics of the brain functional network.

2.5.5. Fatigue Detection Based on Mean HbO Features in fNIRS

fNIRS measures changes in the absorption of near-infrared light by brain tissue to indirectly reflect brain activity. In this study, we used the mean value of the oxygenated hemoglobin (HbO) feature to reflect the degree of fatigue in the MI tasks under two different visual cue conditions: TPVCs and VRVCs. To account for the delay in the hemodynamic response and individual differences in response time, we calculated the mean feature values for three different time windows (0–10 s, 1–11 s, and 2–12 s). In addition, we used the cognitive load scale to analyze changes in the cognitive load of the subjects.

3. Results

3.1. Classification Accuracy of MI under TPVCs and VRVCs

Table 1. Classification accuracy of four types of MI (LLS, LGS, RLS and RGS) under TPVCs and VRVCs.

Subject	TPVCs (Mean ± SD (%))	VRVCs (Mean ± SD (%))
S1	53.44 ± 5.79	55.63 ± 6.39
S2	58.13 ± 5.93	49.38 ± 7.34
S3	58.13 ± 7.10	48.44 ± 4.72
S4	45.63 ± 7.82	51.56 ± 9.69
S5	39.69 ± 5.32	52.50 ± 3.84
S6	42.50 ± 6.46	59.38 ± 8.07
S7	57.50 ± 6.11	49.69 ± 4.02
S8	50.00 ± 6.61	42.50 ± 6.11
S9	52.50 ± 4.37	52.81 ± 5.20
Mean	50.83 ± 6.71	51.32 ± 6.20

shows the MI classification accuracies of the nine subjects under TPVCs and VRVCs. Under TPVCs, the mean, maximum, and minimum classification accuracies were 50.83%, 71.88%, and 31.25%, respectively. Under VRVCs, the mean, maximum, and minimum classification accuracies were 51.32%, 68.75%, and 37.50%, respectively. There was no significant difference in the subjects’ MI classification accuracies between TPVCs and VRVCs (t-test, p = 0.85).

3.2. Analysis of MI Speed Fluctuations in Subjects under TPVCs and VRVCs

Figure 4 shows the speed fluctuations of the subjects during MI under TPVCs and VRVCs. The standard deviation of MI speed under TPVCs was between 0.50 and 0.55, while under VRVCs, it was between 0.55 and 0.60. The standard deviation of MI speed under TPVCs was significantly smaller than that under VRVCs (t-test, p < 0.05). There was no significant difference in the speed standard deviation under the same visual cue.

3.3. Brain Activation and Connectivity Analysis of MI in Subjects under TPVCs and VRVCs

Figure 5 shows the scalp topography of the MI PSD features for the subjects under TPVC and VRVC conditions, while Figure 6 shows the t-test p-values of the PSD features for different time periods of MI execution under TPVC and VRVC conditions. In Figure 5 and Figure 6, it can be observed that there were no significant differences in the MI PSD features between the TPVC and VRVC conditions within the initial 0.5 s after the start of the experiment. However, significant differences (p < 0.05) were observed in some subjects during the 0.5–2.5 s period.

Figure 7a,b show the binary matrices of the EEG correlation in the motor area between the resting state before/after the TPVC experiment, while Figure 7c,d show them before/after the VRVC experiment. The binary matrices of the EEG correlation between resting states before the TPVC and VRVC experiments are similar. However, compared to TPVCs, the binary matrix of the EEG correlation after the VRVC experiment shows a more noticeable change from the resting state. Figure 7e,f show the brain functional network parameters, and the mean degree and mean clustering coefficient of the brain functional network are significantly higher after the VRVC experiment than after the TPVC experiment (t-test, p < 0.05).

3.4. Analysis of MI Fatigue Level in Subjects under TPVCs and VRVCs

Figure 8 shows the concentration changes in HbO in the frontal area Fp1, Fpz, and Fp2 surrounding the fNIRS channels (as shown in Figure 2c) during the MI period in the TPVC and VRVC conditions. In the TPVC condition, the HbO concentration range in the frontal area during the MI period was between 0.35 and 0.80, with a mean between 0.50 and 0.60. In the VRVC condition, the HbO concentration range in the frontal area during the MI period was between 0.25 and 0.70, with a mean between 0.40 and 0.50. The mean HbO concentration in the frontal area during the MI period in the VRVC condition was significantly lower than that in the TPVC condition (t-test, p < 0.05).

Table 2. Cognitive load scale results.

Dimension	Evaluation Rating (Mean)
	MI under TPVCs	MI under VRVCs
Mental demands	4.56	4.78
Physical demands	2.33	2.44
Temporal demands	4.22	4.11
Performance level	4.11	3.89
Effort level	4.11	5.89
Discomfort level	3.00	5.00
Sum	22.33	26.11

shows the mean cognitive load scores reported by the nine subjects during the MI task. The average cognitive load score for the subjects under TPVCs during MI was 22.33, significantly lower than the average cognitive load score of 26.11 reported by the subjects under VRVCs during MI (t-test, p < 0.05).

4. Discussion

This study investigated the effects of VRVCs and TPVCs on MI performance and stability in subjects. The results show that, for different visual cues, the MI accuracy of all subjects was higher than chance level (25%), indicating that the selected subjects were able to perform the corresponding MI tasks. There was no significant difference in the classification accuracy of MI between VRVCs and TPVCs (Table 1), possibly because the visual cues only prompted or guided the subjects to perform the MI task, but had a low impact on classification accuracy [13]. Otherwise, the potential reasons for the decreased classification accuracy might include the subjects’ lack of familiarity with the BCI system, the absence of prior experience with BCI technology and motor imagery, or insufficient pre-experimental training that hindered the subjects’ ability to perform the motor imagery task effectively. In addition, a questionnaire was employed to offer a comprehensive description of this situation (

Table 3. BCI Ability Assessment Scale for Subjects.

Description	Evaluation Rating
	S1	S2	S3	S4	S5	S6	S7	S8	S9
Familiarity with BCI	4	5	2	6	2	5	8	3	2
Experience with BCI	4	5	2	5	6	3	7	5	6
Experience with MI	4	4	1	3	2	3	5	5	2
Understanding of the MI Paradigm	5	7	6	8	6	5	6	9	7

Higher scores indicate greater familiarity.

).

According to the stability results of the subjects’ MI performance, it was found that, under the same visual cue condition, the speed fluctuations of the four types of MI tasks were similar, which may indicate that the execution stability of different MI tasks is similar under the same visual cue condition. Compared with VRVCs, TPVCs can reduce the speed fluctuations of subjects’ MI performance; that is, the time taken by subjects to perform MI is more similar, and the stability of the subjects performing the MI tasks is higher. This may be because, under TPVCs, subjects can concentrate more on the MI task itself without being distracted by the additional visual information provided by VRVCs. In addition, TPVCs are simpler and easier for subjects to understand and master, which may improve the stability of MI.

The lack of differences in PSD features within the initial 0.5 s after the start of the MI experiment between the TPVC and VRVC conditions may be attributed to the potential influence of the presented crosshair markers on the subjects’ PSD features, as well as the possibility that the subjects did not initiate the corresponding MI task within the 0.5 s timeframe. However, during the four time periods from 0.5 s to 2.5 s, significant differences in PSD features were observed in the majority of subjects, indicating variations in neural activation during MI between the TPVC and VRVC conditions. Nonetheless, these differences might be offset by a small number of subjects who did not exhibit significant differences, particularly when calculating mean values and when generating scalp topography plots. The subjects who did not show significant differences may be less familiar with MI-BCIs or the MI paradigm. In addition, the differences in brain functional networks between the TPVCs and VRVCs may be due to the fact that the subjects received and processed more VRVC information [21,26,27].

This study used fNIRS, which is less susceptible to electric interference from head-mounted VR, to evaluate the level of fatigue during MI. The subjects believed that more mental and physical effort was required for MI under VRVCs [14,15], but they were more satisfied with their performance. They also perceived the weight and blurriness of the head-mounted VR as factors contributing to fatigue and discomfort.

However, this study has some limitations. Firstly, additional metrics may be necessary to comprehensively assess the effects of different visual cues on MI-BCI performance. Additionally, further research is needed to explore the effects of different factors in different VR environments, such as the complexity and interactivity of virtual scenes, on MI-BCIs. Future studies can further address these issues to improve the effectiveness and feasibility of MI-BCIs.

5. Conclusions

This study indicates that TPVCs and VRVCs have different effects on subjects and MI-BCI performance when constructing MI-BCI classification models. Although there was no statistical difference in classification accuracy, the subjects showed more stable MI under TPVCs, while VRVCs provided more prompt information to the brain, which could have had a negative impact on the subjects’ MI performance and increased their cognitive load and fatigue. These findings contribute to providing insights and directions for combining MI-BCIs with VR.