Next Article in Journal
The Relationship between Urban Expansion and Air Pollution Depends on Urban Spatial Morphology in the “2 + 26” Cities of Beijing-Tianjin-Hebei, China
Next Article in Special Issue
Novel Deep-Learning Approach for Automatic Diagnosis of Alzheimer’s Disease from MRI
Previous Article in Journal
The Role of Bone Alkaline Phosphatase and Osteocalcin in Saliva as Indicators of Skeletal Maturity in Children
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Review

Sensorial Feedback Contribution to the Sense of Embodiment in Brain–Machine Interfaces: A Systematic Review

by
Diogo João Tomás
1,2,
Miguel Pais-Vieira
3 and
Carla Pais-Vieira
1,*
1
Faculty of Health Sciences and Nursing (FCSE), Center for Interdisciplinary Research in Health (CIIS), Universidade Católica Portuguesa, 1649-023 Lisboa, Portugal
2
Escola Superior de Saúde Atlântica, 2730-036 Barcarena, Portugal
3
Department of Medical Sciences, Institute of Biomedicine (iBiMED), Universidade de Aveiro, 3810-193 Aveiro, Portugal
*
Author to whom correspondence should be addressed.
Appl. Sci. 2023, 13(24), 13011; https://doi.org/10.3390/app132413011
Submission received: 15 September 2023 / Revised: 3 December 2023 / Accepted: 4 December 2023 / Published: 6 December 2023
(This article belongs to the Special Issue Computational and Mathematical Methods for Neuroscience)

Abstract

:
The sense of embodiment (SoE) is an essential element of human perception that allows individuals to control and perceive the movements of their body parts. Brain–machine interface (BMI) technology can induce SoE in real time, and adding sensory feedback through various modalities has been shown to improve BMI control and elicit SoEe. In this study, we conducted a systematic review to study BMI performance in studies that integrated SoE variables and analyzed the contribution of single or multimodal sensory stimulation. Out of 493 results, only 20 studies analyzed the SoE of humans using BMIs. Analysis of these articles revealed that 40% of the studies relating BMIs with sensory stimulation and SoE primarily focused on manipulating visual stimuli, particularly in terms of coherence (i.e., synchronous vs. asynchronous stimuli) and realism (i.e., humanoid or robotic appearance). However, no study has analyzed the independent contributions of different sensory modalities to SoE and BMI performance. These results suggest that providing a detailed description of the outcomes resulting from independent and combined effects of different sensory modalities on the experience of SoE during BMI control may be relevant for the design of neurorehabilitation programs.

1. Introduction

In the field of cognitive sciences, the ability that enables a person to feel their own body parts, initiate and control their own actions, and perceive mental states as their own is known as the sense of embodiment (SoE) [1,2,3,4,5]. SoE has been identified as a necessary component for achieving health outcomes and behaviors [6,7] and may be compromised under clinical conditions [8,9,10,11]. SoE is not limited to our own physical body but can also be induced through the perception and illusory control of a virtual or robotic body or body parts [12,13,14,15,16,17,18]. While there are various definitions of SoE concerning external bodies, in this study, we adopt the definition provided by Kilteni and colleagues [3]. According to these authors, SoE is a sense that arises when the properties of an external body are processed as if they were the properties of one’s own biological body [3]. Furthermore, it has been demonstrated that some aspects of SoE can be achieved simply by being in control of objects that bear no human resemblance [17,19,20]. Measuring SoE has posed a challenge for empirical research, and several attempts have been made to find standardized psychometric measures [5,21,22]. In addition to subjective measures, there have been endeavors to measure embodiment through electrophysiological recordings [23,24], skin conductance responses (SCRs) [25,26,27], or body temperature measurements [28,29].
SoE can be broken down into three underlying components: the sense of self-location (SoL), the sense of ownership (SoO), and the sense of agency (SoA) [3]. While some studies do not differentiate between these components or employ different terminology, an analysis of the subjective questions used allows associating each of them with one of these three categories. According to Kilteni and colleagues (et al., 2012), the subjective experience of recognizing oneself as the agent of certain behaviors is described as SoA, the feeling that a body (or its parts) belongs to the person is referred to as SoO, and the experience of being situated in the space where one’s body is located is denoted as SoL.
SoO can be considered in a broader sense, where it includes the feeling of mineness not only of the body (body ownership) and its parts (limb-ownership) but also of feelings and thoughts (see Braun et al., 2018, for a review) [30]. Meanwhile, the same authors highlight that SoA allows not only the distinguishing between self- and other-generated actions but also the intention to generate motor activity (i.e., motor imagery). Although there have been multiple studies on the neurophysiological basis of embodiment, its evaluation is usually complemented by the use of questionnaires [31,32,33,34,35].
There are several studies demonstrating that the illusory experience of the body increases with the use of a brain–machine interface (BMI) or brain–computer interface (BCI) [14,15,36,37,38]. However, to be able to understand how BMIs influence embodiment, it is necessary to define and classify BMIs. In general terms, BMI technology enables the use of brain activity (or a proxy) decoded in real time to control an external device [39,40]. The terms BMI and BCI are generally considered synonymous terms [41]. Here, we will adopt the BMI to refer to both BMI and BCI.
There are different types of BMIs and some of them can be categorized as active BMIs or as reactive BMIs [39]. The active BMI is a system that uses neural activity resulting from voluntary activity, as occurs during motor imagery (i.e., thinking about walking). Motor imagery-based BMIs (MI-BMIs) are the most used type of BMI. On the other hand, a reactive BMI is a system that uses brain signals resulting from a reaction to an external stimulus. A very common example of this is the steady-state visually evoked potentials (SSVEPs), where changes in brain activity are evoked in the visual cortex through a visual stimulus flickering at a specific frequency [39]. Other devices that interact with neural activity and embodiment to some degree, or are close to BMIs but do not constitute actual BMIs, will not be considered here. This is due to their potential to result in varying levels of user engagement and embodiment.
Previous studies have suggested that SoE can be increased if multimodal sensory stimulation (visual, tactile, auditory, etc.) is used [15,38,42,43,44]. In the same way, this increase in sensory stimulation plays an important role in improving performance during MI-BMI training sessions [17,35,36,37,38,39,40,41,42,43,44,45,46,47]. SoE increased via sensory feedback while using a BMI has been recognized as beneficial for more efficient MI-BMI training [13,17,45]. However, experimental studies that manipulate the increasing sensory feedback modalities’ effect on SoE during BMI performance are scarce and necessary to assess the effectiveness and efficiency of using these technologies. Specifically, it is unclear whether increasing the number of sensory modalities during BMI increases SoE, as well as what is the contribution of each sensory modality. Studies describing the independent and joint contributions of the different sensory modalities to the SoE during BMI control may be relevant for neurorehabilitation protocols.
In this review, we will examine studies that have employed measures to assess SoE during use of BMIs that included single or multisensory feedback. More specifically, the purpose of the present study is to analyze the contribution of multisensory feedback to embodiment and encoding in studies using BMIs.

2. Methods

The systematic review was carried out according to the Reporting Items for Systematic Review and Meta-Analysis (PRISMA) 2020 checklist [48]. The protocol was registered in the International Prospective Register of Systematic Reviews (PROSPERO) in September 2022 (CRD42022348645).

2.1. Search Strategy and Selection

The search was carried out on PubMed, Web of Science, SCOPUS, and Cochrane databases. The search strategy was formulated based on combinations of three concepts: embodiment AND brain–machine interface AND sensorial feedback. Due to the diversity of nomenclature, the embodiment concept was also searched in the form of volition, ownership, agency, body experience, and presence. The brain–machine interface concept was also searched as brain–computer interface. And finally, the sensorial feedback concept was also searched in the form of sensory stimulation, multisensory, visual, tactile, haptic, vibrotactile, auditory, sound, temperature, virtual reality, and virtual immersion. Search results from each database were merged and sorted for the removal of duplicates. Afterwards, titles and abstracts were screened according to the inclusion/exclusion criteria. The screening process was performed by the authors. The full text of the selected papers was obtained for closer inspection. Any disagreement concerning whether to include a specific study was discussed among all the authors.
The studies selected in the review were based on the following inclusion criteria: (i) studies with the full text published in English; (ii) studies were original research; (iii) studies were only carried out with experiments related to humans; (iv) studies integrated SoE variables; (v) studies integrated BMIs (note that studies not using an actual BMI but only giving participants the impression of using it were also excluded from the search); and (vi) studies integrated at least one sensory feedback modality. Additionally, studies were excluded if they (i) were reviews, (ii) were conference papers, (iii) were not peer-reviewed material, or (iv) did not have accessible full text (also, see Figure 1).
The searching strategy returned 493 articles, 223 of which were duplicates. A total of 2 were in foreign languages, 86 were conference papers, and 1 was not full-text available. From the remaining 181 articles searched, 119 did not have any type of sense of embodiment measures, 33 were reviews or conference papers, and 7 did not use brain–machine interface devices. After full-text reading, 2 papers were excluded for using just a simulation of a BMI. This resulted in a final list of 20 papers in our review. Details of the process are described in the flowchart (Figure 1).

2.2. Data Extraction and Analysis

For research articles that were included in the review, we extracted the following data systematically from each study: (1) sources of studies; (2) types of BMI; (3) SoE-reported measures; (4) modalities of sensorial feedback; and (5) BMI performance. The number of sensorial modalities used for feedback was counted. Also, the numbers of classes used for the task(s) employed in each study were also counted.

2.3. Assessment of Quality

Assessment of quality was performed as in previous studies [49,50]. Briefly, three different researchers independently read and scored each study (scores between 1 and 3) across the five dimensions: (1) research design; (2) methods and analysis; (3) generalizability; (4) relevance of focus; and (5) reliability of findings.
For dimension 1, research design, an evaluation of the experimental groups and variable manipulations (related to the topic of the present review) was conducted. Studies lacking a control group or that were unbalanced were scored as 2, while studies with control group or that were counterbalanced were scored as 3. For dimension 2, methods and analysis, the presence of proper statistical analysis was scored as 3. For dimension 3, generalizability, not only the size but also the existence of an equal number of male and female subjects, as well as age distribution, were considered. Studies with small sample sizes, heterogenous clinical presentations, and age or gender inequality received low scores (1 for single subject, 2 for <15 or only one gender, and 3 for larger and/or more representative samples). In dimension 4, the quality of the study, regarding the present review, was evaluated. An assessment was made regarding the extent to which each study, or its components, addressed the main questions, such as the effects of multisensory feedback on SoE (or SoA, or SoL, or SoO), as well as in BMI performance. Studies that focused solely on one patient were scored as 2. For dimension 5, reliability of findings, the extent to which the study findings can be trusted in answering the study question was considered. A score of 3 was assigned if the experiments conducted, the results obtained, their analysis, and their limitations effectively contributed to the conclusions drawn.
After scoring each paper, the mean score was calculated as 13.75 ± 1.08 (mean ± SD) with scores ranging between 11 and 14.67 points. For manuscripts with dimensions equally scored by all three researchers, the mean of the three values is presented (without the standard deviation, which was SD = 0.0). For dimension 3, generalizability, the researchers disagreed on the scores, so both the mean and standard deviation were presented.

3. Results

Of the twenty studies included in this review, two were carried out with only one participant (see Table 1).
The remaining studies had an average N = 21.8 ± 10.6 subjects (mean and standard deviation), with a min of N = 7 and a max of N = 40 subjects (see Table 2).
The publication date ranged from 2009 to 2022 (see Table 1, Table 2 and Table 3). The score obtained for each paper regarding the 5 dimensions can be found in Table 3.
We will start by describing the type of BMI technology used, then the type of questionnaire used to study SoE, as well as aspects related to the different nomenclature, followed by additional techniques used to objectively evaluate SoE (e.g., SCR). Then, a detailed analysis of sensory feedback modalities, SoE, and BMI performance will be made.
Regarding the type of BMI technology used, 15 studies were MI-based, where brain activity was captured using non-invasive electroencephalography (EEG). One study, a spinal cord-injured (SCI) patient [37], also used an active MI-BMI, but the activity was captured intracortically. Four studies used a reactive BMI, where one of them used P300 technology and the other three were based on SSVEPs (see Table 1). One of the studies that used SSVEP-BMI also used MI-BMI in the experimental design [14]. Here, the authors intended to explore the performance of the BMI and the effect on SoO and SoA when a BMI used neural activity from the sensorimotor areas (MI-BMI) or activity from the visual areas (SSVEP-BMI). Although the performance was slightly higher for the MI-BMI group, this difference was not statistically significant. Significative higher ratings of SoO and SoA were found in the MI-BMI group.
In all studies, subjective questionnaires (7-point or 11-point Likert rating) were used to measure embodiment variables (also see Table 4 for details). One study adopted a classification from 0 to 100 points [18]. In only two studies [19,37], the answers were given by choosing “yes” or “no”. However, in one of these [37], the authors added a second request to rank from 0 to 100 on the degree of certainty in the given answer. The number of questions used in the questionnaires was also very variable. Of the analyzed studies, the number of questions to assess SoE variables ranged from 1 to 26.
Differences in the nomenclature of the various SoE components were present in the studies analyzed. As we have followed the classification proposed by Kilteni et al. (2012), it was necessary to specifically analyze all the questions posed in each of the studies to be able to fit them into the scope of SoO, SoL, or SoA (see Table 1). It was found that there is great heterogeneity in the terminology used. For example, in some studies, the authors used the term SoE when referring to “It was as if the virtual body was my body” and the term sense of control when referring to “I felt in control of avatar’s actions” [18,46]. However, according to Kilteni et al., 2012, this should be included in the SoO and SoA categories, respectively. Among the 20 studies included in this review, 16 had SoO questions, 14 had SoA questions, and 8 studies had SoL questions in their questionnaires. Only five studies had questions covering the three components of the SoE. Other combinations of SoE components can be analyzed in Figure 2.
It is also common in studies related to SoE to include measures of disownership, defined as the experience that a body part does not belong to the subject [61]. In most cases, these measures were focused on assessing the participant’s awareness of their actual body parts. An example of this is the study by Škola and Liarokapis (2018) when the participants were asked about the sense of proprioception related to awareness of the position of their own hands [17]. In two other studies, the authors were interested in assessing the influence of the apparatus illusion on the real body parts. In the work of Perez-Marcos et al. (2009), a proprioceptive drift was objectively measured by asking the participants to indicate, without looking, the location where they perceived the real hand to be placed after a virtual arm-moving task in the BMI [16]. Meanwhile, in the study of Ziadeh and colleagues [20], participants were questioned about the movement of the virtual hands over their own hands.
Apart from these subjective measures for SoE, three additional studies from the same group also assessed the SCR following a threatening stimulus to a non-real body part (injection in the robot hand) [12,52,54]. This procedure intended to assess more objectively how embodied the participant with the external body parts was during the BMI task. For this, the authors measured the physiologic changes (trough the assessment of SCR) in the real body during the BMI-performing task. Across the three studies, the SCR was higher in the experimental conditions with higher scores of reported SoO. In addition, this was also verified for SoA when it was included as a variable [54]. In another study, electromyography (EMG) was used to estimate the amount of muscle activity required to perform a virtual task if it occurred in the real world [16]. The group with higher EMG activity also reported higher levels of SoO.
Providing sensory feedback while performing BMI tasks is one of the presumed strategies used to improve BMI performance itself [62,63]. It is hypothesized that this agreement between stimuli and actions is relevant to the SoE experience. However, few studies explore the contribution of different sensory modalities to SoE. All studies included in this review used visual stimuli as their main sensory modality (see Figure 3). Although the auditory modality was present in six studies, and some form of haptic stimulus was present in four studies, none of them specifically quantified the influence of these stimuli on the SoE variables. Immersive visual feedback was present in 14 studies whereas the others used non-immersive visual feedback. Normally the non-immersive visual feedback was received through a computer screen. Only in one study, the visual feedback was non-virtual with participants observing a robotic hand moving in front of them [36]. Within the visual feedback modality, the immersive type has shown higher levels of SoE compared to the non-immersive type [13]. The congruence or incongruence of the visual stimulus with the MI action performed seems to have a strong impact on SoE. Congruent visual stimuli seem to increase SoO [12,16,52]. However, differences between studies were found for SoA. In one study, no significant differences were present in SoA levels between incongruent and congruent visual stimulus conditions [16]. In contrast, another study reported a negative impact of visual incongruence on SoA [19]. These differences may find some explanation regarding the type of visual stimulus that was provided. Although both used a non-immersive form of visual feedback, in the study by Perez-Marcos et al. (2009), the feedback was based on the movement of a virtual hand, while in Evans et al. (2015), the feedback congruence was associated with the displacement of a virtual bar on a screen. In this second study, it was also verified that larger delays negatively impact the SoA. However, for delays under 1000 ms, no significant differences were found [19]. This information becomes relevant to understand the acceptable limit of delay in this type of BMI technology, since any feedback addition has some amount of expected delay due to the normal computational processing. In the only study where the congruency of an auditory stimulus with a visual stimulus was explored, no significative impact on the SoE was found [18].
The realism of sensorial feedback seems to enhance more realistic responses of the participants that are exposed to it [15,44,64]. In a case study, the exposure of an SCI patient to a very realistic immersive virtual environment combined with auditory and thermal stimuli consistent with the scenario and with tactile vibratory stimuli coherent with the action (walking) has been shown to provide high levels of SoE [15]. Many attempts to provide realism to a BMI experience have been performed by several research groups. For example, it has been proposed that the control of robotic hands with a more human-like shape through MI-BMI may induce higher SoO than controlling robotic hands with the shape of mechanical tweezers [53]. Nevertheless, both conditions seem to induce high scores of SoA that do not differ significantly from each other. Also, in another study in which the MI-BMI task consisted in popping virtual balloons with virtual hands or with virtual blocks, both conditions showed high scores of SoA but did not differ among themselves [20]. In this same study, it was also found that the illusory induction of a sense of movement in one’s own hands was greater in the virtual hands’ condition than in the virtual blocks’ condition.
The BMI tasks present in the studies analyzed typically involve training and evaluation phases. Interestingly, these studies tended to adopt realistic conditions in their evaluation phases, but the training phase for the acquisition of sensorimotor activity related to MI typically followed a standard protocol using simplistic arrows and a bar graph as visual feedback [65]. Exceptions to this are the studies by Škola and Liarokapis (2018) and Pais-Vieira and colleagues (et al., 2022). Škola and Liarokapis (2018) also decided to explore if the introduction of a realistic scenario already in the training phase leads to a higher rate of SoE. However, they concluded that the more realistic experience during the training phase does not seem to significantly affect the SoO and SoA scores during the experience evaluation phase [17]. Meanwhile, Pais-Vieira and colleagues (et al., 2022) used a highly realistic scenario in the training and evaluation phases to ensure that spinal cord injury patients maintained high levels of engagement throughout the multiple sessions that constituted the experimental protocol [15].
Only four of the twenty studies included some type of haptic stimulus [15,37,56,57]. Three applied the stimulus in the form of vibration while one applied it in the form of electrostimulation. The vibratory feedback was applied in two studies, not intended to replicate the real tactile sensation expected in an action performed, but its application intended to be consistent with the timing of some event in the virtual scene. In the study of Pais-Vieira et al. (2022), the BMI task allowed an avatar of the subject taking steps. The vibration stimulus matched the moment that the sole of the avatar’s foot touched the ground when walking and was delivered to the participant’s forearm [15]. Meanwhile, in the study of Škola and colleagues (2019), the vibratory stimulus was applied to the participant’s hand and was consistent with triggering a weapon from a spaceship. Both studies reported high levels of SoO and SoA despite the very different conditions tested [57]. Differences were found for the SoL where the apparatus of Pais-Vieira et al. (2022) seems to have also induced high levels of SoL. It should be noted, however, that multimodal feedback was used (visual, auditory, and tactile) and the contribution of each modality was not individually assessed.
The application of a vibratory stimulus as feedback to mimic the sensation induced by the action if it was truly performed was explored in the study of Tidoni, Gergondet, et al. (2017) [46]. A vibratory stimulus was applied in the right bicep’s brachial tendon of the subjects, inducing the proprioceptive illusion of downward extension of the elbow while performing a similar movement with a virtual arm via MI-BMI. This condition was compared with a vibratory stimulus applied over the bone (not inducing a proprioceptive illusion). Despite the specificity of the stimulus application, no significant differences were found in SoE between the different conditions.
A very interesting case study using an SCI patient combined somatosensory feedback in the form of muscular electrostimulation with visual feedback through the visualization of a virtual hand [37]. Here, the authors explored the different combinations between the congruence and incongruence of the different stimuli with MI action during BMI. The authors reported that somatosensory congruency was more effective in driving SoA. Ratings were higher when both feedback signals were congruent as compared to both being incongruent. When visual feedback was incongruent but somatosensory feedback was congruent, higher levels of SoA were reported as compared to the condition where visual feedback was congruent but somatosensory was not.
The present review also aimed to explore the relationship between SoE variables and BMI performance. Of the 20 studies, 6 of them attempted to establish a relationship between reported SoE values and task performance in BMI. A positive relationship was found between SoO and BMI performance [13,45,52,58]. Also, the SoA showed a significant correlation with BMI performance [13,19]. Regarding SoL, two studies found a positive relationship with BMI performance [13,58] while another one found no significant correlation [55]. The studies analyzed here focused mainly on the effects of congruence, synchrony, and likeness of stimuli and scenarios in SoE.
A comparison between the number of sensory modalities, embodiment, and BMI performance does not support the existence of a clear relation between levels of embodiment, or its components, and BMI performance (refer to Table 4). Out of 17 studies where a comparison between BMI performance and SoE (or one of its components) values was possible, 11/17 = 64.7% involved a single modality. Moreover, 11/11 = 100% studies used had visual or visual immersive as the feedback sensorial modality. All of these eleven studies were associated with an increase in SoE or one of its components, but from the nine studies that reported both the values of embodiment and of BMI performance [13,17,19,45,51,52,53,54,58], five (5/9 = 55.56%) were associated with an increase in performance [17,45,53,54,58] and three (3/9 = 33.33%) [13,19,52] had no effect on performance. Only in one study (1/9 = 11.11%) [51], a detrimental effect in BMI performance occurred. Therefore, studies using a single feedback sensorial modality were all based on visual feedback, all reported increases in embodiment or one of its components, and approximately half reported improvements in BMI performance.
A total of eight studies included two sensorial modalities (8/20 = 40.00%) [14,18,20,36,37,55,56,57]. From these, 5/20 = 25.00% included visual (or visual immersive) and auditory feedback [14,18,20,36,55] and 3/20 = 15.00% included haptic feedback [37,56,57]. In two studies where visual feedback was paired with auditory feedback, an improvement in BMI performance and in SoE (or one of its components) occurred [14,18]. In the remaining three studies, either no difference in BMI [20,55] or in SoE [36] was reported. Meanwhile, in the three studies where haptic feedback was paired with visual feedback, no improvement in SoE nor in BMI performance was reported in one case [56], and no values were reported in another [57]. The third study with two haptic modalities [37] tested somatosensory and visual feedback and revealed that the congruent somatosensory feedback prevailed over incongruent visual feedback, namely increasing SoE and BMI performance. This study was performed in a single patient using intracortical recordings. Only one study included four different types of sensory feedback (visual, auditory, haptic vibratory, and haptic thermal) [15], but no difference in embodiment or in BMI performance was reported. It is noteworthy that this study was performed on a patient and included only 10 sessions.
Lastly, of the 20 studies analyzed here, gender bias was present in 12 of them. In three studies, the number of males more than doubled the number of females [19,20,51], and one studied only females (N = 29) [14]. This male gender bias most likely reflects the increased number of male patients previously reported [66] and highlights the need for more studies in the female population [67].

4. Discussion

This review has analyzed studies using BMIs to determine if an increase in the number of sensory modalities is associated with increased SoE and improved BMI performance. Most studies employed motor imagery-based BMIs through EEG recordings and included only one type of feedback, either visual or visual immersive. This type of feedback was consistently associated with increased SoE, but only 55.56% of the cases were associated with an improvement in BMI performance. Studies that combined two different types of sensory feedback either used visual (or visual immersive) and auditory feedback or, alternatively, visual (or visual immersive) and haptic feedback in the forms of vibration or electrical stimulation. While most studies utilized one or two types of sensorial feedback in an EEG motor imagery-based BMI, a small number explored different approaches such as SSVEPs, intracortical recordings, or incorporated more than two types of sensory feedback. Additionally, a relatively limited number of studies were conducted in patients. Lastly, a noticeable bias towards male participants was observed. The studies analyzed in this review do not support the notion that an increased number of sensory modalities enhances SoE and BMI performance. However, they also highlight the fact that, to date, no study has systematically explored the influence of different sensorial modalities in SoE and BMI performance.
SoE, particularly the sense of ownership (SoO) over external objects, has been investigated using the rubber hand paradigm [42]. In this paradigm, users have one of their hands hidden but exposed to tactile stimuli while simultaneously observing a substitute rubber hand. During the experiment, the rubber hand receives the same stimulus at the same time as the participant’s hidden hand, leading the participants to attribute the proprioceptive sensation to the observed stimulus rather than the one delivered to their own skin. Some researchers have proposed that the SoO experienced towards the rubber hand contributes to the SoA. In other words, participants who feel a strong SoO over the rubber hand also report a high perceived SoA, believing they could control the movements of the rubber hand if they desired [42,68]. Several studies included in this review reported a suggested connection between SoO and SoA [17,20,53,54]. However, previous research has shown that a strong SoO over a rubber hand can occur without feeling agency over its movements [69]. Also, visual representations resembling a human body or body part have been found to enhance the SoE compared to more abstract representations with subjects reporting feeling less embodied by a virtual block [20] or a robotic tweezer [53] than by a human-like hand.
Several research groups have made efforts to develop BMI tasks with more realistic actors such as robots or avatars. Many studies have focused on exploring the effects of congruent and incongruent sensory feedback on actions and their influence on SoE. This has been examined in relation to visual [12,16,19,37,52], auditory [18], or haptic [37,56] congruency/incongruency of feedback. These studies support the idea that the sense of agency is a fundamental component of embodiment processes and is influenced by sensorimotor congruence in the executed action, with sensory input playing a crucial role [19]. It appears that, as long as some congruent sensorial feedback is provided, the SoO [68] and SoA [70] can be induced in participants. In other words, these participants believe they are controlling a task through a BMI when, in fact, they are not. Therefore, it is possible to induce some SoO in an additional bodily part, such as a third hand, without losing the SoO in their real hands [68]. These studies suggest that the intention to control a BMI may recruit both SoO and SoA.
In general, congruent visual stimuli have been found to elicit higher levels of SoO [12,16,52] and SoA [19,37]. However, it is still inconclusive whether congruence of vibratory feedback [18] or auditory feedback stimuli [56] leads to a higher level of SoE. Only one study included in this review, which focused on an SCI patient, found that the congruence of the haptic feedback might have a greater influence on the induction of SoA than the congruence of the visual feedback [37]. However, it should be noted that the haptic stimulus used in that study involved muscular electrostimulation, which induces movement through muscular contraction and can be considered haptic–kinesthetic feedback. This type of feedback cannot be directly compared to haptic–tactile feedback, such as that resulting from vibration, as they have very different characteristics.
Interestingly, despite the importance of congruence in the feedback stimuli, the presence of visual feedback, even if incongruent, appears to have a more positive effect on SoE than its absence [52].
In addition to the feedback related to the action or the virtual/robotic actor, studies have also focused on the realism of the virtual scenario where that action took place. For example, the study by Pais-Vieira and colleagues [15] incorporated auditory and thermal stimuli that were coherent with a highly realistic virtual reality environment. Similarly, in the study by Legény and colleagues (et al., 2011), the visual elements necessary for the operation of an SSVEP-BMI were contextualized [51].
The results of the present review support the notion that visual sensorial feedback is beneficial for the SoE and that multisensory feedback combining visual and auditory or visual and haptic feedback tends to be beneficial for SoE, though not necessarily for BMI performance. As elegantly demonstrated in the single-SCI-patient study by Serino and colleagues [37], it is likely that the interplay of different sensorial modalities may be critical at specific points in time. Lastly, while the present review does not support the hypothesis that multisensory feedback necessarily improves SoE and BMI performance, the study of Pais-Vieira and colleagues [15], performed in a single SCI patient, suggests that including visual (immersive), auditory, vibratory, and thermal feedback is not detrimental to embodiment and BMI performance. However, any extrapolation of findings from these latter studies needs to be approached with caution due to the small number of patients and the fact that SCI patients already have an altered SoE.
After analyzing the studies associated with SoE and multisensory feedback during BMI control, it is proposed here that a detailed examination of the effects of each type, as well as the combination, of sensory modalities is crucial for our understanding of the neural basis of SoE (and SoO, SoA, and SoL) and how it relates to BMI performance. To achieve this, it is critical to systematically evaluate the effects of removing and adding each sensory modality, or combinations of modalities, in various types of tasks (SSVEPs, MI, P300) and with different types of neural signals (EEG, functional magnetic resonance (fMRI), intracortical recordings, etc.). We suggest that a series of experiments using within-subject designs could help control for individual differences in physiological parameters.
Additionally, this review highlights that only a limited number of studies have been conducted in SCI patients [15,18,37,56]. These studies, although conducted in a small number of patients (N = 1–8), allowed for the examination of SoE in pathological conditions and provided significant insights that could not otherwise be studied. Therefore, it is relevant for future studies to specifically address the role of multisensory feedback in SoE during BMI control in SCI and other patients. It is noteworthy that a large fraction of the BMIs analyzed here required users to engage in active motor imagery, with instructions to avoid making actual movements. However, in the context of rehabilitation, motor imagery BMIs are typically employed to promote or facilitate specific motor activities [15,37]. Consequently, users are instructed to attempt a set of pre-defined movements. This difference in goals should be considered in future studies examining the role of multisensory feedback in embodiment and BMI performance.
BMIs based on neural activity recorded invasively or non-invasively will inevitably result in significantly different decoding and experimental setup details, which may influence SoE. The present review included only one study with intracortical neural recordings [37], revealing that the dynamics between sensory and motor cortices during BMI control are crucial for the SoA, especially if visual feedback is incongruent. This study underscores the importance of recognizing that BMIs based in EEG recordings, while highly practical and reproducible, lack the ability to extract neural information (i.e., single- or multi-unit activity) with high spatial resolution.
Lastly, the concepts of SoE, SoO, SoL, and SoA can vary between authors leading to different questionnaires [31,32,33,34,35]. Therefore, the present review must be cautiously considered since the terms used by each author may present some degree of variation.

5. Conclusions

The number of BMI studies has significantly increased in the last two decades, but the incorporation of SoE measurements in experimental designs remains relatively scarce. The individual studies analyzed here suggest that greater realism, such as more immersive scenarios, greater human similarities of the virtual/robotic avatar, and greater coherence of the feedback all contribute to higher levels of SoE and enhance the embodiment experience. Despite these individual results, the larger group of studies analyzed here does not support the notion that an increased number of sensorial modalities will lead to increased SoE and improved BMI performance. It should be noted, however, that no study has systematically explored the influence of the different sensorial modalities in SoE and BMI performance. Therefore, we propose that it is necessary to perform experimental studies that separately test the cumulative and isolated contributions of multimodal feedback in inducing SoE.

Author Contributions

Conceptualization, D.J.T., M.P.-V. and C.P.-V.; methodology, D.J.T. and C.P.-V.; validation, D.J.T., M.P.-V. and C.P.-V.; formal analysis, D.J.T. and C.P.-V.; writing—original draft preparation, D.J.T., M.P.-V. and C.P.-V.; writing—review and editing, D.J.T., M.P.-V. and C.P.-V.; visualization, D.J.T., M.P.-V. and C.P.-V.; supervision, M.P.-V. and C.P.-V.; project administration, D.J.T., M.P.-V. and C.P.-V.; funding acquisition, M.P.-V. and C.P.-V. All authors have read and agreed to the published version of the manuscript.

Funding

Project CIIS I&D—UIDB/04279/2020—Base; Project iBiMED—UIDB/04501/2020—Base.

Data Availability Statement

Not applicable.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Blanke, O.; Metzinger, T. Full-Body Illusions and Minimal Phenomenal Selfhood. Trends Cogn. Sci. 2009, 13, 7–13. [Google Scholar] [CrossRef] [PubMed]
  2. De Vignemont, F. A Self for the Body. Metaphilosophy 2011, 42, 230–247. [Google Scholar] [CrossRef]
  3. Kilteni, K.; Groten, R.; Slater, M. The Sense of Embodiment in Virtual Reality. Presence 2012, 21, 373–387. [Google Scholar] [CrossRef]
  4. Lee, K.M. Presence, Explicated. Commun. Theory 2004, 14, 27–50. [Google Scholar] [CrossRef]
  5. Longo, M.R.; Schüür, F.; Kammers, M.P.M.; Tsakiris, M.; Haggard, P. What Is Embodiment? A Psychometric Approach. Cognition 2008, 107, 978–998. [Google Scholar] [CrossRef] [PubMed]
  6. Kim, S.Y.; Prestopnik, N.; Biocca, F.A. Body in the Interactive Game: How Interface Embodiment Affects Physical Activity and Health Behavior Change. Comput. Hum. Behav. 2014, 36, 376–384. [Google Scholar] [CrossRef]
  7. Thorpe, G.; Arthur, A.; McArthur, M. Adjusting to Bodily Change Following Stoma Formation: A Phenomenological Study. Disabil. Rehabil. 2016, 38, 1791–1802. [Google Scholar] [CrossRef]
  8. Fuentes, C.T.; Pazzaglia, M.; Longo, M.R.; Scivoletto, G.; Haggard, P. Body Image Distortions Following Spinal Cord Injury. J. Neurol. Neurosurg. Psychiatry 2013, 84, 201–207. [Google Scholar] [CrossRef]
  9. Lewis, J.S.; Kersten, P.; McCabe, C.S.; McPherson, K.M.; Blake, D.R. Body Perception Disturbance: A Contribution to Pain in Complex Regional Pain Syndrome (CRPS). Pain 2007, 133, 111–119. [Google Scholar] [CrossRef]
  10. Lotze, M.; Moseley, G.L. Role of Distorted Body Image in Pain. Curr. Rheumatol. Rep. 2007, 9, 488–496. [Google Scholar] [CrossRef]
  11. Pleger, B.; Ragert, P.; Schwenkreis, P.; Förster, A.F.; Wilimzig, C.; Dinse, H.; Nicolas, V.; Maier, C.; Tegenthoff, M. Patterns of Cortical Reorganization Parallel Impaired Tactile Discrimination and Pain Intensity in Complex Regional Pain Syndrome. Neuroimage 2006, 32, 503–510. [Google Scholar] [CrossRef] [PubMed]
  12. Alimardani, M.; Nishio, S.; Ishiguro, H. Humanlike Robot Hands Controlled by Brain Activity Arouse Illusion of Ownership in Operators. Sci. Rep. 2013, 3, 2396. [Google Scholar] [CrossRef] [PubMed]
  13. Juliano, J.M.; Spicer, R.P.; Vourvopoulos, A.; Lefebvre, S.; Jann, K.; Ard, T.; Santarnecchi, E.; Krum, D.M.; Liew, S.L. Embodiment Is Related to Better Performance on a Brain–Computer Interface in Immersive Virtual Reality: A Pilot Study. Sensors 2020, 20, 1204. [Google Scholar] [CrossRef]
  14. Nierula, B.; Spanlang, B.; Martini, M.; Borrell, M.; Nikulin, V.V.; Sanchez-Vives, M.V.; Taylor, J.; Farina, D. Agency and Responsibility over Virtual Movements Controlled through Different Paradigms of Brain−computer Interface. J. Physiol. 2021, 599, 2419–2434. [Google Scholar] [CrossRef] [PubMed]
  15. Pais-Vieira, C.; Gaspar, P.; Matos, D.; Alves, L.P.; da Cruz, B.M.; Azevedo, M.J.; Gago, M.; Poleri, T.; Perrotta, A.; Pais-Vieira, M. Embodiment Comfort Levels During Motor Imagery Training Combined with Immersive Virtual Reality in a Spinal Cord Injury Patient. Front. Hum. Neurosci. 2022, 16, 909112. [Google Scholar] [CrossRef]
  16. Perez-Marcos, D.; Slater, M.; Sanchez-Vives, M.V. Inducing a Virtual Hand Ownership Illusion through a Brain-Computer Interface. Neuroreport 2009, 20, 589–594. [Google Scholar] [CrossRef]
  17. Škola, F.; Liarokapis, F. Embodied VR Environment Facilitates Motor Imagery Brain–Computer Interface Training. Comput. Graph. Pergamon 2018, 75, 59–71. [Google Scholar] [CrossRef]
  18. Tidoni, E.; Gergondet, P.; Fusco, G.; Kheddar, A.; Aglioti, S.M. The Role of Audio-Visual Feedback in a Thought-Based Control of a Humanoid Robot: A BCI Study in Healthy and Spinal Cord Injured People. IEEE Trans. Neural Syst. Rehabil. Eng. 2017, 25, 772–781. [Google Scholar] [CrossRef]
  19. Evans, N.; Gale, S.; Schurger, A.; Blanke, O. Visual Feedback Dominates the Sense of Agency for Brain-Machine Actions. PLoS ONE 2015, 10, e0130019. [Google Scholar] [CrossRef]
  20. Ziadeh, H.; Gulyas, D.; Nielsen, L.D.; Lehmann, S.; Nielsen, T.B.; Kjeldsen, T.K.K.; Hougaard, B.I.; Jochumsen, M.; Knoche, H. “Mine Works Better”: Examining the Influence of Embodiment in Virtual Reality on the Sense of Agency During a Binary Motor Imagery Task With a Brain-Computer Interface. Front. Psychol. 2021, 12, 806424. [Google Scholar] [CrossRef]
  21. Gonzalez-Franco, M.; Peck, T.C. Avatar Embodiment. Towards a Standardized Questionnaire. Front. Robot. AI 2018, 5, 74. [Google Scholar] [CrossRef]
  22. Schwind, V.; Knierim, P.; Haas, N.; Henze, N. Using Presence Questionnaires in Virtual Reality. In Proceedings of the Conference on Human Factors in Computing Systems—Proceedings, Association for Computing Machinery, Glasgow, Scotland, 4–9 May 2019. [Google Scholar] [CrossRef]
  23. Franco, M.G. Neurophysiological Signatures of the Body Representation in the Brain Using Immersive Virtual Reality. 2014. Available online: http://hdl.handle.net/10803/359383 (accessed on 19 February 2023).
  24. Alchalabi, B.; Faubert, J.; Labbe, D.R. EEG Can Be Used to Measure Embodiment When Controlling a Walking Self-Avatar. In Proceedings of the 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), Osaka, Japan, 23–27 March 2019. [Google Scholar] [CrossRef]
  25. Armel, K.C.; Ramachandran, V.S. Projecting Sensations to External Objects: Evidence from Skin Conductance Response. Proc. R. Soc. B Biol. Sci. 2003, 270, 1499–1506. [Google Scholar] [CrossRef]
  26. Ehrsson, H.H.; Rosén, B.; Stockselius, A.; Ragnö, C.; Köhler, P.; Lundborg, G. Upper Limb Amputees Can Be Induced to Experience a Rubber Hand as Their Own. Brain 2008, 131, 3443–3452. [Google Scholar] [CrossRef]
  27. Tsuji, T.; Yamakawa, H.; Yamashita, A.; Takakusaki, K.; Maeda, T.; Kato, M.; Oka, H.; Asama, H. Analysis of Electromyography and Skin Conductance Response During Rubber Hand Illusion. In Proceedings of the 2013 IEEE Workshop on Advanced Robotics and its Social Impacts, Tokyo, Japan, 7–9 November 2013; pp. 88–93. [Google Scholar]
  28. Kammers, M.P.M.; Rose, K.; Haggard, P. Feeling Numb: Temperature, but Not Thermal Pain, Modulates Feeling of Body Ownership. Neuropsychologia 2011, 49, 1316–1321. [Google Scholar] [CrossRef] [PubMed]
  29. Llobera, J.; Sanchez-Vives, M.V.; Slater, M. The Relationship between Virtual Body Ownership and Temperature Sensitivity. J. R. Soc. Interface 2013, 10, 1–11. [Google Scholar] [CrossRef] [PubMed]
  30. Braun, N.; Debener, S.; Spychala, N.; Bongartz, E.; Sörös, P.; Müller, H.H.; Philipsen, A. The senses of agency and ownership: A review. Front. Psychol. 2018, 9, 535. [Google Scholar] [CrossRef]
  31. Witmer, B.G.; Singer, M.J. Measuring presence in virtual environments: A presence questionnaire. Presence 1998, 7, 225–240. [Google Scholar] [CrossRef]
  32. Wolpaw, J.R.; Ramoser, H.; McFarland, D.J.; Pfurtscheller, G. EEG-based communication: Improved accuracy by response verification. IEEE Trans. Rehabil. Eng. 1998, 6, 326–333. [Google Scholar] [CrossRef] [PubMed]
  33. Sanchez-Vives, M.V.; Spanlang, B.; Frisoli, A.; Bergamasco, M.; Slater, M. Virtual hand illusion induced by visuomotor correlations. PLoS ONE 2010, 5, e10381. [Google Scholar] [CrossRef]
  34. Friedman, D.; Pizarro, R.; Or-Berkers, K.; Neyret, S.; Pan, X.; Slater, M. A method for generating an illusion of backwards time travel using immersive virtual reality—An exploratory study. Front. Psychol. 2014, 5, 1–15. [Google Scholar] [CrossRef]
  35. Peck, T.C.; Gonzalez-Franco, M. Avatar embodiment. A standardized questionnaire. Front. Virtual Real. 2021, 1, 575943. [Google Scholar] [CrossRef]
  36. Caspar, E.A.; de Beir, A.; Lauwers, G.; Cleeremans, A.; Vanderborght, B. How Using Brain-Machine Interfaces Influences the Human Sense of Agency. PLoS ONE 2021, 16, e0245191. [Google Scholar] [CrossRef] [PubMed]
  37. Serino, A.; Bockbrader, M.; Bertoni, T.; Colachis Iv, S.; Solcà, M.; Dunlap, C.; Eipel, K.; Ganzer, P.; Annetta, N.; Sharma, G. Sense of Agency for Intracortical Brain-Machine Interfaces. Nat. Hum. Behav. 2022, 6, 565–578. [Google Scholar] [CrossRef]
  38. Pais-Vieira, C.; Gaspar, P.; Matos, D.; Gago, M.; Azevedo, M.J.; Poleri, T.; Perrotta, A.; Paisvieira, M. Multimodal Visual, Auditory, Thermal, and Tactile Feedback During Brain-Machine Interface Use by a Spinal Cord Injury Patient. In Proceedings of the Human Interaction & Emerging Technologies (IHIET-AI 2022): Artificial Intelligence & Future Applications, Nice, France, 21–23 April 2022. [Google Scholar] [CrossRef]
  39. Mudgal, S.K.; Sharma, S.K.; Chaturvedi, J.; Sharma, A. Brain Computer Interface Advancement in Neurosciences: Applications and Issues. Interdiscip. Neurosurg. 2020, 20, 100694. [Google Scholar] [CrossRef]
  40. Lebedev, M.A.; Nicolelis, M.A.L. Brain-Machine Interfaces: From Basic Science to Neuroprostheses and Neurorehabilitation. Physiol. Rev. 2017, 97, 767–837. [Google Scholar] [CrossRef] [PubMed]
  41. Wolpaw, J.R.; Millán, J.d.R.; Ramsey, N.F. Brain-Computer Interfaces: Definitions and Principles. Handb. Clin. Neurol. 2020, 168, 15–23. [Google Scholar] [CrossRef] [PubMed]
  42. Botvinick, M.; Cohen, J. Rubber Hands ‘Feel’ Touch That Eyes See. Nature 1998, 391, 756. [Google Scholar] [CrossRef] [PubMed]
  43. Pazzaglia, M.; Galli, G.; Lewis, J.W.; Scivoletto, G.; Giannini, A.M.; Molinari, M. Embodying Functionally Relevant Action Sounds in Patients with Spinal Cord Injury. Sci. Rep. 2018, 8, 15641. [Google Scholar] [CrossRef]
  44. Slater, M.; Khanna, P.; Mortensen, J.; Yu, I. Visual Realism Enhances Realistic Response in an Immersive Virtual Environment. IEEE Comput. Graph. Appl. 2009, 29, 76–84. [Google Scholar] [CrossRef]
  45. Penaloza, C.I.; Alimardani, M.; Nishio, S. Android Feedback-Based Training Modulates Sensorimotor Rhythms during Motor Imagery. IEEE Trans. Neural Syst. Rehabil. Eng. 2018, 26, 666–674. [Google Scholar] [CrossRef]
  46. Tidoni, E.; Gergondet, P.; Kheddar, A.; Aglioti, S.M. Audio-Visual Feedback Improves the BCI Performance in the Navigational Control of a Humanoid Robot. Front. Neurorobot. 2014, 8, 20. [Google Scholar] [CrossRef] [PubMed]
  47. Zhang, B.; Zhou, Z.; Jiang, J. A 36-Class Bimodal Erp Brain-Computer Interface Using Location-Congruent Auditory-Tactile Stimuli. Brain Sci. 2020, 10, 524. [Google Scholar] [CrossRef] [PubMed]
  48. Page, M.J.; McKenzie, J.E.; Bossuyt, P.M.; Boutron, I.; Hoffmann, T.C.; Mulrow, C.D.; Shamseer, L.; Tetzlaff, J.M.; Akl, E.A.; Brennan, S.E.; et al. The PRISMA 2020 Statement: An Updated Guideline for Reporting Systematic Reviews. BMJ 2021, 372, n71. [Google Scholar] [CrossRef]
  49. Connolly, T.M.; Boyle, E.A.; MacArthur, E.; Hainey, T.; Boyle, J.M. A systematic literature review of empirical evidence on computer games and serious games. Comput. Educ. 2012, 59, 661–686. [Google Scholar] [CrossRef]
  50. Vieira, C.; Pais-Vieira, C.; Novais, J.; Perrotta, A. Serious Game Design and Clinical Improvement in Physical Rehabilitation: Systematic Review. JMIR Serious Games 2021, 9, e20066. [Google Scholar] [CrossRef]
  51. Legény, J.; Abad, R.V.; Lévuyer, A. Navigating in Virtual Worlds Using a Self-Paced SSVEP-Based Brain-Computer Interface with Integrated Stimulation and Real-Time Feedback. Presence 2011, 20, 529–544. [Google Scholar] [CrossRef]
  52. Alimardani, M.; Nishio, S.; Ishiguro, H. Effect of Biased Feedback on Motor Imagery Learning in BCI-Teleoperation System. Front. Syst. Neurosci. 2014, 8, 52. [Google Scholar] [CrossRef]
  53. Alimardani, M.; Nishio, S.; Ishiguro, H. The Importance of Visual Feedback Design in BCIs; from Embodiment to Motor Imagery Learning. PLoS ONE 2016, 11, e0161945. [Google Scholar] [CrossRef]
  54. Alimardani, M.; Nishio, S.; Ishiguro, H. Removal of Proprioception by BCI Raises a Stronger Body Ownership Illusion in Control of a Humanlike Robot. Sci. Rep. 2016, 6, 33514. [Google Scholar] [CrossRef]
  55. Vourvopoulos, A.; Bermúdez i Badia, S. Motor Priming in Virtual Reality Can Augment Motor-Imagery Training Efficacy in Restorative Brain-Computer Interaction: A within-Subject Analysis. J. Neuroeng. Rehabil. 2016, 13, 69. [Google Scholar] [CrossRef]
  56. Tidoni, E.; Abu-Alqumsan, M.; Leonardis, D.; Kapeller, C.; Fusco, G.; Guger, C.; Hintermuller, C.; Peer, A.; Frisoli, A.; Tecchia, F.; et al. Local and Remote Cooperation with Virtual and Robotic Agents: A P300 BCI Study in Healthy and People Living with Spinal Cord Injury. IEEE Trans. Neural Syst. Rehabil. Eng. 2017, 25, 1622–1632. [Google Scholar] [CrossRef]
  57. Škola, F.; Tinková, S.; Liarokapis, F. Progressive Training for Motor Imagery Brain-Computer Interfaces Using Gamification and Virtual Reality Embodiment. Front. Hum. Neurosci. 2019, 13, 329. [Google Scholar] [CrossRef]
  58. Choi, J.W.; Huh, S.; Jo, S. Improving Performance in Motor Imagery BCI-Based Control Applications via Virtually Embodied Feedback. Comput. Biol. Med. 2020, 127, 104079. [Google Scholar] [CrossRef] [PubMed]
  59. Slater, M.; Steed, A.; McCarthy, J.; Maringelli, F. The influence of body movement on subjective presence in virtual environments. Hum. Factors Ergon. Soc. 1998, 40, 469–477. [Google Scholar] [CrossRef] [PubMed]
  60. Roberts, R.; Callow, N.; Hardy, L.; Markland, D.; Bringer, J. Movement imagery ability: Development and assessment of a revised version of the vividness of movement imagery questionnaire. J. Sport Exerc. Psychol. 2008, 30, 200–221. [Google Scholar] [CrossRef] [PubMed]
  61. De Vignemont, F. Embodiment, Ownership and Disownership. Conscious Cogn. 2010, 20, 82–93. [Google Scholar] [CrossRef] [PubMed]
  62. Shen, X.; Zhang, X.; Huang, Y.; Chen, S.; Yu, Z.; Wang, Y. Intermediate Sensory Feedback Assisted Multi-Step Neural Decoding for Reinforcement Learning Based Brain-Machine Interfaces. IEEE Trans. Neural Syst. Rehabil. Eng. 2022, 30, 2834–2844. [Google Scholar] [CrossRef] [PubMed]
  63. Suminski, A.J.; Tkach, D.C.; Fagg, A.H.; Hatsopoulos, N.G. Incorporating Feedback from Multiple Sensory Modalities Enhances Brain-Machine Interface Control. J. Neurosci. 2010, 30, 16777–16787. [Google Scholar] [CrossRef]
  64. Yu, I.; Mortensen, J.; Khanna, P.; Slater, M. Visual Realism Enhances Realistic Response in an Immersive Virtual Environment—Part 2. IEEE Comput. Graph. Appl. 2012, 32, 36–45. [Google Scholar] [CrossRef]
  65. Renard, Y.; Lotte, F.; Gibert, G.; Congedo, M.; Maby, E.; Delannoy, V.; Bertrand, O.; Lécuyer, A. OpenViBE: An Open-Source Software Platform to Design, Test, and Use Brain-Computer Interfaces in Real and Virtual Environments. Presence Teleoperators Virtual Environ. 2010, 19, 35–53. [Google Scholar] [CrossRef]
  66. McColl, M.A.; Charlifue, S.; Glass, C.; Lawson, N.; Savic, G. Aging, Gender, and Spinal Cord Injury. Arch. Phys. Med. Rehabil. 2004, 85, 363–367. [Google Scholar] [CrossRef]
  67. Raguindin, P.F.; Muka, T.; Glisic, M. Sex and Gender Gap in Spinal Cord Injury Research: Focus on Cardiometabolic Diseases. A Mini Review. Maturitas 2021, 147, 14–18. [Google Scholar] [CrossRef] [PubMed]
  68. Bashford, L.; Mehring, C. Ownership and Agency of an Independent Supernumerary Hand Induced by an Imitation Brain-Computer Interface. PLoS ONE 2016, 11, e0156591. [Google Scholar] [CrossRef] [PubMed]
  69. Braun, N.; Emkes, R.; Thorne, J.D.; Debener, S. Embodied Neurofeedback with an Anthropomorphic Robotic Hand. Sci. Rep. 2016, 6, 37696. [Google Scholar] [CrossRef] [PubMed]
  70. Lynn, M.T.; Berger, C.C.; Riddle, T.A.; Morsella, E. Mind Control? Creating Illusory Intentions through a Phony Brain-Computer Interface. Conscious Cogn. 2010, 19, 1007–1012. [Google Scholar] [CrossRef] [PubMed]
Figure 1. Flowchart of the selection process.
Figure 1. Flowchart of the selection process.
Applsci 13 13011 g001
Figure 2. Number of studies per SoE dimension assessed.
Figure 2. Number of studies per SoE dimension assessed.
Applsci 13 13011 g002
Figure 3. Number of studies per type of stimulation.
Figure 3. Number of studies per type of stimulation.
Applsci 13 13011 g003
Table 1. Feedback modalities and SoE measures.
Table 1. Feedback modalities and SoE measures.
Author(s)Type of BMIFeedback ModalitySense of Embodiment (SoE) MeasuresOther Measures
Sense of Self-LocationSense of OwnershipSense of Agency
Perez-Marcos et al., 2009 [16]EEG-based via MIVisualnoyesyesProprioceptive drift
EMG deltoid muscle
Legény et al., 2011 [51]EEG-based via SSVEPsVisualyesnoyes--
Alimardani et al., 2013 [12]EEG-based via MIVisual (immersive)noyesnoSkin conductance responses
Alimardani et al., 2014 [52]EEG-based via MIVisual (immersive)noyesnoSkin conductance responses
Evans et al., 2015 [19]EEG-based via MIVisualnonoyes--
Alimardani et al., 2016 [53]EEG-based via MIVisual (immersive)noyesyes--
Alimardani et al., 2016 [54]EEG-based via MIVisual (immersive)noyesyesSkin conductance responses
Vourvopoulos & Bermúdez i Badia, 2016 [55]EEG-based via MIVisual (immersive) + auditoryyesnono--
Tidoni et al., 2017 [56]EEG-based via P300Visual (immersive) + haptic (vibratory)yesyesyes--
Tidoni et al., 2017 [18]EEG-based via SSVEPsVisual (immersive) + auditoryyesyesyes--
Škola & Liarokapis, 2018 [17]EEG-based via MIVisual (immersive)noyesyes--
Penaloza et al., 2018 [45]EEG-based via MIVisualnoyesno--
Škola et al., 2019 [57]EEG-based via MIVisual (immersive) + haptic (vibratory)noyesyes--
Juliano et al., 2020 [13]EEG-based via MIVisual/visual (immersive)yesyesyes--
Choi et al., 2020 [58]EEG-based via MIVisual (immersive)yesyesno--
Nierula et al., 2021 [14]EEG-based via MI and SSVEPsVisual (immersive) + auditorynoyesyes--
Caspar et al., 2021 [36]EEG-based via MIVisual + auditoryyesyesyes--
Ziadeh et al., 2021 [20]EEG-based via MIVisual (immersive) + auditorynoyesyesSubjective proprioception
Serino et al., 2022 [37]IntracorticalVisual + haptic (electrostimulation)nonoyes--
Pais-Vieira et al., 2022 [15]EEG-based via MIVisual (immersive) + auditory + haptic (vibratory + thermal)yesyesyes--
Table 2. Summary of studies included.
Table 2. Summary of studies included.
Author(s)Aims/Objectives of StudySampleMethodsMain Results
Perez-Marcos et al., 2009 [16]To explore whether the control of a virtual arm through a non-invasive BCI can induce the illusion of ownership, proprioceptive displacement, and agency towards that arm, in the absence of tactile sensory stimulation.N = 16
(healthy participants)
Age: 26.1 ± 9.4 (Mean ± SD)
Two groups with different visual feedback conditions:
Group 1: virtual hand moves congruently with the motor imagery attempt.
Group 2: virtual hand moves randomly and independently of the participant’s performance.
Sense of ownership and EMG deltoid activity higher in group 1.
Sense of agency with high levels but not different between groups.
Proprioceptive drift not significant in either of the two group’s conditions.
Legény et al., 2011 [51]To study the usability of SSVEP-based BMIs in virtual environment navigation.N = 17
(healthy participants)
Age: 25.5 ± 4.3 (Mean ± SD)
Four experimental conditions:
Condition 1: “arrow” visual trigger without real-time visual feedback of user’s brain activity.
Condition 2: “arrow” visual trigger with real-time visual feedback of user’s brain activity.
Condition 3: “butterfly” visual trigger without real-time visual feedback of user’s brain activity.
Condition 4: “butterfly” visual trigger with real-time visual feedback of user’s brain activity.
Senses of self-location and agency significantly or near significantly higher in condition 4 than the other conditions.
Alimardani et al., 2013 [12]To explore if sense of agency and body ownership illusions can be induced for a pair of BMI-operated human-like robotic hands without proprioceptive updates of real motions from operators’ sensations.N = 40
(healthy participants)
Age: 21.13 ± 1.92 (Mean ± SD)
Two experimental conditions:
Still condition: The robot’s hands did not move at all throughout the whole session, although a subject performed motor imagery according to cues.
Match condition: The robot’s hands moved when the subject performed the MI.
At the end of each test session, for both conditions, a syringe was injected into the robot’s hand.
Sense of ownership higher in “match condition” compared with “still condition”.
Higher skin conductance response in “match condition” compared with “still condition” during the syringe injection.
Alimardani et al., 2014 [52]To investigate the inducement of body ownership illusion for a pair of BMI-operated human-like robotic hands under different presentations of feedback.N = 40
(healthy participants)
Age: 21.13 ± 1.92 (Mean ± SD)
Two experimental conditions:
Still condition: The robot’s hands did not move at all throughout the whole session, although a subject performed motor imagery according to cues.
Match conditions: The robot’s hands moved only in those trials that the classification result was correct and in accordance with cue.
Raw condition: The robot’s hands moved according to the classification results in all trials. In case of wrong result that was not in accordance with cue, the robot’s opposite hand moved.
At the end of each test session, for both conditions, a syringe was injected into the robot’s hand.
Sense of ownership higher in “match condition” than in “still condition” and “raw conditions”.
Sense of ownership higher in “raw condition” than in the “still condition” while operating the robot hands, but no significative differences between these two conditions when the robot’s hand was injected.
Higher skin conductance response in “match condition” compared with the other conditions but just statistically significant when compared with “still condition”.
Positive correlation between sense of ownership while operating the robot hands and the BMI’s performance.
Evans et al., 2015 [19]To explore the sense of agency for BMI-mediated actions.Study 1:
N = 8
(healthy participants)
Age: 26.5 ± 3.5 (Mean ± SD)
Study 2:
N = 7
(healthy participants)
Age: 26.0 ± 2.3 (Mean ± SD)
Study 1: Control the right/left displacement of a virtual bar by imagining clasping the right/left hand under six different visual feedback delay conditions: 0 ms, 750 ms, 1500 ms, 2250 ms, 3000 ms, or 3750 ms. This feedback was also presented as congruent (displacement direction according to the intention) or incongruent (displacement direction opposite to the intention).
Study 2: Control the right/left displacement of a virtual bar by imagining clasping the right/left hand under six different visual feedback delay conditions: 0 ms, 250 ms, 500 ms, 750 ms, 1000 ms, or 3750 ms.
Study 1: Sense of agency higher for congruent than incongruent feedback.
For congruent feedback, sense of agency is higher when the delay is lower.
For incongruent feedback, sense of agency is low and it is not dependent on the delay conditions.
BMI performance cannot be explained using the level of sense of agency.
Study 2: Sense of agency not significatively different between the conditions under 1000 ms.
Lower sense of agency for the delay condition of 3750 ms compared with the others.
No significative differences in BMI performances between conditions.
Alimardani et al., 2016 [53]To assess the impact of embodiment on motor imagery learning during BMI control.N = 38
(healthy participants)
Age: 23.8 ± 8.2 (Mean ± SD)
Control of BMI-operated robotic arms in two followed sessions: Initial session with a positive visual feedback bias, and a subsequent session with feedback associated with the real performance.
Geminoid group: Participants initially operated Geminoid’s hands (human-like) and in a subsequent session proceeded to operation of the Arm Robot (robotic tweezers).
Arm Robot group: Participants BMI-operated only the robotic tweezers in both sessions.
Sense of agency was greater in the positive visual feedback bias session compared to the real performance visual feedback for both groups.
Sense of ownership, for Geminoid group, was significantly higher in the session with the human-like hands compared with the one with robotic tweezers. In the Arm Robot group, there were no significant differences between sessions.
Alimardani et al., 2016 [54]To investigate the interference of proprioceptive afferences in the body ownership illusion when mismatching with visual feedback.N = 30
(healthy participants)
Age: 21.51 ± 1.73 (Mean ± SD)
To operate human-like Geminoid robot hands, participants performed two sessions in a random order:
MoCap session: Subjects grasped their own right or left hand to control the robot’s corresponding hand.
BMI session: Subjects performed a right or left motor imagery task and controlled robot’s hands without actual motions.
In both sessions, the visual feedback had a certain amount of delay.
At the end of each session, a syringe was injected into the robot’s hand.
Higher senses of agency and body ownership in the BMI session.
Skin conductance responses revealed that the operators’ reactions to a painful stimulus (injection) were significantly stronger in the BMI sessions.
Vourvopoulos & Bermúdez i Badia, 2016 [55]To explore the role of motor priming in virtual reality in BMI-operated virtual arms.N = 9
(healthy participants)
Age: 27.0 ± 2.0 (Mean ± SD)
To perform MI of circular movements of arms for a garage door opening under three BCI conditions in a randomized order:
VR condition: performing MI, receiving visual and auditory feedback trough a virtual environment.
VR + MP condition: using real arm movements while performing MI, receiving visual and auditory feedback trough a virtual environment.
Control condition: performing MI, receiving a visual standard feedback through arrows and bar.
VR + MP and VR conditions share high scores of sense of self-location.
For BMI performances, no significative differences between VR + MP and VR with control condition.
No significant correlation between BMI performance and sense of self-location.
Tidoni et al., 2017 [56]To explore the role of proprioceptive feedback in healthy people and those living with SCI during a BCI-based social interaction task.Study 1:
N = 8
(healthy participants)
Age: 27.0 ± 3.5 (Mean ± SD)
Study 2:
N = 10 + 8
(healthy participants)
Age: 29.33 ± 2.87 (Mean ± SD)
(SCI participants)
Age: 28.0 ± 5.19 (Mean ± SD)
Study 1: Participants immersed into a virtual environment in two experimental conditions:
MovI+: Vibration applied in right bicep’s brachial tendon (inducing proprioceptive stimulation with illusion of downward extension of the elbow).
MovI-: Vibration applied over the bone close to bicep’s brachial tendon (proprioceptive stimulation without illusion).
Study 2: Participants with vibration applied in right bicep’s brachial tendon in two conditions:
Virtual: controlling an avatar in a virtual environment.
Robot: controlling a robot with the itself perspective.
Study 1: No significative differences in BMI’s performances and SoE measures between conditions.
High levels of sense of self-location and sense of agency.
Study 2: Healthy participants: No significative differences in BMI’s performances and SoE measures between conditions. High levels of sense of agency.
SCI participants: SoE experience did not differ relative to healthy participants but had found a more variable performance in the control of the virtual avatar and the robotic surrogate.
Tidoni et al., 2017 [18]To explore the use auditory combined with visual feedback in virtual navigation to the subjective experience in terms of BMI usability and feelings of ownership over the controlled surrogate.N = 14 + 3
(healthy participants)
Age: 25.8 ± 6.0 (Mean ± SD)
(SCI participants)
Age: 27.0 ± 4.0 (Mean ± SD)
Participants control a humanoid robot walking and grasping bottles. Visual feedback was given from its own perspective combined with four auditory stimulus conditions:
Foot Sync: steps sound with the visual feedback.
Foot Async: steps sound asynchronous with the visual feedback.
Beep Sync: beep sound synchronous with the visual feedback.
Beep Async: beep sound as asynchronous with the visual feedback.
Healthy participants: High levels of sense of agency and low levels of sense of embodiment and sense of self-location. Higher accuracy in the grasping bottles phase with footstep sound condition relative to a beep sound condition. No differences were found between synchronous and asynchronous.
SCI participants: Reduced control of the robot when asynchronous auditory feedback was matched with the robot’s movements.
Škola & Liarokapis, 2018 [17]To explore the use of a more realistic feedback during the MI-BMI training process in comparation to the traditional neurofeedback mediated via a simple symbolic representation.N = 30
(healthy participants)
Experimental group: Training phase was placed into a virtual reality environment observed from a first-person view of a human-like avatar, and their rehearsal of MI actions was reflected by the corresponding movements performed by the avatar.
Control group: Training phase instructions were delivered using the standard protocol with arrows, and feedback was displayed as extending blue bar, continuously changing according to the classifier decision.
Sense of agency was slightly higher in the experimental group than for the controls.
Sense of ownership was higher for the controls, but with a very small difference.
In both groups, there was a similar number of participants with scores that could be considered as “embodied participants group”. This suggests that virtual reality experience during training did not affect ratings in the evaluation phase of the experiment. Similar tendency is present for the agency statements.
Penaloza et al., 2018 [45]To investigate an alternative BMI training protocol that uses a human-like android robot (Geminoid HI-2) to provide realistic feedback.N = 27
(healthy participants)
Age: 21.5 ± 1.69 (Mean ± SD)
Two groups control an android robot (Geminoid HI-2) in a grasping hand task trough two different protocols:
Classical Training Protocol (CTP): Calibration–Training–Evaluation.
Android Feedback Training Protocol (AFTP): Pretraining–Training–Calibration–Evaluation (pretraining consists of rehearsing the kinesthetics of hand movements and memorizing the physical sensation).
Sense of ownership was significantly higher in the group with AFTP than the one with CTP.
Strong correlation between AFTP group performance and sense of ownership.
Moderate correlation between CTP group performance and sense of ownership.
Škola et al., 2019 [57]To investigate the use of gamification in MI-BMI training.N = 19
(healthy participants)
Age: 26.0 ± 2.78 (Mean ± SD)
The gamified VR scene was set inside a cockpit of a spaceship containing a simplistic control panel. The objective was to trigger weapons aiming for the destruction of asteroids using MI of the left or right hand, depending on its source position. Feedback was provided using three modalities: (1) movements of the avatar, (2) vibrations, and (3) providing information about trial accuracy (score).Positive, moderately strong rating of sense of ownership and sense of agency.
Juliano et al., 2020 [13]To explore the role of embodiment on neurofeedback performance using HMD-VR versus a computer screen.N = 12
(healthy participants)
Age: 24.4 ± 2.7 (Mean ± SD)
Participants under three blocks of conditions:
Block 1: controlling the virtual arm with brain activity on the computer (screen);
Block 2: controlling the virtual arm with brain activity in a head-mounted display virtual reality (HMD-VR) system;
Block 3: controlling the virtual arm with actual arm movements in a head-mounted display. (IMU): control.
Higher levels of embodiment in the HMD-VR condition.
For the HMD-VR condition, a significant relationship between embodiment and neurofeedback performance was reported.
Choi et al., 2020 [58]To explore a novel control scheme in which virtually embodiable feedback is
provided during control to enhance performance.
N = 14
(healthy participants)
Training phase: During the MI period, the virtual hands executed the movement corresponding to the task.
Control conditions (EFCS and SCS): The device moved in a virtual route track based on the real-time EEG signals. Repeated left-hand grasping and right-hand grasping MIs were mapped to left rotation and right rotation of the device, respectively.
Feedback was given in two different conditions:
EFCS: virtual hands are shown and execute the movement that is classified;
SCS: does not show the virtual hands.
Participants expressed great levels of sense of ownership and sense of self-location. They were able to perform MI better during the EFCS than during the SCS with statistical significance.
Participants found the virtual hands to be helpful for performing MI during the training phase and during the EFCS.
Significant positive linear relationships between classification accuracy and sense of ownership and sense of self-location were shown for EFCS. No statistically significant relationships were found for SCS.
Nierula et al., 2021 [14]To investigate agency and responsibility by studying the control of movements of an embodied avatar, via BMI technology, in immersive virtual reality.N = 29
(healthy participants)
Age: 21.5 ± 2.6 (Mean ± SD)
Participants went through three conditions:
Observe: passive observation of the virtual arm performing the task;
MI-BMI: control of the movement through motor imagery;
SSVEP-BMI: control of the movement through steady-state visually evoked potentials.
Sense of agency was higher in MI-BMI than SSVEP-BMI.
Sense of agency was higher in SSVEP-BMI than in “Observe”.
Sense of ownership was higher in MI-BMI than SSVEP-BMI and “Observe”.
Sense of ownership was not statistically different between SSVEP-BMI and “Observe”.
BMI performance in MI-BMI was slightly higher than SSVEP-BMI.
Caspar et al., 2021 [36]To investigate whether using brain–machine interfaces influences the human sense of agency.Study 1:
N = 27
(healthy participants)
Age: 23.78 ± 2.68 (Mean ± SD)
Study 2:
N = 30
(healthy participants)
Age: 23.77 ± 2.76 (Mean ± SD)
Study 1: Participants had to press a keyboard button to produce a sound. They were then asked to estimate and report, in ms, the duration of the delay between their keypress and the resulting tone. This was proceeded using a real hand or controlling a robotic hand through BMI.
Study 2: Same protocol as study 1, but participants were BMI-trained for two consecutive days, at the same hour of the day.
Study 1: The interval estimates in the real hand condition and the robotic hand condition were not significantly different.
For the robotic hand, results indicated a higher score for sense of agency.
Study 2: BMI performance higher on day 2 than on day 1. Sense of ownership, sense of self-location, and sense of agency did not differ between the two days.
Ziadeh et al., 2021 [20]To investigate whether higher levels of ownership from a humanoid hand in VR can enhance the perceived agency users feel over hand’s movements during an online MI-BMI task.N = 22
(healthy participants)
Age: 24.0 (Mean)
Performing a virtual task (popping balloons).
Group 1: First block popped with the virtual hands and the second block popped with virtual flying blocks.
Group 2: First block popped with virtual flying blocks and the second popped with the virtual hands.
Similar BMI performances between virtual hands and blocks.
Virtual hand induced higher sense of ownership and proprioception levels than blocks.
Sense of ownership and performance significantly predicted sense of agency.
Proprioception correlated with performance in the virtual hand but not the block’s condition.
Serino et al., 2022 [37]To explore sense of agency for intracortical brain–machine interfaces.N = 1
(SCI participant)
Age: 24
Experiment 1: Visual (V) was used to provide visual feedback, consisting of a life-sized virtual arm on a monitor superimposed over the participant’s right arm, matching the location and dimensions of the participants real arm, which was occluded from view.
Experiment 2: NMES (S) was used to provide somatosensory feedback; the patients upper limb muscles were electrically stimulated so they could feel, but not see, the selected movement.
Experiment 3: Combined V and S to provide visual–somatosensory feedback. In half of the trials, sensory feedback was congruent with the cued action, while in the other half, it was incongruent.
Experiments 1 and 2: Congruent visual and congruent somatosensory feedback resulted in more frequent agency responses versus incongruent conditions. Confidence was modulated via somatosensory congruency (higher for somatosensory congruent than incongruent). The effect of visual feedback congruency on confidence ratings was not significant.
Experiment 3: Somatosensory congruency was more effective in driving the sense of agency and the associated confidence. Ratings were higher when both feedback signals were congruent as compared to both being incongruent. When visual feedback was not congruent but somatosensory was congruent, higher levels of sense of agency and confidence were present when compared to the condition with congruent visual feedback but incongruent somatosensory feedback.
Pais-Vieira et al., 2022 [15]To explore embodiment comfort levels during motor imagery training combined with immersive virtual reality in a spinal cord injury patient.N = 1
(SCI participant)
Age: 52
Walking and stopping with an avatar in a virtual environment.
Protocol with three phases:
(a) Habituation;
(b) EEG baseline and neural data acquisition for classifier training;
(c) Testing real-time decoding of neural activity without control of avatar.
High levels of senses of ownership, agency, and self-location.
The participant could generate higher levels of neural commands associated with “Walk” and “Stop”.
Subjective reports describe this experience as being positive.
In three sessions involving water scenarios, participant reported his legs feeling cold. Not exclusive of thermal feedback.
Table 3. Quality assessment.
Table 3. Quality assessment.
Quality Assessment: (Classify from1 to 3; 1 = Low; 2 = Medium, and 3 = High)
ReferencesPerez-Marcos et al., 2009 [16]Legény et al., 2011 [51]Alimardani et al., 2013 [12]Alimardani et al., 2014 [52]Evans et al., 2015 [19]Alimardani et al., 2016 [53]Alimardani et al., 2016 [54]Vourvopoulos & Bermúdez i Badia, 2016 [55]Tidoni et al., 2017 [56]Tidoni et al., 2017 [18]Škola & Liarokapis, 2018 [17]Penaloza et al., 2018 [45]Škola et al., 2019 [57]Juliano et al., 2020 [13]Choi et al., 2020 [58]Nierula et al., 2021 [14]Caspar et al., 2021 [36]Ziadeh et al., 2021 [20]Serino et al., 2022 [37]Pais-Vieira et al., 2022 [15]
1. How appropriate is the research design for addressing the question, or sub-questions, of this review (higher weighting for inclusion of a control group)?33333333333323333322
2. How appropriate are the methods and analysis?33333333333333333333
3. How generalizable are the findings of this study to the larger population with respect to the size and representativeness of the sample?1.33 ± 0.581.33 ± 0.582.67 ± 0.582.67 ± 0.581.33 ± 0.582.67 ± 0.582.67 ± 0.581.33 ± 0.581.67 ± 0.582 ± 0.002.67 ± 0.582.67 ± 0.582 ± 0.001.67 ± 0.581.67 ± 0.582.67 ± 0.582.67 ± 0.582.33 ± 1.151 ± 0.001 ± 0.00
4. How relevant is the particular focus of the study (including conceptual focus, context, sample, and measures) for addressing the question or sub-questions of this review?33333333333333333322
5. To what extent can the study findings be trusted in answering the study question(s)?33333333333333333333
Total score (5–15)13.3313.3314.6714.6713.3314.6714.6713.3313.671414.6714.671313.6713.6714.6714.6714.331111
Table 4. Embodiment and BMI evaluation.
Table 4. Embodiment and BMI evaluation.
StudyNeural SignalFeedbackN ModalitiesCompares Sensorial ModalitiesBMI ClassesSoE EvaluationBMI%
(Mean/Median)
Embodiment and BMI Conclusion
Perez-Marcos et al., 2009 [16]EEG-based via MIVisual1No2Botvinick and Cohen, 1998 [42]N/AMI Visual fb
SoO ↑, BMI% N/A
Legény et al., 2011 [51]EEG-based via SSVEPsVisual1No3 (L, R, Forward)Slater et al., 1998 [59]N/ASoO may improve ERD
SoL↑, SoA↑, BMI% ↓
Alimardani et al., 2013 [12]EEG-based via MIVisual (immersive)1No2Two Qs
Q1: “feel (…) your own hand received the injection?
Q2: Feel as if they were your own hands?
N/ABody ownership illusions can be induced without the correlation of multiple sensory modalities
SoO ↑, BMI% N/A
Alimardani et al., 2014 [52]EEG-based via MIVisual (immersive)1Yes2Two Qs
Q1: Feel (…) your own hand received the injection?
Q2: Feel as if they were your own hands?
FakeP = 60.78
Raw = 49.22
Match = 54.37
FakeN = 50.47
MI improved with + bias feedback, SoO
SoO ↑, BMI% =
Evans et al., 2015 [19]EEG-based via MIVisual0 vs. 1No2SoA “ I was controlling the cursor”. “Yes”, “No”Cong = 76→Incong~79; Visual = 76.7→None = 53.4MI congruent Visual fb
SoA ↑, BMI% =
Alimardani et al., 2016 [53]EEG-based via MIVisual (immersive)1Yes2(pre-) Botivnik and Cohen., 1998 [42];
(post-) Qs: Q2: (…) where your hands?
Q3: (…) operation (…) was easier?
Geminoid: 1.31→1.08 Robot: 1.48→0.68BMI’s potential in inducing stronger agency-driven illusions
SoO ↑, SoA ↑, BMI% ↑
Alimardani et al., 2016 [54]EEG-based via MIVisual (immersive)1No2Two Qs: (Q1) Could you operate the robot’s hands according to your intentions?
(Q2) Did you feel as if the robot’s hands were your own hands? 7-point
SoA:
Hum: S3 = 4.58→S4 = 3.05
(p < 0.001)
Rob: S3 = 4.47→S4 = 3.21 (p < 0.001)
SoO:
Hum: S3 = 4.36→S4 = 2.53
(p < 0.001)
Rob: S3 = 4.0→S4 = 3.53
(p = 0.18)
Improved BMI learning with visual humanoid
SoO ↑, SoA ↑, BMI% ↑
Vourvopoulos & Bermúdez i Badia, 2016 [55]EEG-based via MIVisual (immersive) + auditory2Yes2Witmer and Singer 1998 [30]; Roberts et al., 2008 [60]VRMP  = 51.29
VR  = 53.61
Control  = 50.1
VR and MP can enhance the activation of brain patterns present during overt motor execution
SoL N/A, BMI% =
Tidoni et al., 2017 [56] *EEG based via P300Visual (immersive) + haptic (vibratory)2Yes9Friedman et al., 2014; [34] Sanchez-Vives et al., 2010 [33]BMI = 86.06
VR = 83.33
BMI = 95.00
VR Robot = 93.75
Proprioceptive feedback did not contribute to alter performance measures and body ownership sensations
SoO =, SoA =, BMI =
Tidoni et al., 2017 [18] *EEG-based via SSVEPsVisual (immersive) + auditory2Yes6 (5 + 0)Wolpaw et al., 1998 [32]; Sanchez-Vives et al., 2010 [33](Dist. to bottle)
Foot = 1.481
Beep = 1.975
Paired visual auditory (foot) improved BMI performance
SoA ↑, SoO =, SoL =, BMI ↑
Škola & Liarokapis, 2018 [17]EEG-based via MIVisual (immersive)1No212 Qs: SoO, SoA, otherVR = 58.3
MI-BMI = 52.9
SoA was higher, performance was higher in VR
SoA ↑, SoO = N/A, BMI% ↑
Penaloza et al., 2018 [45]EEG-based via MIVisual1No2SoO: Did you feel that robot’s hands were your own hands?Android = 61.38
Classical = 52.38
MI robotic hand Visual fb
SoO ↑, BMI% ↑
Škola et al., 2019 [57]EEG-based via MIVisual (immersive) + haptic (vibratory)2No2Botvinick and Cohen, 1998 [42]; Longo et al., 2008 [5]MI = 75.84SoO correlated with EEG modulation
SoA N/A, SoO N/A, BMI% N/A
Juliano et al., 2020 [13]EEG-based via MIVisual/Visual (immersive)1Yes2Witmer and Singer, 1998 [42]Screen = 80.95
VR = 83.33
VR fb improved embodiment but not BMI perf.
SoE ↑, BMI% =
Choi et al., 2020 [58]EEG-based via MIVisual (immersive)1No310 Qs: SoO, SoLEmbodied = 53.27
Standard = 39.99
Embodiable feedback generates SoO and SoL and improves BMI performance
BMI% ↑(L/R)
Nierula et al., 2021 [14]EEG-based via MI and SSVEPsVisual (immersive) + auditory2Yes27 Qs: my body, agency, responsibilitySSVEP = 90.9
MI = 87.4
(p = 0.052)
MI
SoA ↑, SoO ↑
SSVEPs
SoA ↑, SoO ↓, BMI% ↑
Caspar et al., 2021 [36]EEG-based via MIVisual + auditory2No2Kalckert and Ehrsson, 2012;
Longo et al., 2008 [5]
Day 1 = 59.47
Day 2 = 61.72
Sensorimotor information may not be the most important cue for generating a sense of agency
SoA =, SoL =, BMI N/A
Ziadeh et al., 2021 [20]EEG-based via MIVisual (immersive) + auditory2No2Skola, 2019 [57]Hand = 53
Blocks = 54
Avatar increased SoO and SoA
SoA ↑, SoO ↑, BMI% =
Serino et al., 2022 [37] *IntracorticalVisual + haptic (electrostimulation)2Yes4Q1: sense of agency Q2: confidenceVisual
Cong V = 93.8, incong = 5.2
Somat
Cong. = 97.5, incong = 8.8
Vi inc. + Somatos. Cong.,
(somat. prevails)
Somat. Cong. ↑ SoA, BMI ↑ (soma+ vis− versus soma− vis+)
Pais-Vieira et al., 2022 [15] *EEG-based via MIVisual (immersive) + auditory + haptic (vibratory + thermal)4Yes2Peck and Gonzalez-Franco, 2021Sleeve = 82.50
No Sleeve = 73.50
(p = 0.2857)
Multimodal stimulation not detrimental for performance or embodiment
SoA =, SoL =, SoO =, BMI% =
* Includes patients. N/A indicates Non Applicable or unable to access data. Up and Down arrows indicate an increase or decrease, respectively.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Tomás, D.J.; Pais-Vieira, M.; Pais-Vieira, C. Sensorial Feedback Contribution to the Sense of Embodiment in Brain–Machine Interfaces: A Systematic Review. Appl. Sci. 2023, 13, 13011. https://doi.org/10.3390/app132413011

AMA Style

Tomás DJ, Pais-Vieira M, Pais-Vieira C. Sensorial Feedback Contribution to the Sense of Embodiment in Brain–Machine Interfaces: A Systematic Review. Applied Sciences. 2023; 13(24):13011. https://doi.org/10.3390/app132413011

Chicago/Turabian Style

Tomás, Diogo João, Miguel Pais-Vieira, and Carla Pais-Vieira. 2023. "Sensorial Feedback Contribution to the Sense of Embodiment in Brain–Machine Interfaces: A Systematic Review" Applied Sciences 13, no. 24: 13011. https://doi.org/10.3390/app132413011

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop