How Immersed Are You? State of the Art of the Neurophysiological Characterization of Embodiment in Mixed Reality for Out-of-the-Lab Applications

Abstract

Mixed Reality (MR) environments hold immense potential for inducing a sense of embodiment, where users feel like their bodies are present within the virtual space. This subjective experience has been traditionally assessed using subjective reports and behavioral measures. However, neurophysiological approaches offer unique advantages in objectively characterizing embodiment. This review article explores the current state of the art in utilizing neurophysiological techniques, particularly Electroencephalography (EEG), Photoplethysmography (PPG), and Electrodermal activity (EDA), to investigate the neural and autonomic correlates of embodiment in MR for out-of-the-lab applications. More specifically, it was investigated how EEG, with its high temporal resolution, PPG, and EDA, can capture transient brain activity associated with specific aspects of embodiment, such as visuomotor synchrony, visual feedback of a virtual body, and manipulations of virtual body parts. The potential of such neurophysiological signals to differentiate between subjective experiences of embodiment was discussed, with a particular regard to identify the neural and autonomic markers of early embodiment formation during MR exposure in real settings. Finally, the strengths and limitations of the neurophysiological approach in the context of MR embodiment research were discussed, in order to achieve a more comprehensive understanding of this multifaceted phenomenon.

1. Introduction

Mixed Reality (MR) represents one of the most innovative and fascinating frontiers in the field of modern technology. It is a fusion of the real world with the virtual one, allowing the creation of new environments where people, physical, and digital objects coexist and interact in real time. This combination of elements offers immersive and engaging experiences that can radically transform how we interact with our environment and digital technologies. Therefore, to make a comprehensive understanding of MR, it is important to distinguish between two related but distinct concepts: Augmented Reality (AR) and Virtual Reality (VR). In fact, the integration of such two concepts constitutes the definition of MR. Typically, both the VR and AR rely on the usage of Head Mounted Display (HMD) for providing the virtual environments to users. Within AR, users see the real world through a technological device such as a smartphone or smart glasses, with digital elements overlaid on this real world, such as graphics, videos, or holograms. On the other hand, in VR, users are completely immersed in a digital environment through devices like VR headsets, where the perception of the real world is completely replaced by that of the virtual world. MR sits between these two extremes, by offering a combination of experiences from both AR and VR. In other words, MR encompasses both AR and VR experiences, depending on how much digital elements overlap with or completely replace the physical reality. This allows users to interact with virtual objects in the context of the real world, generating a hybrid experience that leverages the benefits of both technologies.

In recent years, MR has garnered increasing interest across a wide range of fields, from entertainment to education, industry, and healthcare [1,2,3,4,5]. Since the objective of the MR consists in providing an immersive experience that blends elements of the real and virtual worlds, it has to be considered that not all MR devices and equipment (i.e., haptic gloves, audio systems, vibrotactile stimuli accessories, etc.) are effective in creating engaging and immersive experiences for users. One key factor influencing the effectiveness of an MR device is the concept of embodiment, which refers to the sensation of being fully immersed and integrated into the virtual environment [6]. Embodiment is essential to ensure that users feel engaged and involved during the MR experience, which in turn can positively impact the effectiveness and acceptance of MR devices. MR-based approaches became consistently employed in several operational environments as the central support for training programs. In this regard, it has to be considered that such an approach was already significantly adopted by the aviation industry for pilot and air traffic controller trainings [7,8,9,10,11,12,13], and, more generally, by the industry sector in which operators are requested to perform activities in high-risk environments, such as healthcare, construction, manufacturing, and emergency services [11,14,15,16,17]. Other relevant application fields of such technologies are related to learning and educational environments. In fact, the selection of MR technologies may overcome limitations of traditional training and learning programs, allowing users to experience and recognize possible hazards and thus learn how to manage their behavior, emotions, and instinctive reactions, to avoid them [9,18]. In the era of the metaverse, it is in fact expected that such technologies will be able to completely replace some tasks (e.g., training, learning, and education) that up to now had been done just in real settings.

Other extremely topical applications of MR are focused on healthcare. These include the use of MR technology in scientific rehabilitation, as well as in cognitive and motor function assessment [19,20]. In this regard, different scientific contributions demonstrated that MR provides excellent support for rehabilitation sessions, helping to alleviate pain and reduce muscle fatigue [21,22]. This is particularly beneficial for patients undergoing long-term rehabilitation, as it can improve their overall experience and outcomes. Additionally, MR technology can offer immersive and engaging environments that motivate patients to persist with their therapy, ultimately leading to more effective and efficient recovery processes [23,24].

Considering the recent technological advancements of MR systems in terms of software and hardware integration, it is possible to imagine how the integration of additional multisensory feedback could be used to replicate real training and learning scenarios, and to induce same kind of reactions. But what about user perception and cognition? Could this multisensory integration be able to generate a simulated training and learning environment, able to mimic the same kind of perception (i.e., embodiment [25]) that the user would experience in a physical place, with the final aim to elicit, and so learn how to manage eventual risky behaviors, inducing thus an effective training and learning? In this regard, the existing scientific literature shows opposite findings. For example, some studies show that healthy users can sense increased cognitive load when first using MR technology [26,27], while other studies have indicated that gender differences may influence perceived workload during MR use, as males prioritized virtual tasks, whereas females prioritized virtual and physical tasks equally [28]. Despite these initial challenges, other research suggests that both MR and VR technologies generally enhance attention and cognitive functions [29,30].

Considering these findings, the perception of the technology employed appears crucial to humans’ attention and cognitive load. Therefore, the concept of embodiment becomes central in understanding these effects. Previous scientific works [25,31,32,33] referred to the concept of embodiment as the sense of presence. More recently, the concept of embodiment was proposed also as the combination of three principal components: the sense of self-location (SoL), the sense of ownership (SoO), and the sense of agency (SoA) [34]. These contributions described the diverse perception of the body (SoO), the space in which the body is located (SoL), and the experience of recognizing oneself as the agent of certain actions (SoA). Indeed, such an approach would provide more specificity to the embodiment characterization, even if it must be noted that, at the best of our knowledge, the above-mentioned three principal components were not characterized by neurophysiological models, but only through subjective and behavioral measurements [35,36,37]. For this reason, the neurophysiological features related to the embodiment variations that emerged from the present review will be discussed in terms of transversal embodiment characterization.

In general, research indicates that more sophisticated simulations (higher immersion) result in increased presence. Such a concept has been widely studied in virtual environments and has demonstrated a significant impact on user experience. However, evaluating embodiment poses a complex challenge. Typically, studies assessing different degrees of immersion find higher presence in more immersive MR systems compared to less sophisticated setups. In this regard, embodiment has been traditionally assessed through questionnaires and behavioral parameters, providing a subjective evaluation of the user experience [38]. However, these methods may be limited by their subjectivity and lack of objectivity. In fact, the employment of subjective-based approaches is susceptible to bias and inaccuracies inherent in human perception and interpretation [8,39,40,41].

In recent years, there has been growing attention to the use of neurophysiological signals to assess embodiment in MR, in parallel with the increasing integration of neuroscience applications within virtual environments [1,2,4].

Signals such as electroencephalography (EEG), electrocardiography (ECG), electrodermal activity (EDA), and photoplethysmography (PPG) provide an objective and measurable assessment of the user’s brain, cardiac, and physiological activity during the MR experience [27,31,38,42,43,44,45]. These neurophysiological signals allow for a better understanding of the physiological indicators in several application fields, especially within the MR operational scenario design [16,17]. This aspect makes the neurophysiological-based approach fully compatible for assessing embodiment in MR.

This scientific work aims to provide a review of the state of the art in the evaluation of embodiment in MR, with particular regard to neurophysiological characterization, examining the latest investigated approaches and methodologies used to objectively evaluate the underlying mechanisms of this phenomenon.

Through critical analysis of the existing literature, the aim is to identify current challenges and future perspectives for the evaluation and consequent optimization of embodiment in MR devices. A thorough understanding of embodiment and its neurophysiological correlates is essential to guide the development of MR devices that offer an optimal user experience and have a positive impact on user health and well-being.

2. Material Selection

The material selection for the present review was primarily guided by the need to ensure coherence and relevance within the context of the embodiment investigation. In order to identify the most relevant studies, these academic search engines were employed: PubMed, Scopus, and Google Scholar. These tools enabled access to a wide spectrum of scientific articles, ensuring comprehensive coverage of the available literature. Additionally, some articles were selected through an analysis of the bibliography within articles already identified as pertinent. This approach allowed for the identification of further sources that could be relevant to the topic at hand. Particularly, attention was directed towards evaluating embodiment using both biomedical and non-biomedical signals. However, during the selection process, several articles were excluded based on specific criteria to ensure the validity and reliability of the collected data.

Firstly, it was necessary to exclude studies involving an insufficient number of participants, i.e., less than 12, to ensure robust statistical analysis. Additionally, exclusion of unhealthy participants was performed to avoid potential result distortions due to altered physiological conditions. Therefore, studies involving subjects with health issues or pathological conditions were excluded. Finally, articles specifically addressing pathologies or medical conditions were excluded, as the focus of the present research was on embodiment in more generic contexts not linked to specific conditions. The selection criteria allowed for a focus on studies that met the coherence and relevance requirements for the embodiment investigation, while ensuring the reliability and validity of the collected data.

3. Experimental Protocols Design

This section provides an extensive overview of the different experimental protocols design approaches for the neurophysiological characterization of embodiment in virtual environments. More specifically, a classification was made based on the types of tasks that included comparisons between Real Life (RL), VR, and AR for exploratory tasks, exposure to heights, and gaming. Additionally, types of tasks involving avatar control, both in error monitoring and in perception of social entities and emotions, were identified.

Comparison between Different Virtual Environments

The totality of the scientific works considered among this review included a consistent portion of the experimental protocol based on spatial navigation. Indeed, this aspect is crucially connected to the perception of sense of presence. In this regard, it has to be observed that also the type of virtual environment can play a role within the embodiment perception [32,33]. More specifically, three main environments can be identified among the different considered scientific works:

• The physical environments, consisting of the representation of the experimental environment in RL. Among the considered scientific contributions, this experimental environment was considered as the gold standard for the embodiment assessment in MR.
• The photographic environment, consisting of the bidimensional representation of the selected experimental environment. This condition was designed by Juan Luis Higuera-Trujillo and colleagues [46] for evaluating the sense of presence grade within a bidimensional digitalized environment.
• The 360-degree environment, consisting of the tridimensional representation of the selected experimental environment. Again, this condition was proposed by Juan Luis Higuera-Trujillo and colleagues [46] and Marin-Morales and colleagues [47] for assessing embodiment in the static tridimensional environment, which was hypothesized to lead to an increase of the sense of presence with respect to the static bidimensional representation.
• The VR environment, consisting of the virtual and interactive representation of the selected experimental environment. This condition was selected by the totality of the considered scientific works.
• The AR environment, consisting of the integration of digital elements, typical of VR environments, with physical elements, typical of the RL environment. Such a condition was included in the experimental design of the large majority of the considered scientific papers within the present review [5,12,27,43,44,47,48,49].

Among the approaches proposed by the considered scientific works, Slobounov and colleagues [50] investigated the cortical activity modulation in 2D versus 3D virtual reality environments. In this context, the bidimensional and tridimensional virtual environments were compared in terms of cortical activity modulation, in order to derive relevant insights regarding the embodiment of the participants. More specifically, the participants were exposed to a virtual corridor and navigated through it using a wireless joystick to reach a specific location within the target room. This task was divided into two phases: the encoding phase, during which participants were shown the navigation route to memorize, and the retrieval phase, during which they navigated to the position of the target room.

Concerning the comparison between static and interactive virtual environments, Benjamin Schöne and colleagues [38] proposed research for evaluating the neural correlates changings when executing exposure tasks at height in VR, RL, and photographic environments. In particular, the researchers exposed the participants to a height of 33 m in the RL condition through a fire truck’s cherry picker. Analogously, such a condition was replicated in VR, through 3D-360° videos, and in bidimensional images provided through a PC screen. Similarly, Joanna Kisker and colleagues [43] investigated the neurophysiological embodiment of participants when performing a spatial navigation task coupled and not coupled with height exposure. In this context, the experimental protocol foresaw the comparison between the VR and RL conditions. Other approaches in scientific literature were identified in the gaming context. In this regard, Wen Huang and colleagues [51] proposed a comparison between VR and a bidimensional environment representation, provided through a PC desktop screen, in terms of sense of presence perceived by the players. Further interesting research on the embodiment neurophysiological characterization were focused on the Brain Computer Interface (BCI) applications [52,53]. In this regard, it has to be considered the relevant contribution associated with the research proposed by Julia M. Juliano and colleagues [33], which investigated the perceived embodiment by participants while dealing with BCI in immersive VR. In particular, such experimental protocol foresaw the control of a virtual arm with brain activity on the computer screen and through an HMD for VR, and the control of the same virtual arm through the actual arm movements with an HMD.

A second relevant branch of experimental design related to the neurophysiological embodiment characterization in MR corresponds to the avatar control-based tasks. This paradigm aims to investigate how individuals perceive and interact with avatars or virtual bodies within MR settings. In this regard, Bilal Alchalabi and colleagues [54] investigated the EEG-based measurement of embodiment when controlling a walking self-avatar. More specifically, the experimental protocol foresaw that participants were instructed to either physically control the movements of a virtual avatar in real time, mentally simulate the avatar’s movements without physical action, or simply observe the avatar’s movements. This multifaceted approach allowed researchers to explore the nuances of embodiment and motor imagery within a MR setting. A second interesting research in this context was performed by Enea Francesco Pavone and colleagues [32]. They designed an experimental protocol for errors monitoring in the actions of an avatar seen from a first-person perspective. In particular, the participants in the study were engaged in an immersive setup, consisting of a virtual dining room with two mugs on a table, observed through HMD. The virtual environment featured two avatars, one in first-person perspective and the other in third-person perspective. Therefore, this experimental design assessed the neurophysiological embodiment through error monitoring while executing the requested activity in MR in first-person perspective and in third-person perspective. A similar approach was performed by Porssut and colleagues [27], who designed an experimental protocol for assessing the neural correlates associated with embodiment while performing movement of a digital wrist when the immersivity within the virtual environment was broken. More specifically, the experiments foresaw the disruption of the wrist movement representations within the virtual environment. Such a condition followed a monitoring phase in which participant embodiment was evaluated while performing the digital wrist movements within a coherently responding virtual scenario.

Finally, another relevant aspect characterizing the experimental scenario consists of the impact of social entities within the virtual environment on embodiment. In fact, human beings are inherently social creatures, and our interactions with others profoundly influence how we perceive ourselves and our surroundings. In MR, the presence of social entities, such as virtual avatars representing other individuals or artificially intelligent agents, can enhance or diminish the sense of embodiment. This aspect was investigated by Maia Garau and colleagues [55] by performing an experimental protocol in which the participants’ perceptions of agents across different conditions was evaluated, with a focus on the level of responsiveness exhibited by these agents and how it influenced participants’ treatment of them as either social entities or mere objects. Practically, the experimental design foresaw the interaction with a virtual agent behaving differently (i.e., static, moving, responsive, and talking conditions). A consistent similar approach was conducted by Juanzhi Lu and colleagues [48], who investigated the threat perception impact on embodiment in MR when interacting with an avatar who differently behaved in four experimental conditions (i.e., static, moving, responsive, and talking conditions). The following

Table 1. Overview of the different experimental scenario designs in terms of virtual environments that emerged from the scientific literature review.

Environments Comparison	Main Task	Experiments
Real life vs. VR vs. AR [27,38,47,55,56,57]	Walking activity Motor imagery activity Spatial navigation Threat management	6
360° vs. VR [32,58]	Motor activity	2
2D vs. 3D vs. VR [27,43,44,48,57]	Spatial navigation Motor imagery activity Cognitive activity Social interaction	5
VR vs. AR [26,36,37,49,59,60,61]	Walking activity Motor imagery activity Spatial navigation	7
2D vs. VR vs. AR [33,38,46,56]	Spatial navigation Motor imagery activity Cognitive activity	4
2D vs. VR [31,44]	Spatial navigation Motor imagery activity Cognitive activity Social interaction	8

resumes the different MR experimental scenarios investigated by the scientific literature considered through this review:

4. Mixed Reality Instrumentation Selection

As described within the Introduction section, the MR can be defined as all the digital environments generated by the employment of VR and AR technologies. Therefore, the equipment selection for providing the MR scenario must be envisioned according to these two aspects. Concerning the considered scientific research focused on the VR, the use of a range of sophisticated devices has played a fundamental role in providing engaging and realistic experiences to the involved participants. Among the primitive equipment (i.e., MR equipment developed less recently, lacking in terms of integrated tracking modules, low latency, and low refreshing rate) used, the HTC Vive headset, in its two models Pro (HTC, Taoyuan, Taiwan) and Cosmos (HTC, Taoyuan, Taiwan), stood out for its visual quality and precise tracking, allowing users to immerse themselves in detailed and interactive virtual worlds. This headset provided a high-quality VR experience, with wide peripheral vision and crisp resolution, thus contributing to making the experience engaging and immersive. Similarly, the Oculus Rift (Consumer Version 1) HMD (Meta, Menlo Park, CA, USA), along with the Unity 3D game engine (Unity Technologies, San Francisco, CA, USA), enabled users to explore virtual worlds rich in detail and interactivity. The advanced display capability of the Oculus Rift headset contributed to creating an engaging and realistic VR experience, enabling participants to fully immerse themselves in detailed virtual environments. Additionally, the Samsung Gear VR HMD (Samsung, Seoul, South Korea) was employed in different scientific studies [17]. In these contexts, the researchers selected such a device for its wireless compatibility with other different wearables included within the Samsung software ecosystem (Gear VR 2.0.1 and above), in order to provide a more immersive virtual environment. Regarding this primitive equipment, it has to be noted that additional systems were employed for the hand tracking of the participants. As an example, the Vicon optoelectronic motion capture system was frequently selected [10,17,62] for pairing the hand tracking with the virtual scenario provided by the above-described HMD for VR.

Recently, the tech industry offered a more advanced VR HMD, integrating hand tracking features. This is the case of the more recent models produced by Meta (Meta, Menlo Park, CA, USA), which are the Meta Quest 2, Meta Quest Pro, and Meta Quest 3. High-end, but still commercial, HMD are the XR-4 and VR-3 developed by Varjo (Varjo Technologies Oy, Helsinki, Finland). These devices integrate several camera-based systems for hand tracking in real time. This feature provided a consistent advantage to the recent scientific research, since they were allowed to reduce the invasiveness of the whole VR equipment perceived by the participants and, therefore, to increase the overall immersivity.

Concerning the AR equipment selection, it has to be noted that the above-mentioned Meta HMD were also employed for provided AR experiences in scientific research [4,11,57]. Additionally, the Microsoft HoloLens (Microsoft, Redmond, WA, USA) was also consistently selected by researchers for designing experimental protocols focused on AR technology. Similarly, also the Google Glass (Google, Mountain View, CA, USA) and Magic Leap One (Magic Leap, Plantation, FL, USA) were selected, albeit to a lesser extent, within the considered scientific contribution [31,57].

5. Measurement and Parameters Selection

The measurement of embodiment in MR was investigated under different perspectives by the scientific community. Indeed, such a concept is directly related to the realism level of the proposed MR environment. In this regard, Chalmers and Ferko [63] theorized that the realism level associated with a MR environment can be defined as the combination of perceived realism and measured realism. The perceived realism corresponds to how the user perceives the MR scenario as close to real life, while measured realism represents how the selected MR hardware and software reproduce real life. Concerning this latter aspect, Hoorn and colleagues [64] demonstrated that there is a consistent lack of objective methods for evaluating the measured realism associated with the MR. Therefore, the measurement of perceived realism, i.e., embodiment, appears even more crucial.

5.1. Subjective Measurements

Subjectively measuring embodiment is significant to understanding the perception of the sense of presence in virtual environments. To do so, the scientific literature proposed a consistent number of questionnaires and methodologies that explored the perceived level of embodiment. In this regard, the Embodiment questionnaire [33] was the only one aimed at directly assessing the sense of embodiment and has a form adapted from the one proposed by Bailey and colleagues [65] and Banakou and colleagues [66]. Such a questionnaire contains questions relating to two features of embodiment, namely, self-embodiment and spatial embodiment. The first aspect was defined as the extent to which the participant perceives a virtual body component, while the second one represented the degree to which the participant feels immersed in the virtual environment. Another key tool identified in scientific literature for collecting subjective data on embodiment is the Presence Questionnaire, adapted from Witmer and Singer [34] and revised by the UQO Cyberpsychology Laboratory. This questionnaire presents targeted questions to evaluate the realism of the virtual environment, the participant’s ability to act within it, the quality of the interface, and the participant’s self-assessment of performance. The considered scientific works highlighted other subjective measurements for the sense of presence evaluation, such as the IGroup Presence Questionnaire (IPQ). Such a questionnaire provided a subjective measure of the perception of being within a virtual environment [67,68]. Among the considered works, it was proposed as divided into four subscales: general presence, spatial presence, involvement, and realness. Similarly, the Slater-Usoh-Steed (SUS) questionnaire [67] corresponds to a validated method comprising six statements, which are evaluated on a seven-point Likert scale. This approach includes the participant’s sense of being inside the simulated environment, the degree to which the environmental simulation is considered the dominant reality, and how far the simulated environment is remembered as a place. It is often used in conjunction with the Presence Questionnaire (PQ) developed by Witmer & Singer [34]. Within experiments requiring emotional involvement of the participant, which constitute a consistent portion of the considered scientific works within the present review, participants were asked to complete the Self-Assessment Manikin (SAM) [69], a non-verbal, image-based questionnaire designed to assess the emotional reactions of subjects, to measure a person’s affection and feelings in response to an object or event. In the context of perceived emotions and feelings’ impact on embodiment in MR, it is crucial to subjectively assess the participants’ state of anxiety and the cognitive and psychological workload during the virtual experience. In this regard, the State-Trait Anxiety Inventory (STAI) [70] is a psychological inventory comprising 40 self-assessment questions on a four-point Likert scale. It gauges two types of anxiety: state anxiety and trait anxiety. Similarly, more recent research was performed by including within the experimental measurements collection the Profile of Mood States (POMS) [71] and Social Avoidance and Distress (SAD) [72]. Moreover, the considered gold standard by the scientific community for assessing cognitive and psychological workload is the NASA Task Load Index (TLX) [73]. The following

Table 2. Overview of the subjective questionnaires related to embodiment and the sense of presence evaluation that emerged from the considered scientific literature.

Questionnaire	Year	Items	Experiments
Acrophobia Questionnaire (AQ)	1977	20	2
State-Trait Anxiety Inventory (STAI)	1981	40	3
Simulator Sickness Questionnaire	1993	16	2
Barfield et al. Questionnaire	1993	3	2
Slater-Usoh-Steed Questionnaire (SUS)	1994	6	3
Self-Assessment-Manikin (SAM)	1994	6	1
Sensation Seeking Scale, Form V (SSS-V)	1996	40	2
Positive and Negative Affect Schedule (PANAS)	1996	20	3
Kim and Biocca Questionnaire	1997	8	1
Witmer and Singer Presence Questionnaire (PQ)	1998	19	3
Immersive Tendencies Questionnaire	1998	18	1
Questionnaire on Presence and Realism (QPR)	1998	10	1
Dinh et al. Questionnaire	1999	14	2
Murray et al. Questionnaire	2000	5	2
Nichols et al. Questionnaire	2000	9	1
Reality Judgment and Presence Questionnaire (RJPQ)	2000	18	1
Lombard & Ditton Questionnaire	2000	103	1
Gerhard et al. Questionnaire	2001	19	2
Igroup Presence Questionnaire (IPQ)	2001	14	3
Swedish Viewer-User Presence (SVUP)	2001	150	1
ITC Sense of Presence Inventory (ITC-SOPI)	2001	63	1
Krauss et al. Questionnaire	2001	42	1
Swedish User-Viewer Presence Questionnaire	2001	19	3
Schroeder et al. Questionnaire	2001	10	2
Experimental Virtual Environment-Experience (EVEQ)	2002	124	1
E2I Scale Development	2002	9	2
Cho et al. Questionnaire	2003	4	3
Nowak and Biocca Questionnaire	2003	29	1
Sas and O’Hare Questionnaire	2003	34	1
MEC Spatial Presence Questionnaire (MEC-SPQ)	2004	16	1
Bouchard et al. Questionnaire	2004	1	1
Presence-Involvement-Flow Framework (PIFF)	2004	15	1
Template Presence Inventory (TPI)	2009	42	1
Virtual Experience Test (VET)	2010	24	1
Spatial Presence (P) and Self-presence (SP)	2016	5	3
Embodiment Questionnaire (EQ)	2020	10	1

includes all the subjective questionnaires related to embodiment and the sense of presence assessment in MR that emerged from the scientific literature review:

5.2. Behavioural Measurements

Among the selected studies, behavioral data analyses provide complementary information to questionnaires, useful in validating initial hypotheses. Among the numerous behavioral parameters, those related to participants’ performance in the MR environment are common. One key parameter is latency time, defined as the period between a stimulus and the participant’s behavioral response. This metric offers insights into the speed and effectiveness of user reactions in the virtual environment. A short latency time may indicate high embodiment with prompt responses, while a longer time may suggest more cognitive processing or uncertainty, reflecting lower embodiment. Similarly, the transversal time corresponds to an additional behavioural parameter correlated with embodiment [57]. This parameter was defined as the duration required to complete a specific task or action within the virtual environment. Measuring the transversal time may allow for evaluating the efficiency and competence of users in performing assigned activities, providing valuable insights into their familiarity with the virtual environment and their navigational skills. The discomfort response time was another particularly relevant behavioural parameter selected by the most recent approaches for evaluating embodiment in MR [27,57]. Such a parameter was calculated by observing when the participant responded to a stressful stimulus, i.e., the approach of an avatar in virtual reality, in order to evaluate the participant reaction to stress or discomfort within the MR environment. Finally, another crucial behavioural parameter emerging by the recent scientific literature is the walking time. Indeed, such a parameter was strictly associated with the experimental protocol, including consistent spatial navigation activities requested to the participants [31,57]. The following

Table 3. Overview of the behavioral parameters related to the embodiment assessment in Mixed Reality identified among the considered scientific literature.

Behavioural Parameter	Definition	Experiments
Latency time	Time period between an input or stimulus and the participant’s behavioral response.	11
Transversal time	Duration required to complete a specific task or action within the virtual environment.	5
Discomfort response time	Time period for obtaining the participant’s response to a stressful stimulus.	6
Walking time	Time period required for completing spatial navigation tasks in MR.	12

provides an overview about the behavioural parameters that emerged from the considered scientific literature:

6. Neurophysiological Characterization of Embodiment in Mixed Reality

While the above-described subjective and behavioral measurements estimated the human perception of embodiment in virtual environments, it is crucial to note that such approaches are prone to a subjective bias in terms of embodiment perception, which could negatively impact its objective evaluation [74,75]. Therefore, as confirmed by the considered scientific literature, the employment of neurophysiological measurements for characterizing embodiment in MR goes beyond the subjective perception of humans, and therefore captures the intrinsic correlates deriving from a more or less profound sense of immersion in MR [27,40,48,57,76,77]. Therefore, the present section provides an overview of the selected equipment for the neurophysiological data collection among the considered scientific contributions and, subsequently, a description of the measurement and principal outcomes related to the neurophysiological characterization of embodiment in MR.

6.1. Equipment Selection for Neurophysiological Signals Collection

As introduced in the first section of the present review, in the context of MR research, the acquisition of neurophysiological signals such as EEG, ECG, PPG, and EDA is crucial for objectively understanding participants’ experiences, particularly embodiment. Many studies used moderately high-density EEG devices for data recording. Notably, the Brain Products actiCAP EEG System (Brain Products GmbH, Gilching, Germany) was frequently employed to assess the sense of presence in VR and AR environments [57]. The system offers flexible configurations with up to 256 channels, enabling researchers to capture detailed brain activity with high spatial resolution. To meet MR HMD compatibility and wearability requirements, the studies typically used a maximum of 32 active electrodes arranged according to the international 10–20 system. Similarly, the Neuroelectrics Enobio (Neuroelectrics, Barcelona, Spain) was also chosen to investigate neural correlates of embodiment in MR [57]. Such a system was equipped with up to 32 EEG channels, and it was consistently employed for its capability for real-time synchronization with AR and VR environments. In this context, also high-density EEG systems were employed for objective embodiment characterization. Bahavan and colleagues [61] selected the Neuroscan SynAmps RT amplifiers by Compumedics and a fabric cap with integrated electrodes (Electro-Cap International) equipped with 60 active gel-based EEG channels, positioned according to the 10-10 system. Similarly, Mar González-Franco and colleagues [60] employed g.USB-Amp amplifiers (g.tec medical technologies, Austria) and 57 active electrodes for measuring the emotional impact on the sense of presence within the VR environment. Conversely, other recent scientific works were conducted by employing low-density EEG systems. Such systems consistently advanced in terms of performance and accuracy in recent years, and they are the most compatible with out-of-the-lab applications, such as the ones focused on MR approaches, since their invasiveness grade is minimal. In this regard, different recent scientific works were performed by selecting the Emotiv EPOC+ device [45,59]. Such a system was equipped with 14 water-based EEG channels, and it offered compatibility with Unity and Unreal Engine, facilitating integration with AR and VR applications for immersive research experiences.

Other crucial neurophysiological signals for the objective embodiment assessment in MR are the ones deriving from cardiac activity (i.e., ECG and PPG). More specifically, among the different considered scientific contributions, the ECG signal was collected through the usage of a various range of equipment directly connected to the EEG amplifier. In this regard, Schöne and colleagues [38] selected three active sensors connected via a BIP2AUX adapter by Brain Products. Similarly, Lu and colleagues [48] employed EKG ProComp+ sensors by Thought Technologies Ltd. on the torso. More wearable solutions were selected within more recent scientific approaches by collecting the PPG signal. In this regard, several recent scientific papers [46,47,51] investigated the cardiac correlates to embodiment in VR and AR environments by using the PPG collected through the Empatica E4 (Empatica, Milan, Italy). Such a device consists of a wristband equipped with PPG sensors on its back, which adhere to the participants’ wrist’s skin. In the same context, researchers [4,62] also employed the Shimmer GSR3+ (Shimmer Sensing, Dublin, Ireland) for PPG collection in a realistic MR environment. Such a device was equipped with PPG capabilities for monitoring blood volume changes in peripheral blood vessels using green LEDs and photodetectors to capture signals from the participant’s skin. Empatica and Shimmer Sensing solutions are the least invasive and most compatible with MR applications due to their minimal interference with participants’ activities and wireless connectivity. These portable devices, the Empatica E4 and Shimmer GSR3+, were also used for EDA collection in MR embodiment studies. The Empatica E4 employed two stainless steel electrodes to measure skin conductance, while the Shimmer GSR3+ used sensors on participants’ fingers for EDA collection.

6.2. Neurophysiological Measurements

This section summarizes the neurophysiological measures identified in the scientific literature for assessing embodiment in MR. As discussed earlier in this review, the literature consistently focuses on using EEG, ECG, PPG, and EDA to objectively characterize embodiment, providing insights into brain, cardiac, and electrodermal system activities.

Concerning the brain activity evaluation, i.e., the EEG analysis, the scientific literature contains a consistent number of works based on the time-related EEG features. More specifically, the Error-Related Negativity (ERN), the Error-Positivity (PE), and the N400 were the most selected parameters by the researchers for the neurophysiological evaluation of embodiment [48,49,57]. The ERN was defined as a component of brain electrical activity that occurs after a person makes an error during a task. It is characterized by a negative deflection in brain electrical activity, usually within the first 100 milliseconds after the error is detected. ERN was often observed primarily in the fronto-central regions of the brain and has been associated with error perception and recognition. In the context of embodiment, ERN can be studied to better understand how the brain reacts to errors during interaction with MR environments and to assess the participant’s experience. PE was defined as a component of brain electrical activity that occurs after a person has consciously detected an error [48]. It was characterized by a positive deflection in brain electrical activity, usually around 200–400 milliseconds after an error was committed. PE was often observed primarily in the parietal region of the brain, with a prominent peak at electrode Pz. In research contexts of embodiment assessment, PE can be studied to better understand how the brain perceives and responds to errors during interaction with virtual environments. Its measurement provides valuable information about subjective error perception and the participant’s experience in the MR environment. Finally, the N400 was defined as a component of brain electrical activity that typically occurs around 400 milliseconds after the presentation of a linguistic stimulus, such as a word or phrase [48]. It is characterized by a negative deflection in brain electrical activity, usually observed in central-parietal EEG channels. In the context of the neurophysiological characterization of embodiment, it was demonstrated that the N400 can be studied to better understand how the brain processes concepts and information related to body and action during interaction with virtual environments [48,57].

Concerning the cardiac (i.e., PPG and ECG) and electrodermal (i.e., EDA) activity monitoring for the neurophysiological embodiment assessment, the scientific literature highlighted that the most common selected parameters consisted of the Heart Rate (HR), the Heart Rate Variability (HRV), the Skin Conductance Level (SCL), and the Skin Conductance Response (SCR). Regarding the first two parameters, related to the cardiac activity collected through ECG or PPG tracking systems, the works proposed by Lu and colleagues [48], Kisker and colleagues [43], and Schöne and colleagues [38] applied the Welch’s method for the frequency analysis of the ECG and PPG signals. This approach estimated the HR and HRV parameters. More specifically, the HR parameter was estimated from the ECG or PPG signal by preprocessing the signal for identifying and correcting the signal artifacts [78] and, subsequently, by identifying the R-peaks for computing the RR interval. Similarly, the HRV parameter was defined as the variation in the time intervals between heartbeats (i.e., RR intervals). Its computation relies on the identification of Very Low Frequency (VLF) [<0.04 Hz], Low Frequency (LF) [0.04–0.15 Hz], and High Frequency (HF) [0.12–0.4 Hz] spectral components of the ECG/PPG signal. Several scientific approaches defined HRV as the ratio between the LF and HF [38,43]. Concerning the EDA-derived features, the Continuous Decomposition Analysis resulted in the most common approach for evaluating the SCL and SCR parameters from EDA, as proposed by the study performed by Higuera-Trujillo and colleagues [46] and by Marucci and colleagues [25]. The EDA decomposition computed the slow varying component of such a signal, i.e., the SCL, which is associated with the overall level of sweat gland activity [79,80,81], and its most rapidly variable component, i.e., the SCR, which is associated with the transient changes in skin conductance [82]. This approach was technically implemented through the use of the Ledalab suite [83], developed for the Matlab environment. In this regard, it must be noted that the SCL was the unique investigated EDA-derived parameter when such a signal was collected through the Empatica E4, as proposed by the most recent research [78,84], due to the low sample frequency of the device (i.e., 4 Hz).

6.3. Neural Correlates of Embodiment in Mixed Reality

Concerning the neural correlates to embodiment in MR, the scientific literature offers a wide range of contributions. In fact, it was demonstrated that the electroencephalographic signal possesses the highest information content with respect to embodiment when compared to other neurophysiological measures. Indeed, in the large majority of the considered scientific contributions, it was considered as the main element of the analysis. Such scientific outcomes related to the neural derived features can be divided into two main categories, i.e., spectral and temporal features, according to the kind of performed signal processing (i.e., frequency domain analysis and time domain analysis). In this regard, it must be noted that the above-mentioned EEG temporal features correspond to the so-called time-locked EEG features, i.e., specific patterns or components of electrical activity in the brain that are consistently related to certain events or stimuli, such as Event-Related Potentials (ERPs).

Concerning the spectral neural parameters, interesting research was proposed by Dey and colleagues [85]. The researchers selected a wearable EEG system for the signal collection, and they identified the specific frequency band and EEG channels directly related to the embodiment level in the VR environment. More specifically, the frontal and occipital region was the most sensible to the embodiment variations along the experimental protocol. The researchers demonstrated that the following EEG-based indexes were correlated with the embodiment variations in VR [85]:

$$TA={\displaystyle \frac{{PSDtheta}_{AF3,AF4}}{{PSDalpha}_{P7,P8}}}$$

where ${PSDtheta}_{AF3,AF4}$ and ${PSDalpha}_{P7,P8}$ corresponded to the EEG Power Spectral Density (PSD) computed within a 2-second-long time window in the theta and alpha EEG frequency band, respectively, within the frontal and parietal regions, respectively. It must be considered that the above-mentioned index is fully coherent with the mental workload definition provided among different previous scientific contributions in the context of the Human Factors neurophysiological characterization [86,87,88,89]. In particular, the same definition was adopted by Marucci and colleagues [25] in their work centered on a VR environment characterized by different sensorial dimensions. In this context, the researchers observed coherent results concerning the relationship between the mental workload and the embodiment level. Another interesting outcome reported by the same study performed by Dey and colleagues [85] consists of the significant correlation between the EEG spectral features computed in beta and alpha EEG frequency bands, as the following:

$$BA={\displaystyle \frac{{PSDbeta}_{FC5,FC6}}{{PSDalpha}_{P7,P8}}}$$

where ${PSDbeta}_{FC5,FC6}$ and ${PSDalpha}_{P7,P8}$ corresponded to the EEG PSD computed within a 2-second-long time window in the beta and alpha EEG frequency band, respectively, within the fronto-central and parietal regions, respectively. Other research adopting the frequency domain analysis of the EEG signal demonstrated how the spectral EEG features can be employed for the embodiment characterization in MR through machine learning approaches. This was the case of the research proposed by Krugliak and colleagues [26] in an AR environment. The researchers employed a mobile EEG system, and they optimized a neural network based on EEG features computed within the theta, alpha, and beta frequency bands among the frontal and parietal regions. The generated model exhibited the highest accuracy (i.e., above 70%) in terms of the embodiment estimation when including the above-mentioned EEG spectral features. Similarly, Marin-Morales and colleagues [47] proposed a connectivity analysis based on EEG spectral features for estimating the embodiment variations across different MR scenarios. More specifically, the researchers defined the EEG spectral features as the PSD computed within the delta, theta, alpha, and beta frequency bands, computed among the frontal, central, and parietal regions. A slightly different approach was selected by Juliano and colleagues [33], who performed a spectral analysis of the EEG signal by defining the region of interest according to the requested activities to the participants immersed in MR. Since the main task consisted of walking along the virtual environment, the researchers identified the median EEG PSD computed among the EEG channels corresponding to the motor cortex (i.e., C3 and C4). Therefore, the results demonstrated how the normalized PSD computed among these regions within the alpha and beta bands were consistently correlated with the embodiment variations across the different MR scenarios. Additionally, the research conducted by Romero Soto and colleagues [59] defined a specific EEG spectral index correlated with embodiment while playing VR games. In particular, the researchers validated that the EEG PSD computed in alpha band across the left frontal region was associated with the sense of presence within the VR game. Finally, further interesting research was proposed by Gonzalez-Franco and colleagues [60], who defined a readiness potential index as the EEG PSD asymmetry computed on C3 and C4 channels within the alpha band. Such an index was demonstrated to be significantly correlated with the immersion level within the VR scenario and, therefore, with embodiment characterizing the participants. The following

Table 4. Overview of the EEG spectral correlates of the embodiment variations in MR that emerged from the considered scientific literature.

Method	Frequency Bands and Regions	Results
Normalized EEG PSD ratio	Theta, alpha Frontal, and parietal	High capability of identifying high-embodiment conditions in MR.
Normalized EEG PSD ratio	Beta, alpha Fronto-central, and parietal	High capability of identifying high-embodiment conditions in MR.
Neural network developed with EEG spectral features	Theta, alpha, beta Frontal, and parietal	Highest accuracy (i.e., above 70%) in tracking the embodiment variations.
Connectivity analysis based on EEG spectral features	Delta, theta, alpha, beta Frontal, central, and parietal	High accuracy (i.e., above 75%) in detecting the embodiment variations, especially in social VR interactions.
Normalized EEG PSD	Alpha Central	Significant and high correlation with the embodiment variations during walking activities in VR.
EEG PSD asymmetry	Alpha Central	Significant and high correlation with the embodiment variations during physical activities in VR and AR.

resumes the main results concerning the EEG temporal features correlated with the embodiment variations in MR:

Concerning the second category of neural correlates to embodiment in MR identified in scientific literature, i.e., the EEG-derived temporal features, the ERP features were the most correlated with embodiment variations across the VR and AR scenario. It must be underlined that such kind of neurophysiological features, defined as time-locked, were assessed within VR and AR environments through MR equipment able to accurately synchronize the stimulus events with the neurophysiological signals recording. In this regard, Lu and colleagues [48] demonstrated how the N170 in the temporal region and the N3 in the fronto-central region as the EEG temporal components significantly and highly correlated with events occurring within the virtual scenario while the participants were experiencing high embodiment. In this context, the research proposed by Porssut and colleagues [27] confirmed the consistent relationship between the early and mid-latency EEG ERPs and the embodiment experienced in virtual environments. More specifically, the researchers demonstrated that the ERN, Pe, and N400 were significantly representing the events corresponding to the embodiment disruption within the AR scenario. In this regard, Pavone and colleagues [32] performed a study in which the relationship between the ERN and N400 and low levels of embodiment in VR was confirmed. In particular, the researchers observed that such two ERP components were significantly and highly correlated with the VR scenario in which disruption were introduced. Similarly, Gonzalez-Franco and colleagues [60] performed a research based on a VR scenario foreseeing the interaction with a virtual hand. They demonstrated that the amplitude of the P450 component was significantly and directly correlated with the subjective perception of sense of presence with respect to the virtual hand. With regard to experimental scenarios including the interaction between the participants and digital entities, such as the avatar, the study performed by Lu and colleagues [48] demonstrated that the amplitude of the N170 component was significantly and highly correlated with the interaction toward a negative emotional avatar and, therefore, with the highest embodiment perception by the participants. The following

Table 5. Overview of the EEG ERP components sensitive to the embodiment modifications in MR that emerged from the considered scientific literature.

Latency	ERP Components	Results
Early components	N170	Highest-embodiment perception, especially with emotional MR stimuli.
	PE	High correlation with events, including embodiment disruption.
	VPP	High correlation with high-embodiment subjective perception.
	N3	Significant correlation with high-embodiment perception, especially during the interaction with avatars.
	Pe	High correlation with events of embodiment disruption, especially during the interaction with the virtual arm and/or hand.
Late components	N400	The amplitude of such a component was highly correlated with the condition characterized by low-embodiment.
	P450	The amplitude of such a component was highly correlated with the high-embodiment perception while interacting with the virtual arm and/or hand.

resumes the main results concerning the EEG temporal features correlated with the embodiment variations in MR:

6.4. Cardiac and Electrodermal Correlates of the Embodiment in Mixed Reality

Concerning the cardiac correlates to the embodiment experienced in MR, the study performed by Marín-Morales and colleagues [47] revealed that an increased activity in the LF band corresponded to a low-embodiment MR condition. Conversely, the increased activity in HF was associated with a moderate- or high-embodiment MR condition. Such research demonstrated that the above-mentioned cardiac-related features were the most accurate (70.39%) in assessing different embodiment levels in MR. However, it must be observed that the proposed approach was validated only in an MR context relying on arousing stimuli. In the context of the embodiment evaluation in VR, where the height exposure was foreseen, Kisker and colleagues [43] demonstrated that the increased normalized HR was associated with an increased spatial presence. These findings were corroborated by research performed by Dey and colleagues [85], who observed a significant increase of the normalized HR while the participants were experiencing high spatial embodiment in a VR scenario focused on walking activities. This significant increase of the normalized HR was observed also by Lu and colleagues [48] when the participants experienced the interaction with an avatar in a VR scenario, which was hypothesized to induce a higher embodiment level. Other interesting cardiac correlates with embodiment in MR were observed by Higuera-Trujillo and colleagues [46], who demonstrated that the combination of normalized HF (nHF) and HRV parameters was significantly correlated with the sense of presence in VR. More specifically, such research revealed that this cardiac-related index, defined as the following definition, was negatively correlated with the levels of realism increase:

$$nHF-HRV={\displaystyle \frac{meannHF-{HRV}_{stimuli}}{meannHF-{HRV}_{baseline}}}$$

where ${HRV}_{stimuli}$ and ${HRV}_{baseline}$ corresponded to the HRV parameter, defined as the ratio between LF and HF parameters, evaluated along the stimuli (i.e., during the execution of the requested spatial activities in VR environment) and baseline (i.e., during a resting state in VR environment) conditions, respectively.

Concerning the electrodermal correlates of embodiment in MR, the above-mentioned research performed by Higuera-Trujillo and colleagues [46] revealed also that the normalized SCL parameter was partially and significantly correlated with the increase of sense of presence in VR when the participants were performing interactive and emotional experiences with an avatar. Such an index was defined as the following:

$${Phasic}_{EDA}={\displaystyle \frac{mean{Phasic}_{EDA-stimuli}}{mean{Phasic}_{EDA-baseline}}}$$

where $mean{Phasic}_{EDA-stimuli}$ and $mean{Phasic}_{EDA-baseline}$ corresponded to the SCL parameter estimated along the stimuli (i.e., during the execution of the requested spatial activities in VR environment) and baseline (i.e., during a resting state in VR environment) condition, respectively. In the same context, the study performed by Marucci and colleagues [25] showed the significant impact of both the tonic and phasic components on the load condition. Additionally, this research revealed elevated arousal in the high perceptual load condition relative to the low perceptual load condition. These results do not provide direct information on the sense of presence, but it can be reasonably assumed that the high perceptual load experimental condition is more stressful and arousing. It can be posited that a higher level of arousal may lead to an increased sense of presence, given that emotional involvement during complex tasks is generally high [90,91]. A final important finding obtained by Dey and colleagues [88] related to the autonomic correlates to embodiment must be considered. In fact, such researchers demonstrated that the EDA-based features correlated to embodiment in MR are strictly dependent from the virtual scenario. In particular, such correlations between the EDA-related features and embodiment can be observed only if factors as arousal, anxiety, or stress are included within the virtual scenario. In other words, it was not possible to determine a correlation with the sense of presence in studies in which these sensations were not elicited.

The following

Table 6. Overview of the autonomic (i.e., cardiac and electrodermal) correlates of the embodiment variations in MR that emerged from the considered scientific literature.

Type	Feature	Results
Cardiac	Normalized HR	High correlation with the increase of spatial presence in VR and AR.
	Normalized HF	High correlation with the sense of presence in VR.
	Normalized LF	Negative and significant correlation with the increase of embodiment in VR.
Electrodermal activity	Normalized SCL (i.e., phasic EDA)	Positive and significant correlation with the embodiment increase in VR and AR.

provides an overview of the different cardiac and electrodermal correlates of the embodiment variations in MR that emerged from the considered scientific literature:

7. Discussion

The presented review reported the approaches adopted in scientific research regarding the evaluation of embodiment in MR. As described in the Introduction section, MR is a technology of enormous potential in many kinds of applications, especially in the era of the metaverse, in which it will be possible, and more convenient from many points of view, to perform in the virtual world many activities (e.g., training) that until now were performed exclusively in physical environments. In this regard, the great versatility and effectiveness to generate virtual scenarios with critical situations with a very high realism could, in fact, revolutionize current training and learning practices of users. Therefore, the present review aimed at identifying the scientific works proposing approaches for analyzing the effects of MR on cognition, and assessing how the MR is perceived as close to the real world by the users (i.e., embodiment).

The presented work was conducted with a particular focus on the neurophysiological evaluation of embodiment in MR. In fact, with respect to subjective and behavioral ones, the neurophysiological-based measurements offer deeper insights into the underlying neural mechanisms, detect subtle changes in embodiment perception, and potentially reveal unconscious processes. Among the wide range of emerged methods, it appears to be clear that the measurements derived from the cerebral activity are the most promising for obtaining an objective index for evaluating embodiment variations within virtual environments. In this regard, several EEG spectral indexes were demonstrated to be significantly sensitive to embodiment variations, both in VR and AR environments. Recent research showed how the increase of the EEG PSD computed in alpha band is frequently related to the high-embodiment experience in MR. Similarly, the increase of the EEG PSD computed in theta and alpha bands within the central region is significantly correlated with high-embodiment perception while interacting with virtual body entities. In fact, previous research demonstrated that the central region is strictly representative of the motor cortex activity. Therefore, the cerebral activity increase within such a region can be consistently related to high-embodiment perception, since it represents the fact that the user is perceiving the virtual entity (e.g., a virtual arm and/or hand) as part of his own physical body, as confirmed by the high and significant correlation between the neurophysiological and behavioral features [33,60]. In this context, it was also deeply investigated the impact of the emotional interaction on the sense of presence in MR. More specifically, different research demonstrated how the combination of EEG spectral features computed in theta and alpha bands are significantly sensitive to embodiment variations while interacting with emotive avatars in MR [31,47]. Additionally, a further consistent contribution that emerged from the scientific literature showed how the time-locked EEG features are capable of providing a significant contribution to the embodiment assessment in MR. More specifically, the considered scientific works validated the significant correlation between embodiment variations experienced within virtual environments and the specific EEG ERP components [25,32,48,49,50]. In this regard, different research revealed the direct and consistent relation between early and late ERP components and different types of events related to embodiment variations. This was observed within both virtual scenarios designed for increasing the sense of presence and virtual scenarios, including embodiment disruptions [27]. Therefore, the considered scientific contributions identified the EEG ERP components associated with embodiment increase and the ones associated with embodiment disruption due to the virtual scenario modifications.

Considering other measurements besides the neural correlates, the different considered scientific contributions demonstrated how the autonomic-derived parameters, such as the ones related to the cardiac and electrodermal activities, are correlated with the sense of presence variations within VR and AR environments. More specifically, the scientific literature exhibited how the increase of HR and its variability (i.e., the HRV) can be related to high-embodiment perception in MR. Similarly, it was marginally demonstrated how the EDA-derived features, especially the SCL, can be representative of embodiment variations within MR environments. This finding was demonstrated to be consistent with MR scenarios including social or physical interactions. Such an aspect is coherent with the related scientific research based on the Human Factors neurophysiological characterization. In fact, the variations of EDA-related features, with a particular regard to the SCL, were largely validated to be correlated with arousal, anxiety, and stress variations. Therefore, the considered scientific literature clearly demonstrates that autonomic features (i.e., cardiac and EDA-derived parameters) are prone to objectively characterize embodiment in MR. In fact, different previous research showed that cardiac and EDA-based features are representing the human cognitive and emotional aspects variations within the perception of adaptive multisensory environments.

Taken all together, the scientific evidence related to neural and autonomic correlates of embodiment in MR highlighted that only a part of them is compatible with out-of-the-lab application. More specifically, EEG-derived approaches based on time-locked features (i.e., ERP) cannot be selected for embodiment characterization in naturalistic MR environments, while other EEG-based measurements compatible with the out-of-the-lab scenario, such as the spectral features proposed by Dey and colleagues [85], showed a promising reliability in objectively estimating embodiment variations in MR. Concerning autonomic-based approaches, the considered scientific work showed promising methods in the context of embodiment evaluation in out-of-the-lab MR environments. In fact, autonomic signal collection (i.e., EDA and PPG) can be performed through wearable devices, which are easy to wear even in naturalistic settings without negatively interfering with experimental activities.

A further significant aspect that emerged from the presented literature review is related to the embodiment conceptual characterization. As mentioned within the Introduction of this work, under the psychological point of view, the literature describes embodiment as the combination of the sense of self-location (SoL), the sense of ownership (SoO), and the sense of agency (SoA) [34]. In this regard, there is only evidence of these contributions evaluated through subjective and behavioral parameters [92] and, therefore, this paves the way for further future investigations in terms of neurophysiological characterizations of the SoL, SoO, and SoA. Finally, another relevant factor that emerged from the presented literature review is related to the side effect of the MR on the perceived embodiment. In fact, different recent studies [34,93,94] highlighted how the required MR equipment and systems can negatively impact the human perception of the generated digital environment and, therefore, impair the positive embodiment level. This aspect confers even more significance to the objective evaluation of embodiment in MR.

In conclusion, the presented work highlights the potential of techniques like the ones based on EEG, PPG, and EDA analysis in objectively capturing the dynamics of embodiment. However, these findings should be critically considered within the broader theoretical framework of embodiment and neural mechanisms. Embodiment in MR is deeply intertwined with theories of embodied cognition, which posit that cognitive processes are rooted in the body’s interactions with the environment. The neurophysiological signals discussed in this review reflect this embodiment, as they capture the brain’s real-time response to virtual stimuli, reinforcing the theory that cognition is not just a brain-bound process but a dynamic interaction between the brain, body, and the environment. In this regard, the sense of embodiment, as characterized by self-location, ownership, and agency, provides a tangible manifestation of these theoretical concepts, aligning with the notion that the body is central to the cognitive experience. Therefore, the emerged findings suggest that the signals reflecting embodiment in MR are the ones underlying the neural mechanisms in which embodiment relies. The EEG data, for example, reveal how specific brain regions are involved in processing visuomotor synchrony and the integration of visual feedback from a virtual body. These findings imply that embodiment in MR may be underpinned by neural processes similar to those involved in body ownership and agency in real-world settings, but further research is needed to fully delineate these mechanisms.

As of now, the synthesis of these findings suggests that the best approaches to studying embodiment in MR should integrate both neurophysiological measures and subjective and behavioral reports to capture the full spectrum of the user experience in MR. In fact, the neurophysiological-derived features provide objective insights, while the subjective and behavioral measurements, even if they are not able to reveal unconscious processes, offer a necessary contextual understanding of how the neurophysiological signals-related measurements translate to the felt experience of embodiment.

Limitations and Future Trends

Besides all the presented scientific outcomes, which are clearly promising and effective in the context of the embodiment evaluation in MR environments, the presented review underlined that there is still a lack of a transversal and objective approach for the neurophysiological embodiment assessment.

In fact, the performed literature review in the context of embodiment neural correlates highlighted that, even if the validated EEG-derived features appeared to be reliable in objectively evaluating embodiment in MR, the state-of-the-art methodologies are for the most part based on time-locked EEG features. Such an approach might limit the applications fields, since the time-locked EEG features evaluation, such as ERP, requires specific experimental design constraints, such as the accurate synchronization between the EEG traces and specific stimuli to add to the MR environment. This aspect might limit the validated embodiment methods evaluations in out-of-the-lab applications. Concerning the identified spectral correlates of embodiment in MR, the scientific literature proposed promising approaches. However, such approaches rely on EEG-derived spectral features also connected to Human Factors variation (e.g., mental workload), which might not be necessarily related to embodiment variations, but directly correlated with the human perception and reaction to the requested activities within MR environments.

Similarly, the present work highlighted that several cardiac and electrodermal features were identified to be highly and significantly correlated with embodiment variations across different MR scenarios and applications, but still consistently dependent on the specific phenomena characterizing the activities included in the considered digital environment. In other words, the autonomic parameters identified in scientific literature are sensible to embodiment variations in MR within out-of-the-lab applications, but, at the same time, are still strictly correlated to the other cognitive variations induced by the specific experimental scenario (e.g., the SCL parameter was demonstrated to be significantly sensible to stress and arousal variations elicited by the performed experimental task [7,79,95,96,97], independently from the embodiment level). This aspect could lead to consistent misinterpretation in the context of the embodiment evaluation within different experimental environments.

By considering all the above-mentioned findings, it appears to be clear that all the emerged neurophysiological variables cannot always and exclusively be associated with embodiment variations in MR, since they were demonstrated to be correlated with other HFs variations according to the experimental context. Therefore, the scientific gap consists of determining a standardized and transversal approach for the objective assessment of embodiment in digital environments. This could also be achieved by furtherly investigating the conceptual subcategorization of embodiment suggested by Witmer and colleagues [34] in their work, for which embodiment could be assessed through the evaluation of the different senses of agency, ownership, and self-location. In fact, the single evaluation of each component through a neurophysiological-based method would provide an objective measurement of the specific neural correlates associated with each component. More specifically, previous studies demonstrated that the SoA is associated with the EEG activity in premotor cortex, supplementary motor area, and parietal lobes [98,99]; while the SoO is associated with the neural activity in primary somatosensory cortex [35,36]; and the SoL is impacting on the activity in temporoparietal junction and hippocampus [35,37]. Therefore, in terms of future research, an interesting approach could consist of designing an experimental protocol including an exact dual experience between MR and real life, i.e., by requiring to the participants to perform the same experimental activities both in MR and in real life. Such an approach investigates the robustness of the embodiment neurophysiological characterization even with respect to the experimental task impact on the participants’ HFs. Additionally, such a multimodal approach would provide a significant robustness to the embodiment assessment when evaluated in out-of-the-lab MR environments, since it would allow the objective estimation of the singular embodiment components to finally combine them and make the embodiment evaluation unbiased from the eventual impact of mental and cognitive states variations related to other phenomena.

8. Conclusions

The presented review considered a consistent number of scientific contributions representing the most updated state of the art regarding the neurophysiological characterization of embodiment in MR. Several aspects were discussed, both in terms of digital environments design and in terms of neurophysiological signal-processing methodologies and technologies selection. This work identified several neural, cardiac, and electrodermal correlates of embodiment variations assessment in MR, and it determined an accurate overview of the optimal approach to be selected for objectively evaluating embodiment within specific MR applications. The assessed state of the art highlighted that different methodologies have already been used to evaluate embodiment. The large part of such approaches relies on the neural embodiment correlates estimation, especially based on time-locked EEG features. In terms of application fields, it was consistently demonstrated how embodiment in MR can be objectively evaluated within operational environments [15,65], but also in more controlled research contexts [98,99,100], especially in physical and cognitive rehabilitation applications.

However, this work also underlined the research gaps that are still present in this context. There is in fact the lack of a generalized approach for the objective embodiment assessment in MR, independently from the specific operational task, and the cognitive variations induced by the task itself, that could affect the embodiment assessment. Future works should be functional to propose embodiment-assessment methods independent from the MR scenario design. This could pave the way for crucial advancements toward the MR application optimization and the potential maximization of this technology within its several application fields.