1. Introduction
Exposure to immersive environments creates different sensations with different intensities in participants. The participant’s responses to the stimuli sent by the virtual environment reflect the effect it causes. During exposure, the participant’s brain activity is stimulated and changes in this activity can be measured by electroencephalography. The analysis of the results of the registered signals reflects the effect provoked on the participant [
1,
2].
Since the changes in brain activity are a participant’s response, usually involuntary, to the stimuli sent by the system, the interest in using them as an element of interaction becomes relevant. If the system can detect and interpret these values, it can use them to become more interactive, adapting its behavior to the participant’s reactions. Stimuli trigger emotions and emotions generate reactions. The inclusion of interactivity during the experience by real-time Biofeedback mechanisms increases the feeling of presence in the virtual environment [
3].
The real-time data obtained on the user’s physiological aspects allows for determining how the stimuli affect him. On the other hand, when the user receives information in real time about a certain aspect of his physiology, he can determine how his mental changes can influence his state [
4].
The introduction of biofeedback systems in the design of a simple immersive environment transform it into an Emotionally Adaptive Immersive Environment, where the user experience can be optimized through the continuously adaptation of stimuli to the user emotional state. The quantity and intensity of the stimuli are determined through an adaptive affective algorithm, which collect, interpret, and convert the user’s physiological data [
5].
The main objective of this project is to develop a software tool that incorporates a dynamic and parameterizable algorithm to be used as a core engine in the real-time Bio-feedback process, this software tool is the element responsible for making the connection between the Looxid Link device coupled to VR HeadSet and the unity platform that manages the immersive environment.
2. Materials and Methods
2.1. Materials
Table 1 describes the hardware and software used to develop the project. For a non-evasive electroencephalogram device (Looxid Link
TM Mask for VIVE) coupled with VR Headsets HTC Vive Pro
TM, the software was developed using the C# programming language.
2.2. Methods
Unity3D software was used as a development and interconnection platform between the Looxid Link device coupled to VR HeadSet and the VR application. Using the C# programming language, an algorithm was developed to create a real-time Bio-feedback core-engine generator.
The algorithm recognizes the values of the biological signals registered in each of the EEG channels of the Looxid Link device; these values reflect the participant’s states of relaxation and attention, allowing the definition of value intervals, with the attribution to each one of interactive variables that can be used in the Biofeedback process.
The development of the algorithm included five aspects. Access the values of the EEG signals detected by the Looxid Link device through sensors AF3, AF4, AF7 AF8, Fp1, and Fp2, and Fpz. Group the values detected by each of the sensors into frequency ranges aggregating these values into two indicators, relaxation and attention. Assign an interaction variable to each frequency range. Assign the desired functions to each interaction variable.
Parameterize the period between readings of the values of the interaction variables allows for determining the desired degree of interactivity.
3. Results
The software incorporates a dynamic and parameterizable algorithm to be used as a core-engine in the real-time biofeedback process. The algorithm recognizes the values of the biological signals registered in each of the EEG channels of the Looxid Link device, identifying the participant’s states of relaxation and attention and allowing the definition of value intervals, with an interaction variable attributed to each one that can be used in the Biofeedback process by VR applications.
4. Discussion and Conclusions
One of the applications of the developed core-engine can focus on the participant’s states of relaxation and attention. The Looxid Link device determines these two states in real time during exposure, allowing them to be used as an interactive element. The algorithm uses these values and, according to predefined intervals, readjusts the intensity of the stimuli generated by the system or triggers new stimuli.
This feature has two important advantages. It increases the level of interactivity generated by the application, as this results from the real-time reading of the participant’s brain activity that was generated by the emitted stimuli, with the new stimuli generated adapting to the participant’s real reactions. The other advantage is the interoperability of the algorithm, which can be adapted to different virtual reality applications, simply by parameterizing the intervals according to the desired amount and sensitivity of the system. After this parameterization, it is possible to assign which stimuli are intended for each interaction variable.
Author Contributions
Conceptualization, R.C., P.V.G.; methodology, R.C., P.V.G.; validation, P.V.G.; investigation, R.C.; writing—original draft preparation, R.C.; writing—review and editing, R.C., P.V.G.; visualization, R.C., P.V.G.; supervision, A.M., J.P.; project administration, R.C. All authors have read and agreed to the published version of the manuscript.
Funding
This research received no external funding.
Institutional Review Board Statement
Not applicable.
Informed Consent Statement
Not applicable.
Acknowledgments
This research was carried out and used the equipment of the Psychosocial Rehabilitation Laboratory (LabRP) of the Research Center in Rehabilitation of the School of Allied Health Technologies, Polytechnic Institute of Porto.
References
- Tori, R.; Kirner, C.; Siscoutto, R. Fundamentos e Tecnologia de Realidade Virtual e Aumentada; Editora SBC: Porto Alegre, Brazil, 2006; ISBN 85-7669-068-3. [Google Scholar]
- Sá, C.; Gomes, P.V.; Marques, A.; Correia, A. The Use of Portable EEG Devices in Development of Immersive Virtual Reality Environments for Converting Emotional States into Specific Commands. Proceedings 2020, 54, 43. [Google Scholar] [CrossRef]
- Kumar, A.; Killingsworth, M.A.; Gilovich, T. Waiting for Merlot: Anticipatory Consumption of Experiential and Material Purchases. Psychol. Sci. 2014, 25, 1924–1931. [Google Scholar] [CrossRef] [PubMed]
- Barandas, M.; Gamboa, H.; Fonseca, J.M. A Real Time Biofeedback System Using Visual User Interface for Physical Rehabilitation. Procedia Manuf. 2015, 3, 823–828. [Google Scholar] [CrossRef] [Green Version]
- Gomes, P.V.; Marques, A.; Donga, J.; Sá, C.; Correia, A.; Pereira, J. Adaptive model for biofeedback data flows management in the design of interactive immersive environments. Appl. Sci. 2021, 11, 5067. [Google Scholar] [CrossRef]
Table 1.
Hardware and software used to develop and implement the project.
Table 1.
Hardware and software used to develop and implement the project.
Equipment | Specifications |
---|
Computer | CPU: Intel® CoreTM i7-9700K (3.60 GHz–4.90 GHz) |
Graphic card: NVIDIA® GeForce® RTX 2080 Ti |
Memory: 64 GB RAM |
VR Headset HTC Vive ProTM | High resolution Dual AMOLED 3.5” diagonal screens |
1440 × 1600 pixels per eye (2880 × 1600 pixels combined) |
Refresh rate: 90 Hz; Field of view: 110 degrees |
Integrated microphones with 3D Spatial Audio |
Four SteamVR Base Station 2.0: 10 m × 10 m |
VIVE Wireless Adapter |
Looxid LinkTM Mask for VIVE | EEG sensors; Looxid Link Hub |
6 channels: AF3, AF4, AF7, AF8, Fp1, Fp2 |
1 reference: FPz at extended 10–10 system |
Dry electrodes on flexible PCB; Sampling rate: 500 Hz |
Resolution: 24 bits per channel (with 1 LSB = 0.27 V) |
Filtering: digital notch filters at 50 Hz and 60 Hz, 1–50 Hz |
Digital bandpass; Real-time data access |
Raw EEG data: 500 Hz (with/without filter options) |
Feature indexes (alpha, beta, gamma, theta, delta): 10 Hz |
Mind indexes (attention, relaxation, balance): 10 Hz |
Software | Unity Personal 2020.3 LTS |
C# programming language |
| Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).