Next Article in Journal
Detection of DoS Attacks in an IoT Environment with MQTT Protocol Based on Intelligent Binary Classifiers
Previous Article in Journal
Survey on Preprocessing Techniques for Big Data Projects
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Proceeding Paper

The Influence of Brain Activity on the Interactive Process through Biofeedback Mechanisms in Virtual Reality Environments †

by
Rita Costa
1,‡,
Paulo Veloso Gomes
2,*,‡,
António Correia
2,‡,
António Marques
2,‡ and
Javier Pereira
3,‡
1
LabRP-CIR, Psychosocial Rehabilitation Laboratory, Center for Rehabilitation Research, School of Engineering, Polytechnic Institute of Porto, 4200-072 Porto, Portugal
2
LabRP-CIR, Psychosocial Rehabilitation Laboratory, Center for Rehabilitation Research, School of Health, Polytechnic Institute of Porto, 4200-374 Porto, Portugal
3
CITIC, Research Center of Information and Communication Technologies, Talionis Research Group, Universidade da Coruña, 15008 A Coruña, Spain
*
Author to whom correspondence should be addressed.
Presented at the 4th XoveTIC Conference, A Coruña, Spain, 7–8 October 2021.
These authors contributed equally to this work.
Eng. Proc. 2021, 7(1), 15; https://doi.org/10.3390/engproc2021007015
Published: 7 October 2021
(This article belongs to the Proceedings of The 4th XoveTIC Conference)

Abstract

:
This work focuses on the development of a software link interface tool between the Looxid Link Device coupled to the HTC Vive Pro VR HeadSets and the Unity platform, to generate real-time interactivity in virtual reality applications. The software incorporates a dynamic and parameterizable algorithm to be used as a core-engine in the real-time Biofeedback process, recognizing the values of the biological signals registered in each of the EEG channels of the Looxid Link device. The values of EEG frequencies detected in real time can be used to generate elements of interactivity, with different frequencies and intensities.

1. Introduction

Exposure to immersive environments creates different sensations with different intensities in participants. The participant’s responses to the stimuli sent by the virtual environment reflect the effect it causes. During exposure, the participant’s brain activity is stimulated and changes in this activity can be measured by electroencephalography. The analysis of the results of the registered signals reflects the effect provoked on the participant [1,2].
Since the changes in brain activity are a participant’s response, usually involuntary, to the stimuli sent by the system, the interest in using them as an element of interaction becomes relevant. If the system can detect and interpret these values, it can use them to become more interactive, adapting its behavior to the participant’s reactions. Stimuli trigger emotions and emotions generate reactions. The inclusion of interactivity during the experience by real-time Biofeedback mechanisms increases the feeling of presence in the virtual environment [3].
The real-time data obtained on the user’s physiological aspects allows for determining how the stimuli affect him. On the other hand, when the user receives information in real time about a certain aspect of his physiology, he can determine how his mental changes can influence his state [4].
The introduction of biofeedback systems in the design of a simple immersive environment transform it into an Emotionally Adaptive Immersive Environment, where the user experience can be optimized through the continuously adaptation of stimuli to the user emotional state. The quantity and intensity of the stimuli are determined through an adaptive affective algorithm, which collect, interpret, and convert the user’s physiological data [5].
The main objective of this project is to develop a software tool that incorporates a dynamic and parameterizable algorithm to be used as a core engine in the real-time Bio-feedback process, this software tool is the element responsible for making the connection between the Looxid Link device coupled to VR HeadSet and the unity platform that manages the immersive environment.

2. Materials and Methods

2.1. Materials

Table 1 describes the hardware and software used to develop the project. For a non-evasive electroencephalogram device (Looxid LinkTM Mask for VIVE) coupled with VR Headsets HTC Vive ProTM, the software was developed using the C# programming language.

2.2. Methods

Unity3D software was used as a development and interconnection platform between the Looxid Link device coupled to VR HeadSet and the VR application. Using the C# programming language, an algorithm was developed to create a real-time Bio-feedback core-engine generator.
The algorithm recognizes the values of the biological signals registered in each of the EEG channels of the Looxid Link device; these values reflect the participant’s states of relaxation and attention, allowing the definition of value intervals, with the attribution to each one of interactive variables that can be used in the Biofeedback process.
The development of the algorithm included five aspects. Access the values of the EEG signals detected by the Looxid Link device through sensors AF3, AF4, AF7 AF8, Fp1, and Fp2, and Fpz. Group the values detected by each of the sensors into frequency ranges aggregating these values into two indicators, relaxation and attention. Assign an interaction variable to each frequency range. Assign the desired functions to each interaction variable.
Parameterize the period between readings of the values of the interaction variables allows for determining the desired degree of interactivity.

3. Results

The software incorporates a dynamic and parameterizable algorithm to be used as a core-engine in the real-time biofeedback process. The algorithm recognizes the values of the biological signals registered in each of the EEG channels of the Looxid Link device, identifying the participant’s states of relaxation and attention and allowing the definition of value intervals, with an interaction variable attributed to each one that can be used in the Biofeedback process by VR applications.

4. Discussion and Conclusions

One of the applications of the developed core-engine can focus on the participant’s states of relaxation and attention. The Looxid Link device determines these two states in real time during exposure, allowing them to be used as an interactive element. The algorithm uses these values and, according to predefined intervals, readjusts the intensity of the stimuli generated by the system or triggers new stimuli.
This feature has two important advantages. It increases the level of interactivity generated by the application, as this results from the real-time reading of the participant’s brain activity that was generated by the emitted stimuli, with the new stimuli generated adapting to the participant’s real reactions. The other advantage is the interoperability of the algorithm, which can be adapted to different virtual reality applications, simply by parameterizing the intervals according to the desired amount and sensitivity of the system. After this parameterization, it is possible to assign which stimuli are intended for each interaction variable.

Author Contributions

Conceptualization, R.C., P.V.G.; methodology, R.C., P.V.G.; validation, P.V.G.; investigation, R.C.; writing—original draft preparation, R.C.; writing—review and editing, R.C., P.V.G.; visualization, R.C., P.V.G.; supervision, A.M., J.P.; project administration, R.C. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Acknowledgments

This research was carried out and used the equipment of the Psychosocial Rehabilitation Laboratory (LabRP) of the Research Center in Rehabilitation of the School of Allied Health Technologies, Polytechnic Institute of Porto.

References

  1. Tori, R.; Kirner, C.; Siscoutto, R. Fundamentos e Tecnologia de Realidade Virtual e Aumentada; Editora SBC: Porto Alegre, Brazil, 2006; ISBN 85-7669-068-3. [Google Scholar]
  2. Sá, C.; Gomes, P.V.; Marques, A.; Correia, A. The Use of Portable EEG Devices in Development of Immersive Virtual Reality Environments for Converting Emotional States into Specific Commands. Proceedings 2020, 54, 43. [Google Scholar] [CrossRef]
  3. Kumar, A.; Killingsworth, M.A.; Gilovich, T. Waiting for Merlot: Anticipatory Consumption of Experiential and Material Purchases. Psychol. Sci. 2014, 25, 1924–1931. [Google Scholar] [CrossRef] [PubMed]
  4. Barandas, M.; Gamboa, H.; Fonseca, J.M. A Real Time Biofeedback System Using Visual User Interface for Physical Rehabilitation. Procedia Manuf. 2015, 3, 823–828. [Google Scholar] [CrossRef] [Green Version]
  5. Gomes, P.V.; Marques, A.; Donga, J.; Sá, C.; Correia, A.; Pereira, J. Adaptive model for biofeedback data flows management in the design of interactive immersive environments. Appl. Sci. 2021, 11, 5067. [Google Scholar] [CrossRef]
Table 1. Hardware and software used to develop and implement the project.
Table 1. Hardware and software used to develop and implement the project.
EquipmentSpecifications
ComputerCPU: Intel® CoreTM i7-9700K (3.60 GHz–4.90 GHz)
Graphic card: NVIDIA® GeForce® RTX 2080 Ti
Memory: 64 GB RAM
VR Headset HTC Vive ProTMHigh resolution Dual AMOLED 3.5” diagonal screens
1440 × 1600 pixels per eye (2880 × 1600 pixels combined)
Refresh rate: 90 Hz; Field of view: 110 degrees
Integrated microphones with 3D Spatial Audio
Four SteamVR Base Station 2.0: 10 m × 10 m
VIVE Wireless Adapter
Looxid LinkTM Mask for VIVEEEG sensors; Looxid Link Hub
6 channels: AF3, AF4, AF7, AF8, Fp1, Fp2
1 reference: FPz at extended 10–10 system
Dry electrodes on flexible PCB; Sampling rate: 500 Hz
Resolution: 24 bits per channel (with 1 LSB = 0.27 μ V)
Filtering: digital notch filters at 50 Hz and 60 Hz, 1–50 Hz
Digital bandpass; Real-time data access
Raw EEG data: 500 Hz (with/without filter options)
Feature indexes (alpha, beta, gamma, theta, delta): 10 Hz
Mind indexes (attention, relaxation, balance): 10 Hz
SoftwareUnity Personal 2020.3 LTS
C# programming language
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Costa, R.; Gomes, P.V.; Correia, A.; Marques, A.; Pereira, J. The Influence of Brain Activity on the Interactive Process through Biofeedback Mechanisms in Virtual Reality Environments. Eng. Proc. 2021, 7, 15. https://doi.org/10.3390/engproc2021007015

AMA Style

Costa R, Gomes PV, Correia A, Marques A, Pereira J. The Influence of Brain Activity on the Interactive Process through Biofeedback Mechanisms in Virtual Reality Environments. Engineering Proceedings. 2021; 7(1):15. https://doi.org/10.3390/engproc2021007015

Chicago/Turabian Style

Costa, Rita, Paulo Veloso Gomes, António Correia, António Marques, and Javier Pereira. 2021. "The Influence of Brain Activity on the Interactive Process through Biofeedback Mechanisms in Virtual Reality Environments" Engineering Proceedings 7, no. 1: 15. https://doi.org/10.3390/engproc2021007015

Article Metrics

Back to TopTop