1. Introduction
Virtual reality technology has rapidly gained prominence in various fields, such as training, simulation, medical applications, and entertainment [
1,
2,
3]. VR growth has been fueled by hardware and software advancements like affordable headsets, making it more accessible and immersive [
4]. As VR becomes more integrated into different sectors, so does the importance of its user experience. However, a significant challenge to this experience is the prevalence of cybersickness, a type of motion sickness specific to VR environments. Cybersickness manifests through symptoms like nausea, dizziness, and disorientation, which significantly limit the adoption and effectiveness of VR technology. For instance, in training simulations, cybersickness can hinder the learning process, while in medical applications, it can affect the precision and effectiveness of procedures. Therefore, addressing cybersickness is not just a matter of comfort but is crucial for VR applications’ safety, performance, and overall success [
5].
Cybersickness, like traditional motion sickness, arises from a sensory inconsistency between what is seen in a virtual environment and what the body perceives physically. This inconsistency leads to symptoms ranging from mild headaches and dizziness to severe nausea and vomiting. This phenomenon is a major concern in sectors where virtual reality is used for critical purposes.
Evaluating cybersickness has traditionally relied on subjective methods, such as the Simulator Sickness Questionnaire (SSQ) and the Motion Sickness Susceptibility Questionnaire (MSSQ) [
6,
7]. Further advancements included the postural instability tests and physiological monitoring, which fail to capture real-time sickness development or, in some cases, are susceptible to bias. These traditional approaches are criticized for their retrospective or indirect nature while providing an unclear picture of cybersickness during VR exposure. These methods also suffer from reduced ecological validity, struggle to distinguish causes from effects, and account for individual physiological differences [
8,
9].
Modern technologies that use biomedical signals like heart rate, galvanic skin response (GSR), eye tracking, and machine learning modeling have begun to address these limitations by enabling more accurate, real-time measurements and predictions of cybersickness. However, a notable knowledge gap exists in understanding the impact of demographic factors such as age, gender, and individual susceptibility to cybersickness. This gap hampers the development of tailored VR experiences that could minimize cybersickness prevalence and severity. Incorporating demographic variables into assessments alongside physiological signals and other measures allows for more precise, individualized predictions and tracking of susceptibility over time and across different VR sessions. A multimodal approach, combining demographic data with techniques like biomechanical monitoring, shows promise for effective cybersickness tracking and mitigation.
Combining demographic variables into cybersickness assessments helps account for known risk factors, such as age, gender, and exposure time, which affect individuals’ susceptibility to cybersickness [
8,
10,
11]. Machine learning models can make more accurate and individualized predictions about cybersickness by combining demographic data with other measures, like physiological signals or eye tracking. This also enables tracking changes in susceptibility over time and across different VR sessions [
12,
13]. A multimodal approach that combines demographic variables with other techniques is the most promising solution for real-time tracking and mitigation of cybersickness [
14]. In addition, demographic information is easy to collect and includes established risk factors like gender, age, and inherent motion sickness susceptibility, alongside the well-correlated duration of VR exposure to sickness severity. Machine learning algorithms can uncover complex relationships between these factors and cybersickness outcomes, offering a straightforward approach to risk assessment [
15].
This study investigates the predictive power of demographic data on the occurrence of cybersickness among virtual reality users. By focusing exclusively on demographic variables such as age, gender, and inherent motion sickness susceptibility, we aim to uncover patterns and correlations that could forecast the likelihood and severity of cyber sickness. This approach is predicated on the hypothesis that demographic characteristics can serve as reliable indicators for designing more personalized and comfortable VR experiences.
The primary research objectives of this study are twofold:
To investigate the correlation between demographic data and cybersickness occurrence, determining if specific demographic profiles are more susceptible to experiencing discomfort in VR environments.
Identifying which demographic factors predict the likelihood and severity of cybersickness provides a foundation for future VR design considerations that prioritize user well-being.
By concentrating on demographic information, this research seeks to establish a foundational understanding of how such data can be effectively utilized to anticipate cybersickness outcomes. This focus on demographic variables centers on enhancing the theoretical framework within which we understand user interactions with VR technologies. The expectation is that an innovative comprehension of demographic influences on cybersickness will pave the way for abundant research and development strategies in the future. This study contributes significantly to the field by laying the groundwork for a more inclusive approach to VR system design. Recognizing and accommodating the diverse needs and sensitivities of a broader user base through personalized VR experiences could improve user satisfaction and expand the accessibility of VR technologies. Ultimately, by advancing our knowledge of the demographic predictors of cybersickness, we open new avenues for research and innovation in creating VR environments that are more universally enjoyable and engaging.
This paper is structured as follows.
Section 2 provides a detailed overview of VR technology, the emergence of cybersickness as a concern, and a review of existing research.
Section 3 explains our research methodology design and data collection methods. It also gives an overview of the analysis techniques used in the study.
Section 4 presents the survey findings, highlighting demographic factors correlating with cybersickness through statistical analysis.
Section 5 interprets the results, relates them to existing literature, and explores their implications.
Section 6 suggests avenues for further research, and the conclusion concisely summarizes key findings, their significance, and potential benefits, reinforcing the study’s contributions to the field of VR and cybersickness research.
3. Methodology
The study employed a structured approach to identify cybersickness in users experiencing a virtual environment. The steps involved designing a tailored web-based survey for data collection, preprocessing the collected data, and subsequently analyzing it using ML classification techniques. Utilizing an online survey methodology facilitated efficient data collection from a diverse participant pool, ensuring the reliability and validity of the gathered data. This approach allowed for firsthand insights from individuals engaged with VR technologies, broadening the study’s scope. The methodology, as shown in
Figure 1, commenced with survey design and participant selection followed by data acquisition, including demographics and subjective responses. After preprocessing and filtering, the data underwent correlation analysis and feature extraction. Machine learning classification was then applied to progress to the prediction and validation phase to evaluate the model’s accuracy.
3.1. Survey Design and Implementation
The survey was created using Qualtrics [
31], a web-based survey software, with the goal of collecting detailed demographic information from respondents. Our survey was shared through a comprehensive approach to ensure a diverse and inclusive participant response. The methods employed included email campaigns, targeted distribution lists, and strategic social media postings to effectively reach a broad audience while also targeting specific communities and groups that could provide valuable demographic and health-related perspectives for our study. This information includes age, gender, location, educational background, profession, experience with virtual reality, frequency of virtual reality usage, and types of virtual reality environments used. Additionally, the survey aimed to determine participants’ initial impressions and associations with virtual reality, as well as their experiences of discomfort, sickness, or dropout during VR sessions and their underlying causes. The chosen variables were carefully selected to investigate how different demographic factors and levels of VR exposure might affect a person’s susceptibility to cybersickness. The survey utilized a mix of question formats, including multiple-choice, Likert scales, and open-ended questions.
3.2. Data Acquisition
The data for this study were obtained through a comprehensive online survey designed to capture subjective experiences of cybersickness among virtual reality users. The survey collected a diverse range of information from participants, including the following:
Demographic Data: Participants provided demographic details such as age, gender, country of residence, educational background, and occupation. These variables were gathered to investigate potential correlations between demographic factors and susceptibility to cybersickness.
VR Awareness and Experience: Information regarding participants’ awareness of VR technology, frequency of VR experiences, and types of VR environments they have been exposed to was collected. These data points shed light on the relationship between prior VR exposure and the likelihood of experiencing cybersickness.
Subjective Experiences: Participants reported their subjective experiences related to VR usage, including first impressions, associations with VR, experiences of sickness or discomfort, instances of dropout from VR environments, and perceived causes of these adverse effects. The qualitative data provided valuable insights and perceptions of cybersickness among users. The survey employed a combination of multiple-choice questions, Likert scales, and open-ended responses to capture the diverse range of data effectively. Rigorous data cleaning and preprocessing techniques were applied to ensure the integrity and usability of the collected data for subsequent analysis and machine learning applications.
3.3. Data Preprocessing
Data preprocessing is a crucial step in ML pipelines, as it can significantly impact an ML model’s performance and interpretability. Rigorous data preprocessing steps were undertaken to ensure data integrity and usability for machine learning applications. The preprocessing and application of ML classification on the data involved some steps that are shown in
Figure 2.
To ensure the highest quality and reliability of the data, we refined the original dataset of 245 entries to 148 responses by applying strict criteria that required complete responses across all 17 columns. The process included handling missing data through imputation or deletion.
To optimize the dataset for machine learning analysis, we applied data normalization techniques and derived new variables or features from existing ones.
On our dataset, we first performed correlation analysis to identify the relevant features that exhibit a strong relationship with the target variable, “Sickness”. This step helps reduce dimensionality and mitigate the curse of dimensionality by eliminating irrelevant or redundant features [
32].
After identifying the relevant features (Age, VRAware, Num of VR experiences, VRArea, Exposure, VRType, DropOut, Discomforts), we employed one-hot encoding to handle categorical variables. One-hot encoding is a process of converting categorical variables into a form that machine learning algorithms can effectively utilize. This technique represents each distinct category as a binary vector, allowing the model to treat them numerically while retaining their non-linear relationships with the target variable [
33].
In our machine learning classification problem, where the goal is to predict a target variable “Sickness” based Age, VRAware, Num of VR experiences, VRArea, Exposure, VRType, DropOut, Discomforts. The machine learning model would learn the relationship between the features and the target variable “Sickness”.
In a classification problem, the objective of a model is to learn a mapping function
f: X → Y, where X represents the input feature space, and Y represents the set of possible class labels (e.g., “Sickness” or “No Sickness”).
The function f is learned from the training data, which consists of labeled instances (x, y), where x is a vector of feature values, and y is the corresponding class label. The machine learning algorithm aims to find the optimal parameters of the function f that minimize a predefined loss function (e.g., cross-entropy loss for classification) on the training data. Once the model is trained, it can be used to predict the class labels for new, unseen instances based on their feature values.
In our case, the trained model can then be used to predict the likelihood of an individual experiencing cybersickness based on their demographic and VR-related characteristics. We achieved a final dataset that is diverse, representing a range of ages, genders, nationalities, educational backgrounds, and experiences with VR and occupational fields. This diversity was crucial for gaining a comprehensive understanding of varied experiences and perspectives related to VR across different population segments.
After acquiring data through the survey, a series of refined data processing steps address missing values and categorize variables for computational efficiency. Key features were then identified via correlation analysis, serving as inputs for our classification model. The models were then trained, tested, and validated for their application for sickness prediction, which was integral to our research.
Figure 2 details an overview of the data preprocessing and the classification steps used in the research.
3.4. Machine Learning Models
Machine learning classifiers are well-suited for predicting cybersickness in VR because they are great at learning patterns and relationships in data. Standard classifiers like KNN, SVM, decision trees, random forests, logistic regression, and neural networks are all effective options that can handle both categorical and numerical input features. These classifiers have also been used in existing research to predict cybersickness in VR environments effectively.
We adopted this method because it excels at detecting complex, non-linear patterns in multidimensional data, which is crucial for gaining a comprehensive understanding of the factors that contribute to cybersickness. By leveraging machine learning’s predictive capabilities, we were able to anticipate the likelihood of cybersickness based on a range of variables, such as demographics and VR usage patterns [
34]. These algorithms processed and analyzed the data with impressive efficiency, identifying the most influential factors impacting cybersickness through feature importance analysis. The analysis was performed using scikitlearn, and numpy, the free machine learning libraries for python [
35,
36].
Furthermore, machine learning’s ability to model interactions between different variables allowed for a deeper understanding of how combined factors influence the experience of cybersickness. The insights gained from this analysis will be used to customize VR experiences. Notably, the statistical rigor of ML models ensured the reliability and credibility of our research findings. Validation techniques like cross-validation provided a strong foundation for our conclusions.
3.5. Model Evaluation
Evaluating a classifier’s performance using a single metric, such as accuracy, can be misleading, especially in imbalanced datasets [
37].
When predicting cybersickness using machine learning classifiers, the dataset is likely to be imbalanced, with a smaller number of individuals experiencing cybersickness compared to those who do not. In such cases, overall accuracy can be misleading as a classifier that predicts the majority class (individuals not prone to cybersickness) will have high accuracy but fails to identify the minority class (individuals prone to cybersickness) [
38,
39]. Metrics like balanced accuracy consider the accuracies for each class separately, and precision and recall for the minority class become crucial [
40,
41]. Correctly identifying individuals likely to experience cybersickness is more important than correctly identifying those who are not, making precision and recall for the minority class precious metrics.
Additionally, misclassifying an individual as not prone to cybersickness when they are may lead to discomfort, adverse effects, or premature termination of the VR experience. While misclassifying an individual as prone to cybersickness when they are not may result in unnecessary precautions. The relative costs of these errors can be evaluated using precision and recall. The F1 score, which balances precision and recall, provides a single metric that captures both aspects [
42]. By considering multiple metrics like accuracy, balanced accuracy, precision, recall, and F1 score, researchers and developers can gain a comprehensive understanding of the classifier’s performance in predicting cybersickness, allowing for informed decision-making and model selection based on the specific requirements and priorities of the application [
43].
The ML classifiers for the dataset were evaluated using metrics like accuracy, precision, recall, and F1 score. These metrics provided a comprehensive assessment of the models’ performance, considering their overall accuracy. To further enhance our analysis and ensure our prediction reliability regarding cybersickness, we implemented Leave One Out Cross Validation (LOOCV). For small datasets like the one with only 148 participants, LOOCV can be incredibly helpful. LOOCV preserves all available data for training and testing by holding out one instance for each iteration. It also reduces the risk of overfitting and provides a more reliable estimate of the model’s overall performance [
22]. Through the utilization of LOOCV, we were able to precisely estimate the performance of our machine learning models, resulting in accurate and generalized findings. This meticulous evaluation was essential in determining the effectiveness of the model in accurately predicting cybersickness, taking into account the diverse range of variables included in the dataset.
4. Results
The analysis of our dataset through various machine learning classifiers yielded insightful results regarding the predictive capabilities concerning cybersickness. The initial application of these classifiers provided promising results, with Random Forest achieving the highest balanced accuracy of 80.56% and an F1 score of 82.35%, underscoring its robustness in managing large volumes of data and mitigating overfitting through its ensemble approach. A more detailed overview of the balanced accuracy and F1 score are listed in the
Table 1.
A decrease in performance metrics was observed upon applying LOOCV. Although there was a noticeable decrement in both balanced accuracy and F1 scores across all classifiers—with KNN and SVM experiencing drops to 68.59% and 65.80% in balanced accuracy, respectively—the significance of these results lies beyond mere numerical values. The performance metrics post-LOOCV are presented in the
Table 2. Applying LOOCV provides a more transparent, generalized understanding of each classifier’s predictive power in real-world scenarios, where data are seldom perfect or complete. While the observed decrease in classifier performance following the LOOCV application may initially seem alarming, it reinforces our analytical approach’s validity and robustness. By adhering to a rigorous evaluation standard, our study provides valuable insights into the demographic predictors of cybersickness in a virtual environment.
The decline in performance metrics post-LOOCV highlights the importance of employing rigorous validation methods in machine learning studies, ensuring the reliability and credibility of the findings. This study’s innovative use of LOOCV to evaluate the classifiers’ performance emphasizes our commitment to methodological excellence and contributes to the burgeoning field of cybersickness research in virtual reality.
Moreover, this study’s methodological framework—particularly the strategic choice and application of machine learning classifiers followed by an intensive validation process—sets a new precedent for future research in this domain. It underscores the potential of employing advanced machine learning techniques and rigorous validation methods to unravel the complexities of cybersickness, paving the way for more personalized and effective mitigation strategies in virtual reality environments.
Figure 3 shows the Balanced Accuracy for the same set of classifiers. This indicates that for most models, balanced accuracy decreases after LOOCV, with Naive Bayes showing the most significant drop. This suggests that LOOCV has a more pronounced effect on Balanced Accuracy, potentially due to its sensitivity to the distribution of classes.
Figure 4 illustrates the F1 Scores for KNN, SVM, RF, LR, and NB classifiers. The performance is relatively consistent before and after LOOCV, with slight variations that suggest LOOCV had a minimal impact on F1 Scores.
5. Discussion
The results from our study indicate a strong correlation between demographic data and the incidence of cybersickness in virtual reality users. Specifically, the performance metrics of the machine learning classifiers, with RF emerging as the top performer, suggest that demographic factors such as age, gender, and prior VR exposure can predict the likelihood and severity of cybersickness.
Our prediction results for cybersickness using demographic factors and VR experience-related features are in line with existing research findings. The selected features, such as age, gender, VR exposure duration, and prior VR experience, have been consistently identified as influential factors in cybersickness susceptibility across multiple studies [
8,
10]. This finding challenges the conventional reliance on biomedical and behavioral data for cybersickness prediction [
25,
26], highlighting the potential of demographic data as a standalone predictive tool. The high F1 scores and balanced accuracy achieved by the classifiers demonstrate the effectiveness of this approach in accurately classifying individuals at risk of cybersickness.
This research offers a new avenue for creating personalized VR experiences that minimize cybersickness, enhancing user comfort and safety. By incorporating demographic data in the design and development process, VR applications can be tailored to reduce the risk of cybersickness among users, broadening the accessibility and appeal of VR technologies. This approach promises users comfortable and enjoyable VR experiences, with reduced risk of discomfort and dropout. Researchers can leverage these insights further to explore the implications of demographic factors on VR experiences, paving the way for more inclusive and ethical research methodologies that prioritize user privacy and comfort.
Our findings complement the existing body of research on cybersickness prediction. While previous studies have acknowledged the relevance of demographic factors, they are treated as secondary to biomedical or behavioral data. Our research underscores the predictive power of demographic data, offering a less invasive and more accessible approach to mitigating cybersickness. Our results align with the work of Luong et al. [
30], who found that gender and VR experience significantly influenced cybersickness susceptibility but went further by demonstrating the efficacy of machine learning techniques in utilizing these factors for prediction.
6. Limitations and Future Work
Our research in predicting cybersickness through machine learning classifiers and demographic data provides significant insights yet encounters notable limitations. The study’s scope, limited by the diversity and size of the dataset, may only partially represent the broad spectrum of VR users’ experiences, potentially affecting the generalizability of our findings. Additionally, the performance variability observed with the stringent LOOCV validation suggests that our models may face challenges in real-world applicability, highlighting the complex, multifaceted nature of cybersickness. These limitations underscore the dynamic evolution of VR technology, which could alter the factors contributing to cybersickness, thus necessitating continuous refinement of our predictive models. Future research can expand the sample size and demographic diversity to enhance the generalizability of findings. Investigating the interaction of demographic and behavioral data could yield richer insights into cybersickness prediction. Exploring the application of more advanced machine learning and artificial intelligence techniques, such as deep learning, could further refine the accuracy and reliability of forecasts. Finally, longitudinal studies examining the long-term effects of VR exposure on cybersickness susceptibility will provide valuable insights for developing more user-friendly VR technologies.
While using demographic data in predictive models for cybersickness can yield practical benefits, it raises ethical concerns regarding privacy and potential bias. Personal information such as age, gender, and occupation could be considered sensitive data, and its misuse or unauthorized access could violate individuals’ privacy rights. Additionally, relying heavily on demographic factors in predictive models carries the risk of perpetuating societal biases and discriminatory practices, as certain groups may be disproportionately associated with higher cybersickness susceptibility. We implemented robust data protection measures, obtained consent from users, and continually monitored and mitigated any emergent biases in the models to ensure the ethical and responsible use of such predictive systems.
7. Conclusions
This study uses demographic data and ML techniques to predict cybersickness within virtual reality environments, questioning the traditional dependency on biomedical and behavioral indicators. By harnessing the predictive capabilities of ML algorithms, this study emphasizes the substantial potential of variables such as age, gender, and VR exposure in forecasting the incidence of cybersickness. The methodology adopted here circumvents the ethical and privacy concerns associated with invasive data collection and heralds a paradigm shift towards a more accessible and simplified predictive framework. This pioneering approach facilitates a nuanced understanding of cybersickness, advocating for a recalibration of VR development practices towards inclusivity and user-centric design.
Furthermore, the implications of our findings are far-reaching, proposing a recalibrated approach for VR developers to tailor applications more attentively to a diverse demographic landscape, potentially broadening the scope of VR adoption and enhancing the overall user experience. The comparative analysis of various ML models contributes critical insights into the strategic selection of predictive algorithms, enriching the academic and practical discourse on VR. In sum, this research advances the scholarly conversation surrounding VR technologies and cybersickness and delineates a forward-looking trajectory for the field. It advocates for a more inclusive, data-informed approach to VR design, underscoring the pivotal role of demographic factors in the preemptive identification and mitigation of cybersickness, thus setting a new precedent for future research and development within the burgeoning domain of virtual reality.