Next Article in Journal
Effects of Meteorological Factors on Waterbird Functional Diversity and Community Composition in Liaohe Estuary, China
Next Article in Special Issue
User Perceptions and Needs Analysis of a Virtual Coach for Active and Healthy Ageing—An International Qualitative Study
Previous Article in Journal
Protecting Human Health from Climate Change: Legal Obligations and Avenues of Redress under International Law
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
IJERPH
Volume 19
Issue 9
10.3390/ijerph19095389
ijerph-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Review

Virtual Reality (VR) Technology for Treatment of Mental Health Problems during COVID-19: A Systematic Review

by
Muhammad Hizri Hatta
1,
Hatta Sidi
1,
Chong Siew Koon
2,
Nur Aishah Che Roos
3,
Shalisah Sharip
1,
Farah Deena Abdul Samad
1,
Ong Wan Xi
2,
Srijit Das
4 and
Suriati Mohamed Saini
1,*
1
Department of Psychiatry, Universiti Kebangsaan Malaysia Medical Centre, Cheras, Kuala Lumpur 56000, Malaysia
2
Department of Psychiatry, Hospital Sultanah Nur Zahirah (HSNZ), Kuala Terengganu 20400, Malaysia
3
Faculty of Medicine and Defence Health, National Defence University of Malaysia, Sungai Besi, Kuala Lumpur 57000, Malaysia
4
Department of Human & Clinical Anatomy, College of Medicine & Health Sciences, Sultan Qaboos University, Al-Khoud, Muscat 123, Oman
*
Author to whom correspondence should be addressed.
Int. J. Environ. Res. Public Health 2022, 19(9), 5389; https://doi.org/10.3390/ijerph19095389
Submission received: 14 March 2022 / Revised: 18 April 2022 / Accepted: 27 April 2022 / Published: 28 April 2022
(This article belongs to the Special Issue Technology-Based Interventions to Foster Healthy Aging, Health Behaviors, and Health Care)
Browse Figures
Review Reports Versions Notes

Abstract

:
There was a surge in psychological distress and emotional burnout during the COVID-19 pandemic. Virtual reality (VR) is helpful as a psychological intervention whilst maintaining physical or social distancing. The present systematic review assessed the role of VR as a psychological intervention tool for mental health problems during the COVID-19 pandemic. We conducted a systematic review that followed the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) guidelines. This study used the search-related terms: (Virtual reality OR simulated-3D-environment OR VR) AND (covid! or corona!) AND (mental* OR psychologic* OR well* OR health*) AND (intervention) on six databases, i.e., MEDLINE, PsycINFO, Ovid Medline, EMBASE, ACM digital library, and Cochrane Central Register of Controlled Trials (CENTRAL) from the inception date until 23 June 2021. We finally included four studies in the systematic review out of the 379 references imported for screening. These studies reveal that VR is beneficial as a psychological tool for intervention in individuals with mental health problems. Immersed in the telepresence, interacting in a 3-D format compared to a 2-D layout, having a sense of enjoyment and engagement, activating an affective-motivational state, “escaping” to a virtual from the real world are pivotal faucets of VR as a psychological tool for intervention.

1. Introduction

The coronavirus disease 2019 (COVID-19), caused by the inexplicable SARS-CoV-2 virus, was first identified in Wuhan, China, in December 2019, and gradually spread to the other parts of the world [1]. The disease was found to be extremely contagious and resulted in high mortality. Since then, the virus has swiftly spread everywhere in China and other parts of the globe. Subsequently, the World Health Organization (WHO) declared this infectious condition a global pandemic in March 2020 [2]. The devastating pandemic resulted in the loss of livelihoods due to prolonged lockdowns, and this had a negative effect on the global economy [2]. There was a mass spread of disease in early March 2020, and then, it became worse [3]. Many countries enforced restrictions or movement control orders (MCOs) from March 2020 to June 2020, and until 2022 [4].
The MCO imposed a ban on the public from socializing and taking part in any events or gatherings. These included cultural, educational, social, and religious activities such as praying and gathering at religious facilities [5]. Sports activities, such as walking outside the house, cycling, jogging, work-related (for non-essential sectors), and scholarly activities, such as face-to-face classes, were restricted and strict social distancing methods were imposed [6,7]. There was much psychological and emotional distress among individuals. Studies also observed the change in emotions and factors associated with emotional distress immediately as well as following the onset of the pandemic [8].
Depression, grief, anxiety, anger, irritability, and emotional burnout throughout the MCO period was observed during the pandemic [6]. The public, patients, healthcare workers, and medical personnel were all at risk of contracting a viral infection on a regular basis. This situation predisposed them to a higher risk of major mental illnesses, which included depression, anxiety, post-traumatic, and sleep disorders during the challenging time of the pandemic [9,10,11,12,13,14]. A study which investigated the MCO and COVID-19 pandemic’s psychological impact, reported a high prevalence of psychological morbidity in a cohort of university students during this pandemic [15]. Because of the limitation of face-to-face psychological interventions, there was a need for different apps for digital reformation to treat individuals with mental health problems.
COVID-19 is a very contagious disease and it spreads rapidly. There was a directive from the administration to remain at home and avoid going out. An individual could not easily access natural beauty and attractions for any purpose, including recreation. Because virtual environments can generate the required visual, cognitive, and social links to connect the individual to the living place, they can evoke favourable psychological responses [16]. Various research studies have found that VR-based therapies may be beneficial for various mental health disorders [17,18]. VR has been employed as a distraction device to overcome patients’ stress [19,20,21].
To date, there has been a paucity of information on how psychological interventions such as digital reformation, i.e., the use of apps to treat patients with COVID-19-related mental-health problems, will be helpful. The lack of published literature has added to the knowledge gap in this specialty area. The majority of studies on psychological intervention were conducted online, without an interactive session to help patients deal with their problems. To the best of our knowledge, there are no studies that assessed the psychological intervention provided by digital interactive navigation during the COVID-19 pandemic. Many literature reviews on major psychiatric illnesses, such as major depressive illness, anxiety disorders, and other stress disorders, did not include the role of VR. It is pivotal to perform a systematic review to provide a comprehensive and up-to-date review of the role of VR as a psychological intervention for mental health problems during the COVID-19 pandemic.

2. Material and Methods

This systematic review conformed to the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) guidelines [22,23] and was registered in the International Prospective Register of Systematic Reviews (PROSPERO) under registration number CRD42021265380.

2.1. Search Strategy

Search strings were developed and conducted across the following electronic databases: PsycINFO, Ovid Medline, Embase, ACM digital library, and Cochrane Central Register of Controlled Trials (CENTRAL) from December 2019. We conducted the last search on 23 June 2021. The search-related terms incorporating the Boolean and the truncation were as follows: (Virtual reality OR simulated-3D-environment OR VR) AND (covid! or corona!) AND (mental* OR psychologic* OR well* OR health*) AND (intervention).
We restricted the searches to human studies and publications in the English language. No geographical restriction was applied.

2.2. Eligibility Criteria

A study was eligible for inclusion if it met the following criteria. Regarding the study design, only primary studies such as case–control studies, cross-sectional studies, cohort studies, and randomized controlled trials were considered.
Regarding the respondents, the criteria included adult subjects with or without underlying psychological distress, including depression, stress, anxiety, and psychosis secondary to COVID-19, a relative affected by COVID-19, or being infected by COVID-19. No gender, race, or ethnicity restrictions were applied. The criteria for interventions/exposure included psychological interventions, e.g., counselling, family-based intervention, psychotherapy, behavioural therapy, and positive activity intervention (PAI) delivered using VR in an inpatient or outpatient setting. The comparator includes any conventionally delivered psychological interventions. The outcomes included improvements in psychological wellbeing. The outcomes were measured by subjective report or objectively assessed using tools such as the Depression Anxiety Stress Scale (DASS), General Health Questionnaire (GHQ), and Beck Depression Inventory (BDI).
The exclusion criteria included (i) studies conducted on children and adolescents; (ii) intervention in the form of other immersive technology, e.g., augmented reality (AR); (iii) abstracts, conference proceedings, case reports, case series, and reviews; and (iv) studies that could not report the outcome of interest.

2.3. Selection Process

We exported all search results into a reference management software, Covidence [24]. The program automatically takes out all duplicate studies. Two independent reviewers (MH and CSK) screened titles and abstracts for eligibility. The full texts of the eligible records were then obtained, and screened for eligibility according to the PICOS framework with the help of two independent reviewers. To achieve consensus, we resolved any doubts or conflicts by discussion with a third reviewer (H.S.).

2.4. Methodological Quality

We assessed the quality of each of the included studies according to the McMaster Critical Appraisal Tool for quantitative studies. We also added questions on the randomisation for quality assessment of randomised-controlled trials (Appendix A). Each question was rated with a “yes,” “no,” “not addressed,” or “not applicable.” Every “yes” answer was given a score of 1 point, and the overall score differed depending on the study design. The intervention category of the Australian National Health and Medical Research Council’s (NHMRC) evidence hierarchy was also used to determine the level of evidence in the included studies.

2.5. Selection Process

We exported all search results to reference management software, Covidence (Covidence systematic review software, Veritas Health Innovation, Melbourne, Australia; available at www.covidence.org, accessed on 28 July 2021). The program automatically removes duplicate studies upon import. Titles and abstracts were screened, and then full texts were analysed for eligibility following the inclusion criteria by two independent reviewers (M.H., C.S.K. and F.D.). Any doubts or conflicts were resolved by discussion between the two reviewers, and we thus reached a consensus.

2.6. Data Extraction

Three independent reviewers (M.H., C.S.K. and F.D.) performed a data extraction standardized form created in Microsoft Excel (Microsoft Corporation, Redmond, WA, USA). The first author, year of publication, number of participants, age, country of origin, study setting, exposure, comparator, length of follow-up, and outcome were extracted from each included study. We contacted the authors via email to obtain additional information if necessary. Any discrepancies in the extracted data were resolved by a third reviewer (H.S.).

2.7. Methodological Quality

We assessed the quality of each of the included studies according to the McMaster Critical Appraisal Tool for quantitative studies [25]. A revised version was produced by removing the domain-assessing interventions, as they were not relevant to this review. Questions on randomisation were also added for the quality assessment of randomised-controlled trials (Appendix A). Each question was rated “yes,” “no,” “not addressed,” or “not applicable.” Every “yes” answer was given a score of 1 point, and the overall score differed depending on the study design. The intervention category of the Australian National Health and Medical Research Council’s (NHMRC) evidence hierarchy [26] was also used to determine the level of evidence of the included studies.

2.8. Data Synthesis

The study quality and characteristics of interest were tabulated and narratively described.

3. Results

Study Characteristics

Out of the 379 references imported for screening, 29 duplicates were removed. A total of 350 studies were screened against titles and abstracts. Subsequently, 302 studies were excluded, and among the 48 studies assessed for full-text eligibility, 44 studies were excluded because of the following reasons, i.e., wrong study design (21) (e.g., case reports), wrong intervention (6) (e.g., augmented reality (AR), face-to-face consultation), wrong outcomes (6) (e.g., education satisfaction), wrong patient population (5) (e.g., healthy or not related to mental health), wrong setting (2) (outside the time frame of the COVID-19 pandemic), no outcome findings (1) (e.g., research protocol with no outcome results), trials registration (1), wrong indication (1), and commentators (1). A total of four studies were finally included (Figure 1). We summarized the key results of the study characteristics, which include information about the intervention, its type, duration, setting, and effectiveness in Table 1.

4. Discussion

To the best of our knowledge, this study was the first review to explore the role of VR as a psychological intervention for individuals with mental health problems. We included important studies by Yang et al. (2021), Siani et al. (2021), Waller et al. (2021), and Kolbe et al. (2021) in our final search criteria based on the systematic review study [27,28,29,30]. We could not synthesize the data for a meta-analysis because of the vast differences (heterogeneity) between the studies, i.e., different populations, assessment tools, different outcomes, and the duration of studies.
Regarding the four studies, their quality was fair to good. The studies by Yang et al. (2021), Siani et al. (2021), Waller et al. and Kolbe et al. (2021) [27,28,29,30] showed fair (both cross-sectional study and comparative study without concurrent controls, type III) and good (RCT, type II) quality, respectively.

4.1. The Role of 360° Virtual Tour in Psychological Stress Reduction

This study constructed a four-level model to examine how a 360° virtual tour can reduce people’s psychological stress on two types of presence, i.e., the sense of existence or telepresence, and affective-motivational, referring to the emotions and rewards during the extraordinary period of the COVID-19 pandemic. Yang et al. (2021) [30] employed the partial least squares (PLS) analysis to test the moderating effect of a person’s involvement among two hundred and thirty-five respondents. This study showed that telepresence (TP) had a more significant impact on generating affective-motivational states (AMS) than the sense of presence. Among the factors, enjoyment [31,32,33] showed the highest effect on satisfaction with the 360° virtual tour experience and stress reduction. The engagement of an experience and AMS moderated the impact of TP on satisfaction with the 360° virtual tour adventure. Yang et al. (2021) [30] reported VR research outcomes by differentiating between the concepts of ‘sense of presence’ and ‘telepresence’ and demonstrating how VR technology can influence an individual’s psychological and mental well-being.
For practical implications, when designing a 360° virtual tour, many software developers (SD) emphasised the experience of a ‘real’ situation in the VR platform. However, users have a sense of ‘being there’ instead of being in the ‘real place.’ Just like the makers of Marvel/Warner Bros. cinematics, the SD has to plan to produce a sense of ‘being there’ among the viewers to encourage viewers to watch their movies. Respondents to a tourist destination are looking for the importance of ‘being there’, as more than a ‘real’ presence in their memory destination. As they discovered, they enjoyed 360° content more as an ‘ideal place’ destination. For example, they were watching a turquoise sky and a vibrant rainbow. These virtual tour experiences they could never see frequently in the ‘real place’ than in the ‘ideal place’ are gratifying experiences. The enjoyment of physical activity in immersive VR is another way to benefit from digital health creativity [34]. They retained their experiences in the post-processed memory effects. The AMS is considered the response to the 360° virtual tour content. When exposed to the ‘ ideal place’, the respondents could have more gratification and involvement (based on audio, screen clarity, and even the sense of smell of the real places) as part of the AMS process.
The SD has to design the 360° virtual tour experience as an immersive experience as much as possible so that participants can imagine themselves as tourists. VR developers could co-design with the respondents to integrate their interests and improve the flow elements of the 360° virtual tour content. It is time to recommend reducing psychological stress during and after the COVID-19 pandemic. The 360° virtual tour also offers an experience for senior citizens with mobility issues [30]. For instance, senior citizens who cannot take a long-distance trip may benefit from the virtual tour and help promote an old person’s mental well-being.

4.2. VR Video Games as a Form of Recreational Use during the Lockdown

Siani et al. [28] aimed to evaluate the effects of VR activities on users under lockdown because of the COVID-19 pandemic. They investigated the recreational use of VR during the lockdown period and gathered users’ opinions on its impact on their physical and mental healths. The researchers employed non-parametric tests to evaluate the statistical significance of the responses provided by the 646 participants. The survey results showed that VR use significantly increased during the lockdown period for most participants, who expressed overwhelmingly positive opinions on the impact of VR activities on their mental and physical wellbeing. Interestingly, the self-reported intensity of physical activity was considerably more strenuous in VR users than in console users.
Video games were the most popular VR activity [28], followed by physical fitness, social media (SM), videos, and meditation. Most of the respondents spent 1–4 h per day playing VR video games and up to 2 h/day using VR for physical fitness. Most of the respondents used up between 1 to 4 h a day for playing VR video games and a VR for physical fitness up to 2 h/day. Interestingly, VR video games require substantial physical effort. Less than 20% of those polled spent more than an hour/day using VR for video, social media, or meditation activities. Even with brief VR sessions, such as a 10 min 3D 360° video daily for a week, regular engagement could relieve lockdown-induced boredom and anxiety and subsequently foster a positive psychological state. Regardless of gender and age, VR use increased during lockdown and is associated with a positive opinion on the usefulness of VR to keep respondents busy and improve their psychological and physical well-being. Siani et al. (2021) [28] found a variation in self-reported workout intensity between respondents using gaming consoles and VR headgear. They linked the latter to a significantly more intense exercise than the former. These findings imply VR may be a more effective device in promoting training/activity than traditional consoles.
Siani et al. (2021) [28] provided novel insights into how the recreational use of VR can successfully alleviate the negative impact of lockdown periods on the population’s mental and physical wellbeing. VR activities help users keep themselves occupied and physically active under the restrictions imposed by the lockdown. VR headsets have become a mainstream entertainment device in many households, because of their increasingly affordable price and high-tech accessibility. Therefore, researchers, policymakers, and healthcare workers should consider designing and implementing intervention strategies as potential aids in public health to mitigate the negative consequences of prolonged lockdown periods. Providing the population with the means to engage in VR activities to keep them occupied and physically fit could be a promising strategy to minimise the decline in mental and physical well-being. Therefore, this digital health intervention might ease the current pressure on healthcare and medical workers.
Given the current uncertainty regarding the duration and course of the pandemic, as well as the possibility of intermittent lockdowns in the upcoming years, the outcome of this study could have a significant impact on the development and deployment of VR-based strategies which aim to help the population cope with prolonged social distancing, particularly with regard to vulnerable individuals.

4.3. Meditative Effect of 3D vs. 2D Format

This strategy by Waller et al. (2021) [27] includes meditation instruction. In VR, a person feels the physical presence of an instructor, although this may be lessened when the interventions are employed remotely. This situation might affect one’s meditative experiences. The use of head-mounted displays (HMD) to demonstrate video-recorded instruction (VRI) may enhance one’s sense of psychological presence with the instructor as compared to the presentation via a regular flatscreen (e.g., laptop) monitor. Waller et al. (2021) [27] studied the role of meditating by employing VR in a 360° video of perception in the presence of an instructor.
The researchers evaluated a didactic, trauma-informed care approach to instruction in mindfulness meditation by comparing meditative responses to an instructor-guided meditation. It is delivered face-to-face versus by pre-recorded 360° videos viewed either on a standard flatscreen monitor (2D format) or via HMD (virtual reality (VR) headset; 3D arrangement). All 82 respondents were recruited from a university introductory course. They were asked to experience a 360° video-guided meditation via HMD, i.e., VR conditions in a 3D format. They were also randomly assigned to employ the same meditation either via scripted face-to-face instruction (in vivo (IV) format) or when viewed on a standard laptop display (non-VR condition, 2D design). Researchers documented respondents’ maintenance of focused breathing attention using the meditation breath attention scores (MBAS) throughout each meditation session. Meditating in VR (3D format) was associated with a heightened experience of amazement. Compared to face-to-face instruction (in vivo format), VR meditation was rated as less embarrassing, less entertaining, and extra tiring. When compared to the 2D format, VR meditations were associated with more significant relaxation experiences. The latter was associated with less distractibility from the practice of breathing and less exhaustion. Based on MBAS, there were no significant differences found between VR and non-VR meditation in the respondents’ concentration. Baseline post-traumatic stress symptoms were risk factors for experiencing distress while meditating in either a VR/3D or non-VR/2D instructional format. Approximately half preferred the VR format and recorded 360° video instruction in meditation viewed with an HMD (i.e., VR/3D form), which appears to offer some experiential advantage over instructions given in 2D format. VR meditation may provide a safe—and even preferred—alternative to face-to-face teaching meditation.

4.4. The VR Program Implemented in COVID-19 Recovery Unit (CRU)

Based on Kolbe et al. (2021) [29], VR has seen extended use in acute healthcare and clinical settings in recent years. Many mental health issues were present, including psychoneurological sequelae, adjustment disorder, depression, and stress [4]. The study by Kolbe et al. (2021) [29] suggested that the innovative use of VR is one modality of a part of rehabilitative CRU due to its ability to alleviate distress. Kolbe et al. (2021) [29] discovered positive patient fulfilment and a perceived benefit of VR on patients and medical staff in the CRU, as well as demonstrated the feasibility in logistical functioning and VR content delivery. During the COVID-19 surge in New York City in 2020, the CRU in a large teaching hospital included respondents with three categories of interactive experience, i.e., guided meditation, exploration of natural environments, and cognitive stimulation games.
The participants were surveyed regarding satisfaction and whether they perceived and benefited from the sessions. The study found that 13 patients and 11 staff reported median patient satisfaction scores of 9 out of 10. Ten patients showed to be “extremely satisfied,” with a median staff satisfaction score of 10. All patients responded “yes” to endorsing the VR intervention. A total of 92.3% of patients approved the perceived enhancement of their therapy. A total of 100% of staff agreed to recommend the intervention to others. They also agreed to the role of VR as a perceived augmentation of their well-being. Kolbe et al. (2021) [29] reported that both patients and healthcare providers were extremely satisfied with the perceived benefit of the VR program implemented on a CRU. Participants stated that the use of VR was valuable and beneficial in coping with the sense of isolation and loneliness. They also stated other reasons, as depicted in Figure 2.
VR could be implemented within medical care for COVID-19 patients as an ideal rehabilitation model. Using VR was also logistically and operationally feasible in the CRU. Future work to compare the positive benefits of VR to a novel neuropsychological intervention and rehabilitation is needed.
In summary, Kolbe et al. (2021) [29] discovered that a VR program implemented on a CRU is extremely satisfying for both patients and staff, with perceived benefits for improving patient treatment and healthcare staff well-being.
The acceptance of VR use in the CRU has not only made patients more interested in the ongoing use but has also generated staff interest in the expansion of VR use in inpatient therapeutic settings. The next challenge would conduct a more extensive prospective study to assess whether exposure to the VR content could promise better outcomes, such as mood elevation, anxiety control, sleep regulation, pain management, and a reduction in feelings of isolation.

4.5. Shared Elements for VR in the Psychological Intervention for a Patient with Mental Health Problems

The studies by Yang et al. (2021), Siani et al. (2021), Kolbe et al. (2021), and Waller et al. (2021) [27,28,29,30] shared similar or related elements in the role of VR as a psychological intervention for a patient with mental health problems. The studies concluded that VR had a beneficial effect as a tool for digital intervention among distressed individuals. In this systematic review, we found that immersion plays a crucial role in the mediation of therapeutic benefits [29]. “Immersion” refers to the perception of being physically present in a non-physical world [35]. The encircling environment experienced by the user-generated perception in the VR system will interact with stimuli such as images and sounds that provide an engaging atmosphere. The interactive experience between the users and the digital platform in a 3-D setting (VR design) is preferable to a 2-D scene (i.e., flat screen). The VR platform as a digital health intervention is also helpful for users who want a sense of self-empowerment (ability to perform the intervention at their convenience or leisure) vs. predetermined regimented therapy (performing the intervention at a given specific time and by a particular instructor or therapist). This freedom and flexibility in terms of time and space are crucial for the use of VR as a tool for in-demand psychological intervention, especially among distressed individuals who are constrained by busy schedules and the need to attend a face-to-face consultation.
Our review also found that engaging in the VR scenario and interacting with the stimuli during the immersion process is pivotal to the beneficial effects of the psychological intervention [36]. Activities such as gaming, guided medication, virtual tours, and probably gauging the biological response could be helpful in the process of the intervention. However, there is a knowledge gap between how long a person should engage in the VR platform (i.e., 30 min or more) and what kind of physical activity in the VR system would provide benefits (gamification, fitness activity, or walking on a virtual tour)? Additionally, what kind of activity could be helpful to gauge users’ biological responses? Is it the respiratory rate, blood pressure, or pulses under different techniques and approaches (control breathing technique vs. guided medication) that would be helpful?
The future direction of studies on the role of VR for psychological intervention for an individual with mental health issues should complement the existing body of knowledge that VR could be necessary for a more traditional style of face-to-face consultation settings [37,38,39,40,41]. VR may serve as a reachable and immersive way to bring practical clinical interventions to hospitalized patients, mainly during the ongoing COVID-19 pandemic across the world. It is pivotal to take note that from our systematic review findings, i.e., being immersive in the telepresence settings, engaging in an interactive process in a 3-dimensional format, engaging and activating an affective-motivational state by “escaping” to a virtual world are important elements to contemplate in VR software development for a psychological tool intervention.

4.6. Limitations of VR as a Digital Health Intervention: For Patients and the Healthcare Staff

Regarding the users who use the VR intervention such as healthcare staff, Kolbe et al. (2021) [29] outlined further limitations, including staff time pressure when medical and healthcare personnel are involved. This situation may pose a practical restriction on their utilization of the VR platform as a digital health intervention. VR is undoubtedly a unique context for patients via a self-directed experience rather than within a therapeutic session [42]. VR is affordable for many people, but not all patients are accessible through these treatment modalities [43]. For example, in a remote place with internet connection problems, the use of VR is minimal, or if the Wi-Fi connection is slow, the intervention could be inadequate.

4.7. Strength and Limitations of the Study

To the best of our knowledge, this is the first systematic review investigating the role of psychological intervention among individuals with mental health problems. The thoroughness of the findings is one of the major advantages of this study. There was a systematic literature search into all accessible resources, minimizing selection bias, and avoiding subjective selection bias. On the other hand, systematic reviews would provide with experts’ intuitive, experiential, and explicit perspectives on focused topics when narrated by experts in specific research areas [44]. In an earlier narrative review, the beneficial effects of VR were stated to promote positive well-being [44].
As with all other studies, the value of a systematic review may depend on what it accomplished in the research, i.e., what was observed, and the clarity of information reporting. The observation of the quality of systematic reviews varies based on the robustness of the included studies, which may limit the readers’ ability to assess the strengths and weaknesses of each inclusive study [45]. Even though systematic reviews are deemed to be the best evidence for making the best answer to a research question, there are certain intrinsic flaws related to them, such as the selection bias and heterogeneity of the included studies, i.e., diverse objectives outcomes (perceived self-satisfaction vs. relaxed state), different study settings (inpatient setting vs. online survey), different measurement tools (perceived self-rating assessment vs. qualitative thematic analysis), and different design platforms (2-D vs. 3-D), wherein a meta-analysis cannot be synthesized.

5. Conclusions

The COVID-19 pandemic has caused significant changes to most aspects of our lives. Because of the quarantine enforced by governments and authorities worldwide, people suddenly had to adapt their daily routines, including work, study, diet, leisure, and fitness activities, to the new circumstances. There is a necessity for the remote delivery of psychological and mental health interventions such as relaxation techniques, breathing exercises, and biofeedback.
This pandemic has inspired the need for creativity and substitution in medical and healthcare benefit delivery modalities. In addition, the following stages would seek to distinctly measure the comparative effects of diverse types of modules within the VR system, i.e., in vivo (traditional face-to-face) vs. virtual reality. Types of the frequency use (self-empowerment vs. default/regimented program), number of users (single vs. multiple players), the schema of the interaction (didactic coaching vs. free and easy trip, i.e., 360° virtual tours) are numbers of the practical troubleshooting that need to be solved. This step is essential to regulating patient satisfaction in a user-friendly, affordable, and medically feasible therapeutic modality. As we are not sure which mental health problems, i.e., mood disorders, anxiety, or psychosis maybe benefit from digital technology intervention, further studies are pivotal to measure specific mental health problem outcomes. We suggest embarking on a specific intervention for each mental health problem in order to evaluate and determine the effectiveness of the digital technology intervention.

Author Contributions

Conceptualization—M.H.H., H.S., C.S.K., S.M.S., N.A.C.R., S.S., F.D.A.S., O.W.X. and S.M.S.; methodology—M.H.H., H.S., C.S.K., S.S., F.D.A.S. and S.M.S.; writing—M.H.H., H.S., C.S.K., N.A.C.R., S.S., F.D.A.S., O.W.X., S.M.S. and S.D.; editing—H.S. and S.D. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

This study was approved by Universiti Kebangsaan Malaysia Research Ethics Secretariat (protocol code: NF 21-155 and date of approval: 29 December 2021).

Informed Consent Statement

Not applicable.

Conflicts of Interest

The authors declare no conflict of interest.

Appendix A. Modified McMaster Critical Review Form

McMaster Critical Review Form is modified to suit the studies included in this review, which are mainly etiological studies, so the intervention domain was taken out. Two questions are added for quality assessment of prospective cohort studies, i.e., were the groups randomized, and was randomization appropriately done. Providing 1 point for every answer of “Yes”, making the total score 13. In the case of non-prospective cohort studies, those 2 questions added will be not applicable (NA) and the total score will be reduced to 11.
Table
DomainsYesNoNot AddressedNot Applicable
Study purposeYesNoNaNA
1. Was the purpose stated clearly?
Literature reviewYesNoNaNA
2. Was relevant background literature reviewed?
SampleYesNoNaNA
3. Was the sample described in detail?
4. Was sample size justified?
5. Were the groups randomized?
6. Was randomization appropriately done?
OutcomeYesNoNaNA
7. Were the outcome measures reliable?
8. Were the outcome measures valid?
ResultsYesNoNaNA
9. Were results reported in terms of statistical significance?
10. Were the analysis of method(s) appropriate?
11. Was clinical importance reported?
12. Were drop-outs reported?
ConclusionsYesNoNaNA
13. Were conclusions appropriate, given study methods and results?

References

  1. Kumar, A.; Singh, R.; Kaur, J.; Pandey, S.; Sharma, V.; Thakur, L.; Sati, S.; Mani, S.; Asthana, S.; Sharma, T.K.; et al. Wuhan to World: The COVID-19 Pandemic. Front. Cell. Infect. Microbiol. 2021, 11, 596201. [Google Scholar] [CrossRef]
  2. Cascella, M.; Rajnik, M.; Aleem, A.; Dulebohn, S.C.; Di Napoli, R. Features, Evaluation, and Treatment of Coronavirus (COVID-19); StatPearls Publishing: Treasure Island, FL, USA, 2022. [Google Scholar]
  3. New Straits Times. COVID-19: Movement Control Order Imposed with Only Essential Sectors Operating. 2020. Available online: https://web.archive.org/web/20200316224520/https://www.nst.com.my/news/nation/2020/03/575177/covid-19-movement-control-order-imposed-only-essential-sectors-operating (accessed on 3 December 2021).
  4. Chu, D.K.; Akl, E.A.; Duda, S.; Solo, K.; Yaacoub, S.; Schünemann, H.J. COVID-19 Systematic Urgent Review Group Effort (SURGE) study authors. Physical distancing, face masks, and eye protection to prevent person-to-person transmission of SARS-CoV-2 and COVID-19: A systematic review and meta-analysis. Lancet 2020, 395, 1973–1987. [Google Scholar] [CrossRef]
  5. Chi, X.; Becker, B.; Yu, Q.; Willeit, P.; Jiao, C.; Huang, L.; Hossain, M.M.; Grabovac, I.; Yeung, A.; Lin, J.; et al. Prevalence and Psychosocial Correlates of Mental Health Outcomes among Chinese College Students during the Coronavirus Disease (COVID-19) Pandemic. Front. Psychiatry 2020, 7, 803. [Google Scholar] [CrossRef]
  6. da Silva, F.C.T.; Neto, M.L.R. Psychiatric symptomatology associated with depression, anxiety, distress, and insomnia in health professionals working in patients affected by COVID-19: A systematic review with meta-analysis. Prog. Neuro Psychopharmacol. Biol. Psychiatry 2020, 104, 110057. [Google Scholar] [CrossRef]
  7. South China Morning Post. Coronavirus: Malaysia in Partial Lockdown from March 18 to Limit Outbreak. Available online: https://www.scmp.com/week-asia/health-environment/article/3075456/coronavirus-malaysias-prime-minister-muhyiddin-yassin (accessed on 10 November 2021).
  8. Berge, L.I.; Gedde, M.H.; Husebo, B.S.; Erdal, A.; Kjellstadli, C.; Vahia, I.V. Age and Emotional Distress during COVID-19: Findings from Two Waves of the Norwegian Citizen Panel. Int. J. Environ. Res. Public Health 2021, 18, 9568. [Google Scholar] [CrossRef]
  9. Cao, W.; Fang, Z.; Hou, G.; Han, M.; Xu, X.; Dong, J.; Zheng, J. The psychological impact of the COVID-19 epidemic on college students in China. Psychiatry Res. 2020, 287, 112934. [Google Scholar] [CrossRef]
  10. Cai, H.Z.; Tu, B.R.; Ma, J.; Chen, L.M.; Fu, L.; Jiang, Y.F.; Zhuang, Q. Psychological impact and coping strategies of frontline medical staff in Hunan between January and March 2020 during the outbreak of Coronavirus Disease 2019 (COVID-19) in Hubei, China. Med. Sci. Monit. 2020, 26, e924171-1–e924171-16. [Google Scholar]
  11. Liu, X.; Luo, W.T.; Li, Y.; Li, C.N.; Hong, Z.S.; Chen, H.L.; Xiao, F.; Xia, J.Y. Psychological status and behavior changes of the public during the COVID-19 epidemic in China. Infect. Dis. Poverty 2020, 9, 58. [Google Scholar] [CrossRef]
  12. Liu, S.; Yang, L.; Zhang, C.; Xiang, Y.T.; Liu, Z.; Hu, S.; Zhang, B. Online mental health services in China during the COVID-19 outbreak. Lancet Psychiatry 2020, 7, e17–e18. [Google Scholar] [CrossRef]
  13. Xiao, H.; Zhang, Y.; Kong, D.; Li, S.; Yang, N. Social Capital and Sleep Quality in Individuals Who Self-Isolated for 14 Days during the Coronavirus Disease 2019 (COVID-19) Outbreak in January 2020 in China. Med. Sci. Monit. 2020, 26, e923921. [Google Scholar] [CrossRef]
  14. Forte, G.; Favieri, F.; Tambelli, R.; Casagrande, M. COVID-19 Pandemic in the Italian Population: Validation of a Post-Traumatic Stress Disorder Questionnaire and Prevalence of PTSD Symptomatology. Int. J. Environ. Res. Public Health 2020, 17, 4151. [Google Scholar] [CrossRef]
  15. Sundarasen, S.; Chinna, K.; Kamaludin, K.; Nurunnabi, M.; Mohammad Baloch, G.; Khoshaimet, H.B.; Abid Hossain, S.F.; Sukayt, A. Psychological impact of COVID-19 and lockdown among university students in Malaysia: Implications and policy recommendations. Int. J. Environ. Res. Public Health 2020, 17, 6206. [Google Scholar] [CrossRef]
  16. Zhai, K.; Dilawar, A.; Yousef, M.S.; Holroyd, S.; El-Hammali, H.; Abdelmonem, M. Virtual Reality Therapy for Depression and Mood in Long-Term Care Facilities. Geriatrics 2021, 6, 58. [Google Scholar] [CrossRef]
  17. Valmaggia, L.R.; Latif, L.; Kempton, M.J.; Rus-Calafell, M. Virtual reality in the psychological treatment for mental health problems: An systematic review of recent evidence. Psychiatry Res. 2016, 236, 189–195. [Google Scholar] [CrossRef] [Green Version]
  18. Dellazizzo, L.; Potvin, S.; Luigi, M.; Dumais, A. Evidence on Virtual Reality-Based Therapies for Psychiatric Disorders: Meta-Review of Meta-Analyses. J. Med. Internet Res. 2020, 22, e20889. [Google Scholar] [CrossRef]
  19. Tashjian, V.C.; Mosadeghi, S.; Howard, A.R.; Lopez, M.; Dupuy, T.; Reid, M.; Martinez, B.; Ahmed, S.; Dailey, F.; Robbins, K.; et al. Virtual reality for management of pain in hospitalized patients: Results of a controlled trial. JMIR Ment. Health 2017, 4, e7387. [Google Scholar] [CrossRef]
  20. Bani Mohammad, E.; Ahmad, M. Virtual reality as a distraction technique for pain and anxiety among patients with breast cancer: A randomized control trial. Palliat. Support. Care 2019, 17, 29–34. [Google Scholar] [CrossRef]
  21. Pourmand, A.; Davis, S.; Marchak, A.; Whiteside, T.; Sikka, N. Virtual reality as a clinical tool for pain management. Curr. Pain Headache Rep. 2018, 15, 53. [Google Scholar] [CrossRef]
  22. Page, M.J.; McKenzie, J.E.; Bossuyt, P.M.; Boutron, I.; Hoffmann, T.C.; Mulrow, C.D.; Shamseer, L.; Tetzlaff, J.M.; Akl, E.A.; Brennan, S.E.; et al. The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. BMJ 2021, 372, n71. [Google Scholar] [CrossRef]
  23. Cohen, J.F.; Deeks, J.J.; Hooft, L.; Salameh, J.P.; Korevaar, D.A.; Gatsonis, C.; Hopewell, S.; Hunt, H.A.; Hyde, C.J.; Leeflang, M.M.; et al. Preferred reporting items for journal and conference abstracts of systematic reviews and meta-analyses of diagnostic test accuracy studies (PRISMA-DTA for Abstracts): Checklist, explanation, and elaboration. BMJ 2021, 372, n265. [Google Scholar] [CrossRef]
  24. Covidence Systematic Review Software, Veritas Health Innovation, Melbourne, Australia. Available online: www.covidence.org (accessed on 20 February 2022).
  25. Law, M.; Stewart, D.; Letts, L.; Pollock, N.; Bosch, J.; Westmorland, M. Guidelines for Critical Review of Qualitative Studies. 1998. Available online: https://medfac.tbzmed.ac.ir/Uploads/3/cms/user/File/10/Pezeshki_Ejtemaei/conferance/dav.pdf (accessed on 20 January 2022).
  26. Barratt, A.; Ragg, M.; Irwig, L.; Swinburne, L.; Chapman, S. How to Prepare and Present Evidence-Based Information for Consumers of Health Services: A Literature Review; National Medical Health and Medical Research Council: Canberra, Australia, 1999. Available online: https://www.nhmrc.gov.au/sites/default/files/2018-03/How-to-prepare-and-present-evidence-based-info-for-consumers-of-health-services.pdf (accessed on 20 February 2022).
  27. Waller, M.; Mistry, D.; Jetly, R.; Frewen, P. Meditating in Virtual Reality 3: 360° Video of Perceptual Presence of Instructor. Mindfulness 2021, 12, 1424–1437. [Google Scholar] [CrossRef]
  28. Siani, A.; Marley, S.A. Impact of the recreational use of virtual reality on physical and mental wellbeing during the COVID-19 lockdown. Health Technol. 2021, 11, 425–435. [Google Scholar] [CrossRef]
  29. Kolbe, L.; Jaywant, A.; Gupta, A.; Vanderlind, W.M.; Jabbour, G. Use of virtual reality in the inpatient rehabilitation of COVID-19 patients. Gen. Hosp. Psychiatry 2021, 71, 76–81. [Google Scholar] [CrossRef]
  30. Yang, T.; Lai, I.K.W.; Fan, Z.B.; Mo, Q.M. The impact of a 360° virtual tour on the reduction of psychological stress caused by COVID-19. Technol. Soc. 2021, 64, 101514. [Google Scholar] [CrossRef]
  31. Kim, D.; Ko, Y.J. The impact of virtual reality (VR) technology on sport spectators’ flow experience and satisfaction. Comput. Hum. Behav. 2019, 93, 346–356. [Google Scholar] [CrossRef]
  32. Kim, M.J.; Hall, C.M. A hedonic motivation model in virtual reality tourism: Comparing visitors and non-visitors. Int. J. Inf. Manag. 2019, 46, 236–249. [Google Scholar] [CrossRef]
  33. Kimiecik, J.C.; Harris, A.T. What is enjoyment? A conceptual/definitional analysis with implications for sport and exercise psychology. J. Sport Exerc. Psychol. 1996, 18, 247–263. [Google Scholar] [CrossRef]
  34. Dębska, M.; Polechoński, J.; Mynarski, A.; Polechoński, P. Enjoyment and Intensity of Physical Activity in Immersive Virtual Reality Performed on Innovative Training Devices in Compliance with Recommendations for Health. Int. J. Environ. Res. Public Health 2019, 16, 3673. [Google Scholar] [CrossRef] [Green Version]
  35. Van Damme, K.; All, A.; De Marez, L.; Van Leuven, S. 360° Video journalism: Experimental study on the effect of immersion on news experience and distant suffering. J. Stud. 2019, 20, 2053–2076. [Google Scholar] [CrossRef]
  36. Alyan, E.; Combe, T.; Rambli, D.R.A.; Sulaiman, S.; Merienne, F.; Diyana, N. The Influence of Virtual Forest Walk on Physio-logical and Psychological Responses. Int. J. Environ. Res. Public Health 2021, 18, 11420. [Google Scholar] [CrossRef]
  37. Kefaliakos, A.; Pliakos, I.; Kiekkas, P.; Charalampidou, M.; Diomidous, M. Virtual reality in the rehabilitation of patients with neurological disorders. Stud. Health Technol. Inform. 2016, 226, 45–47. [Google Scholar]
  38. O’Neil, O.; Fernandez, M.M.; Herzog, J.; Beorchia, M.; Gower, V.; Gramatica, F.; Starrost, K.; Kiwull, L. Virtual reality for neurorehabilitation: Insights from 3 European clinics. PM&R 2018, 10, S198–S206. [Google Scholar] [CrossRef]
  39. Gupta, A.; Scott, K.; Dukewich, M. Innovative technology using virtual reality in the treatment of pain: Does it reduce pain via distraction, or is there more to it? Pain Med. 2018, 19, 151–159. [Google Scholar] [CrossRef] [Green Version]
  40. Zeidan, F.; Vago, D.R. Mindfulness meditation-based pain relief: A mechanistic account. Ann. N. Y. Acad. Sci. 2016, 1373, 114–127. [Google Scholar] [CrossRef]
  41. Zeidan, F.; Grant, J.A.; Brown, C.A.; McHaffie, J.G.; Coghill, R.C. Mindfulness meditation related pain relief: Evidence for unique brain mechanisms in the regulation of pain. Neurosci. Lett. 2012, 520, 165–173. [Google Scholar] [CrossRef] [Green Version]
  42. Hwang, N.K.; Shim, S.H. Use of Virtual Reality Technology to Support the Home Modification Process: A Scoping Review. Int. J. Environ. Res. Public Health 2021, 18, 11096. [Google Scholar] [CrossRef]
  43. Alvarez-Lopez, F.; Maina, M.F.; Saigí-Rubió, F. Use of a Low-Cost Portable 3D Virtual Reality Gesture-Mediated Simulator for Training and Learning Basic Psychomotor Skills in Minimally Invasive Surgery: Development and Content Validity Study. J. Med. Internet Res. 2020, 22, e17491. [Google Scholar] [CrossRef]
  44. Hatta, M.H.; Sidi, H.; Sharip, S.; Das, S.; Saini, S.M. The Role of Virtual Reality as a Psychological Intervention for Mental Health Disturbances during the COVID-19 Pandemic: A Narrative Review. Int. J. Environ. Res. Public Health 2022, 19, 2390. [Google Scholar] [CrossRef]
  45. Moher, D.; Liberati, A.; Tetzlaff, J.; Altman, D.G.; PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. PLoS Med. 2009, 6, e1000097. [Google Scholar] [CrossRef] [Green Version]
Ijerph 19 05389 g001 550
Figure 1. Systematic literature search, screening, and relevance assessment conducted according to PRISMA guidelines.
Figure 1. Systematic literature search, screening, and relevance assessment conducted according to PRISMA guidelines.
Ijerph 19 05389 g001
Ijerph 19 05389 g002 550
Figure 2. Examples of some patients’ suggestion in response to their VR experiences.
Figure 2. Examples of some patients’ suggestion in response to their VR experiences.
Ijerph 19 05389 g002
Table
Table 1. Characteristics of the included studies.
Table 1. Characteristics of the included studies.
Study by AuthorsType of Study Sample SizeAgeCountrySettingInclusion CriteriaExclusion CriteriaInterventionExposure Measurement ScaleOutcome MeasurementComparator/
Control		Statistics (e.g., OR/RR, p-Value, 95% CI)VR-Based Intervention
(Outcome)
Waller 2021 [27] RCT 3217–28CanadaSetting was
not defined		Not definedRespondents were not controlled. The respondents underwent evaluation for life events, childhood events, traumatic events, PTSD, and life experiences before commencing the studyThe non-VR group was exposed to (1) conventional face-to-face (in vivo (IV) method), (2) pre-recorded 360° video viewed by a standard laptop computer monitor (2D format), and (3) pre-recorded 360° video viewed through an HMD (VR condition; 3D format)A revised evaluation of the emotional questionnaire, Buddhist Affective States, Meditation Breath Attention Scores, and Meditative Experience QuestionnaireEncountering relaxation, less distractibility from the
process of breathing, and being less fatigue		3D (VR) vs. 2D formatQualitative thematic
analysis		↑
When
compared to the 2D format, VR meditations were associated with a more significant outcome
Siani 2021 [28]Cross-sectional64618–40UK-based with multiple countries’ participationOnline survey on researcher’s personal Facebook and Twitter, Facebook group (Virtual Reality Society, Oculus Virtual Reality), and Reddit channels Not definedNot definedMainly VR-video
games play		N/AIncreased use of VR during quarantine, to study the impact on mental health, devices type, and fitness intensityControl (waiting list) vs. Two-Group Random Assignment Pretest–Posttest DesignMajority were positive about the usefulness of VR for fitness (χ2 = 185.21,df = 4, p < 0.001) and mental health (χ2 = 416.27, df = 4, p < 0.001). The majority of both VR (48.4%) and console (42.1%) users engaged with moderate intensity. A greater proportion of VR users engage in vigorous activity (43.0%) than mild activity (8.6%), a trend which is reversed in console users (38.0% mild, 19.8% vigorous)↑
Kolbe 2021 [29]Cross-sectional24
(13 patients and 11 staff from COCID-19 Rehabilitation Unit (CRU), respectively		N/AUSACOCID-19 Rehabilitation Unit (CRU)(1) Hospitalized patients with +ve COVID-19 PCR test
(2) Medical team deems the patient physically stable and has ongoing medical and
rehabilitative needs
(3) Able to tolerate >30+ min physical therapy (PT)/occupational therapy (OT) each daily
(4) PT or OT recommendation for acute/subacute rehabilitation at the time of discharge
(5) Anticipation of remaining in hospital/rehabilitation for ≥1 week		(1) Sexually not active, severe dementia and active delirium, or 1:1 sitter
(2) must have non-invasive O2 needs of 6 L or fewer, or in case of tracheostomy
patients have achieved “trach collaring” with anticipated ability to downsize/
decannulate		(1) Guided meditation, (2) exploration of natural environments, (3) cognitive stimulation gameA yes or no simple rating scale of 1–10 scores where 10 indicates the highest satisfaction and highest recommendationSatisfaction, perceived enhancementCRU inpatients and staffFor patients:
100% of patients answered “yes” to
recommending the therapy to others, and 92.3% answered “yes” to the perceived enhancement of their treatment;
For staff:
100% of staff answered “yes” to recommending the therapy to others, and 100% answered “yes” to perceived
enhancement of their wellbeing		↑
The use of VR led to
significant decreases in participants’ psychological stress
Yang 2021 [30]Cross-sectional235>18ChinaLocal populace
(in a shopping mall in Zhuhai City)		Not definedNot definedValidation of a theoretical model of the 360 degrees VR: A theoretical construct comprising the following factors: EN, IN, SA, SP, SR, TP are strongly related to each other and may help reduce stress from the COVID-19 pandemicA newly designed questionnaire (translated and back-translated from English to Chinese) on the following features:
(1) Introduction to the 360◦ virtual tours, and then the
respondents were asked to watch a short video of the 360° virtual tours;
(2) Measurement on the stress reduction of the
research model;
(3) Recorded the respondents’ demographic
characteristics such as gender, age, marital status, education, income,
occupation, and so on;
(4) Measuring stress
reduction as a result of using the 360° virtual tours		Enjoyment (EN) Involvement (IN) Satisfaction (SA) Sense of presence (SA) Stress Reduction (SR) Telepresence (TP) factorsNo control group definedPLS-SEM: The sense of presence (SP) and their level of enjoyment (EN) (β = 0.221, t-statistics (t) = 2.256), (SP) with involvement (IN) (β = 0.250, t-statistics = 3.224), SA and SP (β = 0.289, t = 4.099) TP to EN (β = 0.528, t = 5.411) TP to IN(β = 0.466, t = 6.028), TP to SA (β = 0.235, t = 3.246), path coefficients: EN to SA to stress reduction of COVID 0.268
(t = 4.345) and 0.474 (t = 5.904), respectively. 0.164 IN to stress reduction of COVID-19 (t = 2.626)
and 0.158 (t = 2.093), SA to stress reduction from COVID-19, respectively: 0.196 (t = 3.116)		↑
Satisfaction with the 360° virtual tour experience and stress reduction
↑ = increased benefit and advantages; VR = virtual reality; OR = odds ratio; RR = relative risk; 95% CI = 95% confidence interval; β = Beta statistics; PLS-SEM = partial least squares structural equation modelling; LDT = letter-detection test.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Hatta, M.H.; Sidi, H.; Siew Koon, C.; Che Roos, N.A.; Sharip, S.; Abdul Samad, F.D.; Wan Xi, O.; Das, S.; Mohamed Saini, S. Virtual Reality (VR) Technology for Treatment of Mental Health Problems during COVID-19: A Systematic Review. Int. J. Environ. Res. Public Health 2022, 19, 5389. https://doi.org/10.3390/ijerph19095389

AMA Style

Hatta MH, Sidi H, Siew Koon C, Che Roos NA, Sharip S, Abdul Samad FD, Wan Xi O, Das S, Mohamed Saini S. Virtual Reality (VR) Technology for Treatment of Mental Health Problems during COVID-19: A Systematic Review. International Journal of Environmental Research and Public Health. 2022; 19(9):5389. https://doi.org/10.3390/ijerph19095389

Chicago/Turabian Style

Hatta, Muhammad Hizri, Hatta Sidi, Chong Siew Koon, Nur Aishah Che Roos, Shalisah Sharip, Farah Deena Abdul Samad, Ong Wan Xi, Srijit Das, and Suriati Mohamed Saini. 2022. "Virtual Reality (VR) Technology for Treatment of Mental Health Problems during COVID-19: A Systematic Review" International Journal of Environmental Research and Public Health 19, no. 9: 5389. https://doi.org/10.3390/ijerph19095389

APA Style

Hatta, M. H., Sidi, H., Siew Koon, C., Che Roos, N. A., Sharip, S., Abdul Samad, F. D., Wan Xi, O., Das, S., & Mohamed Saini, S. (2022). Virtual Reality (VR) Technology for Treatment of Mental Health Problems during COVID-19: A Systematic Review. International Journal of Environmental Research and Public Health, 19(9), 5389. https://doi.org/10.3390/ijerph19095389

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop