Psychoeducation on Stress and Anxiety Using Virtual Reality: A Mixed-Methods Study

Round 1

Reviewer 1 Report

This manuscript is an interesting study in the use of virtual reality in various therapeutic techniques in mental health. Important advantages, are the low cost of the methods and their contribution to minimizing stigma, as the authors comment. In addition, studies in this area are relatively limited and any further efforts are welcomed.

Comments:

1. The selection of the population is appropriate, although the sample is relatively small, combined with the large number of scales administered. Many studies show that psychoeducation must be tailored to the cognitive level of the recipient to be effective. The modules must be modified to suit the needs of patients, family members, clinicians etc. Thus a module designed for healthcare providers might not be appropriate for the general population of the study. 

2. The text of the report is unclear, lines 169-171. Was an assessment scale used? Was it a clinical criterion? Were any of the participants referred, for psychological support?

3. The link (line 184) does not refer to a website and an error response occurred.

4. Significant part of this manuscript seems to be described in Ref 30 (doi: 10.3389/frvir.2021.620225).

5. The extensive use of Likert scales might reduces the reliability of the results (doi: 10.1111/j.1365-2929.2004.02012.x).

6. The degree of knowledge of the differences between stress and anxiety seems relatively low for a population with the background of those in the study. 

7. In table 4 the SDs are very high and in conjunction with the small population size and low statistical significance, might reduce the reliability of the conclusions.

8.Some of the references could be omitted, either because they do not offer more evidence, or because they are quite outdated for such a contemporary topic.

Author Response

We thank the Editor and the Reviewers very much for the attention and important comments provided. The input was crucial in improving the quality of the manuscript.

As requested by the Editor we revised many of the parts indicated in the ithenticate report. However, it was not possible to change some sections, such as definitions, questionnaire items, technical details, affiliations, or titles of virtual experiences, which can only be the same as in previous papers.

We have revised the full text by following the directions and editing the indicated sections. In the manuscript, we have inserted in red the sections modified or added to the text, in order to make the changes made as clear as possible.  Below are the detailed answers to each comment received by Reviewer.

Reviewer 1

This manuscript is an interesting study in the use of virtual reality in various therapeutic techniques in mental health. Important advantages, are the low cost of the methods and their contribution to minimizing stigma, as the authors comment. In addition, studies in this area are relatively limited and any further efforts are welcomed.

Comments:

1. The selection of the population is appropriate, although the sample is relatively small, combined with the large number of scales administered. Many studies show that psychoeducation must be tailored to the cognitive level of the recipient to be effective. The modules must be modified to suit the needs of patients, family members, clinicians etc. Thus a module designed for healthcare providers might not be appropriate for the general population of the study.

Following this comment, we revised the Implications for Clinical Practice section, specifying the fact that the usefulness of MIND-VR for psychoeducation refers specifically to health care providers (lines 464-465).

2. The text of the report is unclear, lines 169-171. Was an assessment scale used? Was it a clinical criterion? Were any of the participants referred, for psychological support?

We deleted that part from the Participants section, since it had been entered in error. The participant recruitment procedure has been corrected and is now specified (lines 161-168).

3. The link (line 184) does not refer to a website and an error response occurred.

We have corrected the link and inserted the correct web address https://mind-vr.com/free-download/ (line 177).

4. Significant part of this manuscript seems to be described in Ref 30 (doi: 10.3389/frvir.2021.620225).

In the revision we eliminated many parts already described in that previous manuscript (that is now reference 31) (lines 214-215, 265-266).

5. The extensive use of Likert scales might reduce the reliability of the results (doi: 10.1111/j.1365-2929.2004.02012.x).

We have added that aspect and the suggested literature reference in the section on the limitations of the study (lines 447-449).

6. The degree of knowledge of the differences between stress and anxiety seems relatively low for a population with the background of those in the study.

Thanks for this comment. In the forthcoming studies we will conduct on MIND-VR we will try to explore this aspect further.

7. In table 4 the SDs are very high and in conjunction with the small population size and low statistical significance, might reduce the reliability of the conclusions.

We have added a reference to this among the limitations of the study (line 444).

8. Some of the references could be omitted, either because they do not offer more evidence, or because they are quite outdated for such a contemporary topic.

We have removed some of the older references on the topic as suggested (e.g., Pallavicini et al. 2009, Repetto et al. 2011).

Author Response File: Author Response.pdf

Reviewer 2 Report

Dear Authors:

I enjoyed reading the article very much. However, you should make some changes before the article is published:
1. you must identify the type of Mixed Methods used. We recommend reading the following papers:
-Anguera MT, Portell M, Chacón-Moscoso S and Sanduvete-Chaves S (2018) Indirect Observation in Everyday Contexts: Concepts and Methodological Guidelines within a Mixed Methods Framework. Front. Psychol. 9:13. doi: 10.3389/fpsyg.2018.00013.
-A.J. Onwuegbuzie and B. Johnson (2021).  Reviewer's Guide for Mixed Methods Research Analysis. London: Routledge.
2. They should state: (a) whether the study was conducted in accordance with the Declaration of Helsinki (WMA 2000, Bošnjak 2001, Tyebkhan 2003), which sets out the fundamental ethical principles for research involving human subjects. (b) Whether participants voluntarily agreed to participate in the study, and informed consent was obtained in all cases. (c) Whether this study was conducted in compliance with the Standards for Ethics in Sport and Exercise Science Research (Harriss et al., 2020). (d) The Ethics Committee approval reference.
3. They should report the psychometric properties of the questionnaires used.
4. The effect size should be calculated and included in the abstract.
5. In the descriptive data used, together with the mean and standard deviation, indicators of skewness and kurtosis should be included.
6.  In the part of the work where qualitative methodology is used, the system of codes/categories (with expression of its categorical nucleus and its degrees of openness) used should be included. It should also include the results of the quality control of the data performed.
7. The computer program used to perform the analysis and several screenshots should be included. In addition to stating the computer program used to perform the analysis, several screenshots should be included that show both the unit of analysis (and its composition) and the codes used (and their frequencies), the segmentations performed, the networks established, etc.
8. For this qualitative part, we recommend reading the following works:
-Anguera MT, Portell M, Chacón-Moscoso S and Sanduvete-Chaves S (2018) Indirect Observation in Everyday Contexts: Concepts and Methodological Guidelines within a Mixed Methods Framework. Front. Psychol. 9:13. doi: 10.3389/fpsyg.2018.00013.
-Morales-Sánchez, V., Pérez-López, R., Reigal, R.E. and Hernández-Mendo, A. (2020) Mixed-Methods Analysis of Emotional Quality in Sports Organizations: Facial Expressions of Child Users of Sports Services as Data. Front. Psychol. 11:1199. https://doi.org/10.3389/fpsyg.2020.01199

I am awaiting modifications.

Sincerely.

Author Response

We thank the Editor and the Reviewers very much for the attention and important comments provided. The input was crucial in improving the quality of the manuscript.

As requested by the Editor we revised many of the parts indicated in the ithenticate report. However, it was not possible to change some sections, such as definitions, questionnaire items, technical details, affiliations, or titles of virtual experiences, which can only be the same as in previous papers.

We have revised the full text by following the directions and editing the indicated sections. In the manuscript, we have inserted in red the sections modified or added to the text, in order to make the changes made as clear as possible.  Below are the detailed answers to each comment received by Reviewer.

Reviewer 2

I enjoyed reading the article very much. However, you should make some changes before the article is published:

1. you must identify the type of Mixed Methods used. We recommend reading the following papers:
   -Anguera MT, Portell M, Chacón-Moscoso S and SanduveteChaves S (2018) Indirect Observation in Everyday Contexts: Concepts and Methodological Guidelines within a Mixed Methods Framework. Front. Psychol. 9:13. doi: 10.3389/fpsyg.2018.00013.
   -A.J. Onwuegbuzie and B. Johnson (2021). Reviewer's Guide for Mixed Methods Research Analysis. London: Routledge.

As requested, we have specified in the revised version of the paper the type of Mixed-Methods used (lines 18, 147).

2. They should state: (a) whether the study was conducted in accordance with the Declaration of Helsinki (WMA 2000, Bošnjak 2001, Tyebkhan 2003), which sets out the fundamental ethical principles for research involving human subjects. (b) Whether participants voluntarily agreed to participate in the study, and informed consent was obtained in all cases. (c) Whether this study was conducted in compliance with the Standards for Ethics in Sport and Exercise Science Research (Harriss et al., 2020). (d) The Ethics Committee approval reference.

In the revised manuscript we stated (a) the study was conducted in accordance with the Declaration of Helsinki (line 169); (b) Whether participants voluntarily agreed to participate in the study, and informed consent was obtained (lines 166-167); (c) that Standards is not related to our study and was not required by the Ethics Committee; (d) The Ethics Committee approval reference (line 168-169).

3. They should report the psychometric properties of the questionnaires used.

We add details on the psychometric properties of the questionnaires used (lines 226-227, 234, 241-242, 257).

4. The effect size should be calculated and included in the abstract.

Since this is mainly an usability study and we included an higher number of the suggested required number of participants – i.e., the rule associated with Nielsen sounds like “five users are enough to uncover about 85% usability problems” - we did not include the effect size calculation.

5. In the descriptive data used, together with the mean and standard deviation, indicators of skewness and kurtosis should be included.

In the Data Analysis section, we specified that the skewness and kurtosis values were all <1  (lines 285).

6. In the part of the work where qualitative methodology is used, the system of codes/categories (with expression of its categorical nucleus and its degrees of openness) used should be included. It should also include the results of the quality control of the data performed.

In the Data Analysis section, we have deeply revised the part on the methodology used for quantitative data analysis. As now specified in the text (lines 292-295), specifically, qualitative data analysis was conducted using a conventional qualitative content analysis (PMID:16204405). Differently from the summative content analysis, which focuses on counts and quantification of data as indicated by the Reviewer, this methodological approach aims to identify repeated patterns of meaning across a data set. Besides, in the revised text we specified the quality control of the data performed (lines 295-307).

7. The computer program used to perform the analysis and several screenshots should be included. In addition to stating the computer program used to perform the analysis, several screenshots should be included that show both the unit of analysis (and its composition) and the codes used (and their frequencies), the segmentations performed, the networks established, etc.

See the previous comment related to our use of a conventional qualitative content analysis.

8. For this qualitative part, we recommend reading the following works

-Anguera MT, Portell M, Chacón-Moscoso S and Sanduvete Chaves S (2018) Indirect Observation in Everyday Contexts: Concepts and Methodological Guidelines within a Mixed Methods Framework. Front. Psychol. 9:13. doi: 10.3389/fpsyg.2018.00013.

-Morales-Sánchez, V., Pérez-López, R., Reigal, R.E. and Hernández-Mendo, A. (2020) Mixed-Methods Analysis of Emotional Quality in Sports Organizations: Facial Expressions of Child Users of Sports Services as Data. Front. Psychol. 11:1199. https://doi.org/10.3389/fpsyg.2020.01199

We thank you for the important suggested materials, which were very helpful during the review.

Round 2

Reviewer 1 Report

Dear authors thank you for your response and the revised manuscript, and I wish you a successful further research in the area.