The Impact of Virtual Reality-Based Products on Mild Cognitive Impairment Senior Subjects: An Experimental Study Using Multiple Sources of Data
Round 1
Reviewer 1 Report
The authors should spell out the full name of MCI in the title and abstract in its first appearance, otherwise readers who are not familiar with this term will be very confused. So as for VR, NIR, etc. Please proofread carefully and make sure they have their full name at first appearance and only abbreviations afterwards.
The introduction part could be concise and more focus on what and why you are interested in this project.
The description of the participants should be more specific. "Twenty workers were enlisted from the local area" What workers? and what local area? Does volunteer refer to the workers, as well as the subjects? This should be consistent, and the authors should state clearly the criteria of using score of 26.
Table 1 should be revised with publishable format and row names correctly labeled, ie, mean, standard deviation.
Is there any repeated measurement? How can you make sure the results are not by random or error?
Majority of the subject are female, and how do you think if there is any gender bias here?
Do you have any information on subjects' neurological disorders history or family history of neurological disorders?
How do you interpret these instant changes and how confident you are to make the conclusion that "VR products can develop good activation of the brains of MCI older adults" How does this instant change lead to long-term effect, if any? I think the authors should be careful about making these strong conclusions and should be specific about what you observed and obtained.
Author Response
Dear Reviewers,
We appreciate the positive and constructive comments and suggestions that you provided.
We have revised the paper carefully according to you comments. All the revised parts are marked in RED.
The details of responses to reviewers’ comments are listed as follows. We hope the revised paper is in good order for you consideration. We look forward to hearing from you.
Many Thanks and Best Regards
Response to reviewers’ comments
***********************
Point 1: The authors should spell out the full name of MCI in the title and abstract in its first appearance, otherwise readers who are not familiar with this term will be very confused. So as for VR, NIR, etc. Please proofread carefully and make sure they have their full name at first appearance and only abbreviations afterwards.
Response 1: Thanks for the comments. The issue has been changed.
Point 2: The introduction part could be concise and more focus on what and why you are interested in this project.
Response 2: Thank you for your helpful comments. Some content has been removed from this section and the description of the content of interest has been enhanced.
Although some studies have begun to evaluate products focusing on the cognitive and motor perspectives of brain function, many challenges remain. First, cognitive training products are still in their infancy, with a gradually increasing variety of training products appearing in various forms, such as physical and virtual products, and no standard training model. Second, the current assessment of training results produced by these products is not accurate enough, and the assessment tools that are used are not comprehensive. A cognitive assessment based on brain function is the current research trend, and due to the complexity of cognition, different scholars are trying to use richer multi-source data to evaluate cognitive training products, without forming a fixable evaluation system. Third, the training characteristics for MCI elderly are not clear, and target-ed training cannot be conducted. Based on the above considerations, this study used fNIRS to identify changes in the brain hemoglobin concentration of test subjects while carrying out activities based on products in the VR environment with the help of the de-vice’s multisensory features, combined with Kinect device. This was used to obtain behavioral data from the test subjects. We aim to detect the cognitive and motor characteristics exhibited by MCI elderly people during the use of VR products, to determine whether the combination of multiple modalities, using fNIRS and Kinect as well as cognitive scales, can obtain a correlation between their data and some physiological representations of MCI elderly people while using VR products, as well as to enhance multiple forms of cognition with the use of these products.
Point 3: The description of the participants should be more specific. "Twenty workers were enlisted from the local area" What workers? and what local area? Does volunteer refer to the workers, as well as the subjects? This should be consistent, and the authors should state clearly the criteria of using score of 26.
Response 3: Thank you for your helpful comments. The "workers" and "volunteers" have been unified to prevent confusion and to explain the score 26.
Using the score set by the MoCA scale as the standard, Twenty-six or more points was considered normal cognition; volunteers with MoCA scale scores ≤26 were screened for the experiment, and a total of 17 subjects were retained.
Point 4: Table 1 should be revised with publishable format and row names correctly labeled, ie, mean, standard deviation.
Response 4: Thanks for the constructive comments. Table 1 have revised with publishable format and row names correctly labeled, ie, mean, standard deviation.
Table 1. Subjects’ personal information.
|
Characteristics |
Mean |
Standard deviation |
|
Age (years) |
62.35 |
4.94 |
|
Height (cm) |
163.56 |
5.52 |
|
Weight (kg) |
60.42 |
9.67 |
|
MoCA |
22.88 |
2.47 |
Point 5: Is there any repeated measurement? How can you make sure the results are not by random or error?
Response 5: Thanks for the comments. The software page of the device has obvious signal identification tips. After repeated debugging to make sure the signal is stable, and after pre-acquisition to make sure there are no errors in the data before starting the acquisition, thank you for your reminder, and we will pay more attention to these aspects in the subsequent experiments.
Prior to the test, it was determined that each subject was new to the VR product and each subject was provided with a basic understanding of the game to ensure they un-der-stood the requirements and were prepared for the test. Subjects were fitted with fNIRS acquisition devices, and electrodes were tuned until all channels showed green on the product page. The analysis started after the subject loosened up in their seat, and up-per-extremity activity was perceived by the movement catch gear.
After 10 minutes, the subjects entered the task state. The VR device was added to the electrode cap and the electrodes were recaptured; then, the device was debugged again until all channels showed green on the software page, and the Kinect device was switched on to capture upper-extremity motion data.
Point 6: Majority of the subject are female, and how do you think if there is any gender bias here?
Response 6: Thanks for the helpful comments. The factors influencing users' use of the product are indeed complex, but due to the difficulty of subject recruitment, female elderly people are highly motivated and relatively more are recruited, but we believe that the gender factor does not play an important role in this paper, and our main point is to focus on whether the use of VR products can be stimulating for elderly people through the device. At the same time, to further validate the point, in the future we will go into greater efforts to find suitable subjects, which is added to the limitations.
The current study has several limitations. The sample size included in this study is small, which may impact its accuracy. Many factors influence the effectiveness of a users' use of the product; gender may be one of them. The elderly people recruited in this study were mainly female, which may have some influence on the results. Regarding fNIRS data preprocessing, subjectivity when identifying artifacts may influence the results.
Point 7: Do you have any information on subjects' neurological disorders history or family history of neurological disorders?
Response 7: Thanks for the constructive comments. Interviews were conducted on this issue when recruiting volunteers, as explained in the article.
Twenty volunteers were recruited in the community, all of whom were older adults who no longer participated in the workforce. The basic personal information of the sub-jects, such as their age, height, and weight, was recorded before the test, and they were asked whether they had neurological disorders. In this study, a group of older adults without neurological disorders was selected, and only their cognition was judged; their cognitive abilities were quickly assessed using the MoCA scale.
Point 8: How do you interpret these instant changes and how confident you are to make the conclusion that "VR products can develop good activation of the brains of MCI older adults" How does this instant change lead to long-term effect, if any? I think the authors should be careful about making these strong conclusions and should be specific about what you observed and obtained.
Response 8: Thanks for the constructive comments. For some statements in the article that may lack accuracy, the help of the MDPI English editor was requested to make corrections. Some new literature has been added to the article to jointly illustrate the impact of VR products on MCI older adults. The activation of brain regions in MCI older adults relied heavily on information from data collected by the fNIRS device in this study, comparing data from MCI older adults in resting and task states to determine the possible activation of different regions in the brain. Of course, our current study focused on the objective data information that can better reflect MCI older adults while using VR products with the help of these devices to ensure the availability of multi-source data. More research on brain regions will be enhanced in subsequent studies. In addition, immediate and long-term correlations will also be considered.
VR-based products can stimulate the brains of MCI older adults [39,40], especially older adult groups with relatively low cognitive levels [14].
[39] OPTALE G, URGESI C, BUSATO V, et al. Controlling Memory Impairment in Elderly Adults Using Virtual Reality Memory Training: A Randomized Controlled Pilot Study [J]. Neurorehabilitation and Neural Repair, 2009, 24(4): 348-57.
[40] D B, A. S R, K A. Quantitative Measurements of Brain Activations in Virtual Reality Environments; proceedings of the 2022 IEEE International Symposium on Medical Measurements and Applications (MeMeA), F 22-24 June 2022, 2022 [C].
[14] BAUER A C M, ANDRINGA G. The Potential of Immersive Virtual Reality for Cognitive Training in Elderly [J]. Gerontology, 2020, 66(6): 614-23.
Author Response File: 
Author Response.pdf
Reviewer 2 Report
Need to improve English grammar
Need more subject participants in research
Author Response
Dear Reviewer,
We appreciate the positive and constructive comments and suggestions that you provided.
We have revised the paper carefully according to your comments. All the revised parts are marked in RED.
The details of responses to reviewers’ comments are listed as follows. We hope the revised paper is in good order for your consideration. We look forward to hearing from you.
Many Thanks and Best Regards
Response to reviewers’ comments
***********************
Point 1: Need to improve English grammar.
Response 1: Thanks for your helpful comment. This article has been corrected with the help of MDPI's English editors.
Point 2: More disciplinary participants are needed in research.
Response 2: Thank you for your constructive comments. Since the users of the study are elderly people, there are some difficulties in recruiting volunteers, and we are doing our best to expand the number of recruits. The main purpose of this study is to explore multi-source data as a means to evaluate the effect of VR products on the elderly. We believe that the current figures still reflect certain phenomena, and in future studies, we will continue to expand the number of volunteers to Make the number of users studied more convincing.
Author Response File: Author Response.pdf
Reviewer 3 Report
While the underlying idea of this manuscript is valid. I see several concerns regarding the technical approach and the statistical analyses used in the study to support the findings. I feel that analyses presented in the current version of the manuscript are insufficient to validate the functionality and usefulness of the proposed product/tool. Additionally, it would be extremely helpful to understand how the proposed product performs in comparison to other existing frameworks. This information is lacking in both the introduction and discussion of the manuscript.
Author Response
Dear Reviewer,
We appreciate the positive and constructive comments and suggestions that you provided.
We have revised the paper carefully according to your comments. All the revised parts are marked in RED.
The details of responses to reviewers’ comments are listed as follows. We hope the revised paper is in good order for your consideration. We look forward to hearing from you.
Many Thanks and Best Regards
Response to reviewers’ comments
***********************
Point: While the underlying idea of this manuscript is valid. I see several concerns regarding the technical approach and the statistical analyses used in the study to support the findings. I feel that analyses presented in the current version of the manuscript are insufficient to validate the functionality and usefulness of the proposed product/tool. Additionally, it would be extremely helpful to understand how the proposed product performs compared to other existing frameworks. This information is lacking in both the introduction and discussion of the manuscript.
Response: Thank you for your encouraging remarks and valuable comments. The use of VR for timely and accurate assessment of a subject's condition allows for phased adjustment of rehab tasks, resulting in better treatment outcomes. VR is a form of rehabilitation that has benefits not only for the body but also for brain function and cognitive levels. As the need for specific and individualized rehabilitation increases, more accurate assessment tools are needed. Therefore, the combination of VR technology with other methods considered in our study allows for better assessment and interpretation of physiological and psychological changes in patients during rehabilitation. In our study, we also found that some associations between brain activation, movement, and cognition were also identified, demonstrating the validity of the method. In future studies, we will rely on this assessment tool to do more research on the product's functions and design details, and effects. In addition, the proposed development is described more specifically in the article.
Beat Saber is a VR product that combines action, music, and leisure; the product is very popular, and 99% of player reviews are positive. The game is played using joystick control, with two-color light beams, and color blocks appearing on the screen to form a correspondence with the rhythm of the music. The same color beam passes through the color block. To perform the task, the blocks must be cut with the correct speed to complete the level. The motion control of VR should be fully utilized to avoid pressing the controller by mistake. The combination of music rhythm, flying colors, block position, and direction changes, and the relationship between these factors make the game interactive and rich in action. The game also has a simple and clear visual, and a strong sense of music and rhythm. This process requires a large amount of continuous movement of the upper limbs, accurately judging the position of the red and blue blocks, judging the direction of the split block, and accurately judging the speed of the block according to the rhythm. The subject's attention, reflexes, spatial orientation, and body movements are mobilized.
Author Response File: Author Response.pdf
Reviewer 4 Report
This study proposes a method to evaluate the feasibility of monitoring the interaction status of MCI elderly with VR-based products and Kinect. I suggest that VR games utulized should be explained in detail. Also, the contribution of the study must be highlighted and explained in detail. There are lots of typos and grammatical mistakes. The use of English throughout requires significant improvement.
Author Response
Dear Reviewer,
We appreciate the positive and constructive comments and suggestions that you provided.
We have revised the paper carefully according to your comments. All the revised parts are marked in RED.
The details of responses to reviewers’ comments are listed as follows. We hope the revised paper is in good order for your consideration. We look forward to hearing from you.
Many Thanks and Best Regards
Response to reviewers’ comments
***********************
Point: This study proposes a method to evaluate the feasibility of monitoring the interaction status of MCI elderly with VR-based products and Kinect. I suggest that VR games utulized should be explained in detail. Also, the contribution of the study must be highlighted and explained in detail. There are lots of typos and grammatical mistakes. The use of English throughout requires significant improvement.
Response: Thank you for your encouraging remarks and valuable comments. Following your suggestions, the relevant parts of the article were revised, and the help of the MDPI English editor was requested to correct the full text.
Beat Saber belongs to the action, music and leisure combination of VR products; the product is very popular, with 99% of player reviews being positives. The game is carried out using handle-control, with a two-color beam, and color blocks appear on the screen, forming a correspondence with a musical rhythm. The same color beam cuts through color blocks. To carry out the task, the blocks must be cut at correct rate must be cut to complete the level. VR's somatosensory control should be fully utilized to avoid incorrectly pressing the handle. The combination of music rhythm, flying colors, block locations and changing directions, and the relationship between these factors, makes the game interactive and rich in action. The game also has visual simplicity and clarity, and the sense of music and rhythm is very strong. This process requires a lot of continuous movement of the upper limbs, accurate judgment of the position of the red and blue blocks, accurate judgment of the direction of the split blocks, and accurate judgment of the speed of the blocks according to the rhythm. The subject's attention, responsiveness, spatial orientation, and limb movements are all mobilized.
In this study, fNIRS was used to monitor subjects' brain signals at rest and during a task, and Kincet was used to monitor subjects' behavioral patterns during a task, to evaluate VR-based products, focusing on user–task interaction responsiveness. The fNIRS data extracted from fNIRS were analyzed to show that the OL region of the brain was significantly more activated in older adults during the Beat Saber task than in the resting state. The data extracted from fNIRS showed that the OL region of the brain was significantly activated during the Beat Saber task compared to the resting state, and the upper-limb behavioral data recorded by Kinect were significantly correlated with the subjects’ functional brain regions. Therefore, when evaluating VR products, multimodal data can be used to determine the products’ impact on users, focusing on multiple perspectives. The results indicate that VR products activate some brains regions in MCI older adults. When com-paring the degree of interactive response task completion in MCI older adults, it was found that a decrease in cognitive level causes a decrease in motor ability, which can lead to changes in brain activation. This is a new assessment method that can help strengthen the assessment of the products’ effectiveness in helping diagnosis and treatment, and can be applied to the field of rehabilitation product design to assist rehabilitation medications.
Author Response File: Author Response.pdf
Reviewer 5 Report
Dear Authors, Thank you for submitting your work. The research you have begun is one that is necessary to the contemporary world and of undoubted interest. I have few elemnts to comment:
be careful to define well all acronyms and acronyms from their first appearance. It would be very useful to have them all in an short dictionary at the end.
look at some typo in the text such as spaces omitted or Capital letters such as RUHONG GE.
I did not find any explanation of ANOVA test ( where does it come from, what is it for..).
The Wavelet Transform: better clarify the use of this equation in the context of the following part of the book. Actually it is not clear.
Beat Saber: of course, the scientific method assumes that the test is done with multiple video games while here we preferred to proceed with only one. It must surely be well explained why of the choice of Beat Sauber, why was made and whether it is assumed to be deepened with other games in the future.
Some parts of the discussion could be added in the conclusion.
Also the conclusion could be more specific, expecially regarding the future experiments that should be develop to grow the actual ones. What should be transformed and changed?
Thank you for your work
Author Response
Dear Reviewer,
We appreciate the positive and constructive comments and suggestions that you provided.
We have revised the paper carefully according to your comments. All the revised parts are marked in RED.
The details of responses to reviewers’ comments are listed as follows. We hope the revised paper is in good order for your consideration. We look forward to hearing from you.
Many Thanks and Best Regards
Response to reviewers’ comments
***********************
Point 1: be careful to define well all acronyms and acronyms from their first appearance. It would be very useful to have them all in a short dictionary at the end.
Response 1: Thanks for the constructive comments. The issue has been changed.
Explanation of the main terms covered in the article.
|
Abbreviation |
Full Name |
|
MCI |
mild cognitive impairment |
|
VR |
virtual reality |
|
fNIRS |
functional near-infrared spectroscopy |
|
WT |
wavelet transform |
|
WA |
wavelet amplitude |
Point 2: look at some typos in the text such as spaces omitted or Capital letters such as RUHONG GE.
Response 2: Thanks for the helpful comments. The article has been corrected with the help of the MDPI English editor.
Point 3: I did not find any explanation of the ANOVA test ( where does it come from, what is it for..).
Response 3: Thanks for the helpful comments. Following your suggestions, it has been added to the article.
One-way ANOVA can provide a comparative analysis of N classifications, which uses the mean of the values for a particular characteristic to see if there is a significant difference between groups.
Point 4: The Wavelet Transform: better clarify the use of this equation in the context of the following part of the book. Actually, it is not clear.
Response 4: Thanks for the helpful comments. Following your suggestions, it has been added to the article.
The quantization of the fNIRS signal can be estimated using the Hilbert transform, Fourier transforms or wavelet transform, which is mathematically equivalent when applied to spectral analysis [27]. However, the use of the wavelet transform provides an adjustable window, thus providing an intuitive visualization of the time-frequency domain as well the high resolution for the high- and low-frequency components [28].
[27] BRUNS A. Fourier-, Hilbert- and wavelet-based signal analysis: are they different approaches? [J]. Journal of Neuroscience Methods, 2004, 137(2): 321-32.
[28] WANG W, WANG B, BU L, et al. Vigilance Task-Related Change in Brain Functional Connectivity as Revealed by Wavelet Phase Coherence Analysis of Near-Infrared Spectroscopy Signals [J]. Frontiers in Human Neuroscience, 2016, 10.
Point 5: Beat Saber: of course, the scientific method assumes that the test is done with multiple video games while here we preferred to proceed with only one. It must surely be well explained why the choice of Beat Sauber, why was made, and whether it is assumed to be deepened with other games in the future.
Response 5: Thanks for the constructive comments. Following your suggestions, it has been added to the article.
Beat Saber belongs to the action, music, and leisure combination of VR products; the product is very popular, with 99% of player reviews being positive. The game is carried out using handle control, with a two-color beam, and color blocks appear on the screen, corresponding with a musical rhythm. The same color beam cuts through color blocks. To carry out the task, the blocks must be cut at the correct rate must be cut to complete the level. VR's somatosensory control should be fully utilized to avoid incorrectly pressing the handle. The combination of music rhythm, flying colors, block locations, and changing directions, and the relationship between these factors, make the game interactive and rich in action. The game also has visual simplicity and clarity, and the sense of music and rhythm is very strong. This process requires a lot of continuous movement of the upper limbs, an accurate judgment of the position of the red and blue blocks, accurate judgment of the direction of the split blocks, and accurate judgment of the speed of the blocks according to the rhythm. The subject's attention, responsiveness, spatial orientation, and limb movements are all mobilized.
Point 6: Some parts of the discussion could be added to the conclusion.
Response 6: Thanks for the constructive comments. Following your suggestions, it has been added to the article.
In this study, fNIRS was used to monitor subjects' brain signals at rest and during a task, and Kinect was used to monitor subjects' behavioral patterns during a task, to evaluate VR-based products, focusing on the user–task interaction responsiveness. The fNIRS data extracted from fNIRS were analyzed to show that the OL region of the brain was significantly more activated in older adults during the Beat Saber task than in the resting state. The data extracted from fNIRS showed that the OL region of the brain was significantly activated during the Beat Saber task compared to the resting state, and the upper-limb behavioral data recorded by Kinect were significantly correlated with the subjects’ functional brain regions. Therefore, when evaluating VR products, multimodal data can be used to determine the products’ impact on users, focusing on multiple perspectives. The results indicate that VR products activate some brain regions in MCI older adults. When comparing the degree of interactive response task completion in MCI older adults, it was found that a decrease in cognitive level causes a decrease in motor ability, which can lead to changes in brain activation. This is a new assessment method that can help strengthen the assessment of the product’s effectiveness in helping diagnosis and treatment and can be applied to the field of rehabilitation product design to assist rehabilitation medications.
Point 7: Furthermore, the conclusions could be more specific, especially about future experiments that should be developed to develop actual experiments. What should be shifted and changed?
Response 7: Thank you for your constructive comments. Based on your suggestion it was added to the article.
In future research plans, the selected topics can be expanded based on the application of this method in rehabilitation medicine. Other signal processing methods can also be examined to improve data accuracy. When researching VR products, a combination of multimodal data can be used to further experiment with the interface composition, context, and interaction with VR products to find VR products that are more suitable for older adults with MCI.
Author Response File: Author Response.pdf
Round 2
Reviewer 1 Report
Comments and concerns have been addressed. Thank you for your revision.
Author Response
Thank you for your encouragement and recognition.
Reviewer 3 Report
I appreciate the authors addressing my concerns. After reading the revised manuscript along with the reviewer response, I believe that the paper warrants publication. I still do believe that justifications for the exact statistical tests used are needed. Please include that information for the final version.
Author Response
Dear Editor and Reviewers,
We appreciate the positive and constructive comments and suggestions that you provided.
We have revised the paper carefully according to your comments. All the revised parts are marked in RED.
The details of responses to reviewers’ comments are listed as follows. We hope the revised paper is in good order for your consideration. We look forward to hearing from you.
Many Thanks and Best Regards
Response to reviewers’ comments
***********************
Point: I appreciate the authors addressing my concerns. After reading the revised manuscript along with the reviewer's response, I believe that the paper warrants publication. I still do believe that justifications for the exact statistical tests used are needed. Please include that information in the final version.
Response: Thank you for your encouraging remarks and valuable comments. We have added detailed descriptions of the statistical methods used and marked them in blue.
One-way ANOVA can provide a comparative analysis of N classifications, which uses the mean of the values for a particular characteristic to see if there is a significant difference between groups. The experiment involved six brain region indicators, eight behavioral indicators, and two task states, and the multi-source data needed to be compared between and within groups. For the analysis of the variance of data from two groups and above, one-way ANOVA met the need for statistical analysis in this study. To determine the degree of activation, one-way ANOVA tests were performed on the data from the subjects’ task and resting states to explore the implication of different states on the degree of activation of brain regions during the game. For Kinect data metrics, a one-way ANOVA test was conducted, looking at within-group interaction and completion rates to analyze the effect of behavior on different states of task completion. Pearson correlation analyses were conducted for behavior and brain region activation. The statistical significance level (P) for all analyses was set at 0.05.
Author Response File: 
Author Response.docx
