Effectiveness of a Serious Game Design and Game Mechanic Factors for Attention and Executive Function Improvement in the Elderly: A Pretest-Posttest Study
Round 1
Reviewer 1 Report
Thank you for this interesting piece of work. I encourage you to revise the result section. Please round up to two digits when presenting descriptives. Please also avoid multiple ANOVAs an t-Tests and provide instead one repeated measurement ANOVA and ONE MANOVA in order to avoid alpha error accumulation. Please also provide measures of reliability for dependant variables. Present all descriptive results in ONE table.
Author Response
Dear Reviewer 1,
Please kindly see the attachment file.
thank you very much.
Author Response File: 
Author Response.pdf
Reviewer 2 Report
In the present paper, the authors proposed a serious game for cognitive training of the elderly in terms of attention and executive function. They also conducted an 8-week experiment, and the findings indicated that there is a significant enhancement in both attention and executive functions after the training sessions. The game mechanic factors that were measured were difficulty, pressure, and competition. The difficulty and the pressure factor were shown to have an effect, in contrast to the competition factor.
The general idea of this study is very interesting, and the evaluation seems to be well-organized. However, my main concern is that although all the necessary information is inside the text, the presentation seems to be not very solid and its quality seems average. For example, the authors want to analytically explain something, which leads to the repetition of the same info, resulting in the text becoming somewhat tedious. More specifically:
- In Introduction, Section 1.1, the first 2 paragraphs say almost the same thing, please rephrase them and make it one paragraph.
- In Section 2.2, you repeat the same info with the introduction, please rephrase them and present them more solid.
- In Results, you repeat the same pattern of how to present the results for each test. It would be great if you could rephrase them
- The paired sample correlation show that it is significantly different (p<0.01) (Table 4). The paired sample t-test result shows that it is also significantly different (p<0.001) (Table 5).
- The mixed repeated-measure ANOVA result shows that the effect of the pre-test and the post-test is significantly different (F[1,32]=1649.438, p<0.001). The independent variable (Difficulty) is also significantly different (F[1,32]=174.918, p<0.001). The independent variable (Pressure) is also significantly different (F[1,32]=168.781, p<0.001). The interaction effect (Difficulty and Pressure) is also significantly different (F[1,32]=25.329, p<0.001) (Table 6)
The parts that are mentioned are indicative, I have noticed this pattern in the whole manuscript.
Another point that I would like to mention, is that authors support that “There has not been any research done in this area.” (line 237). Please check the literature review, since there are previous works that use computer-based cognitive training to improve attention and executive function in elderly people. Some suggestions:
- Tamara Simpson, David Camfield, Andrew Pipingas, Helen Macpherson & Con Stough (2012) Improved Processing Speed: Online Computer-based Cognitive Training in Older Adults, Educational Gerontology, 38:7, 445-458, DOI: 10.1080/03601277.2011.559858
- Gajewski, P. D., Thönes, S., Falkenstein, M., Wascher, E., & Getzmann, S. (2020). Multidomain Cognitive Training Transfers to Attentional and Executive Functions in Healthy Older Adults. Frontiers in human neuroscience, 14, 586963. https://doi.org/10.3389/fnhum.2020.586963
- Kueider AM, Parisi JM, Gross AL, Rebok GW (2012) Computerized Cognitive Training with Older Adults: A Systematic Review. PLoS ONE 7(7): e40588. https://doi.org/10.1371/journal.pone.0040588
Another point is that you should put the citations inside the text in the correct place. For example, when you mention the authors’ names, write the citation after the names, not at the end (e.g., Rowe & Kahn (line 52), Baltes & Baltes (line 57), Ball (line 71), Dehaene (line 88), etc. Similarly, in line 64 “In the cognitive neuroscience of aging, there are three patterns of age-related changes in cognitive behavior:”, add the citation in the beginning, not in the end. Please make a cross-check of the citations in the whole text, since the aforementioned are indicative.
One last important comment is that you should link your findings with the previous studies from the literature review, in the Discussion section. For example (also indicative), “other studies have shown that traditional arithmetic calculation do not enhance executive function” you could add the corresponding citation.
Finally, the manuscript needs a careful revision for language and grammar.
Some other comments:
- In line 52 you mention that there are two popular explanations. It would be clearer if you add (a), (b) inside the text.
- In line 61 you mention “Therefore, these studies indicate that increases in high cognitive functions and improvements in cognitive function through health-related activities for the elderly are part of the main features of successful aging.”. Could you please rephrase it since you mention two times “cognitive function”?
- In line 117 you mention “serial subtraction tasks and serial addition tasks both require”, please rephrase it to “both serial subtraction tasks and serial addition tasks require”
- In line 396 you mention “This can be adjusted or gradually increased during the training to fit what is considered a suitable number of competitors. Because participant’s skill might increase during the training”, maybe to join these two sentences?
Author Response
Dear Reviewer 2,
Please kindly see the attachment.
Thank you very much.
Author Response File: Author Response.pdf
Reviewer 3 Report
Dear Authors,
I would like to congratulate you about your study that I found original. I present my comments
about the manuscript below for your information.
1. Title
The title should include the type of study
ABSTRACT:
how old is old age?
The abstract should include the methodology: type of study, with which measurement scale the variables have been assessed ....
Results: the results are imprecise: how many volunteers completed the study, what are the results based on the measurement tools selected?
The summary should include a conclusion.
In general, the abstract does not synthesise the information that has been developed in the manuscript.
INTRODUCTION
The introduction is very long and does not justify the need for intervention with a serious game. Why is it necessary to design serious games to maintain or improve cognitive ability? What previous work has explored this field, what were the results and what does the designed serious game provide? These answers are not answered in the introduction.
The last point of the introduction should be the aim of the study.
The design of the serious game should be part of the methodology, as it is the form of intervention.
METHODOLOGY
The type of study, the non-randomisation of the sample is not clearly described.
Recruitment is not described. Where do the participants come from, do they live in the community, in a nursing home...?
DISCUSSION.
The first point of the discussion should include the aim of the work.
CONCLUSION
The text of the conclusion includes statements that are not derived from the work carried out. The conclusion should be reduced and limited to the findings of the research.
Author Response
Dear Reviewer 3,
Please kindly see the attachment.
Thank you very much.
Author Response File: Author Response.pdf
Reviewer 4 Report
The authors of the current article have developed a serious game involving mental arithmetic calculations aimed at improving attention and executive functions in old people. The authors report that playing this game five days per week for a total of eight weeks improved participants' attention and executive function. Moreover, the level of difficulty and pressure (and their interaction) are relevant factors regarding participants performance.
The current work is of interest and relevant and cognitive decline is a topic that must be researched in order to improve cognition and quality of live in ageing. It is worthy to develop new strategies and tools that facilitate cognitive training. Nevertheless, I have some concerns about the present work that the authors must address:
1. Introduction:
Statement "In the cognitive neuroscience of aging... life-long declines, late-life declines, and life-long stability" (lines 64-65) requires citations.
The introduction is quite long. For instance, section 1.3 provides excessive details about serial tasks. Just mentioning the different serial procedures, a short description and some references will be enough. In section 1.6 it is not necessary to fully explain the Yerkes-Dodson law. A short description will be enough.
2. Materials and Methods:
Information from lines 247-253 is repetitive.
(Line 268) The statement "The serial subtraction task and serial addition task alternately show in each round" is confusing. Readers can understand that subtraction and addition were shown in alternative trials. This is what I initially understood. The same for the statement in line 290 "They process alternately". Should be clarified.
Section 2.2. I have some concerns regarding players could choose the difficulty, pressure and competition levels. This is a source of bias that could affect participants outcomes. How the authors can control that results from the most exigent conditions are not due to personal characteristics of those participants who choose these conditions?
Afterwards, in section 2.3.1. it is said that "...the difficulty level was set based on each participant's evaluation of their skill and their acceptability of the difficulty level. The difficulty level can be adjusted or gradually increase during the period of training, because the participant's skill might increase during training". A new concern is: people changed the play conditions during the experiment from less to more exigent conditions? Therefore it is difficult to know if changes in attention or executive function assessments are related to the time playing the less or the most exigent condition of the game. Game conditions should have been maintained stable during the experiment.
3. Results:
There is a typo in table 4. Sig. 0.001 should be ***
In general, for all the ANOVA results, appropriate post-hoc analyses should be done, particularly for the interaction Pre/Post-test * Difficulty * Pressure. Otherwise it is not possible to know the direction of the effect.
The authors should include graphic figures displaying the effect of difficulty, pressure and competition (and their interaction).
4. Discussion:
Some references are required, particularly for statements of lines 598-600, lines 616-626, lines 633-640.
Studies showing the effect of difficulty and pressure in old people should be mentioned or at least this result discussed with regard to old persons cognitive performance. For instance, in lines 649-650 "Previous studies have shown that competition can increase performance" What happens with old people in this regard?
References should be cited in (lines 654-658) "Therefore, interactivity between... other competitors during the game".
Lines 664-666 statement: "This interaction effect... strongly related to the game". However, the lack of post-hoc analyses and figures that provide a visual representation of the effect impedes to know how this interaction affect participants' performance. It should be clarified.
Author Response
Dear Reviewer 4,
Please kindly see the attachment.
Thank you very much.
Author Response File: Author Response.pdf
Round 2
Reviewer 1 Report
Thank you for your revision and you comments. Nevertheless, your statistics are still not OK. Quoting other work with sub-optimal statistics does not make your statistics right. The problem is still accumulation of error term. Please reduce the number of your inference statistical testing, e.g., by using repeated measurement or MANCOVA (e.g., see https://doi.org/10.1002/9781118901731.iecrm0002). Your reliability analysis is also a huge problem, because your scales are far off reliability. Thus, I recommend to exclude such scales. This also has to be addressed in the discussion.
Author Response
Dear Reviewer 1,
Please kindly see the attachment.
Thank you very much.
Best Regards,
Ying-Kit Wong
Author Response File: Author Response.pdf
Reviewer 4 Report
The authors have improved the manuscript according to the previous request. Some editing for English language is required throughout the manuscript due to some mistakes.
Author Response
Dear Reviewer 4,
Please kindly see the attachment.
Thank you very much.
Best Regards,
Ying-Kit Wong
Author Response File: Author Response.pdf
