Blogged into the System: A Systematic Review of the Gamification in e-Learning before and during the COVID-19 Pandemic
Round 1
Reviewer 1 Report (New Reviewer)
Dear Authors
The systematic nature of your study, looking at game elements in gamification and effects of these elements on learner behaviour and their changes over time is of interest to the reader/
However, I am struggling to understand the significance you are placing on changes that took place to gamification prior and post the pandemic. In the lengthy introduction there is little mention of why you think this global event is especially relevant to gamification research and may have led to the changes you proport.
This framing is dominant throughout the article such as the Q1 a-z list. An important consideration here is papers published during or post the pandemic could have been undertaken prior to the pandemic (length of drafting and publication process). The year of publication is poor proxy for changing behaviour when looking at events at such a narrow scale.
The same applies to Q2 which look at effects of game elements on studies published pre- and post- covid. Looking at the post covid papers cited I immediately found two who report that their data collection took place pre- covid. This makes the comparison currently offered moot.
For me the sensible thing would be to remove the COVID anchor from the paper and simply report on changes over time in gamification elements and there reported uses on learning behaviour. I appreciate this is a substantial re-write, but I can think of no simpler way to proceed.
Two other elements
The list of game elements (Q1) is of interest as are the definitions, though the commentary does not report on variations of use / explanation. In a systematic review like this one this would be useful as many of these terms are used inter-changeably by the scientific community.
It is important to note many ‘new’ or more refined uses of gamification elements could be in part to greater understanding and use of the term in education, and educators getting better at designing and evaluating gamification based initiatives.
I wish you success with your revisions,
Best Wishes
Author Response
Please see the attachment.
Author Response File: 
Author Response.pdf
Reviewer 2 Report (New Reviewer)
The main question that this study aims to answer is to analyze scientific papers related to gamification in e-learning before and after the COVID-19 pandemic.
I think the topic is relevant because gamification of the educational process is especially important in e-learning when learners lose motivation. This study shows, among other things, what game elements are most used in e-learning, what factors should have been taken into account in order to develop effective gamification on e-learning platforms. Such research can help practicing educators choose appropriate gamification tools for e-learning.
The study provides a thorough analysis of the concept of gamification, its types, and the use of gamification in e-learning. The authors have identified the most used game elements. Other scientists and researchers can use this analysis as a basis for classification, as this part is qualitatively analyzed and well-grounded.
The paper examines how game elements (on e-learning platforms) influence learner behavior. This allows practitioners to choose gamification elements that have a positive impact on learners' motivation.
The paper explores the factors that educators should take into account for the effective use of gamification on online platforms. Taking these factors into account, practitioners can organize e-learning more effectively.
In my opinion, the authors used adequate research methods and tools. I have no comments on this part.
In my opinion, the conclusions are well argued.
The conclusions about research on gamification before and during the COVID-19 pandemic are well supported.
The references are quite reasonable.
The table in the appendix that describes the game elements published before and during the pandemic is very interesting.
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
Reviewer 3 Report (New Reviewer)
Thank you for allowing me to review this article.
The first thing that I noticed with this paper was that the in-text citations are not written in APA 7. Many researchers are introduced with their full names and papers with more than 3 authors are not cited with et al. It should be updated to APA 7.
The figure title is not in APA 7.
Many portions of the text are repetitive. I felt as though I was reading the same text over. Additionally, the word 'plethora' was used 19 times. It became distracting.
In general, I believe you have been very thorough with your method. However, the review is missing the entire field of Computer-Assisted Language Learning (CALL) and the sub-field Mobile-Assisted Language Learning (MALL) which both have heavy focus on gamification (think of how mobile apps for language learning, such as Duolingo, Busuu, Babbel, etc. use gamification in language education). There are no CALL related journals referenced for this study, which is a huge field of research (the current focus is now on AI and Chat) and gamification in CALL was very popular well before the pandemic (hence the success of Duolingo).
I would suggest rewriting to include CALL studies and reformatting the paper to APA 7.
I have included some highlights and comments on the attachment. Most of the highlighted text is related to incorrect APA formatting, but I stopped highlighting about half way through.
Comments for author File: Comments.pdf
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
Round 2
Reviewer 1 Report (New Reviewer)
Having read the authors detailed response and the revised manuscript I am now happy to see this paper published.
This manuscript is a resubmission of an earlier submission. The following is a list of the peer review reports and author responses from that submission.
Round 1
Reviewer 1 Report
The authors claim to present a systematic review of the gamification in e-learning before and during the COVID-19 pandemic.
My main problem about this paper is that this objective is not met.
The research questions are presented without any reference to COVID-19, and more importantly, the analysis of the results for these research questions does not - in general - consider the difference between before and during the pandemic. So the paper seems to be a typical systematic review that refers to the pandemic in a few aspects.
Other problem is that all the analyses performed only refer to the number of articles and not to the articles themselves, thus depriving the reader from knowing which specific papers are.
And lastly, the presentation is too textual.
There is no table on the results in the whole document, and the only figure is on the PRISMA guideline.
As a total of 103 articles were studied, some tables and/or figures are sure to facilitate the reader's understanding of this study.
Therefore reading the paper -as it is currently- is very difficult.
To be accepted the paper, the authors should do at least the following:
1.- Differentiate in each research question the before and during the COVID-19 pandemic.
2.- For each section of the results of the 3 research questions, specify not only the number of articles, but which specific articles are concerned, differentiating before and after the pandemic. To this end, a table for each RQ showing for each section those articles would be desirable for ease of reading. This is very useful information for the reader in this type of study.
3.- With respect to the above (2), note that there are sections where they have not put the number of articles (e.g. f, u, v and x) and there is no information about the articles either.
4.- Classify the results of RQ3 as you have done with RQ1 and RQ2.
5.- Improve the discussion and conclusions sections (by the way, better to call this section Conclusions and future work) considering the differentiated results (new section 4) between before and during the pandemic.
6.- Check references and follow the format of the journal. Sometimes they put only the year, sometimes the authors and the year.
7.- Check the wording. Some sentences are poorly constructed or sound bad. Examples: "effective gamification" (summary), "we wanted to find out what are the most commonly used gamification elements in e-learning?" (5. Discussion).
8.- The expression (LIMIT-TO (LANGUAGE, "English") OR EXCLUDE (LANGUAGE, "Spanish") OR EXCLUDE (LANGUAGE, "Russian") OR EXCLUDE (LANGUAGE, "Portuguese") OR EXCLUDE (LANGUAGE, "Arabic")), is it correct? Why is it not enough to include only English? It means that, for example, German and French languages are not excluded, why exclude only these languages and not French and German? Can it be considered a paper written in French or Chinese?
Please, explain this.
9.- At the end of the Introduction section the authors put the structure of the article, but it does not correspond to the real structure. Please put the real sections.
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
Reviewer 2 Report
The authors provide a descriptive/narrative review of game elements/gamification, following a sound systematic methodology. The background is well written and presented. However, unlike the title insinuates, the paper doesn't focus on the contrast between the pre & post covid era outcomes in the literature. The paper feels like a huge collection of information that could have been summarised in a way that a ready could easily get an overview, especially RQ1 & RQ2. The writing style left much to be desired requiring much effort to follow and often lacked a coherent narrative and arguements. RQ 1 and 2 are some how constructively related even tho findings presented make no effort highlight arguements from the perspective of "most-used elements". RQ3, however, is standalone from rest of RQ but the factors presented rely on "effectiveness" which is a loosely defined term through out the literatures they extracted. My core concern with this review is that it provides no new knowledge. The results that the authors have provided are principles of design which are loosely based on the definitions and assumptions, that are already loosely defined (as the authors themselves proclaim). There are no new take-aways from the study that can feed into the next logical step/s of this (ongoing) study. The paper in my opinion perhaps is more logical to state that it attemps to define the term effectiveness in context of gamification of e-learning platforms.
The authors have clearly worked hard and is evident. But i think the narrative needs to be improved and re-written. Furthermore, i think the authors need to re-work on presenting the information they want to convey in a much easier and logical manner. They should also perhaps try to make meaning/model of all the bits of valuable information they have pain stalkingly extracted from these literature.
other issues
1. Citations missing in a lot of places where claims are not backed up. Such as in page 2, para 3 (....significantly each year)
2. Often a fact is presented without elaboration. for e.g. ( the important gap..."game science", what is that gap?)
3. Section 3.1: RQs are re-iterated here, which i think is not necessary
etc.
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
