Gamified Project-Based Learning: A Systematic Review of the Research Landscape

Round 1

Reviewer 1 Report

The manuscript addresses a very interesting systematic review under the title " Gamified Project-based Learning: A Systematic Review of the Research Landscape". The topic of the manuscript, as well as the updated literature review provided by the authors, is appropriate.

The overall assessment of the paper is positive. The authors address the topic of the study in a clear way and rely on a review of the scientific literature that is comprehensive and current. The paper is well-structured and easy to follow.

 

Overall, it is a good piece of work. Thus, I recommend the manuscript for publication.

Author Response

Thanks for your kind comments and recognition. We have improved the manuscript again after the first review around.

Reviewer 2 Report

The project is interesting because it deals with a current topic that is project-based learning through games. Nowadays, there is a strong tendency for corporations of the future to put their computing resources to co-create new solutions based on Games. Point 1. Lines 144 to 146. The methodological triangulation needs to be carried out with great care in order to have distortions or cross-effects of the applied methodologies. Point 2. The authors state in the abstract and between lines 343 and 345 that they used the systematic review. I ask the authors to frame the research carried out in the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) and the inclusion and exclusion criteria. The systematic methodology requires proof of the execution of its steps. In the introductory chapter bring scientific citations to the PRISMA method used. Authors need to detail the period of searches performed, the results returned, the application of inclusion and exclusion criteria to finish with the PRISMA table (https://prisma-statement.org). In addition, the work has a high potential for publication.

Author Response

Thanks for your kind reminder.

point 1: We added this point in Line 153-155.

point 2: Following your suggestion, we rewrote the “methods” part and frame the research carried out in the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA). The revised paper listed the inclusion and exclusion criteria clearly. We added Figure 1 of PRISMA flow diagram of systematic review. More revised details can refer to Line 80-102.

 

Reviewer 3 Report

This manuscript presents a systematic review of journal articles with empirical research on gamification and project-based learning from 2015 to 2020.

I enjoyed reading the manuscript and obtaining a high-level overview of the work done in this specific research subfield. I liked the proposed questions, how the analyses were conducted, and how the results were presented.

I think that the article could be better situation in relation to the relevant existing literature on gamification in education. It cites a few reports from a few years ago, but I recommend mentioning some of the recent systematic reviews on gamification and education. Considering that gamification of PBL is a subset of the more broad topic of gamification in education, I consider that these are relevant sources. Here is a non-exhaustive list with some examples of recent systematic reviews that could be cited. Perhaps it would be interesting to even compare the results of this research with those existing publications.

* Junior, E., Reis, A. C. B., Mariano, A. M., Barros, L. B., de Almeida Moysés, D., & da Silva, C. M. A. (2019). Systematic literature review of Gamification and Game-based Learning in the context of Problem and Project Based Learning approaches. In 11th International Symposium on Project Approaches in Engineering Education (PAEE) 16th Active Learning in Engineering Education Workshop (ALE). Online: https://www.researchgate.net/profile/Ari-Mariano/publication/334046201_Systematic_literature_review_of_Gamification_and_Game-based_Learning_in_the_context_of_Problem_and_Project_Based_Learning_approaches/links/5d3cd21e92851cd0468c5326/Systematic-literature-review-of-Gamification-and-Game-based-Learning-in-the-context-of-Problem-and-Project-Based-Learning-approaches.pdf

* Subhash, S., & Cudney, E. A. (2018). Gamified learning in higher education: A systematic review of the literature. Computers in human behavior, 87, 192-206. Online: https://www.sciencedirect.com/science/article/pii/S0747563218302541

* Toda, A.M., Klock, A.C.T., Oliveira, W. et al. Analysing gamification elements in educational environments using an existing Gamification taxonomy. Smart Learn. Environ. 6, 16 (2019). https://doi.org/10.1186/s40561-019-0106-1

Regarding the different gamification approaches for PBLs, I have one concern regarding "Type 4: Make a game as the activity of PBL". I think it's important to look into the goal of the game as an activity to see if it's being used as a means to learn something else. It looks that at least in Kapralos et al. (2015) (I didn't check the other ones), the project was related to creating a game, but it didn't seem that the project itself was gamified. The course was about game design and development, and a PBL activity was used, but it doesn't seem that the PBL itself was gamified.

Finally, I appreciate that section 5 presents some recommendations for future developments, but I suggest that this section could be expanded. For example, what results led to each recommendation, maybe how future work can tackle each recommendation, and what can be the impact of each of these future research avenues. I believe that recommendations for future research are a valued kind of contribution from this kind of systematic reviews, but they were presented as just a short half-paragraph in the current version of the manuscript.

Author Response

Thanks for your useful suggestion. We read your recommended references carefully some of which had been cited in our work. We compared the contents of previous studies with our work in the discussion part. The revised texts are marked in read.

We totally understand your concerns. We agree with your that some courses are about game design and development, and a PBL activity was used, which should be excluded from “Type 4”. We mainly focused on the cases that some normal courses use “games” as the goal of PBL. We have explained this issue in the corresponding part (see Line 256-260). We also have checked others and did not find this special case.

Thanks for your useful comments which make sense. We have enriched this part as shown in Line 525-539 of the revised manuscripts. We offer several recommendations for future development of GPBL from four dimensions: systematic design framework, teachers’ competency, theoretical underpinnings of learning mechanism, and evaluation system.

Reviewer 4 Report

This study systematically reviews the articles dealing with gamified project-based learning. Such review appears to be useful for teachers in universities and other educational institutions who would like to develop experimental learning methods including game-based learnings.

 

The manuscript is well-written in terms of showing its originality, research questions, methods, and contribution to the academic community while there are some questions about the results and discussions. Therefore, the manuscripts should be revised before publication so that readers can understand the contents and intentions of authors better. 

 

[1] Difference between type 2 and 3 approaches are not clear.

The difference between the type 2 and 3 approaches appears to be nothing but designers of the games. According to the manuscript, the type 2 approach uses games designed by the operators based on PBL materials and the type 3 approach uses ready-made games. While the authors insist that the type 3 approach is more developed or sophisticated (the type 2 and 3 are named “deep gamification” and “total gamification” in Figure 2) readers cannot understand why the use of prefabricated games is a more developed approach.

 

[2] Advantage of design-based learning is not supported by results.

Authors discussed that the design-based learning is an innovative and creative method for GPBL, but this discussion is not linked to the results of literature review. Readers would especially like to know how the learning outcomes presented in Table 2 were different between design-based (type 4) and non-design-based (type 1-3) approaches. 

 

[3] Figure 4 is not easy to understand.

Quality of Figure 4 should be improved so that it can help readers to understand the discussions in section 4. For example, all diamonds in 4 subcategories of Technology Tools have the same color (sky blue) while diamonds in 6 subcategories of Learning Outcomes have different colors. Another example is that the relationships between the colors for each category (red for Technology Tools and orange for Learning Outcomes etc) and the diamond colors for each subcategory are not clear. Readers cannot understand how these settings correspond to the discussions in section 4.

 

Author Response

Thanks for your kind comments. Following your suggestion, we have read some of the recent systematic reviews on gamification and education, and compare the results of our work with those existing publications. We had improved the results and discussions parts seriously.

Thanks for your kind reply. To improve the language quality of the paper, we invited a native English-speaking colleague to do the proofreading of this paper. It is believed that the language quality of this paper should be better than before.

Sorry to confuse you regarding this point. We did not say that the type 3 approach is more developed or sophisticated. The distinguish between type 2 and type is due to the difference between gamification of education and game-based learning (Subhash & Cudney, 2018). The former focuses on the use of gaming elements while the latter refers to use educational games or serious games. We changed the name of “total gamification” into “game-based learning” which is more suitable. We have explained this issue in the manuscripts 

Thanks for your useful comments. You are right that we need more explanation for this conclusion. The reason why we stated this conclusion is that the proportion of Type-4 studies for Category-A and Category-B is more than other types. We have added the following explanation in the manuscript :

“Learning outcomes in Category-E are suitable for all types of GPBL in the section 3.2. The proportion of Type-4 studies for Category-A and Category-B is more than other types. It may imply that making-game as a design-based learning (DBL) can help develop learners’ innovative and creative literacy.”

Thanks for your kind reminder. You are right that we did not deal with the picture in detail. We have revised the Figure to make it clearer.

Reviewer 5 Report

It is quite an enjoyable text to read. I consider that the article is well argued and presents a clear analysis of the issues it sets out to address. 

Please verify the end of the phrase: "Generally speaking, gamified learning consists of two forms: gamification of learning and game-based learning and [65]" (lines: 382-383). 

 

Author Response

Thanks for your kind reminder. Here is a grammar mistake. We have already revised the phrase. Many people are usually confused with the difference between gamification of learning, game-based learning and gamified learning. Therefore, in this manuscript, we cited the literature of Subhash & Cudney (2018) to distinguish the relationship among these terms.

Round 2

Reviewer 2 Report

The reviewer thanks the authors for their attention, making them available for review, and making the PRISMA database available for verification.

 

Some additional points to adjust:

 

Point 1: Figure numbering is incorrect, please adjust.

 

Figure 2. Empirical research paradigm

Figure 3. Gamified approach paradigm based on gamification degree

Figure 2. Gamified approach paradigm based on gamification degree

 

Point 2: Insert an explanatory text between the figures so that the two figures are not left without a text between the two.

 

Point 3: Align the conclusions and the introduction, clearly bringing the methodology used and the revisions inserted in the text.

 

Point 4: Authors need to refine the form of the MDPI, but this step will be carried out together with the editors when the article is finally accepted for publication.

 

 

Finally, the reviewer found the article very interesting and current in the new times of gamified education, especially in higher education.

Author Response

Thank you for your comments and suggestions. I revised my paper again and uploaded it. 

Author Response File: Author Response.docx