**Game-Based Learning and Gamification for Education**

Editor

**Huei Tse Hou**

MDPI • Basel • Beijing • Wuhan • Barcelona • Belgrade • Manchester • Tokyo • Cluj • Tianjin

*Editor* Huei Tse Hou National Taiwan University of Science and Technology Taiwan

*Editorial Office* MDPI St. Alban-Anlage 66 4052 Basel, Switzerland

This is a reprint of articles from the Special Issue published online in the open access journal *Education Sciences* (ISSN 2227-7102) (available at: https://www.mdpi.com/journal/education/ special issues/education game).

For citation purposes, cite each article independently as indicated on the article page online and as indicated below:

LastName, A.A.; LastName, B.B.; LastName, C.C. Article Title. *Journal Name* **Year**, *Volume Number*, Page Range.

**ISBN 978-3-0365-7514-8 (Hbk) ISBN 978-3-0365-7515-5 (PDF)**

© 2023 by the authors. Articles in this book are Open Access and distributed under the Creative Commons Attribution (CC BY) license, which allows users to download, copy and build upon published articles, as long as the author and publisher are properly credited, which ensures maximum dissemination and a wider impact of our publications.

The book as a whole is distributed by MDPI under the terms and conditions of the Creative Commons license CC BY-NC-ND.

## **Contents**



## **About the Editor**

#### **Huei Tse Hou**

Prof. Huei Tse Hou is a distinguished professor at the Graduate Institute of Applied Science and Technology, National Taiwan University of Science and Technology, Taiwan. He is also the director of the Mini-Educational Game Development Group in E-learning Research Center at the National Taiwan University of Science and Technology (NTUST MEG). His research interests involve gamification, behavioral pattern analysis, and game-based learning.

## *Editorial* **Diverse Development and Future Challenges of Game-Based Learning and Gamified Teaching Research**

**Huei-Tse Hou**

Graduate Institute of Applied Science and Technology, National Taiwan University of Science and Technology, Taipei City 106, Taiwan; hthou@mail.ntust.edu.tw

#### **1. Introduction**

Play is one of the most important activities in human life. To promote the motivation and effectiveness of learners, the use of games in various areas of teaching is becoming increasingly common. Among them, game-based learning emphasizes the design of educational games that can achieve learning objectives. On the other hand, gamification focuses on the use of game mechanics and elements (e.g., points, badges, interactive rules, etc.) to make learning activities fun. These two strategies of using games in teaching and learning not only have the potential to enhance motivation and performance, but also promote the key competencies of learners, such as problem solving, collaborative communication, and strategic thinking. Although there has been a great deal of research on the use of games in education, with the development of technology (e.g., metaverse) and changes in the global environment (e.g., the impact of pandemics), the use of educational games or gamified teaching activities in teaching practice needs to be continuously innovated and evaluated to dynamically bridge the gap between academic research and teaching practice at any time. In this regard, this Special Issue focuses on research on the use of educational games or gamification mechanisms in teaching practice, or reviews of previous research in this area.

#### **2. Content of the Special Issue**

This Special Issue contains a total of 13 articles, which can be broadly divided into 3 categories:

1. Recent research on game-based learning applied to various disciplines:

This category contains articles on game-based learning in various disciplines, including game design and empirical evaluation. The forms of games include digital games and board games, and the subject areas include language, vocational orientation, science, etc. Among them, it is worth noting that three of them are related to escape room game mechanics [1–3]. This type of game requires learners to use their knowledge and abilities to perform problemsolving tasks in a specific virtual space, which is a game mechanism with a great potential for developing problem-solving skills. The articles in this Special Issue contain design frameworks and quantitative and qualitative empirical analyses of escape room educational games for readers' reference in design and evaluation.

2. Studies on the use of gamification mechanisms in teaching and learning:

These studies cover the evaluation of the use of gamification mechanisms in actual classrooms and the analysis of possible bottlenecks in their implementation. In addition, they include the analysis of the effectiveness of specific gamification mechanisms, such as badging mechanisms [4], in the curriculum.

3. Systematic reviews of the literature on the use of games in teaching and learning:

These systematic reviews should be useful for understanding the research on the use of games in teaching and learning in the last decade, including the research issues on the

**Citation:** Hou, H.-T. Diverse Development and Future Challenges of Game-Based Learning and Gamified Teaching Research. *Educ. Sci.* **2023**, *13*, 337. https://doi.org/ 10.3390/educsci13040337

Received: 13 March 2023 Accepted: 23 March 2023 Published: 25 March 2023

**Copyright:** © 2023 by the author. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

use of games in education, and the development of games in teaching and learning in specific subject areas (e.g., [5]).

#### **3. Future Challenges and Research Issues**

From the direction of these articles, we can see that the development of game-based learning and gamification education has become increasingly diverse in recent years, and an increasing number of studies have started to review the past literature and explore the current status and bottlenecks of academic research and teaching practice in this field. Accordingly, the following is a list of potential challenges or key research topics for the future use of games in teaching and learning:

1. Remote or blended game-based teaching solutions:

Due to the impact of the pandemic, schools, training institutions, teachers, and students were forced to use online learning tools for learning, thus making the use of remote teaching technology more popular. Therefore, remote or blended educational games or gamified teaching activities may be a potential research area. The timely integration of the advantages of physical and distance courses can lead to more effective and high motivational teaching and learning activities that can break time and space constraints at a lower cost.

2. Highly authentic situated game-based teaching and learning activities with virtualreal integration:

The trend of metaverse and artificial intelligence technology may cause changes and impacts on the theories, tools, evaluation, and promotion strategies for the use of games in education. Among them, the use of artificial intelligence, augmented reality, virtual reality and mixed reality, and various virtual space interactive tools/editors to construct situated role-playing educational games will be a promising trend, and it is expected to combine real or virtual inquiry-based problem-solving activities to achieve learning transfer (e.g., escape room educational games). However, the integration and use of future technologies involves a high degree of complexity. Teachers often need researchers to propose relevant cloudbased tools, design frameworks, and sample cases that have been validated by empirical research as guides to lower the threshold of introduction in the educational field and truly achieve the purpose of promoting game-based learning to on-site teaching practice.

3. Scaffolding-oriented game-based learning mechanism and multidimensional evaluation:

In view of the increasing number of studies reviewing the previous literature and exploring the current status and bottlenecks of academic research and teaching practice in this field, the possible gaps between theory and practice promotion were also found. Possible reasons for these gaps might include that although many studies on game-based learning have found the effectiveness of games, an educational game is an organic body composed of different game elements and mechanisms. Exactly which game mechanisms promote learning effectiveness or motivation, or which mechanisms do not achieve the expected learning-supporting effects, will require more precision in future research in order to fine-tune specific game mechanisms. In this regard, it is a possible trend to combine game mechanisms with scaffolds to assist the learning process, and to analyze the effects of various mechanisms by embedding specific scaffolds (e.g., conceptual scaffolds, peer scaffolds, or metacognitive scaffolds) in the game mechanisms and analyzing the effects of the mechanisms through a multidimensional post-evaluation of the learners (including the effectiveness, psychological factors, behavioral records, and feedback on the usefulness of the scaffolds). This may become an important research direction in the future.

**Funding:** This research was supported by the projects from the Ministry of Science and Technology, Taiwan, under contract number MOST-111-2410-H-011-004-MY3.

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

**Data Availability Statement:** No new data were created or analyzed in this study. Data sharing is not applicable to this article.

**Conflicts of Interest:** The author declares no conflict of interest.

#### **References**


**Disclaimer/Publisher's Note:** The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

### **ę**

**Ȭ** 

 ǰ ¢ ¢ǰ ¢ ŘŘŖřŖśǰ Dz ¢ řŘřŘȓǯǯǯ

**DZ** ¡ ę ǯ ¢ ¡ ě ę ¢ ǰ ǰ ǻǼ Ȭ¡ ¡Ȭ ¢ ǯ Ȭ Ȃ ȬĚ ¢ Ȃ ¢ǯ ę Ȭ ¢ ¢ ě ¡ Ȃ ǻ Ǽ ǰ ę¢ǯ Ȭ ǰ ¢ Ȭ ¡ǯ ę ¢ ǯ ¢ ě ¢£ǰ Ȭ ę ǯ

**¢ DZ** ęDz Dz Dz Dz Dz Dz Ȭ

#### **ŗǯ**

 ę Ȭ Ȭ ǯ ę Ĵ Ĵ Ȭ ǽŗǰŘǾǯ ǻǼ Ȭ ǽřǾǯ ǰ ěǰ ę Ĵǯ ¡ǰ ǯ ǽŚǾ ę ě ę ¢ Ě ¢ ¡ ę ¢ǯ ǰ Ȭ ę ǽśǾǯ ę ę ǯ

 ¢ ę ¢ ǯ ǰ ǰ ¢ǰ ǰ ǰ ¢ Ĝ¢ Ȭ ǽŜǰŝǾǯ ǰ Ĵ Ĵ ǯ ǽŞǾǯ ¡ Ȃ ǽşǾǯ

 ě ǯ ǰ ¡ ę ě ¡ ǰ ě ę ¢ ǯ ¢ ¡ ę ¢ ǯ

**DZ** ǰ ǯȬǯ ę ǯ *ǯ ǯ* **ŘŖŘř**ǰ *ŗř*ǰ şŖǯ ĴDZȦȦǯȦŗŖǯřřşŖȦ ŗřŖŗŖŖşŖ

 DZ 

DZ ŘŘ ŘŖŘŘ DZ ŗŖ ¢ ŘŖŘř DZ ŗŗ ¢ ŘŖŘř DZ ŗŚ ¢ ŘŖŘř

**¢DZ** Ț ŘŖŘř ¢ ǯ ǰ ǰ ĵǯ Ĵ ǻ Ǽ ǻĴDZȦȦ ǯȦȦ¢Ȧ ŚǯŖȦǼǯ

 ę ¢ ǯ ę ¢£ǯ

#### **Řǯ**

#### *Řǯŗǯ ę*

 ę Ȭ ¡ ǽŗŖǰŗŗǾǯ Ĵǰ ę Ȭ ǯ ę ¢ ¢ ǽŗŘǾǯ ȁęȂ ǰ ǰ ǰ ǰ ¢ ǰ ǰ ǰ ¢ǰ ǽŗřǾǯ Ȭ ¢ ę ¢ ǯ Ȭ ǽŗŚǾǯ ǰ ¢ ǰ ǰ ǰ Ȭ¡ǰ ǰ ǰ ǰ ǰ ǰ Ȃ ǽŗśǾǯ ¢ ę Ȭ ǽŗŜǾǰ ¡¢ ǽŗŝǾǰ ¢ ǽŗŞǾǯ

 ǰ ę ¢ ǯ £ ǯ ǽśǾ ¢Ȭ ŘŖŖŞ ŘŖŗş ę Ȭ ę ř ¢ ¢ ǰ ŘŖŗŚǯ ¢ ę ¢ ę ǽŗŚǾDz ę ǰ Ȭ ǯ ǰ Ȭ ǯ ¢ǰ ę ¢ ě Ȭ ǯ

 ę ¢ ǯ ǰ ę Ȭ ǻŘǼ ǰ ěǰ ¡Ȭ ǽŗŚǾǯ ¢ ę ¡ ¡ ¢ǰ ǰ ǰ Ȭ ǰ ǽŗşǾǯ Ř ǰ ę ¢ ě ¢ ǽśǾǯ ǰ ę ǰ ǰ ǰ ǰ ǽŘŖǾǯ ¡ǰ ę ǰ ¢ Ĵ ȬȬ ǰ Ȭ ǽŘŗǾǯ ǰ ę Ȃ ǰ ǰ ǽŘŘǰŘřǾǯ Ȭ ¡ Ȭ Ȭ ¢ ę £ ¢ ǽŘŚǰŘśǾǯ

 ǰ ę ę ǯ Ȃ ę ¢ ǯ í£ ǯȂ ǽŘŜǾ ¢ǰ Ȭ ę ¢ Ĵ ǯ ¢ǰ ę ¢ ę ¢ ǯ £ ǯ ǽŘŝǾ Ȭ Ȭ ę ę Ȭ ǯ ę ¢ ¡ǯ ǰ ě ę ¢ ě ǽŘŞǾǯ ¢ ¢ǯ

ǰ ¢ǰ ǰ Ȭ ¢ ¢ ǯ ǰ ¢ ǽŘŜǾǯ ǰ ę Ȭ ǯ ¢ Ĝ ę ǯ á£Ȭ íȬÛ ǽŘşǾǰ ¢ ę ǰ ¢ǰ ęǰ ¢ǯ ¡ ¢ Ĝ ǰ Ĝ ǰ ę ǯ Ȭ ¢ǰ ¢ ę ¢ ę ǯ ǰ ¢ ę ǯ Ȭ ǰ Ȭ¢ ¢ ę Dz ¢ ¢ £ ¢ Ȭ ¢ ǯ ¢ǰ Ȃ Ȃ Ȭ ę ǯ

#### *ŘǯŘǯ ę DZ ǰ ǰ*

 ¡£ ę ę Ȭ ǽŜǾǯ ǰ ǰ ǻǼ ǰ ę ¢ ǰ Ȭ ¢ ǽřŖǾǯ ¢ ǯ ¢ ¢ Ĝ ę ǽŘşǾǯ ȯ¢ ǰ ǰ ȯ¢ ǯ ǯ ǽřŗǾ ¡ Ȭ ǰ ǰ ěǰ Ȭ Ȃ Ĵ ¡ ǯ ě ǯ Ȭ ę ¢ ęDZ ǰ ǰ ǰ ǽřŘǾǯ ¢ Ȭ Ȭ ǯ

 ǽřŗǰřřȮřŝǾǯ ǰ ȯ ǰ ¢ǰ ¢ȯ ¡ Ě¡¢ Ĝ¢ǯ ¢ ě Ȭ Ȃ Ě ǰ ¢ ǰ ǯ ǰ ¢ ę ǯ ¡ǰ ¢ Ȭ ǯ ¢ ę ǯ ǰ ¢ ¢ ¢ ǯ ǰ ǯ Ȭ ě ¡ ¡ ę Ȃ ǰ ę ę ǯ

 ę ǰ ¢ǰ Ȭ ¡ ǯ ¡ ǻ Ǽ ę ¢ ǯ ę ¢£ ǻ Ǽ Ȭ Ȭ  ǯ ě ę ǰ ǰ ¢£ǯ ¢ ęȬ ǯ DZ


#### **řǯ ¢**

#### *řǯŗǯ*

 ǰ Ȭ ¢ ǯ ŘȬ ǰ ¢ ǯ ǰ ¢ ǰ Ȭ ¢ ǯ Ȃ Ȭ ǽřŞǾǰ ¢¢ ǯ ¡ ¢ ¢ ¢ ¢ǯ ¢ ¢ ǯ ǰ ǰ ¢ Ȭ ¢ǯ ŗŗ ¢ ¢ǯ ¢ Ȯ ǰ řŝśǯŝ řŞřǯřǰ ¢ǰ ¢ ǯ ę ě Ȭ ę¢ ǻ Dz ƽ −ŖǯřŘřǰ ƽ ŖǯŜśŚ ǁ ŖǯŖśǼǯ ¢ǰ Ȃ Ȭ DZ ǰ ǰ ǽřşǾ Řǯ ǰ ǯ ǰ ¢ Ȃ Ȭ ǯ ǰ ŘŖ ǻŗŖ ŗŖ Ȭ Ǽ ǰ ¢ ǻ ¡ Ǽǯ ę ě ę Ȭ ǻ Dz ƽ −ŗǯŜŖŝǰ ƽ ŖǯŘŚŝ ǁ ŖǯŖśǼǯ ǰ ¢ ǻ ƽ řśDz ŗŖ Řś Dz ƽ ŘŖǯřř ¢Ǽ ǻ ƽ ŚŖDz ŗŞ ŘŘ Dz ƽ ŘŖǯŖř ¢Ǽǯ ę ¢ǰ Ȭ ǯ

#### *řǯŘǯ Ȭ ę ¢*

 ¢ ¡ Ȃ Ȭ ę ŘȬ ǯ ǰ Ȭ ǰ ¡ǰ Ě ǰ ę ǰ ¢ ¡¢ ǰ ǯ Ȭ Ȭ ¢ ǰ ę ¢ǯ ǽřŖǾǰ Ȭ ¢ ¢ ǯ ǰ ŚȮś ǯ ę ǻ ř × ř Ǽ ¢ ǯ ǰ ǯ ǰ

 £ ¢ ¢ ǯ ¡ ¢ ¢ ǯ ¢ ǰ ¡ Ȭ ǯ ¢ ǯ ę ę ¢ ǯ ę ¢ ǰ Ȭ ǰ ¡ ę ǯ ǰ Ȭ ǰ ěǰ ǯ ǰ ǰ ǰ ǯ ¢ Ȭ ę Ȭ ¢ ǻ ŗǼǯ

 **ŗǯ** Ȭ ę ¢ ǻ ǰ ǰ Ǽǯ

 ¢ǰ Ȭ ŘŖ Ȭ ǯ ǰ ȬĚ ¡ ę ¢ Ȭ ǯ

#### *řǯřǯ*

 Ȭ ¢ Ȭ Ȃ ȬĚǯ Ȃ Ȭ Ȭ ¢ ǯ ǰ ŘŖ Ȃ Ȭ ǯ Ĝ ŘŖ ŖǯŞŘŝǰ ¢ Ȭ ¢ǯ Ȭ ę¢ ǻ Ĝ ƽ ŖǯŞŗśǼǯ Ȃ ę ¢ Ȭ DZ ȃ ¢ Ě ę ¢ǵȄ ȃ ę ¢ ę ¢ ǵ ¢ ¢ ǵȄǯ

#### *řǯŚǯ ¢*

 ě ę Ȃ ǰ ¢ ǯ Ȭ ¢ ¢ ǻǼ ¡ Ȭ ę Ȃ ǯ Ȭ ǰ ¢ ě Ȭ Ȭǯ ǰ Ȭ Ȃ ȬĚ Ȭ ǯ Ȃ ę ¢ Ȭ ¢Ȃ ě ¢ Ȭ ǯ Ȃ Ě ǰ £ǰ ǰ ĜȬȬ¢ ǰ ǽŚŖǾǯ Ě ¢ ¢ ¡ Ȭ ǯ Ȭ¢ ŞśǯŝřƖǰ ǯ Ȭ ǰ Ȭ ¢ ¢ Ě Ȃ ǯ

#### **Śǯ**

#### *Śǯŗǯ ¢*

 ¢ ě Ȭ ę ¢ Ȭ ǯ Ȭ ǻ ŗǼǯ


 **ŗǯ** Ȭ ǯ

 Ȭ ¢ ǰ ¡Ȃ ¢ Ȭ ǯ Ȭ ǻ¡Ȃ ƽ řǯŚŖŞǰ ƽ ŗǯŗŖŘǰ ƽ ŖǯřŚŝǼǯ Ȭ ¢ ǰ ǰ ě ǻ Ǽ ǻ ȬǼ ǻ ŘǼǯ ę ě ę ǻ ƽ ŘřǯŖŞŞǰ ƽ ŖǯŖŖŖǼǰ ę ě Ȭ ǻ ŘǼǯ

 **Řǯ** Ȭ ǯ


 **Řǯ** ¢ Ȭ¢ ǯ

Ș ǀ ŖǯŖśǯ

 ěǰ ě ¢£ǯ Ȭ Ȭ ¢ ę ě ě ¡ ǯ ę ě ǻ ƽ −ŜǯŘśŜǰ ƽ ŖǯŖŖŖǼ ǻ ƽ −ŖǯŘŞŚǰ ƽ ŖǯŝŝŞǼǯ Ȭ ǰ ¢ ǯ ¢ ę ě ǻ ƽ řŝǯŞřǼ Ȭ ǻ ƽ śŚǯŖŖǼǯ

 ¡ ¢ Ȭ ě ¡ ǯ Ȃ ¢ ę ǯ ǻ ƽ ŗǯřŜŗǰ ƽ ŖǯŘŚŝǼ Ȭ ǻ ƽ ŘǯśřŚǰ ƽ ŖǯŗŗŜǼ Ȭ ¢ ǯ ǰ ę ě ě £ Ȭ ǻ ƽ −śǯŞŜŗǰ ƽ ŖǯŖŖŖǰ Ȃ ƽ ŖǯřŝŘǼ ǻ ƽ −ŗǯŜŖŝǰ ƽ ŖǯŗŗŘǼǯ ǰ ¢ Ȭǯ Ȭ ¢ ę ě ǻ ƽ řŘǯşŞǼ ǻ ƽ śŚǯŖŖǼǯ

 ǰ ǯ ǯ ǰ ¢ ǰ ę ě ǻ DZ ƽ −ŗǯŚŘśǰ ƽ ŖǯŖŜŜDz DZ ƽ −ŗǯŝŖřǰ ƽ ŖǯŘŜśǼ Ȭ ǻ DZ ƽ −ŖǯŖŜŖǰ ƽ ŖǯŘŞŞDz DZ ƽ −ŘǯŞřŘǰ ƽ ŖǯřŖŘǼǯ ě ¢ǯ ǰ Ȃ ¢ ¢ ę Ȭ ǯ

 £ ę ¢ ¢ǯ ŜŖ ¢ǯ ę ę ¢ ǻ řDz ǰ ƽ ŖǯŖŖŞ ǀ ŖǯŖśDz ǰ ƽ ŖǯŖŖŖ ǀ ŖǯŖśǼǯ ¢ ę ǯ

 **řǯ** Ȭ Ȭ ǯ


Ș ǀ ŖǯŖśǯ

#### *ŚǯŘǯ*

 Ȃ Ě ę ¢ Ȭ ǯ ę ¢ ǻŝŝǯŗŚƖǼǰ ǻŝŗǯŚřƖǼǰ ǻřŚǯŘşƖǼǰ Ȭ ǻřŚǯŘşƖǼǯ Ȭ ¡ Ȃ Ě ę ¢DZ ȁ ¢ ¢ Ȭ Ȃ ǻ ǛŞǼDz ȁ¢ ¢ǰ Ȭ ¢ Ȃ ǻ ǛśǼDz ȁ¢ ¢Ȃ ǻ ǛŗşǼDz ȁ ¢ ¡ Ȃ ǻ ǛŗŞǼDz ȁ ¢ ę ¢ Ȃ ǻ ǛŘřǼǯ

 ǰ ǻşŝǯŗŚƖǼ ¢ ę ¢ǯ ęDZ ¢ ǻŚŞǯśŝƖǼDz ¢ ǰ ¢ ǻřŚǯŘşƖǼDz ¢ ¢ ǻŘŞǯśŝƖǼǯ ¡ Ȃ Ě ę ¢DZ ȁ¢ ¢ ¢ Ȃ ǻ ǛŚǼDz ȁ ǰ ¢Ȃ ǻ ǛŜǼDz ȁ ¢ Ȃ ǻ ǛŗŞǼDz ȁ ¢Ȃ ǻ ǛŗŝǼDz ȁ ¢Ȃ ǻ ǛřśǼǯ

#### **śǯ**

 ¢ ȬȬ ę Ȭ ¢ ě Ȃ ¢Dz Ȭ ǯ ¡ ę ¢ ě ěǰ ¡£ ě ǽŚŗǾǯ ǰ ¢ ǻ ěǼ ǻ ěǼ ǯ ¢¢Ȃ ǽŚŘǾ ¢ ě Ř ǰ ǯ ě ¢ ę ¢ ¢ Ȃ ǯ ǰ ě ę ¢ ¢ ¢ǰ ě ¢ ǽŚřǾǯ Ȃ ¢ ǯ Ȃ ǰ ę ¢ Ȭ ǯ ¡ǰ ¢ ¢ ǰ ¢ ¢ ¢ ę ¢ ę Ȭ ǯ ¢ ¢ǯ Ȭ ę ¢ ¢ Ȃ ǯ ę Ȭ ¢ £ ǯ ǽśǾǰ ¢ ęǯ £ ǯȂ ǽŚŚǾ ę ¢ Ȃ Ȭ ¢ǯ ¢ǰ ¢ ¢ Dz ǰ Ȭ ¢ǯ Ȭ Ȃ ǰ ¢ ¢ ¡ ¢ǯ

 ę ě Ȭ Ȭ ǰ ¢ ǰ ě ¡ Ȃ ǯ ǰ ǽŚśǾ Ȭ ¢ Ĵ Ĵ ǰ Ĝ¢ ę ¢ ě ¢ ǯ

ę ě ę Ȭ Ȭ ǯ ¢ ę ę ¢ ǰ ǯ ę £ ǯ ǽŘŝǾǰ ę ę ¢ ¢ǯ

 Ȃ ¢ ǰ ¢ ¡ ǯ ę ¢ ¡ ǯ ǰ Ȭ ¢ ę Ȃ ¢ ǽŗŜǰŗŞǾǯ ¢ǰ ¢ ę ¢ ¢ǰ ǰ ǰ ǰ Ȭ Ȃ ǰ ¢ǯ ę ǰ ě ¢ ¢ Ȭ ǽřřȮřŝǾǯ ǰ ¢ ę ǽŗşǾǯ ¢ ¢ ¡ ¢ Ȭ ǯ ę¢ǰ ǰ ¢ ǯ ǽŚŜǾ ǽŚŝǾǯ ǰ ę Ȭ ¢ Ȭ ¡ ǯ £ ǽŚŞǾ ¢ ǯ ¢ǰ Ȭ ęǰ £ ǰ Ȭ ¡ ǰ ¡ Ȭ ǯ Ě ¢ ¢ ǰ ¡ǰ £¢Ȃ ǽŚşǾ ¢ Ě ǰ ¢ ¢ ǯ Ȭ Ȃ ¡ ¢ ǯ ǰ Ȃ ǰ ę ě Ȭ ǯ

#### **Ŝǯ**

 ¢ǰ ę ¢ Ȭ ¢ Ȃ Ȭ ¢ǯ ¡ ¢ǯ ¢ ę DZ ǻǼ ę ¢ ¢ ě Ȃ Dz ǻǼ ę ę ¢Dz ǻǼ ę ¢ ¢ ¡ ¢ Ȭ ǰ ǰ ¡Dz ǻǼ Ȃ ¢ ę ¢ǯ  ę ¢ ę ¢ ǯ

 ę ǯ ǰ Ȃ ǽşǾǯ ę ǽŗǰŘǾǯ ¢ǰ ¢ ę ǯ ǰ Ȭ ¢ ę ¢ ǯ ¢ ǯ ǻ ǰ ǰ Ǽ ¢ ǽřŖǾǯ ¢ ę ¢ ¢ Ĝ ǽŘşǾǯ ǰ ę ¢ Ȭ ¡Ȭ ǯ ¢ ǰ ¡ǰ Ȭ ǰ ǽśŖǾǯ

 ¢ ě ę ǰ ǯ Ȭ Ȭ ¢ ǯ ę ¢ Ȭ ǰ ę ǯ ¢ ǯ ę Ȭ ¢ ǰ ǰ ǯDz ¢ǰ ǰ ¢ ¢ ě ę Ȃ ǯ ę Ȭ ¢ ¢ ǯ ǰ ¡ ę ǰ ǰ ǰ ǽřŖǾǰ Ȭ ǯ ¢ ¢ Ȭ ę Ȃ ę¢ǯ

**DZ** ¢ ¢ ŗŖŞŖŗřŜǯ

 **DZ** ¢ ¡ ě ę ǯ

 **DZ** ¢ǯ

 **¢ DZ** ¢ ǯ ¢ Ȃ ¢ǯ

**Ě DZ** Ě ǯ

#### **¡**

 ¢ ǻȦȬǼ

ǯ ǻ ¢ ę ǯǼ ǻŘśƖǼ


ǻǼ ¢ǯ

Ǐ ǐŜǯ ǯ Ĝǵ

ǻǼ ǯ ǻǼ ǯ

ǻǼ ¢ ǯ ǻǼ ǯ

Ǐ ǐŝǯ £ ǯ ¢ ¢ ǵ

ǻǼ ǯ ǻǼ ǯ ǻǼ ǯ ǻǼ ǯ

Ǐ ǐŞǯ ȃȄ ¢ǵ

ǻǼ Ĵ ǯ

ǻǼ ǯ

ǻǼ ¢ ¢ ǯ

ǻǼ ¡ ǯ

Ǐ ǐşǯ ȃȄǵ

ǻǼ ¢ǯ

ǻǼ ¢ǯ

ǻǼ ǯ

ǻǼ ¢ǯ

Ǐ ǐŗŖǯ ǵ

ǻǼ Ȧ ǯ

ǻǼ ¢ ǯ

ǻǼ ę ¢ǯ

ǻǼ ǯ

#### 

ǯDz ¢ǰ ǯDz ǰ ǯDz ǰ ǯǯDz ǰ ǯ Ȭ ¢ǯ *ǯ 
 ǯ ǯ* **ŘŖŗŝ**ǰ *ŘŘ*ǰ ŗȮŘŘǯ ǽǾ
ǯDz ǰ ǯ ę DZ ¡ ¢ ě ę ¢ ǯ *ǯ ǯ ǯ* **ŘŖŗŝ**ǰ *Ŝş*ǰ řŝŗǯ ǽǾ
 ǵ ǵ *ǯ ǯ ǯ* **ŘŖŗŜ**ǰ *ŗŜ*ǰ ŗŝȮřŜǯ
ǰ ǯDz ǰ ǯ ę ǵȯ ęǯ Ȭ Śŝ ¢ ǰ ǰ ǰ ǰ Ŝ ¢ ŘŖŗŚDz ǯ řŖŘśȮřŖřŚǯ
ǯDz ǰ ǯǯ ę ¢ DZ ¢ǯ *ǯ ǯ ǯ ǯ ǯ* **ŘŖŗś**ǰ *ŝŚ*ǰ ŗŚǯ ǽǾ
ǯDz ǰ ǯ  *DZ £* Dz DZ ǰ ǰ ǰ ŘŖŗŘǯ
ȬŜ ǯ *ǯ ǯ ǯ* **ŘŖŗř**ǰ *Řş*ǰ řŚśȮřśřǯ ǽǾ
DZ ǰ ¢ǰ ŘŖŗŜǯ
ǯ ǻǼę ęDZ ǯ Ȭ ¢ǰ ǰ ¢ǰ Řŗ ¢ ŘŖŗŚDz DZ ǰ ĵǰ ŘŖŗŚDz ǯ ŘŜŜȮŘŝŘǯ
ǰ ǯȬǯ ě Ȭ ǰ ǰ Ĵ ¢ ǯ *ǯ ǯ ǯ ǯ* **ŘŖŗş**ǰ *řŚ*ǰ ŚŜŘȮŚŞŘǯ ǽǾ
ǰ ǯ ę DZ ¡ ę ££ ǯ *ǯ ǯ* **ŘŖŗş**ǰ *ŗŚŚ*ǰ ŗŖřŜŜŜǯ ǽǾ
ǰ ǯ ¢ ě 
Ř ǯ ǻǼǰ ǰ ǰ Řŗ ŘŖŗřDz ǯ ŚŝȮśŚǯ

 ǯǰ ǯDZ ¢ǰ ǰ ŘŖŗŞǯ
ǰ ǯǯDz ǰ ǯǯDz ǰ ǯȬǯ ¡Ȭ¢ ǯ ş ¢ DZ ǰ £ǰ ǰ ŘŖȮŘŘ ¢ ŘŖŗŞǯ
ěǰ 
ǯ  *¢DZ ¢*Dz DZ ǰ ǰ ǰ ŘŖŖŚǯ
ǰ ǯ ǯ Ȭě Ȭě Ȭě ¢Ȭ DZ ¢ Ȃ Ě Ȭ ǯ *ǯ ǯ ǯ ǯ* **ŘŖŘŗ**ǰ *śş*ǰ śŚŝȮśŝřǯ ǽǾ
ǰ ǯǯDz ǰ ǯ
ǯ ¢ DZ ¢ ¢ǯ *ǯ ǯ ǯ ǯ ǯ ǯ* **ŘŖŗŝ**ǰ *ř*ǰ řŖȮřŜǯ

**ȦȂ DZ** ǰ ¢ Ȭ ǻǼ ǻǼ Ȧ ǻǼǯ Ȧ ǻǼ ¢ ¢ ¢ ¢ ¢ ǰ ǰ ǯ

**Christopher Keller 1,\*, Anna K. Döring <sup>2</sup> and Elena Makarova <sup>1</sup>**


**\*** Correspondence: christopher.keller@unibas.ch

**Abstract:** This study investigates the effectiveness of the serious game *like2be*, which has been developed to support the individual career orientation process of adolescents by broadening their occupational horizon. In this paper, we present results from an intervention study with *n* = 809 adolescents in Swiss schools at the lower secondary education level. To analyze the extent to which cognitive, affective, and motivational factors are stimulated and what influence they have on expanding knowledge about occupations (measured learning outcome), we applied confirmatory factor analysis, multiple linear regression, and a structural equation model. The results indicate that the stimulation of cognitive processes through serious gaming has a statistically significant impact on learning outcome, although such factors as enjoyment, flow experience, or self-perceived benefits in playing *like2be* did not significantly impact gain in knowledge about occupations.

**Keywords:** serious game; game-based learning; *like2be*; career; orientation; career choice; vocational orientation

#### **1. Introduction**

Technological development is advancing rapidly with considerable impact on education worldwide. One such advance in the educational field is learning through the use of serious games. Their implementation in the classroom not only enriches conventional teaching methods but has also become a popular research topic [1]. Serious games contain major pedagogical potential, as they have been explicitly developed for learning purposes [2,3]. They have been used for a number of different educational purposes [4,5], including in the field of vocational orientation [6–8].

Vocational orientation acquires special importance towards the end of compulsory schooling as adolescents need to decide which career they want to pursue. Career choice is a major biographical event for young people as it strongly relates to individual development and life experience [9]. Furthermore, it is embedded in a personal and multi-layered career orientation process that begins in early childhood [10]. In a career choice process, adolescents strive for a professional career that fits particularly well with their career-related personal characteristics. In the context of career research, the level of fit between these characteristics and those of the professional environment (occupational fit) indicates the probability of professional satisfaction, commitment and career stability [11]. However, occupational fit is a complex construct. It emerges and is transformed through interaction with the environment [12] and makes career orientation a lifelong process.

From early childhood, young individuals begin to develop their own personality, self-concept [13], and role in their social environment [14]. In the process, different interests, skills, competencies, and talents emerge, as well as values, strengths, and limitations. These are important indicators for the identification of career-related personality traits and for the formation of a career self-concept with which to find a fitting entry into the occupational world [13,14]. However, children also learn about different professions at an early age. They recognize occupational activities and functions and learn to distinguish between them.

**Citation:** Keller, C.; Döring, A.K.; Makarova, E. Factors Influencing the Effectiveness of Serious Gaming in the Field of Vocational Orientation. *Educ. Sci.* **2023**, *13*, 16. https:// doi.org/10.3390/educsci13010016

Academic Editor: Huei Tse Hou

Received: 5 December 2022 Revised: 20 December 2022 Accepted: 20 December 2022 Published: 23 December 2022

**Copyright:** © 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

Moreover, in interaction with their social–cultural environment, they observe not only who is typically employed in certain professions, but also what prestige is ascribed to certain professions. In consequence, the children develop their first occupational stereotypes (e.g., occupational gender roles), which influence their future career orientation [15].

Because career choice is closely related to individual development and can be influenced by a variety of factors, it is important that young learners engage in career exploration. Research in the field of vocational education and training (VET) shows that their transition into the professional world is particularly successful when they are intensively engaged in in their own career exploration process [16–18]. For this reason, the serious game *like2be* was developed so that adolescents can learn about professions in a playful way and thus broaden their own career horizons [2].

Since serious gaming offers an innovative alternative to conventional learning methods, empirical research in this research field has been intensified in recent years [1]. To date, however, there is a lack of empirical evidence on effectiveness of serious games for career orientation purposes. To overcome this research gap, we conducted a quasi-experimental intervention study in order to discover whether *like2be* effectively supports learning in the context of career choice. The data indicated that the integration of the serious game *like2be* into school lessons can effectively support adolescents in acquiring knowledge about professions, thereby broadening their career horizons [19].

In the present study, we applied a confirmatory factor analysis (CFA) and calculated a structural equation model (SEM) to provide a differentiated answer to the following research question: To what extent do cognitive, affective, and motivational factors influence learning with the serious game *like2be*?

#### *1.1. Serious Games*

The term serious game covers a wide range of attributes. First, a serious game is basically a game, and represents a series of voluntary and enjoyable activities in which players are involved. Furthermore, it is governed by constraints or rules, and it involves certain goals and possibilities for achieving these goals through moves or actions [20,21]. Second, serious games are usually developed by experts explicitly for a specific target group and their abilities and needs, and since concrete learning objectives as well as mediadidactic models are taken into account during development, they lead to an entertaining mental contest and are able to stimulate learning processes effectively [3].

Although serious games do provide entertainment, their main purpose is learning [22–24]. As such, they are primarily used for educational purposes [22], but are occasionally used with the aim of supporting attitude and behavior change, e.g., to discourage smoking or promote recycling [25,26]. The term Serious Game also includes the sub-category Digital Gamebased Learning (DGBL), which refers to an innovative approach to learning skill acquisition and training through computer games (e.g., serious computer games) and therefore has high educational value and potential [26,27]. However, it should be noted that the two terms (Serious Computer Gaming, DGBL) are sometimes used synonymously [28]. In this paper, the terms Serious Game and Serious Gaming are used to refer exclusively to computer-based games.

#### *1.2. Serious Gaming and Subject Learning*

Serious Gaming has become a popular research topic. In particular, the past few years have seen research into whether serious games benefit learning in specific subject areas. Although studies have demonstrated that serious games can be used effectively for various "exotic" educational purposes, such as to reduce school phobia [5], to raise students' awareness of Internet dangers [29], for vocational training purposes [30–33], particularly in commercially oriented businesses [34], or for addiction and disease prevention [35,36], serious gaming has principally been analyzed for its effectiveness in the context of subjectrelated learning.

For example, Byun and Joung [37] showed that serious games significantly benefited the learning of mathematics among students in grades K-12, although the effect

was considered small and the data analysis implied that there may be other options for learning mathematics more effectively than with serious gaming. Likewise, Tokac and colleagues [38] mentioned that serious gaming leads to small but significantly higher learning gains for students in PreK-12th-grade compared to traditional instructional methods. Talan and colleagues [39] similarly described the effect of serious gaming on mathematics learning as moderate, but, unlike in previous studies, they included outcomes from preschool to university in the analysis. Further, in a currently published review, Hussein and colleagues [40] also reported that serious gaming positively impacted mathematics learning in K-12 levels. Although the studies reviewed focus on a wide range of mathematics-related topics, most studies were limited to the effect of serious gaming on learning arithmetic operations.

The effectiveness of serious games for language learning has also been addressed in various studies. Chiu and colleagues [41] found that serious gaming leads to a medium positive effect size with regard to foreign language learning. They noted that drill and practice games result in a small positive learning effect, while meaningful and engaging games result in a large positive effect size. Talan and colleagues [39] also showed a large effect of serious gaming on language learning. They therefore suggest that second language learning is a promising area in which serious gaming could be more effective than traditional media for students from preschool to university. Similarly, Chen and colleagues [42] mentioned large effects of serious gaming on language learning, especially on vocabulary acquisition. In this regard, a gaming approach appears to be superior to conventional learning methods. Similarly to a previous study, their results suggest that the effects of serious gaming on language learning depend on the characteristics of the game design. In particular, adventure games seem to be more challenging for players and thus prove more effective for learning outcomes.

As in learning math and language, research has shown that serious games have a positive impact on science learning achievements. For example, Tsai and Tsai [43] compared science learning using conventional activities with learning supported by playing serious science games. They report that compared to conventional instruction in science classrooms, students learn significantly better with serious games in science class. Also, Riopel and colleagues [44] showed that serious gaming is an effective alternative for science learning. Especially in terms of knowledge acquisition, knowledge learning and knowledge retention, was serious gaming shown to outperform learning with more conventional methods under conditions of similar time investment or engagement. Likewise, Hu and colleagues [45] describe science learning through serious gaming as particularly effective. They also note that serious gaming had a positive effect on cognition, emotion, motivation, and retention and thus led to learning effects. Finally, Lei and colleagues [46] also examined the effects of learning with serious games compared with traditional teaching methods on the academic achievement of students from elementary school through university. They found that students learned substantially more with the use of serious games than under traditional instruction. As possible moderation effects, the authors consider an increase in student autonomy or motivation.

Overall, the state of research shows that serious gaming is used for different purposes in the school context and supports learning, especially subject learning (e.g., mathematics, language or science). Furthermore, it shows that serious gaming can be used to create conditions conducive to learning.

#### *1.3. Serious Gaming and Factors That Promote Learning*

The cultural historian Johan Huizinga noted that the human species was once called *Homo Sapiens*. When man himself realized that he was probably not as reasonable as assumed, *Homo Faber* was created, man the maker. However, since humans have always also been playful beings, the human species, in addition to Homo Sapiens and Homo Faber, can also justifiably be labeled *Homo Ludens*, man the player [47]. Archaeological research has revealed that mankind engaged in play thousands of years ago. Mostly natural materials such as stones, fruit, grains, wood, and later bones were used for different forms of games. Excavations also confirm that the ball, the doll, and toy animals are among the oldest objects of play. Furthermore, ancient games such as nine men's morris or chess are still played with great fascination today [48]. Moreover, it is emphasized by historians, e.g., Retter [49], that as early as approx. 5000 B.C. the peoples of present-day China, India, Persia, Egypt and Greece began to think about playful activities and their importance in education. Consequently, it can be assumed that humans have always had a marked tendency to play games, which gives games and play a considerable potential.

However, the potential of serious games goes far beyond the historical perspective. On the one hand, serious games fit in with trends of the current age, and on the other, they are exclusively available on devices (e.g., smart devices) that have great influence on people's everyday lives. Concerning the professional development of serious games, many studies have reported that serious gaming has a positive effect on specific learning of subject knowledge among learners of different ages. Other studies have focused less on the learning outcome (i.e., subject related knowledge), but instead on factors that can promote effective learning [4], such as an increase in cognitive performance or the stimulation of flow experience, of emotions (e.g., enjoyment), as well as of self-perceived motivating aspects (e.g., learning benefits through serious gaming).

#### 1.3.1. Stimulation of Cognitive Processes with Serious Gaming

In a meta-analysis on serious gaming and its influence on cognitive processes and motivation, Wouters and colleagues [50] found that serious games are more effective in learning and retention, but no more motivating than conventional instruction methods. However Mao and colleagues [51] mention that serious gaming is particularly effective because it often focuses on problem-based learning, where players have to try different strategies to solve problems in order to progress within the game. Additionally, Jong and colleagues [52] found that serious gaming has a positive learning impact when players perceive the game as a challenging task that also offers them the prospect of winning. Furthermore, Sailer and Homner [53] find that gamified learning methods have significant but small overall effects on cognitive, motivational, and behavioral outcomes. Although they point out that gamified learning methods and serious gaming share the same focus on learning value beyond entertainment, but they differ in nature. They also add that collaborative and competitive aspects in learning games are of great importance for cognitive stimulation. Moreover, serious games have been effectively used to increase players' cognitive performance level in terms of fostering attention capacity [54], enhancing short-term and visual memory performance [55], and reducing the fear of learning failure, thus having a positive effect on learning motivation [56].

All studies showed that serious gaming stimulates cognitive processes and that this has a positive effect on learning. Here, the focus is on increasing motivation. A similar relationship exists between enjoyment and effective learning.

#### 1.3.2. Enjoyment through Serious Gaming

Wouters and colleagues [50] mention that it is generally assumed that serious games have a similar motivational appeal, such as high entertainment value, much as commercial computer games have. Nevertheless, they add that serious games cannot compete with commercial computer games in terms of gameplay, game content and game design and therefore have a less motivating effect. This is especially evident in the sandbox game MineCraft. Although it is not a serious game by definition, considerable effort has been put into the development of an Educational Edition in recent years. This edition was developed by educators for educators and includes innumerable possibilities to use MineCraft in a school context in a way that enhances learning processes. Owing to the Educational Edition, MineCraft can be understood as a kind of serious gaming. In various studies, MineCraft is said to have great educational potential, which is related to its high level of enjoyment [57–59]. One indicator for this is its open, creative gameplay which encourages exploration and learning—even requires it [60].

Moreover, self-confidence and self-efficacy are increased through serious gaming [56,61], which is strongly related to the enjoyment of playing serious games, and which contributes to motivational or behavioral learning outcomes [54,61]. Although Breien and Wasson [62] consider the effect of serious games on learning outcomes to be positive in their review, a narrative or story included in the gameplay was found to be a key aspect of the effectiveness of serious gaming. In particular, they identified four narrative or story-related possibilities that increase the positive impact of serious games on enjoyment, engagement, and, consequently, learning: a virtual, quest-based game landscape that can be explored in-game, changeable objects used to overcome challenges and achieve goals, and game-relevant avatars whose own story makes an exciting contribution to the game story, or the integration of real, significant events in human history into the game story.

According to Iten and Petko [29], however, although some scientific evidence suggests that serious gaming may enhance motivation and positive emotions (especially enjoyment), which are strong factors in positive learning outcomes, connections between fun and learning have not yet been fully analyzed by empirical research. They found no clear connection between fun and learning success in their intervention study evaluating the effectiveness of a serious game in Swiss schools. Yet their data showed that greater enjoyment of serious gaming led to greater interest in the learning content, and so they concluded that serious gaming is an effective alternative for introducing a new topic and increasing motivation during the learning process [29].

Thus, studies have shown that serious games can be fun and entertaining and that they consequently promote learning. Nevertheless, it is assumed that they must provide players with a very high level of enjoyment in order to be effective for learning. Another factor related to effective learning is the experience of flow.

#### 1.3.3. Stimulation of Flow Experience with Serious Gaming

Serious gaming can foster intellectual competition in an entertaining way [22–24], and because it is an act of doing (i.e., learning by doing) it contributes to a flow experience. If certain factors are present during the game (such as comprehensible game goals, rules and gameplay, feasible challenges, player concentration, or feedback or assistance), players can fall into a flow state [63]. According to Csíkszentmihályi [64], this state is a situation of complete absorption or engagement in an activity and is of great importance to learning processes. If learners additionally experience a particularly high level of enjoyment while playing a serious game, they can fall into a GameFlow state [65]. Fu and colleagues [63] assume that players in a GameFlow state increase their motivation and engagement, which positively stimulates their learning process. Finally, Wronowski and colleagues [66] reported that students who used a serious game (Deadly Distribution) for the purpose of learning statistics were highly absorbed during gameplay and showed higher levels of engagement and ultimately interest in the subject of statistics than those who used conventional methods.

For those who are completely immersed in an activity, totally focused and exclusively engaged, we talk about the mental sensation of flow. In the context of learning, this creates an environment conducive to learning. Research has shown that serious games can lead to such flow experiences. In addition to stimulating the flow experience, serious games can have an impact on self-perceived benefit through the activity.

#### 1.3.4. Self-Perceived Benefit through Serious Gaming

Although serious games effectively promote learning, the use of supplemental materials appears to further increase learning outcome. Wouters and colleagues [50] noted that serious games were most effective for learning processes when supplemented with other teaching materials rather than when used as the sole teaching method. In a learning process, serious games in combination with specific didactic support (e.g., reflection, modeling, collaboration, modality, feedback, or personalization) led to well-structured prior knowledge that helped learners to build on and continue to learn successfully [50,67]. Similarly, Chen and Law [68] mentioned the benefit of additional support. They analyzed the effectiveness of in-game scaffolding in the serious gaming process and concluded that with additional scaffolding, the positive effect of serious gaming on student motivation and learning performance was significantly enhanced. Moreover, for Mao and colleagues [51], additional support also represents an important aspect of effective learning with serious games. If players receive beneficial feedback while gaming or afterwards, it helps them to better understand and reflect on information use as well as on the decisions made in the serious gaming process, which ultimately supports learning. In order to effectively support learning processes and make learning effects visible, learning options such as serious games must be methodologically and didactically embedded in the school context [69].

In other words, serious games can support learning processes, but purposeful application and the use of additional learning materials can strengthen the learning effect. It appears to be important that additional deepening of the subject matter of the serious games consolidates learning on the one hand, and on the other, the players come to recognize that they can learn successfully through serious gaming.

#### *1.4. like2be—A Serious Game for Vocational Orientation*

In school lessons on "Vocational Orientation", adolescents at the Swiss secondary level are specifically stimulated for an intensive and individual vocational exploration phase, the aim of which is to help them succeed in making the transition to the world of work [70]. An innovative possibility for exploration in the context of career choice is the serious game *like2be*. [6,71] *Like2be* was developed as part of a research project supported by the Swiss National Science Foundation (SNSF) with the involvement of various experts. It is a web-based online game and can be played for free in German, French and Italian *(*www.like2be.ch). In addition, this serious game was developed specifically for adolescents in their career orientation process who will have to make an initial career choice decision by the end of compulsory schooling [2].

With *like2be*, young people can expand their career choice horizons in a playful way. It is a simple point-and-click game without narrative. As players, they take on the role of a personnel agent who must place applicants in suitable jobs or training positions based on their application folders and CVs within a specified time (see Figure 1). A suitable placement is followed by promotion at the end of the game round and the level of difficulty increases (i.e., more vacant jobs to choose from). In the case of unsuitable placements, the player is threatened with dismissal from the virtual job agency [2].

**Figure 1.** The *like2be* Gameplay. (1) Applicant, (2) job or training position, (3) applicant's CV, (4) specified time (game round).

*Like2be* includes different jobs or training positions, a variety of cartoon-style game characters (avatars), and various personal profiles (application folders and CVs). The jobs or training positions integrated into like2be are taken from the official Swiss information portal for career, study, and career guidance (www.berufsberatung.ch). The serious game *like2be* does not contain a specific roster. Instead, a randomization mechanism ensures that all application folders and CVs occurring in the game are randomly assigned to a game character (avatar) each time the game is started. Consequently, players can thereby only play successfully if they compare skills, abilities, and individual wishes of applicants with the job jobs or training positions offer [2].

#### **2. Study Design & Methodology**

In 2021, we conducted a quasi-experimental intervention study with adolescents at the secondary school level to analyze the extent to which *like2be* can broaden young adults' career choice horizons. The total sample included *n* = 809 adolescents from German-speaking Switzerland. Of these, 49.4% were female, 48.2% were male, 2% assigned themselves to another gender. Their average age was 13.77 years (*SD* = 0.82). After incomplete data were excluded, a final sample including 532 adolescents remained. Of these, *n* = 415 were in the intervention groups and played *like2be* in the classroom; 48.7% were female, 50.1% were male, and 1.2% assigned themselves to another gender. Their average age was 13.78 years (*SD* = 0.87). In the control group there were *n* = 117 adolescents who did not play *like2be*, of whom 57.3% were female, 38.5% were male, and 4.3% assigned themselves to another gender. Their average age was 13.62 years (*SD* = 0.61). The participants were surveyed at two measurement points (T1, T2). Between T1 (November 2021) and T2 (December 2021), there was an intervention phase of four weeks. During the intervention phase, the adolescents in the experimental groups played *like2be* twice for two entire lessons of 45 min each. During the two lessons, the teachers were present but did not offer any assistance; except for technical problems. The adolescents in the control group did not play *like2be*.

We found that the intervention with the serious game had a positive effect on expanding knowledge about occupations [19]. In this respect, students in the intervention groups could effectively expand their job-related knowledge with the serious game *like2be* (e.g., knowledge about the job, requirements, or job benefits). Their post-test scores (i.e., knowledge about occupations) were significantly higher than those of the control group.

Since recent research has shown that serious gaming can effectively promote learning on different levels, such as the acquisition of subject-specific learning content, but also the stimulation of factors that promote learning, we focus on the latter. In the present study we aim to describe more precisely to what extent certain factors (cognitive processes, flow, enjoyment, and subjectively perceived benefit) influenced the learning outcome with the serious game *like2be*. Therefore, we tested the following four hypotheses among those students who played *like2be* during the intervention:

**H1.** *The serious game stimulates the cognitive processes and thus supports the expansion of knowledge about occupations.*

**H2.** *The serious game is highly enjoyable and thus supports the expansion of knowledge about occupations.*

**H3.** *The serious game stimulates the flow experience and thus supports the expansion of knowledge about occupations*.

**H4.** *The serious game leads to a high level of self-perceived benefit and thus supports the expansion of knowledge about occupations.*

#### *2.1. Operationalization of the Constructs*

The study included two measurement points (T1, T2) in which the adolescents completed an online questionnaire. At both measurement points, we asked the participants how they would rate their knowledge of each occupation from *like2be* on a 6-point Likert

Scale ranging from "nothing" to "a lot". Since we found a significant group difference showing that the experimental groups learned more than the control group [19], we used the score from the post-test (T2) as the outcome variable for the data analysis.

Further, to evaluate the serious game *like2be* for its effectiveness, we created a new questionnaire based on the "Evaluation of the learning game AWWWARE" questionnaire [29] and the eGameFlow questionnaire [63]. We used items to measure eight of ten latent factors from the scale "Evaluation of the learning game AWWWARE", developed by Iten and Petko [29] based on the EGameFlow Scale (Fu and colleagues [63]). Furthermore, we added items to the scale to measure the latent factor "Challenge" because of its high importance regarding the effectiveness of a serious game [63]. We used the three valid and reliable items with the highest factor loadings from the EGameFlow Scale. Our new scale "Effectiveness Scale for *like2be*" (see Table 1) consisted of nine subscales (factors): (1) Goal clarity (three items): the game objectives should be clear from the beginning and throughout the game. (2) Controlling the game (three items): game controls should be simple so that players can quickly navigate through the game and focus on its content or tasks. (3) Strategic approach (three items): the game should stimulate cognitive processes during play so that successful game strategies are developed. (4) Use of prior knowledge (three items): The game should be a game for everyone. In this respect, all players should be able to play successfully, regardless of their prior knowledge of the subject matter. (5) Flow during gameplay (three items): the game should lead the player into a state of immersion. (6) Enjoyment of the game (three items): the game should be entertaining and fun. (7) Challenge of the game (three items): the game should offer challenges that fit the player's level of skills and the difficulty of these challenges should change in accordance with the increase in the player's skill level. (8) Learning Outcome (four items): the game should increase the level of knowledge or skills of the players while meeting the game objectives. (9) Motivational Outcome (three items): the game should create motivation to explore a certain topic.

**Table 1.** Effectiveness scale for *like2be*.



**Table 1.** *Cont.*

(e): Items excluded for the final adjusted model.

#### *2.2. Scale Design*

To investigate the factorial structure of the Effectiveness scale for *like2be*, we computed a confirmatory factor analysis (CFA) using the lavaan package in R. To determine whether the data were multivariate normally distributed, we applied the Mardia test for skewness and kurtosis [72]. Since both *p*-values were significant, we concluded that the data did not have a multivariate normal distribution. To correct for the violation of the multivariate normal distribution, we performed robust estimation of the model using the Satorra–Bentler adjustments [73,74] for all calculations.

First, we analyzed the basic model (Effectiveness scale for *like2be*) for its model fit. For this purpose, we evaluated the global fit, the local fit, and the parameter estimation. According to Hu and Bentler [74] the fit indices (see Table 2) suggest that the model did not fit the data.

**Table 2.** CFA fit indices of basic model 1.


Chi2: Chi-square, CFI: Comparative Fit Index, RMSEA: Root Mean Square Error of Approximation, CI: Confidence Interval, SRMR: Standardized Root Mean Square Residual.

Second, we analyzed the basic model (Effectiveness scale for *like2be*) for its model fit a second time, but this time we tested a one-factor model. Therefore, we added all 22 items to one factor. However, according to Hu and Bentler [74] the fit indices (see Table 3) suggest that the second model did not fit the data either.

**Table 3.** CFA fit indices of basic model 2.


Chi2: Chi-square, CFI: Comparative Fit Index, RMSEA: Root Mean Square Error of Approximation, CI: Confidence Interval, SRMR: Standardized Root Mean Square Residual.

Third, we reduced the basic model 1 (Effectiveness scale for *like2be*) to an adjusted model with the four subscales (factors): (1) Cognitive learning process, (2) Enjoyment of the game, (3) Flow during the game, and (4) Learning Outcome. The first subscale refers to competitive, challenging, problem-oriented gameplay and other mental processes that stimulate players' cognition during their gameplay (e.g., goal clarity, gaming strategy, use of prior knowledge, learning by doing). In the context of serious gaming research, the stimulation of cognitive processes is considered to be particularly effective for learning because it can improve the attention performance, short-term memory and visual memory [51,53–55]. Since enjoyment of the game was attributed great importance [29,54,61,63], we created a second subscale including all items related to fun, entertainment, enjoyment or items that appeal to players on an emotional level. Although the flow experience was described as a component of motivation [64,75], the state of immersion is considered to be a very important component for the effectiveness of serious games [63] and we therefore created a separate third subscale for flow. Finally, serious games should increase the level of knowledge and skills of players while also meeting the games' objectives to encourage the players to keep playing [63,65]. Consequently, learning outcome as the fourth subscale included all items related to self-perceived learning effect, achievement of game objectives, or learning progress.

Since the adjusted model was improved, but the fit indices still pointed to an insufficient model fit (see Table 4), we identified and excluded certain items (see items with (e) in Table 1) from the model based on the modification indices. Criteria for the exclusion of items were: items that lead to a large improvement in the model and (a) are indistinguishable from other items or ask the same question in different words, (b) can be assigned to more than one factor, or (c) correlate strongly with one or more error terms of other items. We also omitted all items of the initial factor "Controlling the game" because *like2be* is a point-and-click game. Hence, the game control did not present any obstacles, particularly not for adolescents.


**Table 4.** CFA fit indices of adjusted model.

Chi2: Chi-square, CFI: Comparative Fit Index, RMSEA: Root Mean Square Error of Approximation, CI: Confidence Interval, SRMR: Standardized Root Mean Square Residual.

Fourth, we created the final adjusted model, including the four subscales (factors): (1) Cognitive learning process (seven items, Cronbach's α = 0.81), (2) Enjoyment of the game (four items, Cronbach's α = 0.89), (3) Flow during the game (four items, Cronbach's α = 0.80), and (4) Learning outcome (seven items, Cronbach's α = 0.89). The fit indices of the final adjusted model (see Table 5) also showed that the model fit sufficiently. Thus, the CFI was >0.95, the RMSEA was <0.06 and statistically not significant, and the SRMR was <0.08, which, according to Hu and Bentler [74], is a good model fit. Also, in terms of parameter estimation, the model contained no negative variances, and the standardized loadings between the items and the factors were high throughout the model; the loadings within the factors were similarly high (see Table 6).


**Table 5.** CFA fit indices of final adjusted model.

Chi2: Chi-square, CFI: Comparative Fit Index, RMSEA: Root Mean Square Error of Approximation, CI: Confidence Interval, SRMR: Standardized Root Mean Square Residual.

#### **Table 6.** Final adjusted model.


#### *2.3. Data Analysis Methods*

Again, using the lavaan package in R, we conducted a multiple linear regression and structural equation modeling (SEM) to analyze how the four factors identified above (cognitive learning process, enjoyment of the game, flow during the game, and learning outcome) predicted knowledge gain. In this process, the latent (exogenous or non-observable) factors are theoretical constructs that cannot be observed directly. Therefore, latent factors include observable/manifest (endogenous or dependent) variables. In our model, the four dimensions are latent factors (Cognitive learning process, Enjoyment of the game, Flow during the game, Learning outcome) emerged based on the 22 observed variables (C1-L7) (see Table 4).

With SEM, we performed a path analysis to (1) understand more precisely the correlation patterns among factors in our model, and (2) to show how much of the variation in the learning outcome of the intervention with *like2be* can be explained by the latent factors.

#### **3. Results**

Based on our CFA, we regarded clp (cognitive learning process), ejm (enjoyment of the game), flw (flow during the game), and loc (self-perceived learning outcome) as latent factors. To analyze to what extent the four latent factors influenced the increase in knowledge about occupations through playing the serious game *like2be*, we defined knowledge of occupations measured in the post-test as the outcome variable. To analyze to what extent the four latent factors influenced the increase in knowledge about occupations due to playing *like2be*, we defined the knowledge about all occupations from the serious game measured in the posttest as the outcome variable kaj (knowledge about jobs).

To determine the direction and strength of associations among the latent factors, we performed Pearson's correlation. The correlation matrix (see Table 7) shows positive high and statistically significant correlations within the latent factors.


**Table 7.** Correlation matrix of latent factors.

clp: Cognitive learning process, ejm: Enjoyment of the game, flw: Flow during the game, loc: Learning outcome, \*\*\* *p* < 0.001.

First, a multiple linear regression showed that the overall final adjusted model explained 17% of the variance in the outcome variable (*F*(4410) = 20.87, *p* < 0.001). Second, we analyzed the impact of the latent factors using SEM. The initial model fitting indices by using the Satorra–Bentler adjustments for the SEM were as follows: χ2 (df) = 1.284 (221), CFI = 0.960, RMSEA (*p*-value) = 0.043 (*p* = 0.978), SRMR = 0.044. Overall, the SEM model fitted the data well. In this context, the factor clp explained 14% of the outcome variable kaj (β = 0.43, EST = 13.97, SE = 6.11, *z* = 2.29). Additionally, the influence of the latent factor on the outcome variable was statistically significant (*p* = 0.022). However, the three remaining latent factors ejm (β = 0.12, EST = 1.78, SE = 1.94, *z* = 0.91, *p* = 0.361), flw (β = −0.23, EST = −3.91, SE = 3.62, *z* = −1.08, *p* = 0.280), and loc (β = 0.10, EST = 1.93, SE = 2.91, *z* = 0.66, *p* = 0.507) had lower factor loadings than clp, were not statistically significant, and therefore showed no significant effect on the outcome variable (see Figure 2).

**Figure 2.** SEM path model (final adjusted model). clp: kaj: Knowledge about jobs, Cognitive learning process, ejm: Enjoyment of the game, flw: Flow during the game, loc: Learning outcome, \*\*\* *p* < 0.001.

#### **4. Discussion**

In this study, we emphasized the effectiveness of serious gaming for purposes of vocational orientation. In particular, we examined the extent to which cognitive, affective, and motivational factors have an impact on measured learning outcomes in the context of an intervention with the serious game *like2be*.

First, we developed a scale to evaluate the effectiveness of *like2be*. The basic model (Effectiveness Scale for *like2be*) included nine factors (see Table 1). Despite adopting all items from reliable and valid scales (Iten and Pekto [29]; Fu and colleagues [63]), our basic model was found to be inadequate in the course of the CFA. The fit indices did not indicate a good model fit. Consequently, we optimized the model. For this, we reduced our model to four dimensions based on current research. The first dimension included items in relation to measuring the extent of stimulation of cognitive processes (1st factor), which were considered to be particularly effective for learning [51,53–55]. The second dimension referred to items that measured the extent of enjoyment of the game (2nd factor), which were considered to be of great importance in terms of learning success [29,54,61,63]. The third dimension contained items measuring the flow experience (3rd factor), as state of immersion is considered to be a very important component in the effectiveness of serious games [63]. Finally, the fourth dimension included items measuring self-perceived learning outcome (4th factor), which motivates gamers to continue playing [63,65]. After reducing the basic model to four dimensions, we excluded a few items because they were either indistinguishable from other items or severely disrupted the model. Despite the fact that we could show with a CFA that our final adjusted model fits the data well, we should note that it deviates considerably from the factor models in past studies. One possible explanation for this could be that the serious games in the two studies we referred to were not comparable to the serious game *like2be* in terms of gameplay, subject matter, content, controls, etc. However, our final adjusted model is reliable and valid and is suitable for further evaluations of the effectiveness of serious games, especially in the field of vocational orientation.

Second, by applying regression analysis, we investigated the overall impact of the four latent factors on increase in knowledge about occupations. Our results indicate a statistically significant impact explaining 17% of variance. Despite the proportion of explained variance being rather small, cognitive, affective, and motivational factors turned out to positively influence knowledge acquisition and contribute to effective learning with serious games.

Third, we conducted a SEM to analyze the extent to which the four latent factors influence the measured increase in knowledge about occupations. Regarding H1, the results showed that stimulation of cognitive processes while playing *like2be* had a positive and statistically significant effect on increased knowledge about occupations. Therefore, we accept H1, that serious gaming stimulates cognitive processes and thus supports the expansion of knowledge about occupations. Regarding the importance of problem-based learning in gameplay [51], and game challenge, as well as the prospect of winning [52], our data analysis showed that the challenge of achieving the game objectives, the game strategies necessary to do so, and also prior knowledge, all had an impact on the expansion of knowledge. According to Sailer and Homner [53], collaboration and competition are important for cognitive stimulation. Although, *like2be* offers few possibilities for collaborative play, it includes competitive aspects because of the game time limit. Despite not evaluating effectiveness on attention capacity [54], short-term and visual memory performance [55], we conclude that the serious game *like2be* stimulates players' cognition with its competitive, challenging, and problem-oriented gameplay during the game and therefore has a positive impact on knowledge enhancement.

Furthermore, the data analysis indicated that how much players enjoy playing *like2be* has no significant effect on increasing knowledge about occupations. Thus, we reject H2, that the serious game *like2be* is highly enjoyable and supports the expansion of knowledge about occupations. In accordance with Iten and Petko [29], we did not find a clear connection between enjoyment and learning outcome (i.e., increased knowledge) either. In line with Wouters and colleagues [50], we assume that *like2be* was not as entertaining as other commercial computer games and was therefore less motivating. Additionally, the lack of an exciting narrative or story in *like2be* with high exploration, challenge, or success aspects may have led to a lower impact of this serious game on learning outcomes [62].

Moreover, the results of SEM indicated that the flow experience while playing *like2be* had no significant effect on improving knowledge of occupations. Accordingly, we reject H3, that the serious game *like2be* stimulates the flow experience. Although, the serious game *like2be* provided intellectual competition in an entertaining way [22–24] and aspects such as comprehensible game goals, rules and gameplay, plus feasible challenges were given, it neither led to a GameFlow state [63,65] nor to a situation of complete absorption in the gaming activity mentioned by Csíkszentmihályi [64]. A possible explanation is offered by the study results relating to enjoyment. Thus, we assume that players did not experience a particularly high level of enjoyment while playing *like2be* for several reasons.

In terms of H4, the data analysis showed that the extent of subjectively experienced benefits through *like2be* had no significant effect on increasing knowledge of occupations. Therefore, we reject H4, that the serious game *like2be* leads to high level of self-perceived benefit. In this context, *like2be* did not contain in-game scaffolding [68] or did it provide players with a way to ask for feedback [51]. In this respect, *like2be* does not include in-game opportunities for additional reinforcement of the content, which would consolidate learning and help players realize that they are learning successfully through serious gaming. However, there is additional teaching material to deepen learning, following Wouters and colleagues [50], who mention that serious games are most effective for learning when supplemented with additional teaching materials. Furthermore, a set of analog activities for the serious game *like2be* already exists and includes reflection, modeling, collaboration, or personalization activities designed to support learners and help them continue to learn successfully [50,67,69]. The materials have even been favorably evaluated but were unfortunately not part of the current investigation.

#### **5. Conclusions**

Recent studies have shown that serious games can be used effectively for various educational purposes, in particular to promote the acquisition of specific technical knowledge, but also to enhance factors that can promote effective learning. In this study, we focused on the impact of the serious game *like2be* on factors conducive to learning, such as increasing cognitive performance and enjoyment of the game, stimulation of a flow experience and self-perceived benefits through serious gaming.

As a result of our data analysis, we conclude that *like2be* stimulates cognitive processes in players, which consequently has a positive effect on their expansion of knowledge about occupations. However, we were not able to demonstrate that the impact on enjoyment, flow experience, and self-perceived benefit through *like2be* promoted the expansion of knowledge about occupations. Indeed, these findings are in accordance with or can be explained by previous research. For example, enjoyment of a serious game was found to be very important for learning outcome [63], but to provide enjoyment, serious games must include an exciting narrative or story [62]. Additionally serious games must be as highly enjoyable as commercial computer games in order to enhance motivation and thus learning outcomes [50]. Since the fun factor of serious games is not considered competitive with commercial computer games for several reasons [50], it is not surprising that in their intervention study Iten and Pekto [29] found no impact of enjoyment on measured learning outcomes. Considering that enjoyment of the game is a relevant indicator of immersion in an activity [63–65], we assume that the perceived level of enjoyment while playing *like2be* was too low and therefore failed to promote a Flow experience. Further, studies have shown that when players had an in-game opportunity to request scaffolding [68] or feedback [51], learning processes with serious games were effectively enhanced. Due to the fact that *like2be* does not provide in-game scaffolding or feedback, it was not surprising that the data showed no impact on expansion of knowledge about occupations. Furthermore, Wouters and colleagues [50] have shown that additional instructional materials such as reflection, modeling, collaboration, or personalization [50,67,69] can enhance the impact of serious gaming. Although, such additional teaching materials have been developed and are available for *like2be*, their impact was not considered in the context of this study.

On the positive side, our scale (final adjusted model, see Table 6) can be used for future evaluations of the effectiveness of serious games, as it was found to be reliable and valid after testing with confirmatory factor analysis (CFA).

In conclusion, the serious game *like2be* stimulates cognitive processes through its competitive, challenging, and problem-oriented gameplay, consequently promoting effective learning. Thus, it represents an effective tool and should be considered for use in career choice classes.

For the future, we recommend intensifying research in the field of the effectiveness of serious games, especially in the field of vocational orientation. Regarding *like2be*, the effect of the additional teaching material on cognitive, affective and motivational factors conducive to learning should be analyzed. Furthermore, our data analysis shows that the game can be optimized in terms of enjoyment, flow experience, or self-perceived benefits. In this respect, *like2be* should be evaluated with a view to identifying opportunities to add more fun, flow experience indicators, or a scaffolding component to the gameplay. With regard to an optimization of *like2be* as well as the development of other serious games, we therefore recommend a stronger collaboration between game developers, educational experts and the target groups of players. We consider educational experts (e.g., teachers or educational scientists) as ideal contacts for developing specific scaffolding as well as an adequate and exciting narrative. Nevertheless, we suggest asking the target groups pf players (e.g., students) about which aspects make computer games exciting and appealing for them. The findings can be incorporated into the development of new or the optimization of existing serious games and thus generate a high degree of enjoyment and flow experience. **Author Contributions:** Conceptualization, C.K., E.M. and A.K.D.; methodology, C.K., E.M. and A.K.D.; validation, C.K.; formal analysis, C.K. and A.K.D.; investigation, C.K., E.M. and A.K.D.; resources, C.K. and E.M.; data curation, C.K.; writing—original draft preparation, C.K.; writing review and editing, C.K., E.M. and A.K.D.; visualization, C.K.; project administration, C.K. and E.M. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Institutional Review Board Statement:** This study was reviewed and approved by the Ethics Committee and the Data Protection Officer of the University of Basel (date of approval 24 August 2021).

**Informed Consent Statement:** Informed consent was obtained from all subjects involved in this study.

**Data Availability Statement:** The data are not publicly available due to international data sharing agreements.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**


**Disclaimer/Publisher's Note:** The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

## *Article* **Room2Educ8: A Framework for Creating Educational Escape Rooms Based on Design Thinking Principles**

**Panagiotis Fotaris 1,\* and Theodoros Mastoras <sup>2</sup>**


**\*** Correspondence: p.fotaris@brighton.ac.uk

**Abstract:** By immersing learners in a playful, interactive, and engaging experience, Educational Escape Rooms (EERs) have been found to enhance learners' motivation, help them to develop 21st century skills, and improve knowledge acquisition. As research into EERs is still in a preliminary phase, no unified framework about how to design them has been established yet. Additionally, existing frameworks rarely validate the quality and efficacy of the frameworks themselves in terms of usability and usefulness. Therefore, the present paper proposes Room2Educ8, a learner-centred framework for EER design that follows Design Thinking principles. It provides detailed heuristics for empathising with learners, defining learning objectives and constraints, adding narrative, designing puzzles, briefing and debriefing participants, prototyping and playtesting, documenting the whole process, and evaluating the EER experience. A mixed-methods internal validation study based on Instructional Design model validation was conducted with 104 postgraduate students between 2018–2022 to assess the framework's integrity and use. The study findings suggest that Room2Educ8 can be proposed as a valid tool for developing a wide range of EER types that cover a variety of topics. Its well-described and practical steps make it appropriate for educators regardless of a lack of prior experience in EER design.

**Keywords:** escape room; game-based learning; gamification; design thinking; educative innovation; framework; technology-enhanced learning

#### **1. Introduction**

Escape rooms (ERs) are emerging as a new type of learner-centred activity designed to enhance students' learning and 21st century skills in primary, secondary, higher education, and professional development programs [1–3]. An educational escape room (EER) can be defined as an instructional method requiring learners to participate in collaborative playful activities explicitly designed for domain knowledge acquisition, skill development, or behavioural change so that they can accomplish a specific goal (e.g., participants must escape from a physical or virtual room, solve a mystery, find a hidden item, prevent a disaster, break into a vault, etc.) by solving puzzles linked to unambiguous learning objectives in a limited amount of time [4].

Escape room puzzles can be categorised as: (1) cognitive, which make use of the players' thinking skills and logic; (2) physical, which require body movements or the manipulation of artefacts to overcome a challenge; and (3) meta-puzzles, i.e., puzzles that combine results from previous puzzles and are often connected to the narrative in key points of the gameplay [5]. Common puzzles involve unlocking locks with keys and combinations, assembling physical pieces together, unveiling hidden text that reacts to light or heat, interpreting complex ciphers hidden in the text, matching directional locks with directional clues from maps, counting items, placing transparent sheets on top of each other and rotating them until they line up to form letters, navigating mazes, searching for physical objects, or identifying patterns [6].

**Citation:** Fotaris, P.; Mastoras, T. Room2Educ8: A Framework for Creating Educational Escape Rooms Based on Design Thinking Principles. *Educ. Sci.* **2022**, *12*, 768. https:// doi.org/10.3390/educsci12110768

Academic Editor: Huei Tse Hou

Received: 20 September 2022 Accepted: 25 October 2022 Published: 29 October 2022

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

There are various game types that can be used to enable an EER to fit into different classroom settings, including the following [7]:


EERs have been used to introduce, foster, demonstrate, assess, or integrate students' content knowledge and skills [5] into a wide variety of academic disciplines, such as healthcare [9–11], STEM subjects [12,13], computer science [14–16], chemical engineering [17], pharmacy [18], physics [19], mathematics [20], chemistry [21], radiology [22,23], biology [24], sex education [25], teacher education [26], music [27], cultural mediation [28], etc. Additionally, in creative courses students have been asked to become "makers" [29] and develop EERs as a means to demonstrate and improve their creative, artistic, design, and problem-solving skills [30,31]. EERs can leverage the benefits of "competition, challenge, imagination, exploration of the environment, goals to be achieved, interactions (with people and objects) and security" [32]. By immersing learners in a playful, interactive, and engaging experience, EERs enable them to recall, apply, and advance their knowledge [33]. Puzzles within an EER are problem-based and require communication and team-working skills, which are considered intrinsic parts of the way in which adults learn [34], while a robust storyline helps to set the stage, and post-game reflection helps to solidify the learning goals. Various systematic and meta-analysis reviews indicate that, due to their playful nature which favours positive behaviour [35], EERs can enhance learners' motivation, engagement, and time management, increase confidence in critical thinking and decision-making, encourage lateral thinking, improve knowledge acquisition and academic performance, and help in developing logical, spatial, creative, linguistic, interpersonal, and collaborative competencies among players [4,5,33,36]. They can also be designed with elements that simultaneously serve visual, auditory, and kinaesthetic learners, thus covering all learning styles. Additionally, EERs can encourage social interaction, which is very important in the new era of increased hybrid delivery brought upon by COVID-19 [37].

Despite appearing to be a superficial form of entertainment, escape rooms can be grounded in sound educational theory and, when used effectively, act as a low-cost, highimpact resource for a variety of learners. EERs emphasise collaborative learning with activities that require teamwork and communication, force interdependence among multiple individuals who share a clear goal, and provide a built-in opportunity for rapid and unambiguous feedback [1]. From a pedagogical point of view, EERs are based on a social-constructivist approach [38]. Learners construct their own knowledge based on real-time experiences of advancing through several levels of progressive challenges in the escape room; they are called to face new and often complex problems, which can be solved by interacting with their peers and getting support from their tutor. The latter not only provides instructional scaffolding to the learners by facilitating their interaction with the material and with each other [4,13], but also closes the learning loop in a structured debrief, in accordance with simulation best practices [10]. Generation Z students who are considered multimodal learners [39], autodidactic, and are actively seeking activities that make them feel involved [40] will benefit significantly from having several different mediums or channels of information. This experiential and collaborative activity coincides with many of the features associated with a socio-cultural approach to learning [35] and motivates players to practise with hands-on examples as an effective way to increase skill retention.

EERs can also be used within the revised framework of Bloom's Taxonomy to include all six categories of cognitive processes by which thinkers encounter and work with knowledge, comprising "remember", "understand", "apply", "analyse", "evaluate", and "create" (Figure 1). Involving learners in higher levels of cognitive activities can positively impact the levels of engagement and knowledge retention among them [33].

**Figure 1.** Analysis of EERs within the revised Bloom's Taxonomy framework (Adapted from [33]).

EERs frequently expose participants to real-life scenarios with puzzles that fit into the story and involve interacting with a lot of objects in realistic ways, thus bringing greater authenticity to the activity and making it more immersive [32]. Because storytelling is often an entertaining, visual, experiential, and emotionally evoking activity, learners are much more likely to retain the course content taught in story format settings [41–43]. The fact that stories evoke emotions adds to their learning effectiveness; learning experiences associated with emotions are more easily stored [44] and appear to be remembered vividly and accurately, with greater resilience over time [45].Therefore, interactive storytelling is slowly becoming a key factor in EERs. As stated in [46], "a story does what facts and statistics never can: it inspires and motivates".

In an EER that recreates real-life circumstances, participants will also be able to reflect on their own life. Students can experience a situation in which they need to respond to high-stake situations, trust their own and their colleagues' competence, work together as a team, settle differences in opinions, and handle both time constraints and the consequences of not working fast enough [47]. Role playing provided by EERs enables great awareness and ensures a good assimilation of messages. This approach aligns with the paradigm of narrative-centred learning environments [48], which are defined as "a class of game-based learning environments that contextualise educational content and problem solving with interactive story scenarios".

When the EER activity takes place in the environment in which it would normally be applied, e.g., when a medical-themed EER is set up in an actual hospital, it aligns with the situated learning theory [49], which states that situated or scenario-based learning should take place in the context and environment in which it is going to be used. When used as a method of simulation-based education (SBE), EERs can be mapped effectively to Kolb's experiential learning cycle, which suggests that despite individuals' preferred learning methods, experiential interaction with materials produces positive learning outcomes [50]. EERs permit active experimentation in a safe environment, prior to undertaking concrete experiences "in the wild" [51]. Debrief and reflection are essential to learning in Kolb's cycle, and the real value of the EER could be argued to be purely these elements, similarly to other types of SBE [1].

The thrill and excitement of playing an escape game are the results of endorphins being released. Endorphins are the body's natural painkiller. They can also lower stress and anxiety levels, and even create a sense of euphoria. Combined with other neurotransmitters, this helps create an ideal environment for focused learning [52]. Furthermore, in the same way that games help stimulate the production of dopamine, a chemical that is considered to play a key role in motivation, affect, prosocial behaviour, and learning [52,53], EERs that access the same methodologies could result in learning–reward cycles [54] by reinforcing neuronal connections and communications during a learning activity [55]. Finally, unlike the one-size-fits-all lecture, EERs can also be balanced to be appropriate to the learners' skill level [56] to prevent them from becoming frustrated or bored, thus allowing them to experience "flow" or optimal experience, i.e., a highly focused mental state leading to immersion and high performance that is likely to emerge when learning activities are challenging but feasible, have precise goals, and provide clear feedback about performance [57,58].

Researchers have begun to build upon the notion of teachers as designers of learning experiences for students [59]. EERs have the potential to enable new forms of teaching, as evidenced by the rapid increase in publications related to the use of escape rooms for educational purposes, but their design and development for specific learning contexts is a time-consuming task [4], especially for educators without any prior experience in game design. As research in EERs is still in a preliminary phase, no unified framework about how to design them has been established yet. There is a need for frameworks, methodologies, or guidelines especially aimed at EERs [5,25,32,47,60] that could help educators not only in creating these new learning environments, but also in developing design dispositions [61] that will help them adapt to the complexity of teaching in the 21st century [59,62].

EscapED was the first theoretical framework to provide a methodology for creating EERs and interactive game solutions for learning and behaviour change within higher education settings [63]. It consists of six sequential steps (Participants, Objectives, Theme, Puzzles, Equipment, and Evaluation), with each one of them being broken down into other areas for developers to consider at the start of designing their EER. Although the escapED framework has informed the development of various EERs, either in its original form, e.g., [14,15,21,64], or in a modified version, e.g., [26,65], its quality and efficacy in terms of usability and usefulness for developers who wish to use it has not been validated yet.

Another methodology for designing Serious Escape Games for teaching is SEGAM [32]. SEGAM describes how to approach various aspects related to EERs such as constraints, pedagogy, parameterisation, tests, and background. It divides an EER into several levels, with each level representing a stage of the game and having at least one associated riddle that corresponds to one or more educational objectives (diagnostic, formative, summative, or discovery of a notion). However, this methodology was not evaluated and was used to develop only a single EER which was played by 20 students.

Eukel and Morrell presented a cyclic design process to create, pilot, and evaluate EERs that includes five steps: Design, Pilot, Evaluate, Redesign, and Re-evaluate [66]. While this approach offers some generic advice on EER development, it appears to be a simplified and iterative adaptation of the waterfall project management methodology. The provided information for each step lacks depth and there is no evaluation of the proposed method.

Nicholson and Cable [7] proposed a framework that enables the setting of specific learning objectives and individual learning outcomes for students in an escape game by mapping them against seven dimensions (Setting, Social, Story, Skills, Strategy, Simulation, Self) in order to build a cohesive interactive story that provides learning opportunities. Although the authors give instructions on how to build an EER using this framework, they do not provide any information about the framework's own evaluation.

The COMET framework was developed as a step-by-step approach to designing escape room exercises that would meet specific medical knowledge, skills, attitudes, and safety objectives while considering the unique dynamics of an interprofessional team [67]. It comprises five components (Context, Objectives, Materials, Execution, and Team Dynamics) and was piloted in a one-hour workshop aimed at enabling small groups to collaboratively design an interprofessional escape room. Although the COMET framework received generally positive feedback from the participants, its evaluation was very limited due to the small sample size (*N* = 16) and therefore its generalisation will require further reliability and validity testing.

Despite their different approaches, the aforementioned frameworks share one thing in common: they rarely validate the quality and efficacy of the frameworks themselves in terms of usability and usefulness, opting instead to assess the impact on learning of a single prototype escape game that was developed using the particular framework. To address this issue, this paper proposes Room2Educ8, a user-centred, conceptual framework for EER design following design thinking principles that can be adapted to any subject and escape room type. A mixed-methods internal validation study based on Instructional Design model validation was conducted to assess Room2Educ8- s integrity and use.

#### **2. Room2Educ8 Framework**

Room2Educ8 is a conceptual framework that can be easily tailored to fit any subject, set of learning outcomes, and class size by adjusting the escape room type, the puzzles, and/or the narrative. It was specifically designed to offer educators guidance in creating robust EER experiences and has been developed iteratively with pilot testing and refinements of individual elements since 2018. Room2Educ8 is based on design thinking, a process that has already been used as an instructional design method for the development of course content or teaching material [68], in curricular development [69], and as a teaching strategy to achieve subject-specific learning goals [70].

Room2Educ8 aims to allow practitioners to develop their creative confidence, which is required for game-based learning to be fully realised [71], by engaging in hands-on projects that focus on building empathy, promoting a bias toward action, encouraging ideation, and fostering active problem-solving [72]. Its iterative process can be described as a cycle of (1) empathising and observing, (2) defining the problem, (3) contextualising, (4) designing puzzles, (5) briefing and (6) debriefing the participants, (7) prototyping and playtesting, (8) documenting the design process, and (9) evaluating the EER experience (Figure 2). Designers can carry these stages out in parallel, repeat them, reflect, and circle back to a previous stage at any point in the process [73]. These stages were influenced by a typical design thinking process of (1) empathising and observing, (2) defining the problem, (3) creating ideas, (4) prototyping, and (5) testing [74]. Although no prior game design experience is required to use the framework, Room2Educ8 users may benefit from participating in a regular escape room before they start designing their own.

**Figure 2.** Room2Educ8 framework.

#### *2.1. Empathise*

The first step of Room2Educ8 calls for EER designers to gain an understanding of both the people they are designing the EER for and the problem they are trying to solve. Empathising, i.e., intellectually recognising or vicariously experiencing the feelings, thoughts, or attitudes of others [72], is essential, because EER designers create learning experiences for people with given wants and with characteristics different from their own, while often working in design teams composed of people with different skill sets and interests which can affect their prioritisations of interests [75]. Understanding the participants corresponds to the regular "learner analysis" included in most Instructional Design models [76].

EER designers can utilise techniques such as focus groups, interviews, observations, and surveys [77], as well as data from academic records to collect information about their learners. The collected data can then be analysed to identify trends and segments within the overall learning audience. Significant groups of the latter can then be segmented to build learner personas, i.e., fictional characters who represent certain traits and qualities of the target audience for whom the learning experience is designed for [78]. These should be kept in mind throughout the design and development of the EER as they can help designers to identify and understand the learning objectives, challenges and preferences of their learners and tailor the escape room experience with them in mind. A learner persona usually includes a fictionalised name, photo, demographic information, short biography, leadership and character traits, academic needs, primary goals, motivations, frustrations, learning preferences, digital fluency, and relevant quotes from interviews (Figure 3).

**Figure 3.** Sample learner persona for EER about cybersecurity awareness training.

#### *2.2. Define*

The second step of Room2Educ8 synthesises the findings from the empathise stage and carries them into a series of brainstorming sessions to define the following set of constructs that should be closely considered when designing an EER: problem statement; goals; learning objectives; constraints; required knowledge; group size; game type; playtime length; and game position within the curriculum.

A problem statement identifies the gap between the current state (i.e., the problem) and the desired state (i.e., the goal) of a process or product. One way to approach defining a specific problem is to frame it from the learners' perspective and identify the "whos", "whats", and "whys" that exist in the space around the issue, such as asking who is experiencing the problem, what the problem is, and why it matters. An example problem statement is the following: "Employees at a university (who is affected?) need an engaging, memorable, and easy-to-understand Security Awareness Training based on real-life scenarios (need) because they are bored and distracted by their organisation's tedious e-learning training (what is the problem?), thus becoming a big security risk (why does it matter?)".

Defining the problem should be followed by setting up S.M.A.R.T. (Specific, Measurable, Achievable, Relevant, and Time-bound) goals [79], starting with outlining the overall purpose of the EER. Most educators implement EERs to explore an active learning environment, preview, review, or practise material, increase students' motivation and/or engagement, foster learning, and/or develop teamwork and communication skills [5]. Once goals are established, designers should break down what they want to accomplish into smaller, more specific objectives that will help them reach the goal of the room. By answering questions such as those in Table 1, designers should get a sense of direction and know whether the EER could be developed by a single person or would require a team.

**Table 1.** Questions to help define SMART goals for EERs.


Unlike recreational escape rooms, EERs must align with specific and purposeful learning objectives to be effective [80]. These are details of what the participants should have learnt by the time they have finished the EER. Learning objectives should be written in such a way that educators can readily assess if they have been completed. They may describe specific content knowledge and content-related skills (e.g., clinical skills), general skills (e.g., practising or developing teamwork and communication skills, situated problem-solving, critical thinking, reasoning skills, empathising, delegation), affective goals (e.g., performing under pressure, increasing situational awareness), or a combination of them [5]. It is good practice to include learning objectives that everyone should be able to achieve, some trickier ones that most will, and some stretch goals that very few will achieve [81]. Determining

the topics that will be covered in the game and creating tangible objectives allows for the development of an evaluation strategy to assess the players' learning experience [63].

Identifying the constraints that may affect the game's development is also crucial so that designers can focus on suitable ideas and ensure the EER's feasibility. Typical constraints in educational environments that can heavily influence how the EER activity will be developed include time (e.g., the available time to develop the game; the available time for the whole activity to take place, including time to set up, brief, debrief, reset, and facilitate the game), location (e.g., the room should be located as close to the participants as possible), space (e.g., the room may be small or there may be multiple rooms available), game scale, budget, class size, resources, language (e.g., will non-native speakers be able to solve cryptic crossword puzzles?), neurodivergent learners, and curriculum (e.g., do the EER's learning outcomes align with the curriculum?). Tutors are vital for the EER experience, as they usually play the role of the game master whose duties include introducing participants to the game's rules and story, monitoring the team's progression, providing hints during gameplay, and facilitating the debriefing session after the game. Therefore, staff availability should also be considered.

It is accepted practice that any knowledge needed to solve a puzzle in a commercial escape room will be provided within the room itself (e.g., the periodic table, music scales, Morse code, etc.) [82]. However, escape room activities have been proven effective for assessing students' knowledge, applying previously taught information to gain a deeper understanding [10], or practising information retrieval skills. Therefore, designers should decide what knowledge is required to succeed in the game. This can be explicit (i.e., students are given all of the relevant information needed within the game world—no prior subject-specific knowledge is required), assumed (i.e., students are being tested or assessed on what they already know), retrievable (i.e., students use information retrieval skills to find what they need in the real world), or a mix [7].

A key element to having a positive EER experience is doing it with the right group size so as to keep the players in a state of flow [58] with enough puzzles to engage everyone in the group. The size of the group can alter how quickly players move through the game's puzzle path. The average group size for commercial escape rooms is 4.58 people [83]. With more participants, the game should become easier up until a turning point where people who are not engaged in the game are standing around and becoming bored and distracted [82]. However, in an educational setting class sizes are usually too large, classrooms are too small and underfunded, and timetables are too inflexible to allow for small teams to play an escape room without disruption [7]. Conducting escape room activities with large cohorts means that several sessions must take place, which can be a tedious and challenging task [4]. As a result, team size compromises often have to be made, which can affect student participation [3]. Choosing digital, portable, or quick and easy-to-set-up and easy-to-take-down escape room types that are more feasible for a classroom is one way to address these issues [84]. EER designers can refer to Table 2 to decide which escape room type is a better fit to their class sizes, learning outcomes, and time constraints.

The next thing for EER designers to decide is the game's *playtime length*, i.e., the time players spend on the puzzles, not including the briefing before the gameplay and the debriefing afterwards. A time limitation is commonly present in EERs to introduce an element of stress, excitement, and competition. In medical studies, the time constraint is considered not only as a game design aspect, but also an educational aspect, as collaborating under time pressure is a life-saving skill in medical professions. In other disciplines, the restricted time is a way to create social interdependence; everyone must solve all the puzzles in time, so learners are required to spend their time effectively and decide what to focus on. The teamworking and prioritisation practice can be seen as directly linked to developing leadership and management skills [81].


**Table 2.** Advantages and disadvantages of EER types (Adapted from [7]).


**Table 2.** *Cont.*

The playtime of an EER ranges between 15 and 120 min, with most games using the 60 min time limit typically found in commercial escape rooms. In educational settings, it is important that as many students as possible reach all the goals in time, and frustration, dropping out, or trial-and-error behaviour are avoided. Therefore, when considering playtime length, designers should ensure that it allows for a sufficient number of puzzles to be used, offers ample time for students to work as a team, and fits into a classroom time slot [4]. Shorter games require less development time, but longer games can use more meaningful challenges that require more time and effort to be solved.

Finally, designers should decide the EER activity's position within the curriculum. EERs with learning goals solely focused on introducing a subject, general skills, or affective goals, are usually stand-alone activities (e.g., icebreakers, orientation activities to encourage student engagement with library services, induction week activities, playful ways to introduce people to STEM subjects, etc.). Conversely, EERs that are intended to foster content knowledge and related skills are embedded in a course curriculum, usually positioned in addition to lectures at the introduction of the course, at the end of the term to keep the motivation going to the last minute, or to mark a special event in the calendar. EERs with formative assessment goals are positioned mid-term or before the exams [5], while EERs that follow an episodic format may run periodically (e.g., on a weekly basis) for a whole semester. It is recommended to run only one or two EER activities with each class per year, so that the novelty does not wear off.

#### *2.3. Contextualise*

The third step of Room2Educ8 is to place the EER in a particular context which gives meaning to the activities the learners do, provides an authentic reason for escaping, and links the puzzles together in a cohesive storyline so that participants can identify with the game experience and build personal motivations to complete the game [63]. Context includes theme, setting, characters, narrative which contextualises knowledge and skills needed, and environment.

A theme is a necessary component for maintaining the fiction contract with the participants as it ties the puzzles and decorations together and sets the EER's tone, look, and feel [9,85]. It is critical to select the theme early in the design process, as it will dictate the rest of the decisions made about the game, such as the setting, characters, and the tone of the puzzles. More importantly, it will enable designers to target specific competencies and skills. For instance, as well as being fun, a highly imaginative theme can also encourage creative thinking. Mystery themes (e.g., uncovering a murderer) are good for working on problem-solving and decision-making skills. They often prioritise attention to detail and tend to have a more focused and serious feel. EERs with a scientific or technical setting (e.g., a science lab or a factory) can help teams to develop abilities such as strategic planning and delegation. Finally, horror themes are effective in fostering team-working skills under pressure. They encourage adaptability and quick thinking, while also testing the participants' resilience.

The time period and place where the game will be set should be decided next (e.g., a haunted house in Victorian England, a detective's office in New York during the roaring 1920s, a Pharaoh's tomb in Ancient Egypt, etc.), as this can determine what types of elements will be most appropriate to develop the puzzles from. Challenges that use the types of things that are typically found in that particular setting will feel more natural, thus helping immerse the participants in the game world.

A story should be created to support meaningful play [86] and provide an immersive narrative for the activity [1,11] that will be introduced and discovered by the participants in bits and pieces. This will help to solidify the game objective in the minds of the participants and add to the ambience of the game [87]. The story should encompass who the characters are, what conflicts they are facing, and where this is all taking place. A logline that covers the basic elements of a sample EER about raising cybersecurity awareness could be "the participants are a team of private investigators ("who") who use the opportunity of an invitation at their client's CEO's home ("where") to steal evidence of his involvement in the misappropriation of funds and bring him to justice ("what")".

Once EER designers have a story in mind, they should decide about the plot (i.e., how, when, and why everything happens) by asking themselves: Why are the participants in the room? How did they get there? What do they need to do to escape or succeed? What are the consequences of failure? What are the rewards? Why do they need to hurry? Why are there puzzles and clues in the room? Who put them there? How do they fit into the story? Who is the game's facilitator, why are they there, and why are they giving hints? Solving these challenges will make for a very immersive escape room experience that seamlessly integrates characters, story, and puzzles. Examples of basic plot frames that can be used for EER stories include the following [85]:


It is important to keep the plot points in easy-to-understand bite-sized portions and let the beats drive the action and participants towards their goal. Integrating the time factor in the plot is also crucial; many EERs have a one-hour time limit, so designers should ask themselves what story they can tell that culminates in the participants reaching their objective in an hour. Setting up circumstances that generate emotion or presenting participants with dilemmas that play on their sense of justice and morality can make a story even more engaging.

Another consideration at this point is the characters that are part of the plot. In an EER, participants are expected to be an active part of the learning process, so they should assume the starring role (protagonist) in a story that they feel they are influencing, with an outcome they believe they can affect [7]. It is as a direct result of their decisions and actions that the narrative progresses. Once the role of the participants has been decided, the other characters need to be fleshed out as well. EER designers can have those characters communicate their wants and needs through puzzles and/or audio, video, photographs, or written messages integrated into the experience. Adding some complexity to antagonists or even evoking sympathy for them allows the game to have different endings with moral dilemmas, e.g., if the antagonist is a politician who has manipulated an election, but only to avoid a dictator coming into power, then the participants can either keep the secret or reveal everything. This choice may be a starting point for an ethical or philosophical discussion in the debriefing session. All endings need to reach the same learning outcomes, though. Allowing participants to make choices that have direct consequences on how the game

plays out gives them a sense of control over the game and their role in it [33]. Finally, the role of an ally who can provide the participants with hints, tools, and instructions to help them overcome these challenges is usually played by a tutor. This is an opportunity to guide the participants towards success and ensure that they have a good learning experience.

With compelling characters in place, the basic concept of the story needs to be structured into a series of events that the participants can follow. Rich narratives which do not require too much reading and fit the theme and the setting will keep an escape room from just being a random series of puzzles. Often EER narratives are "bookend" narratives, with most of the important story information communicated at the beginning and the end of the game [85]. The core story can be expanded using the model of dramatic structure put forward by German playwright Gustav Freytag in *Die Technik des Dramas* (1863). This model has become commonly known as "Freytag's pyramid". Its application to an EER is illustrated in Figure 4 as a 2D graph in which the x-axis shows progression through the story, and the y-axis shows emotional engagement or tension. The resulting curve depicts the typical dramatic arc rising and falling as a "pyramid" [88]. Freytag's model defines sections of dramatic action, separated by key events, which can be positioned and aligned to sections of the Three Act Structure of a beginning (setup of the conflict), middle (confrontation of the conflict), and end (resolution of the conflict) [89].

**Figure 4.** Dramatic structure of an EER according to Freytag's model.

Act I (beginning/setup for the conflict) starts with *exposition*, which offers background information about the main characters ("who"), the setting ("where"), and the circumstances or time period ("when") to prime the players for the rest of the story, as well as other contextual background information and lore relevant to the action. In an EER, these elements of theme and backstory are often presented during a pre-game introductory briefing by the game master or by a pre-recorded video introduction. The exposition is followed by the *inciting incident*, an event, occurrence, or action that pulls the protagonists out of their normal world and into the main action of the EER. Without it, the protagonists

would not become aware of the conflict, and therefore they would not have the opportunity to resolve it [89]. This sets up the *dramatic question*: What do the protagonists have to do to solve the problem they face? It is often presented to the participants by the game master just at the point that they enter the room and serves to set their objectives in the game. The success (or otherwise) of the participants in achieving the gameplay objectives will determine how the dramatic question is resolved in the narrative. The first act concludes with some turning point that launches the action into Act II.

Act II is where the *rising action* of the story occurs, in which the participants are faced with continuous, escalating conflict as they try to overcome the antagonist. This section of the narrative constitutes the majority of an EER experience, and typically has participants solve puzzles, discover items, and reveal new areas in pursuit of the goal [88]. A sense of increasing urgency can be created by a time countdown, dramatic theatrical music which increases with intensity as time runs out, audio cues from a character or pre-recorded messages, and lighting effects. The dramatic tension in the game increases higher and higher up to the *climax*, which is the emotional peak of the story. It signifies the final moments of the story's overarching conflict and can be represented as the final puzzle (e.g., cutting the wire to defuse the bomb). However, the game should not end immediately after the climax, as participants will not be able to experience the results of their actions in the game world and the story will not be concluded.

In Act III, Freytag's model identifies a period of *falling action* that results from the climax, which can still be exciting (e.g., having defused the bomb during the climax, the participants still need to escape the building) and should lead to the *resolution* of the dramatic question: have the players succeeded in achieving their objective? Finally, the story de-escalates in a *dénouement*, where the events of the climax wind back down into normal life. An outro video at the beginning of the debriefing session that follows the game is an effective way to provide clarity, resolution, and closure by showing participants what happened at the very end of the story. If the EER is a serial story, the video can deliberately end on a cliff-hanger to create a sense of suspense and get everyone excited for the next episode.

As with the theme and narrative, the room's physical environment supports (or detracts from) the activities and overall learning outcomes. Choices about the decoration, physical props, lighting, technology, audio, video, and visuals of both physical and digital game spaces have narratological consequences and must follow the room's theme to prevent cognitive dissonance [90]. A selection of appropriate effects (and music if it makes sense to the environment) in a well-edited escape room soundscape adds another subtle but very effective layer of immersion to any game. Providing on-theme costume accessories and inviting participants to dress up is also an opportunity to encourage immersivity [85].

#### *2.4. Design*

The fourth step of Room2Educ8 involves designing the puzzles that the participants will have to solve to complete the game and meet the learning objectives, deciding upon the game's flow, creating the room layout, choosing appropriate game assets, developing a hint and/or a scoring system, and defining game rules. As every puzzle in an EER should align with a learning objective, designers must determine first which learning outcome each puzzle will support. They must also understand what the participants know before they start the puzzle, and what they should know after completing the puzzle. This will allow for the easier validation and assessment of whether the learning objectives have been achieved at the end of the game experience [63]. Puzzles are opportunities to engage the participants with the story in an interactive way, so the next step is to determine which part of the story the puzzle is aligned with, what the participants perceive about the story before the puzzle, and what they should understand about the story after the puzzle [7]. When creating puzzles, it can often be easier to look at any final meta-puzzle first and then work backwards from this.

Since numbers are often used in lock combinations, finding ways to manipulate numbers is an easy way to add layers of puzzles to an EER. The simplest way to do so is counting, e.g., have participants count how many there are of an object and make that number relevant. Other ways to add numbers to puzzles include giving participants a message written in letters and have them use the numbered keys on a phone to identify which numbers match the letters (e.g., GAME returns 4263), hide numbers in a block of text by replacing some letters with numbers (e.g., cand1e, mak3r, ba5ement, n0rmal), enter numbers into a calculator and then turn it upside-down to read a word (e.g., 35007 upside-down spells the word LOOSE), and use Roman numerals or binary numbers. To communicate letters and words, ciphers can be used to replace each letter with a different symbol, number, or letter (e.g., Caesar, Atbash, columnar transposition, A1Z26, ASCII code, Pigpen, Braille, Morse code, Scytale, etc.). Another common strategy to hide messages with letters is to take a block or line of text and call attention to specific letters or words (e.g., leave some letters in the line lowercase or uppercase, make certain letters a different colour than the surrounding text, place a dot or underline under important letters, etc.). Puzzles should be as self-guided as possible, make their goal easily understood, be clearly linked to clues, relate to the room's theme, propel the narrative, take less than 5 min to solve, and provide clear feedback when solutions are tested [85]. Having an obvious finished state permits the participants to feel successful and boosts their enthusiasm.

To help participants reach a state of flow [58], it is crucial to keep them in a sweet spot between frustration and boredom. If they are frustrated, they will give up because they cannot find a way to engage in the puzzle. Conversely, if participants are not challenged enough, they will get bored and equally give up caring about the game [82,91]. A solution to this is to rate puzzles according to the difficulty of the content and the puzzle itself; designers should make the first puzzle relatively easy to build the participants' confidence and set the stage for success [66,82], then provide them with puzzles of increasing difficulty to keep the tension high. A mix of manageable revision tasks with more difficult new tasks that require some research can maintain a balance between motivation and challenge. Using the design concept of flow helps to create the scaffolding that can take participants from what they already know and make them reach the learning outcomes [7]. Diverse puzzles which challenge participants in different ways as they move through the escape room (e.g., cooperative, logic, sensory, searching, physical tasks, etc.) can target a variety of learning approaches and are more likely to engage multiple team members [10], thus increasing the game's success rate.

Finally, puzzles should be designed with accessibility in mind [82]. Designers must consider how people with disabilities can navigate the game space. If having low lighting is key to the game experience, then using large text with an easy-to-read font and high contrast colours can combat frustration for participants. Similarly, if a puzzle needs to be solved using colours, it can become colourblind accessible by making it also solvable using shapes as well. It is important that participant actions within the room can be observed, as this can help to determine if deficits in a team's performance are due to poor puzzle design or poor teamwork [6].

A key aspect of EER design is ensuring that all individual puzzles contribute and form a greater whole. This essentially creates a puzzle path for participants to follow. When designing the game flow through which participants proceed during the game [86], a popular strategy is to follow a linear path structure, i.e., present to them one puzzle at a time. Solving it will then make the next puzzle available. Linear pathways are easier for participants to understand, the story flows better, and the game can be timed and paced, therefore less guidance is needed, and progression is easier to monitor [32]. Alternatively, in a non-linear game (i.e., a game that uses an open, path-based, or pyramid puzzle structure), multiple puzzles are available to participants all at once, and after all are solved, their outputs can be used to solve the final meta-puzzle. A flowchart showing how puzzles are connected as presented in Figure 5 is an effective way to visualise the puzzle structure.

**Figure 5.** Puzzle structures in escape rooms (adapted from [83]).

To ensure that participants will not miss out on any activities in the game, it is recommended that EERs use a linear model where the entire team will engage with each puzzle together. If there are multiple puzzles available, they should be designed to provide similar experiences, so that as long as participants were engaged with one of the puzzles, they would be able to move closer to the learning outcomes [7].

A room layout with clue placements and arrangements of the puzzles, as well as a flowchart mapping out how players will navigate the room, can help to track the participants' progress and visualise the overall flow of the experience [9]. Cards may be used to provide details of each individual puzzle and its location in the room, starting objects in each location, what prompts participants to start each puzzle, what hints are available, and what clue/reward makes participants go to the next puzzle. These techniques are useful for checking for consistency in the room design, sharing the design with others, and resetting the room.

EER designers should consider any physical and/or digital assets that will be manipulated to solve the puzzles or will be used as clues. These should match the time period, the story, the characters, and the theme the game is set in. Game assets may include the room itself as a space, lock boxes and containers, locks that provide immediate and unambiguous feedback to players (e.g., combination, directional, letter, colour, padlocks, hasps, etc.), envelopes, UV markers and black lights, game tech (e.g., computers, smartphones, GPS, website, app, online answer box, projector, AR/VR, PA system, electronic props with motion detectors, sensors, RFID tags, Arduino, Raspberry Pi, etc.), decoders to validate participants' solutions, a clock or countdown timer to promote a sense of urgency, and narrative elements that embody the theme or setting (e.g., video, audio, printed documents, pictures on the wall, etc.). It is recommended to utilise assets that can be reused and/or are cheap to replace, as well as to produce refill packs with replacements for elements that are used during gameplay in case they are misplaced or malfunction. The physical placement of clues is also important, e.g., putting something above most people's reach when there is nobody tall on the team may cause frustration amongst participants [82]. The "one clue, one use" rule (i.e., each clue or prop is used only once to solve a puzzle and then is retired from the game) can improve the overall gaming experience, because once participants use a clue for a piece of information, they can set it aside and focus on the remaining clues to solve the remaining puzzles, thus reducing their cognitive load [85]. Red herrings (i.e., items that have been deliberately designed to look like puzzles and clues, intentionally forcing participants to waste time on items of no value) should only be used

if their existence ties into a learning outcome and the participants are trying to learn how to identify false information and false leads as part of the activity [7].

Sound and music should not be overlooked when designing EERs, as they can transform a game into a vivid and intense overall emotional experience. In-game sound effects are mostly used for certain events in the game, such as when participants receive a hint, or the countdown clock reaches the last minute of the game. These sounds may fit the room's theme in style, but they should be unusual enough to catch everyone's attention. Conversely, a music soundtrack should be subtle enough to let participants focus on the story without catching their attention.

An EER's success will frequently be built on its hint system. Hints provide an avenue to mitigate the unpredictability of human behaviour and give teams an outlet to progress past difficulties unanticipated by the EER developers [6]. They also help participants of varying levels have similar experiences when playing through a room. To foster a positive learning experience, it is crucial to develop an incremental hint system that offers help to participants when they are stuck and fits the theme and narrative organically. Incremental hints act as metacognitive support [92] in monitoring one's own progress, thus contributing to learners' knowledge-related self-confidence [25], and can be delivered to participants personally (e.g., via a TV screen, through the room's PA system, via a walkie-talkie or phone, on written notes, via an app/website, etc.) or by pre-set hints on apps/websites or on hint cards. Due to space limitations, it is also common for tutors to be present in the same room as the participants throughout the game in order to offer them hints. This approach should be undertaken with caution, however, as it may affect the participants' autonomy in learning [93] or reduce the experience of flow and immersion by interrupting the gameplay. Defining hint rules is recommended, especially for EERs with assessment goals, as hints can artificially influence performance if there are differences in their timing and specificity. A clear hint system with a limited number of hints available can also help the participants to build up resilience and independence, while helping the tutors to stop themselves from interfering. Common hint rules include: teams get a restricted number of hints; the first hint is free, but if more hints are needed, a time penalty is given; participants must earn a hint by passing a knowledge test, solving a puzzle, or finding hint cards or tokens; there is no hint limit, but participants must use a hint button with a cooldown timer; a pre-set hint can be used only if participants have not solved a particular puzzle by a certain time on the game clock [5]. A hint cheat sheet can also be used to provide systematic guidance on the type of hint that is necessary as well as the level of detail that should be provided to teams [6].

Finally, a scoring system can be used to tap into people's natural competitiveness and encourage them to do better. A final score can be awarded based on whether participants were able to finish the game, the time it took them to do so, the number of hints or clues they used to solve the puzzles, or the number of puzzles they solved [85]. Designers should consider whether there will be consequences to participants for any errors they make, e.g., miscalculating a medical dose may result in a two-minute penalty [9]. However, scores leading to tangible rewards (e.g., sweets, stickers, stationery, etc.) should be used cautiously as there is the risk that participants will focus on doing only what needs to be done (e.g., to figure out the code for the locks, instead of engaging fully with a puzzle), which can result in not achieving the learning outcomes. Grades and rewards may send the message that the EER is not going to be an engaging activity in its own right, but a task participants must perform only for the reward [7].

#### *2.5. Brief*

The fifth step of Room2Educ8 is for designers to consider how they are going to inform the participants about the EER's backstory, objectives, and rules. One of the best ways of doing this is to begin the narrative during a 5 to 10 min pre-game briefing. The briefing can be used to provide background information about the main characters, the setting, the time period, and the inciting incident, as well as set up the dramatic question: what do the protagonists have to do to solve the problem they face? This prologue can be in the form of a pre-written script read by the EER facilitator or by the participants themselves. Alternatively, a pre-recorded video introduction can be used to give instructions to the participants and deliver the narrative components. Besides ensuring standardisation across teams and minimising task load on the facilitator, a video can add significantly to the immersiveness of the experience, make participants engage more naturally with the storyline, and heighten the sense of urgency for escaping from the room. Using tropes from films can make it easier to get the participants into the emotional state designers want them to be in when the game starts [82].

A list of rules should also be provided to participants. This may include information about the time limit for successful completion, forbidden items, hint and scoring systems, room boundaries, handling props and furniture, health and safety issues, areas and objects that are out of bounds (e.g., works of art on the walls, light fixtures, air vents, floor grates, etc.), case sensitivity of text entry fields, communication with the game master, acceptable behaviour, consent forms, etc. To deter cheating, facilitators can explicitly request that participants not engage in cheating behaviour in the room. Additionally, they can offer specific examples of behaviours to avoid while using vocabulary that suits the theme where possible, as this can add to the immersion. For example, instead of telling participants not to break the locks in a sci-fi-themed EER, facilitators can advise them "not to interfere with the spaceship's security system". Rules make the game, so it is important that they support the main goal of the EER without making it too hard or too easy to complete. Finally, in a physical EER, an area should be designated for participants to leave their belongings so that they do not have to carry them around during the game.

#### *2.6. Debrief*

The sixth step of Room2Educ8 is for designers to consider how they are going to make participants aware of the learning that occurred during the gameplay. Metacognition, i.e., students' ability to monitor, direct, and review their learning, is a powerful tool to get learners to think about their own learning more explicitly, usually by teaching them to set goals and monitor and evaluate their own academic progress [94]. Learning techniques that have been shown to promote metacognition and enhance memory formation include elaborating, verbalising, and sharing learnt information during and at the end of a learning session. A structured, facilitated debriefing upon the completion of the EER allows for reflection-on-action as described in Kolb's experiential cycle [50]. A good rule of thumb is to reserve one-third of the class time for reflection on the EER activity [7]. The gathered data can also be used in Room2Educ8- s evaluation step to assess the game's success as a subject-specific educational activity and inform any further needed improvements to the overall experience. A recommended debriefing model is the Plus/Delta model which uses two columns; the plus column (+) refers to good behaviours or actions, while delta refers to behaviours or actions that need improvement or change in the future [95]. This technique allows learners to participate in the discussion and is easily utilisable by novice debriefers. More experienced facilitators can use the Advocacy Inquiry model from Debriefing with Good Judgement, in which an *advocacy* is an assertion, observation, or statement, whereas an *inquiry* is a question. When pairing the two together, facilitators act as conversational scientists, stating in their advocacy their hypothesis, and then testing the hypothesis with an inquiry. This is the generic approach that facilitators can use in any scenario: Step (1) notice a relevant result (e.g., something that happened during the EER experience); step (2) observe what actions seemed to lead to the result; and step (3) use advocacy–inquiry to discover the reasoning that produced this result [96].

To provide clarity, resolution, and closure to the story, the debriefing session may begin with an outro video showing participants what happened at the very end of the story, what they did in the game, why doing that was important, and how their actions improved the circumstances of the game's characters.

Next, a reaction phase will allow participants to express and defuse heightened emotions. They are coming out of a high-energy, stressful environment and will be thrilled or disappointed, so it is important to leave them with a positive impression of the EER experience [82]. Sometimes, they may not recognise the learnt skills that were necessary to succeed in the game or may be unable to identify how the lack of those skills led to an obstruction in the team's process. Therefore, the facilitator should guide participants in reflecting on their performance, the game content, the puzzles, the skills needed to solve them, and their overall experience, and then use this discussion to clarify the teaching points. For instance, participants may be asked to describe what they enjoyed about the game, their favourite or most challenging puzzle, a time when they felt particularly proud of themselves or their team, something new that they learnt during the game, how the game related to what they were learning, how solving a puzzle in the game related to solving a problem in the real world, one change that they would like to make to the game, or what they might do differently next time.

As participants may desire feedback on observed team-based skills, open-ended questions can be used to prompt dialogue about leadership, delegation, effective communication, situational awareness, and task assistance [35]. For example, participants may be encouraged to describe what they learnt about themselves during the game, how they contributed to their team, how they made sure their ideas were heard, how their team utilised everyone's strength, a moment when their team worked well together or became frustrated, how their team could have been more effective, and why their team succeeded or failed in completing the challenge. Designers should also have a plan for participant failure, e.g., decide whether the facilitator will disclose answers by guiding the participants through the uncompleted puzzles or will review learning objectives in the debriefing without revealing the EER's secrets [9].

The debriefing may be concluded by giving away revision material (e.g., a revision booklet) which summarises the key learning outcomes the participants explored during the game, and by taking a group photo (or a screenshot when the EER is digital). The group photo is arguably the only shareable thing about an escape room and most participants consider it an important part of the overall experience. It is recommended to give participants original props that fit the room's theme and optimise the photo for sharing on social media.

#### *2.7. Prototype*

The seventh step of Room2Educ8 is for designers to prototype and playtest the EER. Given the challenging nature of predicting human behaviour, prototyping efforts that utilise multiple teams during an EER's development are an effective tool to help estimate the length of time required to complete individual puzzles and the overall length of the game [6]. After developing the ideas and the puzzles, the design team should set out to create simple, cost-effective prototypes of their ideas from their ideation sessions of the previous steps. One large piece of chart paper can act as a surface for drawing a blueprint of the room. Sticky notes with quick descriptions can then be placed on the room's blueprint to mark out the locations of puzzles and clues. The puzzles themselves can be written on sheets of paper and brought out during playtesting when it is time to use them. The goal of the paper prototyping is not to perfect the puzzles but to catch any big errors, check that the puzzle logic makes sense, and that the overarching flow of the room works [82].

Once this internal playtesting has been completed, designers should make another lo-fi testable prototype and invite teams of varying sizes, backgrounds, and levels of prior experience with escape rooms to playtest it (e.g., 2-8 individuals with similar skill sets to the intended learners). They should then get their feedback (e.g., via interviews, questionnaires, etc.), return to the design process to solve any problems that came up during testing, and produce a more refined prototype. This cycle may be repeated several times, so it is recommended to use as few resources as possible in creating the early prototypes, as they will most likely be changed after one play.

Because EERs are usually team-based events, it is important that every participant contribute, otherwise they will feel like they wasted their time. Playtesting will reveal if there are enough puzzles for people who are visual, logical, physical, or other types of thinkers [82]. Each playtest should address issues about realistic playtimes, difficulty levels, puzzle mechanics, the relevance to learning objectives, the quality of hints, and the cohesive nature of the narrative. It is crucial for an EER to have a high win percentage, so testing individual puzzles, paper prototypes, and the full game at different stages, as well as the debriefing structure, will provide important feedback to improve the design and achieve game balance, accessibility, and playability [9].

A Feedback Capture Grid is a structured way of capturing user feedback systematically during playtesting sessions or organising the gathered feedback after the playtest. To start, designers should draw a grid on a piece of paper and divide it into four quadrants labelled "Likes" (positive feedback), "Criticisms" (negative feedback and criticisms about the prototype), "Questions" (questions that the play testers have asked as well as new questions the test session raised), and "Ideas" (any ideas that the testing session has sparked). Then, they should ask play testers to give specific and detailed feedback directly on the grid using sticky notes. Once the grid is full, designers can move into synthesising feedback into clusters or related common themes, brainstorm ideas on dealing with the most important issues, and then create an action item list.

#### *2.8. Document*

The eighth step of Room2Educ8 is for designers to consider how they are going to document the process of developing the EER. A highly descriptive game design document (GDD), created and edited throughout development, can help the design team to refine scope and production needs. A general anatomy of a GDD includes a game overview with general information about the EER and its learning objectives, followed by sections that describe each part of the design (e.g., puzzles, narrative, assets, etc.) in progressively more detail. The document should be consistent, thorough, and specific enough, including illustrations, flowcharts, diagrams, and every other information is required to build the EER, so that it can serve as a blueprint for designing other EERs. A GDD is expected to evolve together with the project as designers find new ideas, uncover new problems, and may even change the overall design while making the game. Therefore, it is important to plan from the beginning to update the documentation as development proceeds.

It is also recommended to produce two additional documents: (1) a facilitator guide, which should contain the learning objectives, briefing and debriefing instructions, game rules, room layout, a game walkthrough with clues and answers for each puzzle, rules, and/or pre-set times for providing hints; and (2) set up/reset instructions containing a visual depiction of the exact location of every object in the room accompanied by clear step-by-step instructions about how to set up and reset the game for another play-through.

#### *2.9. Evaluate*

The ninth and final step of Room2Educ8 is for designers to consider how they are going to evaluate the EER experience and assess whether the EER met its goals, objectives, and learning outcomes, what aspects of the game contributed to or detracted from this, and how the learning experience can be improved. The use of audio/video surveillance equipment or screen recording software to observe and record participants as they complete the room can serve as a data collection method to capture verbal utterances, team processes, and behaviours. However, it comes with the added need for reliable video coding which can be extremely time-consuming [6]. Alternatively, a researcher may watch the teams perform tasks in real-time and take notes. Learner feedback using post-activity interviews, focus groups, surveys, and the debriefing session are common methods to assess participants' perceptions. When the EER is used as a tool to assess knowledge and/or soft skills, learning gains can be measured by means of a pre-/post-/delayed post-knowledge test [5] and/or by a student performance score based on success rate, the number of puzzles solved, and

the number of hints requested. Ideally, studies should follow a mixed methods approach to evaluate an EER experience, as it will provide a better comprehension of their findings by triangulating results and thereby improving the validity of their conclusion. Finally, if students are asked to develop EERs as part of their coursework, they should be provided with a rubric showing the criteria upon which their EERs will be assessed.

#### **3. Room2Educ8 Validation**

*3.1. Methods*

A mixed-methods internal validation study based on Instructional Design model validation [97] was conducted to validate Room2Educ8. The study lasted 4 academic years (2018–2022) and employed a survey and focus groups to assess the framework's integrity and use. Ethical approval was not required for this study as it involved assessing the anonymised student feedback and knowledge from a teaching event. Prior to data collection, the students were informed about the nature of the study and the fact that the study results will be published, gave their consent, and were assured of their anonymity.

Since Room2Educ8 is based on Design Thinking principles, it was embedded in the teaching content of a 13-week-long compulsory module named "Design Thinking" for a postgraduate course in User Experience Design at a British university. This creative module was divided into a theoretical part where the lecturer traditionally exposed the curricular contents, and a practical part where the students became "makers" [29] and worked in groups of four to collaboratively create a one-hour EER experience for their coursework. The required deliverables were a fully working EER prototype (physical, digital, or hybrid) on one of three topics (cybersecurity awareness, information and communication technologies, or information literacy), a report documenting the EER's design process using Room2Educ8, a live demonstration of the EER, and a peer evaluation of each group member's contribution to the project. All students were given 13 weeks to complete the coursework and had to get a mark of 50/100 or higher to pass the module. The coursework was constructed with specific learning objectives in mind, mapped to outcomes from the module's specification document. These objectives included:


Prior to being offered to students, the coursework brief had been peer-reviewed by two lecturers with backgrounds in user experience and game-based learning, respectively, to verify its suitability to the module.

Between the 2018–2022 academic years, four cohorts of 104 students in total (*N* = 104, 48 identified as male, 56 as female) aged 21–32 years old worked in randomly distributed groups of four and created 26 EERs (*N* = 26) for their coursework. A total of 14/26 were digital EERs, 8/26 were physical, and 4/26 were hybrid. A total of 16/26 EERs focused on cybersecurity awareness, 6/26 on information and communication technologies, and 4/26 on information literacy. Before studying this module, 26/104 students had previously completed an escape room, albeit noneducational (25%), 45/104 were only familiar with the escape room concept (43%), and 33/104 had never heard of escape rooms (32%). None of the students had any prior experience with EER design. A total of 25/26 coursework submissions received a grade of 50 or higher and passed the module (96% success rate), with 18/26 getting a distinction grade of 70 or higher (69%).

During the last week of every offering of the Design Thinking module, all groups of enrolled students presented the EERs they had developed for their coursework using Room2Educ8 to the class. Each EER was then playtested by a group of three lecturers who employed the think-aloud protocol [98] to verbalise what they were thinking and doing as they played the game. The added dimension of having players share their thoughts, reactions, pleasure, and frustrations allowed the EER designers to understand the user experience of the game, uncover problems with puzzles, and highlight content that could be improved.

Once all EERs had been playtested, students were invited to participate in the framework's validation. Although this activity was voluntary and not part of the module's assessment, every student agreed to participate as it was an opportunity for them to experience the used research techniques which were relevant to their studies, have their voices heard, and discuss a topic of interest. An anonymised survey of 10 statements developed by the lecturer was employed to measure overall perceptions of Room2Educ8- s clarity, usability, and usefulness. The perception scale was a 5-point Likert scale ranging from "1—strongly disagree" to "5—strongly agree". To support or refute the quantitative findings from the survey, qualitative data were collected through 30-min-long semi-structured focus groups (four students per group) moderated by the lecturer and then analysed using content analysis [99]. Each individual focus group was made up of the four students who worked on the same EER. Indicative focus group questions are the following:


The survey and focus groups were conducted once per academic year (four separate times in total), with a different cohort of students in each offering of the module. A total of 104 students (*N* = 104) completed the survey and participated in 26 focus groups (*N* = 26). Data from the student survey forms were transferred into a Microsoft Excel spreadsheet and descriptive analyses using the total, mean, and standard deviation of feedback scores were performed.

#### *3.2. Results*

Survey results in Table 3 indicated that the framework was very detailed, with clear and understandable steps (*M* = 4.25, *SD* = 0.83) that were easy to follow regardless of lack of prior experience in EER design (*M* = 3.89, *SD* = 0.96). It provided designers with a comprehensive view of EER design (*M* = 4.41, *SD* = 0.60) and could be used to develop a wide range of EER types (*M* = 4.02, *SD* = 0.74) covering a variety of topics (*M* = 4.09, *SD* = 0.66). Using Room2Educ8 increased confidence in EER design (*M* = 4.44, *SD* = 0.63) and helped designers to develop 21st century skills such as teamwork (*M* = 3.94, *SD* = 1.03) and empathy (*M* = 4.27, *SD* = 0.86).

The focus groups also yielded positive results that supported the survey findings. Sample responses are presented in Table 4. In virtually all focus group sessions, participants expressed their initial concerns when they received the coursework brief, as they could not see how designing an educational escape room fitted to the curriculum. However, once they had completed the development of their EER, they could make this connection. One participant stated, "I must admit that I was sceptical about this assignment at first, but by the end I could see how Room2Educ8 can be an effective tool to learn and practise Design Thinking skills." Another major concern was the lack of any game design skills or limited experience with escape rooms. For these participants, the major advantage of Room2Educ8 was its detailed and well-described steps. "Honestly, when we got the coursework brief, I

was sure I was going to fail. I don't play videogames and had no clue what an escape room was before taking this class. To my surprise, the framework with its clear steps made the development of the EER straightforward, even for a noob like me." Another participant added, "At first, I was overwhelmed by the large number of Room2Educ8 steps, but the detailed instructions made them easy to follow." Using Room2Educ8 was also regarded as an effective way to develop teamwork and organisational skills. "I had never met these guys before and was unsure about what to expect. I am not a fan of groupwork, but we gelled very well, and everybody contributed to the project." Finally, the user-centred focus of the framework contributed to honing communication and empathy skills. "The empathise stage has definitely helped me to improve my listening and interview skills. I also got to understand how users feel and why."

**Table 3.** Survey results (*N* = 104).


**Table 4.** Sample focus group responses.


#### **4. Discussion and Conclusions**

As EER design is usually a time-demanding and complex task, the rationale for developing Room2Educ8 was to translate EER design into practical steps that educators and other interested parties with no prior experience with the escape room format could reasonably implement for their own teaching practice. Its prescribed nature also makes it approachable for experienced commercial escape room designers who are considering moving into serious games territory and want to create educational experiences.

According to the study findings, the framework enables the mapping of learning objectives against puzzles and narrative to build a cohesive interactive story that provides contextually immersive learning experiences. Educators and researchers can use Room2Educ8 with any core content subject to develop EERs that reinforce or teach critical concepts using auditory, visual, and kinaesthetic modalities. A framework based on

design thinking has the potential to initiate an innovation aspect and be a useful tool for teacher professionalism, as it can contribute to the development of the creative and adaptive capacities of the escape room designers by encouraging innovative and reflexive thinking [61]. The design thinking approach fosters many of the desirable traits identified as 21st century competencies [100], thus enabling framework users to acquire knowledge, skills, and attributes needed for collaborative problem-solving. Using Room2Educ8 may also contribute to the development of judgement, self-reflection, and practical wisdom, as it seeks to improve the learning experience in an inclusive way by incorporating the views and insights of the learners themselves. The human-centredness of such a framework can serve to nurture qualities necessary for social interaction and the cultivation of empathy. Therefore, Room2Educ8 can also be used by students to design EERs as part of a multi-week project to promote soft skills.

A limitation of this study is that, although the expected target audience for Room2Educ8 is mostly educators, it was used and validated by postgraduate students on a Design Thinking course who did not have a background in education studies, so the framework lacks evidence of widespread use. A broader sample of participants would be a truer reflection of the framework's value; therefore, future works will include similar trials with education students, pre-service teachers, and professional practitioners already working in the education sector in order to observe any similarities or differences towards already tracked reactions to the proposed framework. Another limitation is that students were asked to evaluate the framework in front of their lecturers before their coursework grades were released. Although participation in the framework's validation was voluntary, this "educator bias" may have influenced the students' answers. Room2Educ8 has been used to design EERs covering basic topics on cybersecurity, information and communication technologies, and information literacy. To support the notion that the framework is applicable to any subject, future work should include using Room2Educ8 to design EERs that cover a broader variety of topics, including technically applied courses. Finally, Room2Educ8 was only validated internally, i.e., its validation focused upon the integrity of the framework and its use. To support the study findings, a follow-up external validation addressing the effects of using the framework—the developed EERs themselves, and their impact on learners—will be conducted in the future.

**Author Contributions:** Conceptualization, P.F. and T.M.; methodology, P.F.; validation, T.M.; formal analysis, T.M.; investigation, P.F.; data curation, P.F. and T.M.; writing—original draft preparation, P.F.; writing—review and editing, P.F. and T.M.; visualization, P.F.; supervision, P.F. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Institutional Review Board Statement:** Ethical review and approval were waived for this study due to assessing the anonymised student feedback forms and knowledge from a teaching event.

**Informed Consent Statement:** Informed consent was obtained from all subjects involved in the study.

**Data Availability Statement:** The data that support the findings of this study are available from the corresponding author upon reasonable request.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**


### *Article* **Topic Evolution in the Research on Educational Gamification**

**Jakub Swacha**

Department of IT in Management, University of Szczecin, 71-004 Szczecin, Poland; jakub.swacha@usz.edu.pl

**Abstract:** The research on educational gamification spans many topics of interest. As the total volume of research in this area has greatly increased in the last 10 years, it is interesting to see how the interest in the respective topics has changed over the same period. In this paper, we answer this question by means of keyword analysis performed on 7572 unique keywords extracted from 2203 papers. The obtained results reveal (1) the high popularity of keywords that are non-obviously relevant to gamification, (2) vast disproportions in the volume of research dedicated to different aspects of the same research sub-area, and (3) differing patterns of popularity among the most frequent keywords, as well as (4) keywords introduced and abandoned in recent years. The presented findings bear a number of implications for the future of research on educational gamification.

**Keywords:** gamification in education; literature survey; keyword analysis

#### **1. Introduction**

Educational gamification strives to increase learners' motivation and engagement by incorporating game design elements into educational environments [1]. There is an ongoing debate about the extent and in which educational contexts it delivers what it is expected to [1]. The empirical data indicate that, while not always, more often than not, it does; the most extensive-to-date survey on educational-gamification outcomes reported positive results for 71.4% of the 91 analyzed studies [2].

While educational gamification builds upon the success of gamification in business environments, making use of many techniques first developed for business purposes (primarily, employee and customer engagement [3]), as the interests of teachers and students are much more aligned than those of employers and employees, or vendors and customers, the educational context finds itself free from some of its known drawbacks–in particular, there is no risk of exploitation often associated with gamification in the workplace [4–6].

Once a niche topic, the research on gamification in education amassed in recent years into a large body of knowledge, amounting to 105,000 items reported by Google Scholar for the search terms "education gamification" [7], thus, creating a large opportunity for secondary research. This opportunity did not go unnoticed, and a number of reviews on the research on educational gamification have been published in the last five years. These include systematic literature reviews [1,2,8–32], meta-analyses [33,34], systematic mappings [35–37], and bibliometric surveys [38–45]. None of them, however, focused on how the popularity of the covered topics changed over time.

The gap described above is addressed in this paper, which pursues the aim of understanding the evolution of the topics chosen in the research on educational gamification. The presented study is based on keyword analysis, a capable tool for tracking the evolution of a research area. The potential of this approach stems from the fact that keywords are deliberately selected by authors to correctly express the subject matter of papers ([46] and works cited therein). Keyword analysis has been widely implemented for providing an overview of the development of various research areas, including such diverse fields as animal behavior [47], epidemiology [48], psychiatry [49], management information systems [50], technology forecasting [51], and interdisciplinary research [52]. It has also been

**Citation:** Swacha, J. Topic Evolution in the Research on Educational Gamification. *Educ. Sci.* **2022**, *12*, 640. https://doi.org/10.3390/ educsci12100640

Academic Editor: Huei Tse Hou

Received: 31 August 2022 Accepted: 19 September 2022 Published: 22 September 2022

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2022 by the author. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

successfully applied in the fields of education [53] and educational technology [54–56], game-based learning [57,58], and gamification in general [40,59].

The following research questions have been stated:


The research questions listed above are answered in the six respective subsections of Section 3. Before that, in Section 2, the applied research method and used data sources are described. The final Section 4 concludes the main findings and presents their implications for future research within the area of educational gamification.

#### **2. Materials and Methods**

As the publications on educational gamification are not limited to a few venues or journals, rather than focusing on a preselected set of these, we decided to query bibliographic databases covering thousands of potentially relevant sources. Three providers of such databases have been chosen, two of which because of their overall extensive coverage of the scientific literature (Scopus [60] and Web of Science [61]) and one because of its thematic focus on education (ERIC [62]).

Considering the abundance of publications on educational gamification, in a strive to avoid false positives more than false negatives in the search results, we followed the example of [45] and used just two search keywords: "education" and "gamification".

Unless specified below, the standard query options have been used.

As only incomplete data are available for the year 2022, and the earliest indexed publications were published in 2011, the search period has been limited to 10 publication years: 2012–2021.

In both Scopus and Web of Science, the allowed language has been set to English, and the search was performed on Title, Author Keywords, and Abstract fields. The Index Terms field containing keywords generated by the respective database provider was not included in the search, as doing so resulted in a high number of false positives.

From Scopus, the allowed document types have been limited to "Conference Paper" (1915 items), "Article" (1103 items), and "Book Chapter" (142 items), giving a total of 3160 papers found.

From Web of Science, the allowed document types have been limited to "Proceeding Paper" (1518 items), "Article" (701 items), and "Book Chapter" (21 items); a total of 2203 papers has been found (some papers were assigned to more than one document type simultaneously).

From ERIC, the "Peer reviewed only" option has been set. The query results comprised 227 publications.

The three publication data files were then combined with the duplicates removed. The resulting dataset contained 4324 items, including 3138 that were found in Scopus, 1105 that were found in Web of Science (excluding those also found in Scopus), and 81 that were found in ERIC (excluding those also found in Scopus or Web of Science).

The Author Keywords field for the publications found in Scopus or Web of Science and the Manual Tags field for the publications found in ERIC were then processed to obtain a list of keywords. The first processing step involved converting all keywords and titles into lowercase, in order to allow for case-insensitive matching.

Next, the identified keywords were looked for in each publication's title, and matching publications were marked as relevant to a given keyword even if it was not specified in its own Author Keywords or Manual Tags field. Note that, in the case of keywords containing other keywords, only the longest matching keyword found in the title was considered (e.g., if a title contained "mobile learning", only the "mobile learning" keyword has been assigned to the work, and not "learning").

Next, the keywords used as the search terms (and their variant forms) were removed, as all analyzed works were required to use them to be included in the set, so there was no point in considering them for further analysis. The list of removed keywords comprised the following: "gamification", "education", "gamification in education", "educational gamification", and "gamifying".

Then, the counts for those terms having different spelling variants or forms were aggregated. In total, 56 such alternative forms were identified, including, e.g., "game" and "games"; "Technology Acceptance Model", "TAM", and "Technology Acceptance Model (TAM)"; or "game-based learning", "game based learning", "games-based learning", and "GBL". Finally, the uppercase letters were restored to the proper nouns and acronyms identified in the keywords.

After the above-described pruning, 7572 unique keywords were identified, which occurred 27,272 times in total during the analyzed period. There were 21 keywords which occurred at least 100 times, 58 keywords which occurred at least 50 times, and 158 keywords which occurred at least 25 times during the analyzed period. Looking at the other end of the list, there were 4677 keywords which were encountered only once over the whole analyzed period, 1053 keywords which were encountered only twice, and 448 keywords which were encountered 3 times each. The large number of single-instance keywords is clearly an indicator of a very wide thematic span of the analyzed dataset; among such keywords, there were, e.g., "3d digital storybook", "Ada", "business process simulation", "certification systems", "digital citizenship education", "egyptology", "Filipino sign language", "global warming", "honeymoon effect", "information ecology", "joyful learning", "Kolb cycle", "language fluency", "Mandarin as second language", "nutritional knowledge", "on-the-hour effect", "passive haptics", "quality education", "recurrent neural networks", "scale development", "team formation", "user research", "virtual library", "WordNet", "xMOOC", "young workers", and "zero-cost implementation".

All bibliographic data processing, unless specified otherwise, was performed using dedicated scripts written in Python.

#### **3. Results and Discussion**

#### *3.1. The Evolution of the Yearly Number of Publications*

Before we proceed to the keyword analysis, Figure 1 shows the quantitative development of the educational gamification field during the last 10 years.

As can be observed in Figure 1, the field was characterized by a very fast consistent growth from its beginnings until the year 2019. There is a small drop in 2020 (which could be possibly linked to the general decrease in research efforts due to the COVID-19 pandemic and the turmoil it caused in higher education institutions [63]) and an even smaller rebound in 2021 (possibly indicating that the educational gamification research is slowly getting back on its pre-COVID-19 level).

#### *3.2. The Overview of Educational Gamification Research Topics*

In order to provide an overview of the topics appearing in educational gamification research, a word cloud has been drawn based on the list of the most frequent keywords. The WordClouds.com tool has been used for this purpose [64]. The resulting word cloud is shown in Figure 2.

**Figure 1.** Number of works included in the analyzed dataset per publication year.

**Figure 2.** The world cloud of the keywords most often used in the educational gamification research.

Based on the observations in Figure 2, the following groups of keywords used in the research on educational gamification could be identified:


Note that some keywords can be assigned to more than one group, as they can be used for different meanings, e.g., "application" could mean either software (i.e., member of group 4) or the act of applying some method to solve a given problem (i.e., member of group 5).

The identified groups of keywords allow for the indication of the main directions of research on educational gamification:


#### *3.3. The Most Frequently Used Keywords*

Throughout the entire analyzed period, the top 10 keywords were as follows: "higher education" (used 383 times), "game-based learning" (363), "motivation" (353), "serious games" (314), "game" (298), "learning" (273), "e-learning" (224), "teaching" (204), "engagement" (186), and "use" (149). These 10 keywords together were used 2747 times, which constitutes about 10% of all counted uses. The top 25 keywords (covered in Section 3.5) were used 4393 times, which constitutes about 16% of all counted uses. Table 1 lists the 5 most frequent keywords for each year covered in the analysis.

**Table 1.** The most frequently used keywords for each year covered in the analysis.


As can be observed, with the growing number of papers published each year, the number of appearances of the top keywords grew as well; having three uses was enough to become the most frequent keyword in the area of educational gamification in 2012, whereas it required over 90 uses in 2021.

Only one keyword appeared in the top five list throughout the entire analyzed period: "motivation". As motivation is mentioned in the presented definition of educational gamification [1], this could be expected; what was not expected is the much lower popularity of the second term mentioned in the definition: "engagement", which, although included among the ten most frequent keywords, did not make it into the top five list in any year of the analyzed period. This indicates that engagement (or the rising thereof) is a gamification purpose that is tackled more rarely than motivation (or the building thereof) by educational gamification researchers, despite the fact that, in education, there are multiple sources

of motivation [65], whereas the lack of engagement is often a serious issue, especially in formal education [66].

Interestingly, the two keywords which were missing from the top five (only once, in 2015 and 2020, respectively) were "game-based learning" and "serious games". Together with "game" (having six appearances in Table 1), these keywords have a non-obvious relationship to gamification, which is usually portrayed as a concept different to fullyfledged games [67]. After screening the publications using these keywords, it was found that, while some of them are actually devoted to both serious games and strictly defined gamification, far more of them simply denote the application of games in education as gamification, clearly indicating that not all researchers are aware of the distinction between the two concepts or agree with it. This point of view can be easily understood if we consider gamification as a process of making non-game activities resemble games, with the most extensive form of this process consisting in turning non-game activities into fully-fledged games.

"Higher education" also made it into the top five keywords six times, but was the keyword which took the top spot most often (three times). No other keyword denoting a level of education made it into the top five for any year during the analyzed period. For a comparison, during the whole period, "higher education" was used 383 times as a keyword, whereas "primary education" was used only 39 times and "secondary education" only 33 times. Such a huge difference suggests that educational gamification researchers are most interested in implementing gamification where they themselves teach, and this is most often in higher-education institutions. This is in contrast to the evolution of students' interest in games, which was reported to peak at the middle-school age [68].

While "learning" appears twice in Table 1 (both times in the top spot in the respective years), "teaching" appears only once (and in the fifth spot), and "training" did not qualify for the top five list during any year. This indicates that the educational gamification literature is much more focused on the student's perspective than the teacher's.

Only one keyword denoting a mode of education made it to the top five keyword list: "e-learning" (in four years: 2012, 2013, 2015, and 2016). It was used 224 times during the whole analyzed period; for comparison, in total, only five uses of "traditional learning" and three uses of "face-to-face learning" were found, and not a single instance of "brick-andmortar learning", whereas "blended learning" was used 71 times. This clearly indicates that, despite gamification being used in offline contexts, it is most often applied to computermediated instruction and learning.

#### *3.4. Keyword Co-Occurrence*

Figure 3 presents 40 pairs of keywords with the highest measured value for the Pearson correlation coefficient (the beige line between two keywords indicates such a pair). While most of these pairs consist of two obviously-related concepts (e.g., "educational games" and "educational technology", "engagement" and "motivation", "augmented reality" and "virtual reality", "active learning" and "flipped classroom", "innovation" and "technology", or "game" and "serious games"), some connections bring some new insights, as is stated in the following.


**Figure 3.** The most often co-occurring keywords.

#### *3.5. The Change in Keyword Usage*

Figure 4 compares the number of occurrences for the top 25 keywords in each year during the analyzed period. We can clearly observe the large difference in the number of occurrences between the most frequently used keywords and the remaining ones, and for the first part of the analyzed period, the fast growth in the number of occurrences.

**Figure 4.** The number of keyword occurrences per year during the analyzed period.

In order to make it easier to spot the change in the number of occurrences (also for the less frequent keywords, Figure 5), the data for the same top 25 keywords are presented using relative values (100% denotes the maximum number of occurrences of a given keyword in any year during the analyzed period).

**Figure 5.** The relative number of keyword occurrences per year.

Based on usage trends visible in Figure 5, we can assign each of the top 25 keywords to one of the following groups:


Regarding group 1, and its only representative, it is startling to see how fast and consistent the interest in gamification in higher education was, growing throughout the 10 analyzed years. While this finding is consistent with prior research [31,38,41], none of these indicated the unique status of this research topic. It is interesting to think about how long this trend may continue, and whether the interest in gamification of higher education will form a plateau, staying at a high level, or will decrease more or less rapidly.

Regarding the keywords of group 2, despite there being some fluctuations in popularity (i.e., individual years with the number of publications over or below the trend line), they generally kept growing throughout the analyzed period. Especially worth noting is one member from this list: "evaluation", indicating a growing interest in evaluating educational gamification applications, which is a good sign considering that this aspect of the educational gamification research has been somewhat neglected in the past, as indicated by the fact that only 20 out of 179 educational gamification papers analyzed in [37] included evaluation.

The popularity of the keywords belonging to group 3 seems to have passed its peak. It seems that the volume of writing on motivation and engagement in the educationalgamification context has reached some level of saturation. It is quite surprising to see a fall in the interest in the application of educational gamification to Massive Open Online Courses ("MOOC") and to mobile learning. As for "learning" and "teaching", this may indicate just a change in the selection of the used keywords from very general (as the two mentioned keywords are) to more specific, which is understandable as the field matures.

The group 4 keywords seem to have surpassed a decrease in interest and returned to the path of popularity growth. For "e-learning", the explanation seems straightforward, as the rebound in its popularity can be clearly linked to the outburst of the COVID-19 pandemic and the emergency remote teaching it resulted in [69].

The popularity of keywords in group 5 reached their highest popularity level earlier ("serious games") or later ("augmented reality") and has stayed there since (with small fluctuations). The last member of this group could be seen as unexpected, as educational gamification applications using augmented reality, in spite of many of them being developed (see, e.g., [70]), have not been as widely used as non-educational augmented reality applications such as Pokémon GO (of course, we are aware of the educational use of Pokémon GO, as evidenced, e.g., in [71], but this is not a common practice either).

#### *3.6. The Fresh and Abandoned Keywords*

The list of all keywords has been screened to find the most popular keywords that have started to appear only in the last three years, as well as the most popular keywords that have no longer appeared in the last three years. The results of this screening are shown in Tables 2 and 3, respectively (only keywords with a total of at least seven occurrences in the respective sub-period were included in order to keep the tables compact).



**Table 3.** The most popular abandoned keywords.


As could be expected, "COVID-19" made it to the top of the fresh keywords list. This is understandable, considering the huge disruption it caused to the whole educational domain [72].

The remaining fresh keywords attained much less popularity. The most unexpected is the presence of "meta-analysis" on this list, indicating that there were no meta-analyses on the topic of educational gamification published prior to 2019.

There are no highly popular keywords listed in Table 3, which means none of the wider sections of research on educational gamification ceased to exist. Quite unexpectedly, "persuasive technology" and "pervasive games" top this list, even though the general interest in these topics (i.e., outside of educational gamification) keeps increasing (see, e.g., [73]). Maybe even more puzzling, is the lack of continued interest in "social networking" among the educational gamification researchers, even though social network is often listed as one of primary gamification components (see, e.g., [74]). As for "games and learning", both its component terms remain highly popular keywords, so, apparently, it is just that the educational gamification researchers simply no longer combine them into one keyword.

#### **4. Conclusions and Implications for Future Work**

Gamification in education is a quickly growing area of research, with hundreds of scientific papers devoted to it being published every year. Although this body of literature has been the subject of numerous secondary research studies [1,2,8–45], so far, none of these focused on how the popularity of respective covered topics changed over time.

This paper contributes to addressing this gap via keyword analysis, helping to understand the evolution of the set of topics most often selected in the research on educational gamification.

The obtained results provide a handful of novel insights. Already, the identification of the most frequent keywords throughout the entire analyzed period revealed that two are in a non-obvious relationship to gamification ("game-based learning" and "serious games", which are perceived by some authors as distinct from gamification [67]) and a lack of balance both between student–teacher perspectives (with much more attention given to the student perspective) and the different levels of education (with much more attention given to higher education than to the other levels of education). This brings implications to future work: firstly, unless a significant share of gamification research is to be ignored, a more comprehensive definition of gamification is needed, clearly placing, under its umbrella, the adoption of fully-fledged games for non-entertainment purposes (rather than only game elements). Secondly, more research attention is needed on those aspects of educational gamification that were insufficiently addressed by prior studies, i.e., the gamification of teaching (rather than learning) and the application of gamification at various education levels (rather than higher education only).

The analysis of keyword usage in subsequent years reveals that, despite the consistent growth in the volume of educational gamification research, the popularity of the most frequent keywords has evolved in differing ways. Only 10 of the top 25 most frequent keywords were characterized by steady growth in their yearly number of occurrences, whereas, for the same number of keywords, the number of occurrences decreased in the most recent years. The popularity level of three keywords has stabilized, and two keywords rebounded from a period of lower popularity. The analysis of these patterns also provides some implications for further research, such as the need for a meta-analysis of the models used in educational gamification or the need for the identification of the barriers hampering the spread of the numerous gamified augmented reality applications into educational practice.

We have also identified keywords that came into use only recently and keywords that are no longer used in spite of having some popularity in the past. As the former group indicates the emerging research fronts (the main of which regards COVID-19), the latter directs the educational gamification researchers' attention toward topics that have lost it (in particular, "persuasive technology", "pervasive games," and "social networking").

**Funding:** This research received no external funding.

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

**Data Availability Statement:** Data are available from the authors on request.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**


## *Article* **Let's Escape! The Impact of a Digital-Physical Combined Escape Room on Students' Creative Thinking, Learning Motivation, and Science Academic Achievement**

**Hsu-Chan Kuo 1, Ai-Jou Pan 2, Cai-Sin Lin <sup>3</sup> and Chu-Yang Chang 4,\***


**Abstract:** Digital and physical escape rooms have been suggested as practical and effective approaches to game-based learning and have recently gained momentum. The deficit of scholarly works that simultaneously implement both types of escape rooms legitimizes this study's significance and appropriateness. The researchers systematically combined digital and physical escape rooms and integrated them into fifth-grade science lessons (experimental group N = 22; control group N = 21). Considering that creative thinking is one of the essential competencies in the competitive world, learning motivation is a crucial factor contributing to students' learning, and academic achievement is a criterion for learning outcomes. The Torrance Test of Creative Thinking (fluency, flexibility, originality, and elaboration), the Learning Motivation Scale (value, expectation, affect, and executive volition), and the science achievement exam were used to quantitatively investigate students' learning effectiveness. The results indicated that the experimental group's creative thinking and learning motivation outperformed the control group significantly. Nonetheless, both groups showed no significant difference in science academic achievement. The present study verifies that a digital– physical combined escape room is an effective and practical approach that has the potential to be widely used in schools to benefit students' learning. Some discussions, educational implications, and suggestions for future studies and practices are offered.

**Keywords:** game-based learning; escape room; creative thinking; learning motivation; science academic achievement

#### **1. Introduction**

Children not only enjoy themselves while playing a game but also learn in the process of playing. Rushton and King [1] indicated that play is a pedagogical vehicle for learning STEM subjects. Garaigordobil and Berrueco [2] also found that play can develop children's creative thinking. Game-based learning and gamification are considered feasible methods to develop students' 21st-century skills (e.g., critical thinking and collaboration), stimulate learning motivation, and promote a sense of enjoyment in learning [3,4]. Among various kinds of game-based learning, escape rooms have gained momentum in education practices and research recently. Nicholson [5] defined escape rooms as "live-action team-based games where players discover clues, solve puzzles, and accomplish tasks in one or more rooms in order to accomplish a specific goal (usually escaping from the room) in a limited amount of time". Recreational escape rooms originated in Japan in 2007 and grew rapidly from 2012–2013 [5]. Borrowing from the concept of a recreational escape room, teachers create a compelling narrative and embed knowledge into several puzzles, in which students are required to use course materials and knowledge to solve a series of puzzles and then find a way/code to

**Citation:** Kuo, H.-C.; Pan, A.-J.; Lin, C.-S.; Chang, C.-Y. Let's Escape! The Impact of a Digital-Physical Combined Escape Room on Students' Creative Thinking, Learning Motivation, and Science Academic Achievement. *Educ. Sci.* **2022**, *12*, 615. https://doi.org/10.3390/ educsci12090615

Academic Editor: Huei Tse Hou

Received: 24 August 2022 Accepted: 9 September 2022 Published: 13 September 2022

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

succeed in an escape room [6]. Educational escape rooms have been increasingly integrated into courses since 2017 [7,8], and their application encompasses various fields, including programming [6], mathematics [9], medication [10,11], and social science [12].

Educational escape rooms have received attention in these years and are still in the early stage. Educators usually encounter some challenges when designing escape rooms: the requirement of broad space and specific equipment, the consumption of much time in preparing and conducting escape rooms, and the difficulty in embedding learning objectives into puzzles [13]. In a regular school setting, it is sometimes a challenge for teachers to iron out the abovementioned problems. Considering these, some educators have started using digital escape rooms, in which students can enjoy escape rooms through online platforms everywhere. Teachers can organize an escape room without concern for space and equipment limitations. However, it is believed that physical and digital escape rooms have their own separate strengths and constraints. For example, digital escape rooms cannot provide students with hands-on experience, authentic work environments, or a feeling of "escaping" from a room in the same way physical escape rooms can [14]. To augment student learning effects, educators can conduct digital and physical escape rooms sequentially, letting their strengths complement each other.

Although some studies have claimed that escape rooms can effectively develop students' creative thinking [15], they have not conducted experimental activities to test the effect scientifically. There is a need for empirical evidence about how escape rooms affect students' creative thinking [14]. Recently, creative thinking has received more and more emphasis due to the dire need for innovation in industries. Creative thinking refers to the ability to generate novel and valuable ideas that can lead to positive change [16]. Generally, creative thinking includes four abilities: fluency, flexibility, originality, and elaboration [17]. These abilities are indispensable for students to survive in the highly competitive world. Learning motivation is also a critical factor influencing students' learning. Students with high motivation are willing to overcome every bottleneck and persist in their studies [18]. This study aims to systematically design digital and physical escape rooms, conduct them, and examine their impact on the students' creative thinking, motivation, and academic achievement.

Past scholarly work primarily focused on adults, especially undergraduate students, but the effectiveness of escape rooms on primary school students was seldom explored. Makri et al. [7] reviewed the prior research and indicated that there is a need to investigate the impact of escape rooms on primary school students. Lathwesen and Belova [14] found that most studies did not use a comparison and treatment group design. The empirical evidence of escape rooms' learning effect is still inadequate and requires more comparison and treatment group design in this field. To bridge the research gap, the researchers appropriately designed physical and digital escape rooms based on the escapeED framework [19], which was conducted with reference to the steps of an educational escape room activity [20]. The current study employed a quasi-experimental design to examine the impact of the escape room intervention on primary school students' learning in science lessons. The followings are the questions of the research:


#### *1.1. The Benefits of Digital and Physical Escape Rooms*

Physical and digital escape rooms have their own strengths and constraints. The prior studies found several benefits of using a physical educational escape room in students' learning, for instance, consolidating content knowledge; triggering students' learning interests, enjoyment, and motivation; and cultivating their communication skills [8,14]. Moreover, the immersive environment and the simulation of authentic work situations in escape rooms can provide students with hands-on experience [21]. The immersive learning

environment is a crucial strength for physical escape rooms, as most existing digital settings are considered to be low-tech and cannot provide an immersive experience [14]. However, there are some limitations and challenges when educators design escape rooms, including budget restrictions, classroom and specific equipment availability, the limited number of players, materials that need to be multiplied, and investment of time [7,13,22].

Digital escape rooms have also received tremendous attention recently, as educators can design and conduct a digital escape room without worrying about the limitations of space and specific equipment. As with physical escape rooms, digital escape rooms can also be used to stimulate students' collaboration and motivation and contribute to knowledge acquisition [7,23,24]. However, it is difficult for educators to design complicated digital puzzles as they design a physical escape room. For instance, students do not need to find a tangible item or unlock a real chest in digital puzzles, and some digital puzzles are produced by Google Forms [25,26]. Digital puzzles can be regarded as a battery of tests for individuals in which students do not need to work in a group [24]. The benefit is that they can be easily designed to examine students' content knowledge in specific fields directly. Makri et al. [7] said that "*digital puzzles required players to write or execute codes, allowing them to test and improve skills (e.g., programming skills), whereas physical puzzles are of great help for enhancing both the immersion of the experience and student engagement in the narrative.*" Furthermore, in a remote-learning setting, educators cannot ensure that students complete each puzzle without outside help (e.g., looking up answers on a cell phone or the internet) or "carrying" other students [25].

Past scholarly works have theoretically identified that puzzle-driven escape rooms can help improve a student's learning motivation, key competencies, and many high-order capabilities, including creativity, communication, and academic achievement. The potential effects/impacts and restrictions concerning the employment of physical escape rooms and digital ones are articulated and discussed, which helped the current study to develop a theoretical foundation for the merger of physical and digital escape rooms. Physical and digital escape rooms both have a positive impact on students' learning, and it is believed that both types of escape rooms have their own pros and cons (see Figure 1). The best way to conduct an escape room may be to combine physical and digital rooms, complementing each other and maximizing the students' learning. Thus, based on the past scholarly works and theoretical foundation, the study held physical and digital escape rooms sequentially in elementary school science classes. It is hypothesized, with theoretical support, that the intervention can contribute to the student's creative thinking, learning motivation, and academic achievement.

**Figure 1.** The constraints and strengths of physical and digital escape rooms.

#### *1.2. The Design of Escape Rooms*

The origin of educational escape rooms is an adaptation of recreational games. Therefore, most existing research has not designed escape rooms from theory or frameworks [8]. It is essential for educators to adopt a game-based framework and carefully design escape rooms so that students' game experiences can link to learning goals and key competencies [7]. Clarke et al. [19] proposed the EscapED framework, including six stages that should be considered when designing escape rooms: Participants, Objectives, Theme, Puzzles, Equipment, and Evaluation. In the Participants stage, designers should analyze the demography of targeted players, difficulty levels, and mode. Then, the learning objectives of the games need to be set. These may be academic achievements or soft skills (e.g., creative thinking and communication). Furthermore, to maintain players' motivation and interests, designers should develop a compelling narrative in the Theme stage. In the Puzzle stage, the most challenging part, designers have to design interesting puzzles, and all puzzles should reflect learning objectives. In addition, instructions, rules, and hints should be determined. In the Equipment stage, designers must think about specific equipment and props (including digital devices and real-life actors) needed in the game. The last stage is Evaluation, in which educators have to determine which methods and tests are used to evaluate participants' learning outcomes. Besides the five stages mentioned above, Botturi and Babazadeh [13] identified five game elements (narrative, game-flow, puzzles, equipment, and learning process) and four context elements (players, constraints, evaluation, and debriefing). Educators must consider all elements before conducting escape rooms in their classes.

Once narratives and puzzles are determined, educators should think about the procedure to conduct escape rooms. Abdul Rahim et al. [20] proposed five steps for typical educational escape rooms: (a) a pre-activity test assessment, (b) a game briefing, (c) the ER activity, (d) a post-activity knowledge assessment and perception survey, and (e) a debriefing. Pre- and posttests are essential to examine escape rooms' effects scientifically. Lathwesen and Belova [14] found that only a few studies used pre–post surveys to measure escape rooms' learning effect and affective outcomes. There is a need for more empirical evidence in this field. The game briefing stage refers to instructions before the outset of the game. Moore and Campbell [21] indicated that students might feel confused if a game does not have a clear start. The confusion may lead to students' non-success in games. At the end of escape rooms, a debriefing is a crucial part and cannot be neglected, in which players talk about their feelings, ask questions, and discuss the game [5]. To help students connect game experience and learning objectives, teachers can discuss puzzles and talk about content knowledge behind each puzzle. Without a debriefing stage, the discrepancy between perceived goals and actual goals may be caused.

#### **2. Method and Material**

#### *2.1. Participants*

There were 22 (10 boys and 12 girls) students in the experimental group and 21 (10 boys and 11 girls) in the control group. All the participants were fifth graders (11–12 years old) from two classes in Taiwan. The participants had not experienced an educational escape room; thus, it was a new task and experience for the students.

#### *2.2. The Design of the Escape Rooms*

To ensure educational escape rooms were appropriately developed and reflected learning objectives, the current study followed the six stages of the escapeED framework [19] to design digital and physical escape rooms. Six components in the framework were considered: Participants, Objectives, Theme, Puzzles, Equipment, and Evaluation. A brief description of each stage is shown inTable 1. The researchers thoroughly thought about each stage and carefully designed escape rooms according to each component. The following section introduces a detailed introduction of digital and physical escape rooms.


**Table 1.** The introduction of the six stages (Participants, Objectives, Theme, Puzzles, Equipment, and Evaluation) of the escapeED framework in the study.


**Table 1.** *Cont.*

#### 2.2.1. Digital Escape Rooms

Two digital escape rooms for two science lessons ("Aqueous Solution" and "Force and Motion") were developed and conducted on the Holiyo platform (https://holiyo.tn. edu.tw/game/game\_platform/login.html, accessed on 9 August 2022), which allowed teachers to design their own escape rooms for students. The home page of the escape rooms is shown in Figure 2. Dark red and black were adopted as background colors to create a mysterious atmosphere. The two digital escape rooms each had six puzzles (see Figure 3). Besides the digital escape rooms, the teacher also designed cards for each puzzle. Students worked in groups of three and had to use knowledge about "Aqueous Solution" and "Force and Motion" to solve puzzles. After each digital escape room, a debriefing was conducted to help students reflect on what they learned during the escape room.

**Figure 2.** The homepage of the digital escape rooms.

**Figure 3.** The interface of the digital escape rooms. There were six puzzles in the digital escape rooms. The bottom of the page shows the time left.

The narrative of the first digital escape room for "Aqueous solution" was that students were kidnapped by an evil scientist and put into a mysterious laboratory. There were various solutions and documents scattered on the ground. To escape from the laboratory, students had to discern the acid and alkaline solutions and determine the conductive solutions that could help them open an electric door and then escape. An example of a puzzle is shown in Figure 4. In the puzzle, students had to identify which solutions were acid and alkaline. Students had four cards representing a saline solution, sugar water, soda water, and baking soda, and they had to enter the correct answer to the puzzle. In the "Force and Motion" digital escape room, the game began with the narrative that the students were caught again by the evil scientist and put into a totally different laboratory. This time, students had to use their knowledge of "Force and Motion" to solve six puzzles and then escape from the laboratory.

**Figure 4.** An example of a puzzle and corresponding cards. (**a**) Description of the question and the English translation in the puzzle: You found a solution in a beaker with litmus papers in it. The color of a red litmus paper did not change, but a blue litmus paper turned red. Which solution may be in the beaker? (Enter the name of the solution). (**b**) Description of the cards used in the first digital escape room and the English translation: A saline solution (blue), sugar water (green), soda water (purple), and baking soda (red).

#### 2.2.2. Physical Escape Rooms

The teacher arranged a classroom for a physical escape room, with five puzzles in the room. All props and equipment were scattered around the room (see Figure 5). All puzzles required students to use what they learned from two science lessons, "Aqueous Solution" and "Force and Motion." Students worked in groups of three and had to find items they needed to solve each puzzle, get a code, and then escape from the room.

**Figure 5.** A panorama of the physical escape room.

The five interconnected puzzles used in the physical escape room were as follows: In the first puzzle, students needed to find four conical flasks with different solutions (see Figure 6). Then, students dripped purple cabbage juice into each solution to identify whether the solutions were acid or alkaline. The solutions turned into different colors according to the degree of acidity and alkalinity, and the different colors represented different codes on the wallpaper (see Figure 7). Students decoded the meaning of the colors

according to the clues on the wallpaper and then obtained a password to unlock the first chest (see Figure 8), in which there was a clue for the next puzzle. In order to help the students to link what they learned with the problems encountered in real-life scenarios, the students were asked to solve real-world problems. In the second puzzle, students learned how to measure weight. Students were provided with a spring balance. The chest code was the total weight of the items hidden in dolls scattered in the room. In the hands-on problem-solving task, the students needed to appropriately apply the knowledge of weight and weight measurement to the task (see Figure 9). In the third puzzle, students needed to use litmus papers to examine the acid and alkali levels of different daily solutions (soda water and juice). While acidic water made the litmus paper red, the alkaline solution made it blue. The students used the instrument to identify the acid and alkaline levels of the solutions and find the correct code to unlock the treasure chest (see Figure 10). In the fourth puzzle, students learned about the concept of velocity in daily life (e.g., the meaning of km/hr) and the different characteristics and speeds of different kinds of transportation (e.g., railway and plane). The code was hidden in a world map. Only when the students identified different velocities and transport characteristics could they resolve the puzzle and obtain the code from the map (see Figure 11). In the fifth puzzle, students categorized different kinds of force and movement states in daily life (e.g., kicking a football, dribbling a basketball, pressing clay, stretching a spring). The correct classification enabled the students to find the correct code (see Figure 12).

(a) (b) (c) (d)

**Figure 6.** Conical flasks and test tubes in the escape room: Different conical flasks and test tubes were scattered and hidden in the escape room. Students had to find them in the first puzzle. (**a**) A test tube hidden behind the keyboard. (**b**) A conical flask on a desk. (**c**) A conical flask hidden behind a doll. (**d**) A conical flask hidden under a table.

**Figure 7.** The poster shows different codes behind different colors. Different solutions showed different colors when students dripped the purple cabbage juice into them. Students used the wallpaper to decode the meaning of the colors.

**Figure 8.** Chest in the first puzzle. Once students figured out the code in the first puzzle, they could use the code to unlock the chest and get the clue for the next puzzle.

(a) (b) (c)

**Figure 9.** Dolls and spring balance in the second puzzle. (**a**) A spring balance scale: students were asked to find components of the spring balance scale and assemble them. (**b**,**c**) The two soft toys: Students had to use the spring balance scale to measure the precise weights of the items from the two soft toys.

**Figure 10.** Litmus papers in the third puzzle. Students were asked to use litmus papers to examine the acid and alkali levels of different daily solutions (soda water and juice). The correct identification of the acid and alkaline levels in the solutions enabled the students to unlock the treasure chest and successfully enter the next puzzle—"Motion and Transport." (**a**) Litmus papers in a treasure chest. (**b**) The riddle card in the puzzle.

(a) (b)

**Figure 11.** Different transport cards and a world map in the fourth puzzle. (**a**) Students were asked to identify the different velocities and transport characteristics. (**b**) Once they could identify the different velocities and transport characteristics, they could figure out the puzzle and get a code from the map.

**Figure 12.** Cards for different kinds of force and the last treasure chest. (**a**) Students had to correctly classify various kinds of force in daily life and arrange the cards appropriately. (**b**) The correct classification enabled the students to unlock the last treasure chest.

#### *2.3. Research Instruments*

2.3.1. The Chinese Version of the Torrance Test of Creative Thinking (Chinese TTCT)

The students' creative thinking was measured by the Chinese TTCT [27], one of the most widely used creative thinking tests. The indicators of the test are as follows. (a) Fluency refers to the number of different ideas. (b) Flexibility refers to the diverse categories

of ideas. (c) Originality refers to the novelty (a statistical rarity) of ideas. (d) Elaboration refers to the number of additional ideas added to the responses.

The test included two subtests. (a) Figural test: Respondents were asked to develop as many visual designs as possible based on a given Chinese character, "人 " ("Ren", which means a human in Chinese). The subtest measured respondents' four creativity dimensions: fluency, flexibility, originality, and elaboration. Each dimension's reliability (Cronbach's α) was 0.96, 0.94, 0.86, and 0.91, respectively. (b) Verbal test: Respondents have to think about unusual uses of bamboo chopsticks. The subtest measured respondents' three creativity dimensions: fluency, flexibility, and originality. The reliability (Cronbach's α) of each dimension was 0.99, 0.95, and 0.91, respectively.

#### 2.3.2. Learning Motivation Scale (LMS)

The students' learning motivation was measured using the Learning Motivation Scale (LMS) [28], a five-point Likert scale. There are four subscales with 35 items in total. The subscales are as follows. (a) Value refers to students' perceptions of the importance and usability of the lesson. There are seven items in the section. An example of a question: "I believe that reading academic books is important for students." (b) Expectation refers to students' expectations for their success or failure in their academic learning. There are six items in the section. An example of a question: "I think I can learn academic knowledge well all the time." (c) Affect refers to students' positive/negative affections when learning and studying. There are ten items in the section. An example of a question: "I enjoy reading academic books." (d) Executive volition refers to students' ability to control their behaviors and thoughts to maintain their engagement in academic learning. There are 16 items in the section. An example of a question: "Although I feel tired when doing homework, I persist in writing until I finish it." The reliability (Cronbach's α) of each dimension was 0.89, 0.87, 0.94, and 0.90, respectively.

#### 2.3.3. Science Achievement Exam

The science achievement exam was developed by teachers to assess students' science academic achievement in science lessons thrice a semester. Students had 60 min to complete the test. The first and second achievement tests were used as pre- and posttests in the study. The first test topic consisted of two parts: "The Observation of the Sun" and "The World of Plants," while the second test was composed of "Aqueous Solution" and "Force and Motion." The first test served as a criterion to examine whether the experimental and control group students' science academic performances were similar before the intervention. To ensure the exams' content validity, teachers developed two-way specification tables according to four of Bloom's [29] categories in the cognitive domain when developing the tests, including Knowledge (8 items), Comprehension (10 items), Application (13 items) and Analysis (5 items). The reliability (Cronbach's α) of the first test was 0.78, and the second test was 0.71.

#### *2.4. Experimental Procedure*

A pretest–posttest control group design was used to investigate students' creative thinking, learning motivation, and science academic achievement. The schedule of the experimental procedure is shown in Table 2. Students in the experimental and control groups participated in a ten-week science lesson for 120 min each week, in which two digital escape rooms and one physical escape room were held for the experimental group during the science lesson. The science topics included "Aqueous Solution" and "Force and Motion". Before teaching the first science topic, the teacher conducted the pretest, in which students spent 60 min completing the Chinese TTCT and 20 min on the LMS.


**Table 2.** The schedule of the experimental procedure.

After the pretest, the escape room intervention was implemented on the experimental group. The difference between the experimental and control groups was the implementation of escape rooms. Both groups learned the same science topics taught by the teacher using the same teaching method (didactic instruction and group discussion). The first science topic was "Aqueous Solution," The students learned about the properties of acid and alkaline solutions and how to use acid-based indicators. The first digital escape room was conducted in the experimental group after the students learned the course materials of the first lesson. After the first digital escape room, the science lesson moved to the next topic, "Force and Motion." Students learned what velocity and friction are, how to measure force and weight (e.g., using spring balance), and the application of friction in our lives. The second digital escape room was employed when the lesson was finished. To help students review what they learned in the previous lessons, the teacher designed a physical escape room and then held it in the last week. All escape rooms required students to use what they learned in the science lessons to solve puzzles and "escape" from the room.

After the intervention, a posttest was implemented. Students took the Chinese TTCT for 60 min and the LMS for 20 min. The researchers did not conduct science achievement tests in person, as the tests were conducted by the school to assess whole-school student science learning. After receiving all the data, the researchers analyzed the collected data to examine the impact of the intervention on students' creative thinking, learning motivation, and science academic achievement.

Abdul Rahim et al. [20] proposed that typical educational escape rooms comprise five steps: a pre-activity test/survey, a game briefing, an escape room activity, a post-activity test/survey, and a debriefing. The current study referred to these steps when conducting both digital and physical escape rooms each time. The digital and physical escape room intervention session is shown in Table 3.


**Table 3.** The session of digital and physical escape rooms.

#### **3. Result**

The paired sample *t*-test and covariance analysis (ANCOVA) were used to test the collected quantitative data. The paired sample *t*-test was carried out to examine improvements in the students' creative thinking, learning motivation, and science academic achievement in the experimental and control groups. One-way ANCOVA was performed to determine whether there was a significant difference in students' creative thinking, learning motivation, and academic achievement between the two groups.

#### *3.1. Creative Thinking*

The paired sample *t*-test result of the TTCT (verbal test) is shown in Table 4. The result showed a significant improvement in the experimental group's overall creative thinking, fluency, flexibility, and originality, while no significant improvement was found in the control group's overall creative thinking, fluency, flexibility, and originality. The results indicated that room escape intervention can significantly enhance students' creative thinking. The ANCOVA result of the TTCT (verbal test) is shown in Table 5. There were significant differences between the experimental and control groups' overall creative thinking, fluency, and flexibility, while no statistical difference was found in the aspect of originality. A high effect size was found in overall creative thinking (η2 = 0.41) and each dimension, fluency (η2 = 0.49), flexibility (η2 = 0.56), and originality (η = 0.16).

**Table 4.** Result of the paired sample *t*-test on creative thinking (verbal test).


\* *p* < 0.05 Effect sizes (ES): d = 0.2–0.5 (small effect), d = 0.5–0.8 (moderate effect), and d ≥ 0.8 (large effect) [30].


**Table 5.** Result of ANCOVA on creative thinking (verbal test).

*\* p < 0.05.* Effect size (ES): 0.01 ≤ <sup>η</sup><sup>2</sup> ≤ 0.059 (small effect), 0.059 ≤ <sup>η</sup><sup>2</sup> ≤ 0.138 (moderate effect), and <sup>η</sup><sup>2</sup> ≥ 0.138 (large effect) [30].

The paired sample *t*-test result of the TTCT (Figural test) is shown in Table 6. The result showed a significant improvement in the experimental group's overall creative thinking, fluency, flexibility, and originality, but not in elaboration. No significant improvement was found in the control group's overall creative thinking, fluency, flexibility, originality, and elaboration. The result indicated that the intervention can significantly increase students' creative thinking. The ANCOVA result of the TTCT (figural test) is shown in Table 7. There were statistical differences between the experimental and control groups' overall creative thinking, fluency, flexibility, originality, and elaboration. A high effect size was found in overall creative thinking (η2 = 0.54), fluency (η2 = 0.46), flexibility (η2 = 0.61), and originality (η2 = 0.43), while there was a medium effect in size elaboration (η2 = 0.12). The result indicated that the experimental group students outperformed the control group students in the creative thinking test after the escape room intervention on the experimental group.

**Table 6.** Result of the paired sample *t*-test on creative thinking (figural test).


\* *p* < 0.05 Effect sizes (ES): d = 0.2–0.5 (small effect), d = 0.5–0.8 (moderate effect), and d ≥ 0.8 (large effect) [30].

**Table 7.** Result of ANCOVA on creative thinking (figural test).


\* *<sup>p</sup>* < 0.05 † Nonlinear transformation Effect size (ES): 0.01 ≤ <sup>η</sup><sup>2</sup> ≤ 0.059 (small effect), 0.059 ≤ <sup>η</sup><sup>2</sup> ≤ 0.138 (moderate effect), and <sup>η</sup><sup>2</sup> ≥ 0.138 (large effect) [30].

#### *3.2. Learning Motivation*

The paired sample *t*-test result of the LMS is shown in Table 8. The result showed a significant improvement in the experimental group's overall learning motivation, value, expectation, affect, and executive volition, while no significant improvement was found in the control group's overall learning motivation, value, expectation, and affection. In addition, the control group's executive volition was significantly decreased. The result indicated that escape room intervention can enhance students' learning motivation in all aspects.

The ANCOVA result of the LMS is shown in Table 9. There was a statistical difference in the two groups' overall learning motivation, affect, and executive volition but not in value and expectation. A high effect size was found in overall learning motivation (η = 0.29) and affection (η = 0.20), while a medium effect size was found in value (η = 0.06), expectation (η = 0.07), and volition (η = 0.11). The result indicated that experimental group students had a higher learning motivation, especially the affect and executive volition dimensions, than control group students after the escape room intervention.

**Table 8.** Result of the paired sample *t*-test on learning motivation.


\* *p* < 0.05 Effect sizes (ES): d = 0.2–0.5 (small effect), d = 0.5–0.8 (moderate effect), and d ≥ 0.8 (large effect) [30].


**Table 9.** Result of ANCOVA on learning motivation.

*\* p* < 0.05 Effect size (ES): 0.01 ≤ <sup>η</sup><sup>2</sup> ≤ 0.059 (small effect), 0.059 ≤ <sup>η</sup><sup>2</sup> ≤ 0.138 (moderate effect), and <sup>η</sup><sup>2</sup> ≥ 0.138 (large effect) [30].

#### *3.3. Science Academic Achievement*

The paired sample *t*-test result of the science achievement exam is shown in Table 10. The result revealed a significant improvement in both the experimental and control groups' academic achievement, indicating that students who received and did not receive the escape room intervention had a significant improvement in their science academic achievement.

**Table 10.** Result of the paired sample *t*-test on science academic achievement.


\* *p* < 0.05 Effect sizes (ES): d = 0.2–0.5 (small effect), d = 0.5–0.8 (moderate effect), and d ≥ 0.8 (large effect) [30].

The ANCOVA result of the science achievement exam is shown in Table 11. There was no statistical difference between the experimental and control groups' science academic achievement, and a small effect size was found (η = 0.002). The result indicated that the experimental group students did not outperform the control group students in academic achievement after the escape room intervention in the experimental group. However, both groups' academic achievement could improve significantly.

**Table 11.** Result of ANCOVA on science academic achievement.


*\* p* < 0.05 † Nonlinear transformation. Effect size (ES): 0.01 ≤ <sup>η</sup><sup>2</sup> ≤ 0.059 (small effect), 0.059 ≤ <sup>η</sup><sup>2</sup> ≤ 0.138 (moderate effect), and <sup>η</sup><sup>2</sup> ≥ 0.138 (large effect) [30].

#### **4. Discussion**

The primary aim of the study was to examine the impact of the escape room intervention on students' creative thinking, learning motivation, and academic achievement. There was a significant difference between the experimental and control groups' creative thinking (see Tables 5 and 7). The result also indicated that the experimental group students' three dimensions of creative thinking significantly improved, including fluency, flexibility, and originality (see Tables 4 and 6). Although some articles reported escape rooms' positive effects on students' creative thinking, they seldom scientifically conducted experimental interventions to collect empirical evidence, or they omitted control groups [15]. Therefore, this study bridges the research gap by quantitatively examining the escape room intervention's impact on students' creative thinking. In the study research, students worked in groups to solve several puzzles in an escape room. The problem-solving process requires individuals to think about every possible solution. In addition, teammates' ideas could stimulate their thoughts, helping them consider a matter or a question from various perspectives. As Torrance [31] mentioned, the problem-solving process can be regarded as an exhibition of creative thinking.

Learning motivation is a crucial factor contributing to students' learning. The escape room intervention resulted in better learning motivation for students in the experimental group compared to the control group. Regarding each dimension, the results indicated that there was a significant difference in students' "affect" and "executive volition" but they were nonsignificant in the dimensions of "value" and "expectation". Students who received the escape room intervention had a positive affect when learning. The finding corroborated the prior scholarly work that found escape rooms can contribute to students' positive affections and emotions (e.g., fun, enjoyment, and amusement) when they study [12,23,25]. In the research of Huang et al. [32], they found that digital escape room-infused teaching primarily and positively impacted students' "affect" dimension for learning science but did not impact the "executive volition" dimension. The possible explanation might be that Huang et al. only conducted digital escape rooms (on the Holiyo platform) in the lesson. In addition, according to the students' responses, they suggested adjusting the difficulty level of the game. The research by Huang et al. [32] shed light on the constraints of digital escape rooms. It is challenging for teachers to design complicated digital puzzles and immersive settings as physical ones [14]. As shown in Figure 1, the strength of digital puzzles is in assessing students' knowledge and skills acquisition but not providing an immersive and hands-on experience. Gómez-Urquiza et al. [22] and Macías-Guillén et al. [33] found that students in physical escape rooms are motivated to strive for accomplishments and do high-demand things. The current study conducted digital and physical escape rooms sequentially. In the physical one, students were placed in a messy and mysterious room. They had to find clues and specific items and manipulate various experiments (using litmus paper and a spring balance) to solve every puzzle. The combination of physical and digital escape rooms yielded a more powerful impact on the students' motivation. Students motivated themselves and had a willingness to overcome every bottleneck when learning.

Although no significant difference in academic achievement was yielded between the experimental and control groups (see Table 11), the paired *t*-test results revealed that both the experimental and control group students' science academic achievements significantly improved (see Table 10). Similar results were found in prior studies [21,24,34–37], in which participants who experienced escape rooms had a significant improvement in their learning outcomes, demonstrating that escape rooms can effectively contribute to students' knowledge acquisition and retention of knowledge [9]. However, the prior scholarly work seldom included a control group; thus, it was difficult for them to compare the results with other groups and conclude that an escape room intervention can help students learn better and outperform students who do not experience escape rooms. In contrast to a previous finding [9], no significant difference was yielded in academic achievement between the experimental and control groups. The evidence indicated that whether escape rooms are implemented or not, students can learn well in science lessons. However, it should be noted that the science achievement exams primarily assessed students' knowledge retention, not high-order thinking skills. The results may differ if future studies systematically assess students' high-order thinking, such as synthesis and evaluation. Despite the nonsignificant difference between the two groups, both groups had significantly improved science academic achievement. In addition, according to the result, escape room intervention can contribute to other cognitive and affective skills, such as creative thinking and learning motivation.

#### *Limitations and Implications for Future Study*

The research included a limited number of participants due to the limitations of classrooms and equipment. Future research should include more students if approved by the school administration. Secondly, the research was conducted pretest and posttest but not postpone-test. It is essential to investigate whether escape rooms have a long-term effect on students' learning. Escape rooms are a nascent and emergent game-based teaching method. A novel effect should be noted in this field [10]. Students' learning motivations and affections might not be triggered once they become accustomed to escape rooms. Lastly, future research can include qualitative data to corroborate quantitative data and thoroughly understand why and how escape rooms can impact students' learning, identifying components contributing to their creative thinking, motivation, and academic achievement.

Moreover, the primary aim of an escape room is not only to provide a hands-on experience but also to simulate real-world scenarios. It is essential for educators and researchers to integrate real-world tasks in escape rooms so that students may know how to use their knowledge and understand the link between course materials and daily life. For instance, campus-based escape rooms can be developed to help students apply their knowledge in surrounding areas, manipulate experiments, and come up with creative solutions for real-world problems.

#### **5. Conclusions**

While digital and physical escape rooms are suggested as practical and effective approaches to game-based learning, it is challenging to find studies that implement both types of escape rooms—the deficit of scholarly works legitimizes this study's significance and appropriateness. The study implemented a digital and physical combined escape room to complement each other's strengths, verifying that this approach can improve students' learning quality; fill the research gap; and potentially make theoretical, methodological, and practical contributions to the knowledge and field. Evidence indicated that the digital– physical combined approach can effectively improve students' creative thinking, stimulate their learning motivations (especially their affection and executive volition in learning), and improve their academic achievement. This study also opens a dialogue on the platforming of digital learning and life scenario problem-solving for better educational practices.

**Author Contributions:** Conceptualization, H.-C.K., C.-Y.C. and C.-S.L.; Methodology, H.-C.K., C.-Y.C. and A.-J.P.; Formal Analysis, C.-S.L.; Investigation, C.-S.L.; Resources, C.-S.L.; Data Curation, H.-C.K. and C.-S.L.; Writing—Original Draft Preparation, C.-Y.C.; Writing—Review and Editing, C.-Y.C. and A.-J.P.; Visualization, A.-J.P. and C.-Y.C.; Supervision, H.-C.K.; Project Administration, H.-C.K. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research was funded by the National Science and Technology Council, grant number: 110-2511-H-006-009-MY3 and the Ministry of Science and Technology, Taiwan, grant number: 108-2511-H-006-018-MY2.

**Institutional Review Board Statement:** Ethical approval was obtained from the National Cheng Kung University Governance Framework for Human Research Ethics (Ref. 108-386 2021/01).

**Informed Consent Statement:** Informed consent was obtained from all participants involved in the study.

**Data Availability Statement:** The data presented in this study are available on request from the corresponding author. The data are not publicly available due to participants' privacy and ethical issue.

**Acknowledgments:** The project was funded by Taiwan's Ministry of Science and Technology, grant number 108-2511-H-006-018-MY2. The authors would like to thank all students who participated in the research and the school administration. With their kind help, the project was conducted smoothly and successfully.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**


## *Article* **Investigation of the Effect of Badges in the Online Homework System for Undergraduate General Physics Course**

**James Uanhoro <sup>1</sup> and Shelley Shwu-Ching Young 2,\***


**Abstract:** Badges in education are an increasingly popular phenomenon, and a variety of questions exists as to the abilities and effectiveness of badges. In this study, the effect of digital badges within a Moodle-based online homework system was studied for an undergraduate general physics course at a large research-based university in northeast Taiwan. One hundred and sixty-two participants from two General Physics sessions were involved in this study and divided into two groups through self-selected options. Sixty-eight students in the treatment group could use digital badges in the online homework system, being able to earn one badge per assignment for turning their assignments in earlier than the assignment deadline, while the other students in the control group had no digital badges in the online homework system. The results showed that students in the treatment group turned in their assignments earlier than students in the control group did, and this difference was statistically significant. Further analysis showed that students in the treatment group spaced out their assignment practice more than students in the control group did, and the difference was statistically significant. Additionally, students in the treatment group actively attempted to earn badges, as there was a statistically significant increase in the number of badges earned by students in the treatment group over those in the control group. Based on a questionnaire given to study participants towards the conclusion of the study, the study found that students' perception of badges was positive. These findings corroborate earlier findings by other researchers that badges can be used to motivate specific behaviors in students whilst requiring minimal changes to the course structure. However, further corroborating earlier research is the finding that badges may not be particularly useful to motivate students towards challenging tasks. An earlier study of this course in a preceding academic year found that students are appreciative of the online homework system, and it appears from this study that the primary function of badges within the system is to enhance the experience of students, as well as to motivate timely engagement with assignments.

**Keywords:** digital badges; timeliness; homework system; distributed practice; general physics

#### **1. Introduction**

Gamification is the usage of game elements or features in a non-game context to promote learners' learning engagement, motivation, and performance, and has been proved to induce the learners' change of behavior [1]. The gamification not only influences the structure of the social network but also impact learners' learning success [2]. Among the gamified elements implemented in the educational setting, badges are a game element that is frequently used [3].

Badges in the information era come to form as digital badges, open badges, or educational badges, which have been seen as a visual symbol of achievement, accomplishment, and skills to act as rewards to motivate students' learning within social communities [4], and a method to benefit learners with goal setting [5]. Digital badges, specifically in the world of academia, remain a relatively recent addition to the sphere of online artefacts [6].

**Citation:** Uanhoro, J.; Young, S.S.-C. Investigation of the Effect of Badges in the Online Homework System for Undergraduate General Physics Course. *Educ. Sci.* **2022**, *12*, 217. https://doi.org/10.3390/ educsci12030217

Academic Editor: Huei Tse Hou

Received: 30 January 2022 Accepted: 14 March 2022 Published: 17 March 2022

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

Individuals are more likely to carry out activities that provide social validation that they perceive other users have achieved by earning the badges and would tend to engage in the learning activities with efforts in a more active way [7].

Nowadays, a growing use of digital badges as an innovative instruction and credentialing strategy could be seen in higher education, but digital badging studies have yielded equivocal findings. The involvement of digital badges serves as a stimulus to motivate students' learning performance and engagement, but the means to maintain the effectiveness brought by digital badges are necessary. For example, the more time students spend studying, the more badges they would receive, which ultimately positively promotes their involvement, engagement, and learning experiences [8]. However, the effectiveness of using badge elements declined with the disappearance of novelty effects across the time [4]. Thus, more research is needed to identify effectiveness and optimal implementation tactics [9–12].

Understanding the aforementioned potential and issues of digital badges, the authors conducted this study based on a longitudinal study on implementation of a homework system (HWS) to deal with teaching and learning problems appearing in large-enrollment teaching in undergraduate Physics courses, such as homework tardiness, a perfunctory mentality or even cheating behaviors, heavy workload in homework correction, no immediate feedback to students, etc., at a research-based national university in Northern Taiwan [13]. The HWS was examined for its effectiveness by the target learners and instructors and the results indicated that the system received positive feedback from the users, including improving the homework efficiency and students' learning and saving time on correcting homework, leading to more appropriate teaching interactions. In addition, after the implementation of the HWS, some suggestions for follow-up improvements, such as adding a badge feedback mechanism, were raised to further facilitate active and self-regulated learning. The longitudinal study by Young and Hung [13,14] of an earlier iteration of the same course found that students typically completed their assignments in the final three days before the assignment submission deadline. However, a majority of students self-reported earlier completion dates [13,14], suggesting a disparity between perceived (and/or desired) completion dates and actual completion dates. Additionally, the professor in charge of the homework system expressed a high interest in seeing if the earliness of assignment submissions could be improved. Thus, the purpose of this study was to design badges and explore the effectiveness of integrating the digital badges in terms of helping students' timeliness of homework assignment submission and facilitating learning performance. The following research questions guided this study:


#### **2. Literature Review**

#### *2.1. Teaching of Introductory Physics with Web-Based Homework System*

Physics faculty in higher education institutions (HEIs) often rely on anecdotal experience to guide their teaching practice, and this unwillingness to adopt evidence-based practice is not due to a failure to communicate evidence-based practices on the part of physics education researchers [15,16]. The state of physics education is largely traditional lecture-based, i.e., "teacher-focused with passive students" [17].

Fraser et al. [17] state physics faculty are, more often than not, experts in evidencebased research, yet are often willing to discard evidence-based practices in favor of anecdotes, and further, they argue the adoption of evidence-based practices is an increasing need for Science, Technology, Engineering and Mathematics (STEM). There is great scope for improvement in teaching methods currently used in physics education. Given the increasing need for STEM skills, practice will need to catch up to research for physics education.

When learning is altered to include an online dimension, self-regulated learning could be an issue—a student has greater control of her/his learning. Five learning attributes that improve success in online learning were identified [18], including: motivation; experience with internet technology; time-management skills; study-environment skills; and helpseeking skills. Depending on the learners, different skills may require special attention. As an example, students at a leading research university in Taiwan—the study participants are likely to have experience with internet technology.

An earlier work by Zimmerman [19] argues that self-regulated learning, rather than being seen as comprising skills, should be viewed as processes by which learners transform their mental abilities into academic skills. Following on from this, Zimmerman [19] identified three phases of processes learners go through to transform their mental abilities into academic skills. Forethought phase: There are two major classes of forethought phase processes—task analysis and self-motivation. Task analysis involves goal setting and strategic planning, while self-motivation involves self-efficacy, outcome expectations, intrinsic interest/value, and learning goal orientation. Performance phase: There are two major classes of performance phase processes—self-control and self-observation. Self-control involves the use of imagery, the abilities for self-instruction and attention focusing, and task strategies. Self-observation involves self-recording or self-examination to identify cause and effect. Self-reflection phase: There are two major classes of self-reflection phase processes—self-judgment and self-reaction. Self-judgment involves self-evaluation and causal attribution. Self-reaction involves self-satisfaction or affect and adaptive or defensive responses [19].

Young and Hung [13,14] studied self-regulated learning with a web-based homework system along the following dimensions: goal setting; environment structuring; task strategies; time management; help seeking; and self-evaluation. The results showed that students were cognizant and active across all the dimensions studied, with a majority of students giving markedly positive responses. The only question to which the students responded negatively was when asked if they used the teaching assistant as a means of help seeking; however, students responded positively when asked if they knew how to find the help they needed.

#### *2.2. Digital Badges in Higher Education*

With the advancement of technology, physical badges have gradually transformed into digital badges [4]. Digital badges help to represent skills and achievements of a person and can be used to visually symbolize a skill, an accomplishment, an educational qualification, an interest or a certification [9–12]. Thus, nowadays, digital badges can be found in a growing number of implementations, recognizing learner effort or mastery of learning. In addition to an increasing number of digital badge systems is also the belief that the primary value of digital badges is as a credentialing mechanism. However, badge skeptics have doubted the use of digital badges as credentials, usually with a variation of the question, "Why would a student want to earn a badge?" One common response to the skeptics from digital badge advocates is that future employers or educational organizations will provide employment or credit based on the digital badge [20].

Nevertheless, we have begun to use digital badges in higher education to encourage student persistence by motivating them, recognizing their generic skills, signaling their achievements, and capturing their learning paths. For instance, a badge-based achievement system was introduced into an online learning tool used by college students [21]. A randomized controlled experiment involving over 1000 students found evidence for the positive impact of badges on students' levels of participation. Specifically, badges increased the quantity of students' contributions and the length of time they engaged without decreasing the quality of their contributions. In a study on a university-level online computer science learning environment, badges were implemented to the treatment group and control group. The results show that achievement badges can be used to affect students' behavior positively with the majority of the students reported being motivated by the badges [22]. In

addition, digital badges were integrated into the peer evaluation system and introduced into university classrooms. This system was well received by students, and the digital badges also increased the participation of students in peer evaluation [23]. Moreover, uses of digital badges in the classrooms of Midwestern University and the results showed that the group using the digital badges had significantly better classroom performance than the unused group [24,25].

A number of works have found evidence for the positive impact of badges on students in higher education. However, badge uses in nursing students' performance and participation in bioscience practical class led to declining results, with the disappearance of novelty effects across over time [4]. This indicates digital badging studies have yet yielded equivocal findings. Thus, more research is needed to identify effectiveness and optimal implementation tactics [9–12]. Nevertheless, digital badges have become a popular method of enriching academic experience at many universities. Nowadays, digital badges represent an accomplishment, which appear as icons or logos available online. Introducing digital badges into university courses offer an innovative way to deliver the learning objectives, as well as to motivate learning beyond the classroom [26]. General physics is the foundation of higher engineering education, and students' low motivation to learn is the biggest problem encountered in general physics education and is of great concern. How to use digital badges to associate teaching methods to enhance student participation, motivation and engagement and enhance students' interest in general physics courses has become an important issue of teaching and research [13,26–29]. Given the specific context of associating HWS with digital badges in this longitudinal study, we hope this study could shed light on the uses of digital badges, while the goal of this study focused on developing digital badge system and how such a system might impact on learning in general physics.

#### **3. Materials and Methods**

This section will cover information about the badge design and homework system, requirements to earn a badge, study context, sample size, data collection, management and analysis, etc.

#### *3.1. Badge Design and Homework System*

To meet the context of this study, a full badge system was designed and the look and feel of the assignment chapter badges were designed to reflect the assignment topic. Moreover, a formative feedback of badge design was conducted prior to the start of the semester using this feedback alongside advice from other studies on badges in education [21,28,30–33]. Students in the treatment group were always able to see the available badges at any point in time.

The homework system was hosted on the Moodle Learning Management System (LMS). The Moodle LMS Essential theme was used for the interface design. Each course section was administered in a Moodle LMS course of its own. The badges were made available to the treatment group using the badge feature in Moodle. Each assignment had about 6–9 questions and was assigned using the Quiz module. The questions were of the calculated question type with automated assessment—students knew whether they had passed or failed a question instantly. Each assignment question could be attempted an unlimited number of times without penalty.

There were 16 assignments, but they had shared submission deadlines. Assignments with shared deadlines in chronological order were: Chapters 5–9; Chapters 10–14 and 32; and Chapters 15–19.

Table 1 shows the weekly assignment badges. Additionally, higher-level badges were created to motivate students to earn more badges and to strive for badge accumulation, as shown in Table 2.


**Table 1.** Weekly assignment: topics and badge images.

**Table 2.** Higher-level badges with requirements.



**Table 2.** *Cont.*

*3.2. Requirements to Earn a Badge*

For a student to earn a weekly assignment badge, s/he had two requirements:


This study adopted quasi-experimental methods. The samples self-selected themselves into the two course sections as is customary according to university regulations. Students in the treatment group were provided with an informational document in the system to inform them of these requirements. This full mark requirement was applied because a student's maximum score of three attempts determined the effective score for the student on any given assignment. Without such a requirement, a student could turn in an assignment without taking care to respond to the questions in hopes of getting a badge. Such activity would come without penalty as only the attempt with the maximum grade contributed to the student's effective assignment grade. While this could have placed an extra burden on students who were willing to achieve badges—correctness in addition to timeliness—an earlier study by Hung [14] showed that students routinely achieved the maximum assignment score for each assignment. Additionally, the individuals within the physics department who were responsible for the assignment questions in the homework system tried to make the questions relatively undemanding in an attempt to motivate the willingness of students to attempt questions [14]. This was because the students enrolled in the course were not physics majors.

#### *3.3. Setting of This Study*

The study was conducted at a research-oriented university in northern Taiwan. The course under study was the "General Physics B (I)", which runs yearly in the fall semester. This course is typically offered to first-year undergraduate students drawn from a variety of colleges and departments other than the physics department. The text used for the course was Essential University Physics by Richard Wolfson (Second Edition). Classroom periods were held two times weekly. The course covers twenty chapters from the course text. The course curriculum treats the first four chapters as foundational, and students are expected to be familiar with the material from their pre-university study. The remaining sixteen chapters are primary to the course, and each one of these chapters has an assignment attached to it.

#### *3.4. Sample Size*

In total, one hundred and seventy-seven students had registered into both classes at the close of the course add/drop period. Seventy-nine students were in the treatment group, and ninety-eight students enrolled in the control group. Professors within the physics department developed a pre-test to measure students' knowledge of the topics. This pre-test was administered to students before they began assignments. Of the 79 students enrolled in the treatment group, only 70 students were considered as part of the experiment, as nine students did not take part in the pre-test. Of the 98 students enrolled in the control group, 95 students were considered as part of the study; the other three students did not take the pre-test. Of the 70 students remaining in the treatment group, two students failed to take more than one exam and they were excluded from the analyzed treatment group, resulting in 68 students making up the treatment group. Of the 95 remaining in the control group, one student failed to take more than one exam resulting, in 94 students making up the control group. Thus, the analyzed sample size was 162 students.

The students in both groups came from markedly different colleges and departments within the university—a college contains multiple departments. Across both experiment and control groups, there was very little overlap in the colleges. The majority of students in the control group belonged to college G (n = 83, 88.30%), while no student in the experiment group belonged to this college. Students in the experiment group largely came from two colleges: A (n = 31, 45.59%) and E (n = 33, 48.53%). Additionally, almost every single student in both classes was a student in her or his first year of study. In the control group, there were 87 (92.55%) first year students, while there were 66 (97.06%) first year students in the experiment group.

#### *3.5. Data Collection, Management and Analysis*

This study employed a survey to obtain students' perceptions of badges administered to students in the treatment group. The survey was a modified version of the survey developed by Haaranen et al. [31] for the same purposes. In addition to the original survey, an open-ended question was attached to each close-ended question to generate as much insight and unstructured feedback as possible from respondents. The survey was administered via the homework system.

Assignment data, alongside badge data, were stored on a server located in a graduate student laboratory at the university. Only the researcher and the research supervisor—the university professor—had the credentials required for access to the assignment data saved on the server. The assignment and badge data were retrieved from the study server using SQL scripts. Preparation of the data saved in the homework system for analysis was performed using the Ruby programming language. The output of the data preparation process was in narrow data presentation. All subsequent quantitative analyses were performed using the R programming language. Descriptive statistics were used to analyze timeliness across both groups over the assignments. The timeliness of assignment submissions was treated as panel data; students were the units of analysis, and the different assignment timeliness were the data along the time dimension. The timeliness data were fitted to the panel data regression model, with the study group as the independent variable and timeliness as the dependent variable.

#### **4. Results and Discussion**

The results are presented based on the data collected means of the survey via the homework system and data are interpreted below in reference to the reviewed literature to address each of the research questions raised earlier in the hope to shed light on the uses of badges along with the homework system to facilitate general physic learning.

#### *4.1. Timeliness of Assignment Submissions*

Timeliness of an assignment submission was determined using the positive difference between the time of the assignment submission and the time of the assignment deadline, measured in floating point days. The timeliness used for each assignment was the time

at which a student achieved her or his maximum score of all the student's attempts. If a student achieved this maximum score on more than one occasion, the earliest submission (maximum timeliness) was selected as the timeliness for the assignment.

Figure 1 shows the distribution of submission times of each assignment by study group; the dotted lines show the deadline to earn a badge for each assignment. As shown, the median is always higher in the treatment group than in the control group. A panel data regression model was used to analyze the effect of course selection on timeliness, with the study group as the independent variable and timeliness as the dependent variable. Additionally, the scaled pre-test scores of the students were added to the model as a predictor of timeliness.

**Figure 1.** Submission time for each assignment by class showing badge cut-off deadlines. Units: days.

The resulting panel data regression model was statistically significant (R-Squared = 0.078, Adj. R-Squared = 0.078, F-statistic = 100.49 on 2 and 2372 DF, *<sup>p</sup>*-value: <2.22 × 1016). As shown in Table 3, course selection had a statistically significant effect on timeliness; being in the treatment group decreased the average submission time by about 1.8 days.

**Table 3.** Coefficients of independent variables in linear panel data model (treatment = 1, control = 0).


Significance codes: \*\*\* *p* < 0.001.

Based on the modelling results, the question arises as to the meaningfulness of a 1.8-day average difference between both groups. Timeliness is a useful translational graduate attribute [34]—and an increase in timeliness as an end is worth it—and the modelling results show the ability of a badge system to positively influence this skill. Whether this improvement stays after badges are removed from a student's digital environment is another question, one that is outside the confines of this study. Additionally, for a treatment

that imposes minimal changes to the non-digital dimension of the course, badges appear to be a relatively useful tool.

In relation to the study's first question: "What is the effect of digital badges on the timeliness of assignment submissions within an undergraduate physics course?", the fact that students in both groups came from markedly different colleges (see Figure 1) weakened the internal validity of the results, and thus, our ability to assign cause and effect. Nevertheless, a statistically significant increase in timeliness was observed in the treatment group.

Reversing the treatment and control groups and repeating the experiment could resolve this issue. This would be dependent on the expectation that students in a set of colleges are typically attracted to the exact professors involved in both classes. If the results stay the same in terms of treatment and control group, then it is resolved that the outcomes found at the end of this study are not related to the colleges the students come from but are an effect of badges.

#### *4.2. Attempt to Earn Digital Badges*

In order to find out whether students in the treatment group actively attempted to earn badges, the criteria used to award badges to students in the treatment group were retrospectively applied to the control group to see how both groups fared in terms of acquiring badges.

Across all the periods in the semester, students in the treatment group never obtained fewer badges than students in the control group. To test whether this difference was statistically significant, a two-tailed Wilcoxon Rank Sum test with continuity correction was employed; the badge data were heavily skewed towards zero, as some majority students never earned a single badge.

As shown in Table 4, the mean of the badges earned by students in the treatment group across the semester is significantly higher than the mean of the badges gained by students in the control group. Thus, it appears that students actively attempt to earn badges when given the opportunity. However, there is no significant difference between both groups in terms of earning the higher-level badges except for the Bronze Cup. No student earned the Einstein badge.


**Table 4.** Descriptive statistics and results of Wilcoxon rank sum tests on badge data.

Significance codes: \*\*\* *p* < 0.001, \*\* *p* < 0.01, \* *p* < 0.05.

#### *4.3. Distributed Practice*

Inter-session interval (ISI) was used as a measure of distributed practice [35]. To determine ISI in this study, the course assignments were grouped according to shared assignment deadlines: Chapters 5–9; Chapters 10–14 and 32; Chapters 15–19. The timeliness of assignment submissions within each assignment group for each student were sorted by size, and the positive difference between consecutive timeliness data (after sorting) was used to determine the ISI.

Table 5 presents descriptive statistics for the ISI for each period of the semester. The median and mean ISI are consistently higher in the treatment group as seen. However, the medians are all below one day, i.e., the median spacing between the completion of assignments in all periods of the semester are less than one day.


**Table 5.** Descriptive statistics for the ISI for each period of the semester.

To determine whether the difference in ISI between both groups was statistically significant, a two-tailed Wilcoxon Rank Sum test (Ntreamtment = 803, Ncontrol = 1111) with continuity correction was employed due to the ISIs being highly skewed towards zero as seen in Figure 2. The results indicate a significant increase in ISI in the treatment group (M = 1.44, SD = 2.83) over the control group (M = 0.87, SD = 2.23), W = 399260, *p* < 0.001 (Table 5).

**Figure 2.** Boxplots (without outliers) of ISI for all periods in semester by study group. Units: days.

The analysis specifically answers one of the secondary questions of the study: "What is the relationship between digital badges and the distributed practice of assignments within an undergraduate physics course?" and the presence of badges is associated with an increase in distributed practice. Distributed practice is another translational graduate attribute—like timeliness—implying that an increase in itself is well worth it. Nevertheless, are the gains in distributed practice within the treatment group relative to the control group

large enough to increase a student's long-term recall in a complex subject such as university physics? While this is an empirical question which real-life time constraints did not permit this study to answer, related work by Grote [36] and Rohrer and Taylor [35] show that the ISI values have proven to be an effective determinant of long-term recall for complex learning tasks is in the magnitude of several days to weeks, not fractions of days [35,36]. For the treatment group, the mean ISI across the semester was 1.44 days and the median was 0.2 days. Following on from this, it appears a majority of students would have to have been obtaining badges or close to doing so on a weekly basis for inter-session intervals in the order of weeks to have been observed. However, the question remains open given the absence of evidence required to show what kind of link exists between badges and long-term recall.

#### *4.4. Performance on Assignment and Exams*

The performance of both groups on the assignments and exams is reported; however, there is nothing in the literature that suggests improved or reduced performance. The average assignment score was 94% in the control group and 96% in the treatment group. To test whether the difference in means of exam scores of both groups was statistically significant, a linear regression model with study group as the independent variable was used to predict each exam score. Only for the second exam is there a statistically significant increase in the average grade of a student in the treatment group (see Table 6).

**Table 6.** Regression results using group (Control = 0; Treatment = 1) to determine exam scores.


Significance codes: \* *p* < 0.05.

#### **5. Conclusions and Future Study**

In this quasi-experiment, badges were added to a general physics homework system in an attempt to improve the timeliness of assignment submissions and distributed practice of assignments by students. Our findings show that badges can be used to motivate specific behaviors in students whilst requiring minimal changes to the course structure. These results corroborate results reported by Hakulinen et al. [32] and Denny [21]. However, as found by Denny [21], badges may not be particularly useful in motivating students towards difficult challenges—no student earned the demanding Einstein badge in our study and there was no statistically significant difference between both groups in terms of earning higher-level badges, except for the Bronze Cup (see Table 4). It is possible that badges are effective motivators for low hanging fruit—beneficial tasks that require little effort; further studies are needed to confirm this.

The overall significance of this study is that this is the first study to attempt to estimate the effect of badges on an outcome of interest using panel data modelling. This analysis allowed us to identify the 1.8-day difference in timeliness between students in both groups. While questions may exist as to the meaningfulness of a 1.8-day difference, this number is not to be considered in isolation. The end-of-semester survey on badges revealed that students felt badges were a useful element within their homework system—an element they would like to see return. Moreover, for some students, earning badges was an "honor". The reported benefits in terms of learning outcomes are mixed when an online homework system is introduced into the teaching of general physics.

Despite this, badges were not able to influence exam performance of students in this study [37]. The reported benefits in terms of learning outcomes are mixed when an online homework system is introduced into the teaching of general physics [38,39]. However, major gains are to be found in improved attitudinal stances towards the course under

study [38,39], whilst studying an earlier iteration of the same course studied in this quasiexperiment found that students greatly appreciated the homework system. It appears that badges primarily serve to enhance this experience and motivate their efforts.

Additionally, it might be worth exploring the effects of badges using the switched replication design. This would allow researchers to see whether students retain the behavioral changes they made in the presence of badges once badges are removed.

**Author Contributions:** Conceptualization, S.S.-C.Y.; methodology, S.S.-C.Y. and J.U.; validation, S.S.-C.Y.; formal analysis, J.U.; investigation, J.U.; resources, S.S.-C.Y.; data curation, J.U.; writing original draft preparation, J.U.; writing—review and editing, S.S.-C.Y. and J.U.; visualization, J.U.; supervision, S.S.-C.Y.; project administration, S.S.-C.Y.; funding acquisition, S.S.-C.Y. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research was partially funded by the Ministry of Science and Technology of Taiwan under contract numbers MOST 108-2511-H-007-008-MY3.

**Institutional Review Board Statement:** Ethical review and approval were waived for this study, due to consent provided by the participants while completing the questionnaire.

**Informed Consent Statement:** Informed consent was obtained from all subjects involved in the study.

**Data Availability Statement:** The data that support the findings of this study are available from the corresponding author, upon reasonable request.

**Acknowledgments:** The authors would like to thank all of the professors and students who took part in this study. Without their participation and contribution, this project would not be possible.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**


## *Article* **Enablers and Difficulties in the Implementation of Gamification: A Case Study with Teachers**

**Inês Araújo 1,\* and Ana Amélia Carvalho <sup>2</sup>**


**\*** Correspondence: inesaraujo@fpce.uc.pt

**Abstract:** Although there is ample evidence that gamification can engage students in learning, it is less used than one would expect. This raises the question of the difficulties teachers face in planning and implementing gamification in their classes. What enables teachers' implementation of gamification? These questions were addressed through a case study, and data were collected along the four phases of a teachers' training course. The first phase approached the gamification concept, the Octalysis Framework, the types of gamification, and digital tools. The second phase focused on planning the gamification activity, the third one on implementing it with their students in school, and the fourth phase on sharing and reflecting on their experience. It was possible to identify enablers and difficulties that influenced the planning, such as personal background, time available, and technical conditions in the classroom. Most teachers opted for a platform with digital tools that allowed them to apply all the desired features to their gamified activities. The most complex gamified activities were related to the teacher's gaming experience, use of digital tools, and risk-taking.

**Keywords:** gamification; Octalysis Framework; education; teacher training

#### **1. Introduction**

Gamification has proved to be a valuable strategy for teaching, with a positive impact on learning [1,2], but it has also revealed itself as a complex research theme. Previous research identified 586 possible relationships among a wide set of variables [3]. Over the years, research has addressed aspects such as the reasons and the elements that make games so motivating and how they can be implemented in different contexts [4,5], including education [6]. Games became an interesting new area of study for education, discovering new ways to motivate students, since "the elements of challenge, control, and update in games have the potential to sustain students' motivation when playing games" [7] (p. 10).

Games provide different feelings, such as pleasure and immersion, often becoming addictive, as they are based on theories of human behavior and motivation [4,6,8–10]. Based on the emotions and engagement created by games, a new concept arises, gamification [5,11,12], defined by Kapp [6] (p. 12), for the education context, as a methodology of "using game-based mechanics, aesthetics, and game thinking to engage people, motivate action, promote learning, and solve problems".

Nowadays, gamification continues to be a wide field of research in education, particularly in computer science teaching [1,2,13]. Hamari [14] pointed out that most of the studies in gamification give evidence for improving engagement and motivation. In these studies, some concerns associated with competition among students, difficulties with the design, and problems in updating the activity log were identified.

It should be noted that the studies mentioned above mainly focus on the impact of game mechanisms on achievement and progression [1,2]. Many of them involve experiences with software developed for specific contexts, which makes the generalization of research results difficult [1].

**Citation:** Araújo, I.; Carvalho, A.A. Enablers and Difficulties in the Implementation of Gamification: A Case Study with Teachers. *Educ. Sci.* **2022**, *12*, 191. https://doi.org/ 10.3390/educsci12030191

Academic Editor: Huei Tse Hou

Received: 31 January 2022 Accepted: 7 March 2022 Published: 9 March 2022

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

There are several issues related to gamification that need further research and are also difficult to understand because "teachers can heavily influence the process of gamification" [15] (p. 25). Studying the enablers and difficulties that teachers may have during the planning and implementation of gamified activities is the focus of our study.

#### *1.1. Types of Gamification*

Kapp [6,16] distinguishes two types of gamification that can be applied: structural and content. Structural gamification corresponds to the application of game mechanisms to existing content without changing it [16]. Content gamification corresponds to the reformulation of information, dynamics, and content itself through game design elements [16]. It is intended to make the content more game-like, giving context to it or developing activities such as games. This is a more elaborate type of gamification that requires better preparation and investment.

#### *1.2. Octalysis Framework of Gamification*

Tondello et al. [17] have compared the most cited frameworks of gamification. They identified the twelve most used motivational dimensions. The Octalysis Framework includes ten of them across the eight Core Drives, making it the most comprehensive. It also emphasizes emotions, making it simpler to assimilate by those who have little experience with games.

The Octalysis Framework was developed by Chou [8] based on his experience as a player. For him, human motivation can be triggered by at least one of the eight Core Drives (CD) described below:


There are rules and game mechanics that promote the emotions that characterize each Core Drive, making it possible to plan activities that can engage students through the type of motivation that easily grabs their attention.

It is important to note that the Octalysis Framework [8] organizes the Core Drives in two ways:


Our research aims to identify enablers and difficulties that teachers may have during the planning and implementation of gamified activities, highlighting the decisions they need to make throughout this process and the problems they need to overcome.

#### **2. Materials and Methods**

#### *2.1. Training Course*

A training course for teachers, named "Strategies and digital tools to motivate students to learn through gamification" [18], was outlined with four phases, and it ran for five months.

A workshop modality was chosen to provide teachers with moments of learning, planning, and application in their classes. It required availability beyond the training course hours, which was not always easy to conciliate. The training course had a total of 40 h, from which 20 h were of autonomous work. The training combined face-to-face classes and online sessions.

Over the four phases of the training course, teachers were invited to be reflective about their difficulties, options, and students' reactions to gamification. During the first phase, trainees acquired gamification concepts. In the Octalysis Framework, the types of gamification, namely structure and content gamification, were approached, and they explored platforms and digital tools. Gamification was applied during training to engage participants, allowing them to experience it and see how it works [18]. During the second phase, trainees planned a gamification activity. They were asked, as Kapp [6] suggested, to identify a problem to be solved. Then, they had to define the tools to be used during gamification and outline the activity. In phase three, trainees applied the gamified activity with their students in school and reported the ongoing process. In phase four, the trainees were invited to share students' reactions and their experiences, with possible improvements discussed and the main needs highlighted. To conclude the course, they had to submit a final report.

#### *2.2. Case Study*

According to Alsawaier [19], it is essential to understand gamification holistically through studies carried out that use mixed methods. The author concluded that most of the studies involving gamification use quantitative methodologies, focusing on statistical analysis, and on the quantitative data resulting from the application, namely game metrics and the reward system. He also indicated that the use of qualitative methodologies is something rare, as well as the use of mixed methods, but necessary to understand gamification.

The main advantage of a case study methodology is the possibility to describe a phenomenon that is still not well known. Other advantages are related to the fact that it is appropriate for small-scale research, limited in time, and the fact that it is an open method, which can be very useful for future interventions and aid decision-making, considering the phenomenon studied [20]. This type of methodology is suitable when analyzing new situations or finding out the how and why of certain events [21].

The aim of this case study is to understand the enablers and difficulties of teachers when they create gamified activities and, afterwards, implement them with their students.

During the training course, data were collected through a questionnaire, participant observation, video recording, and reports produced by trainees (Table 1).

During the first session, a questionnaire of characterization and an informed consent form were filled in by the participants. In the following sessions, data were collected through video recording, namely the planning of the gamified activities sessions, and the final session, where the trainees reported on their experience and their students' reactions. All the documents produced (planning, reports of implemented activities, and final report) were also collected for content analysis.


**Table 1.** Data collection during the training phases.

#### *2.3. Participants*

Five participants (P) finished the training course. All of them were teachers of the Portuguese 3rd Cycle of Basic Education (students aged 12 to 15 years old), two were female, and three were male (Table 2). Their ages ranged from 37 to 55 years, with different levels of experience. The oldest participant had 32 years of service (P1), and the youngest had 9 years of service (P5). Regarding gaming experience, three participants reported that they played games of strategy (P3), discovery (P3), puzzle (P2), and simulation (P4).

**Table 2.** Participant information.


The participants taught different subjects: Biology and Geology, History, Physical Education, and Special Education. The group was regarded as heterogeneous, taking in consideration all the above-mentioned characteristics.

#### **3. Results**

#### *3.1. Plan and Implementation of Gamification*

All participants applied their planned activities with their students (Table 3). P1, a geology and biology teacher, decided to address indiscipline amongst students. He used the Educaplay platform, creating a group for his class. Students had to complete different challenges about the content learned in the classroom.

P2, a special education teacher, used the same platform, Educaplay, to promote reading. Her students had to solve games on Educaplay about the stories they read.

P3, a history teacher, created a more complex gamification activity, using different digital tools and analogic games. The students were invited to conduct role-playing games and find the motives that guided the Marquis of Pombal to make the decision to govern Portugal during the 18th century.

P4, a physical education teacher, challenged his students to answer the questions presented in the Edmodo platform about curiosities in sports.

P5, also a history teacher, needed to raise awareness of World War II (WWII) events. She mentioned that many students think that WWII is like a movie; that it is not real. She created a quest sequence using Bluerabbit, Educaplay, and Youtube. Students had the role of a journalist that reported on events in WWII.



Table 3 shows that all participants had different gamification goals. Most of the participants opted for Content Gamification [6], as they changed the usual way of teaching, using different tools or platforms to create game-like activities. Only one participant (P4) applied Structural Gamification [6], the simpler type of gamification. He sent a sports question to his students every week, on an established day and hour.

#### *3.2. Platforms and Tools Used*

Participants chose platforms and digital tools from a list previously presented to them, such as Educaplay, Edmodo, Bluerabbit, Class Craft, Habitica, Kahoot, Quizizz, Plickers, Edpuzzle, Playposit, Nearpod, ActivelyLearn, Classflow, Pear Deck, Thinglink, among others. They had to choose platforms that could respond to their gamification goals.

#### 3.2.1. Educaplay

Educaplay is a platform that allows the preparation of activities based on different games such as crosswords, memory, matching, fill the blanks, unscramble letters or words (Figure 1). The teacher assigns students to a group, and they solve the activities.

**Figure 1.** Games on Educaplay.

For each activity completed, students received points that were shown in a leaderboard of the activity or on the group (CD2). Feedback was frequent through sounds, graphic effects, and points (CD3). Students could improve their points on the leaderboard (CD4). Usually, they wanted to be better than their classmates (CD5). Teachers could set rules in their own activities, such as limiting the time, number of attempts, and providing some tips (CD6). As in all games, students were afraid of losing, which motivated them to reach the highest number of points (CD8). The existence of a countdown in all activities and sound effects that alert to errors contributed to focusing the student (CD8).

This platform was used by P1, P2, and P5. They used this platform in different ways according to their gamified activity. P1 used all features of Educaplay. He created a group for his class and invited students to sign up. Then, he provided collections of activities as they progressed. Students had to pay attention during their classes to be able to successfully complete the activities in the weekly assignment. However, only a few students completed the tasks weekly. Due to this situation, P1 decided to solve the activities in class. When students saw their marks refreshed on the leaderboard, their enthusiasm was outstanding. They were engaged and completed all the tasks.

P2 required different conditions because her students had special education needs. One of the limitations of the Educaplay free account was the publicity on the platform that could distract her students. To supplant this, the activities created were embedded in a Blog and, afterwards, in the class, students were invited one by one to complete each activity. Students did not have to register on Educaplay as they completed the activities as a guest. To create competition (CD5) between the students, P2 registered the points achieved in an excel sheet and informed them at the end of the class.

P5 used Educaplay to create challenges that she embedded on Bluerabbit.

#### 3.2.2. Edmodo

Edmodo is a social platform developed for education, where it is possible to create groups of students and deliver tasks (CD3) that can be evaluated by teachers. This allows social interaction by posting information, commenting on peers' posts, or adding reactions (CD5). It is possible to classify the tasks completed and award badges for the achievements (CD4).

P4 used Edmodo as a forum space, where students could comment on the posts delivered by the teacher (CD5). Each post was made available on a specific day of the week. Each post required a correct answer about curiosity in sports (CD6). The first student to answer correctly would win that challenge (CD5).

P3 used Edmodo to disclose all information about the activities presented to his students.

#### 3.2.3. Bluerabbit

Bluerabbit is a gamified platform created to be used in education (Figure 2). It is possible to create a class, where students can become characters in a mission (CD1). The system of points and progress is already defined (CD2). It is possible to create a storytelling where students complete different quests, missions, or side-quests to achieve the aims of the narrative (CD3). It is possible to give students badges and rewards that they can exchange afterwards for some help (CD4). It is possible to group students in project quests where they can collaborate (CD5). The quest, missions, and side-quests can be blocked to only be accessed after some conditions are satisfied, such as achieving a specific level, having coins to pay the entry, or completing a task to increase points (CD6). By blocking access to new quests, Bluerabbit enhances the curiosity to find what the next step is or what happens in the story (CD7). Finally, no student likes to lose, so they try to complete every task (CD8). In Bluerabbit, it is also possible to give negative tickets that make students lose points if something wrong is detected (CD8), such as cheating or failure to comply with any other rule defined in the classroom.

**Figure 2.** Bluerabbit activity created by P5.

P5 used most of the features of Bluerabbit. Each student assumed the role of a journalist that reported facts of WWII. The mission was to inform all people about what was happening in Europe. P5 used videos available on YouTube from those times, newspaper news, and biographies. Some quests had activities built in Educaplay.

#### 3.2.4. Tools Used

P3 used several tools besides the Edmodo platform. Five tasks were assigned to students. The narrative was available in Edmodo, with the aim of the mission, the role of the students (CD1), and the tasks (CD6). For the first task, a treasure hunt was created using the app Huntzz (CD1, CD2, CD3, and CD7). Each activity was performed, and the coins received were registered in Flippity Progress Indicator (CD2). Those coins were needed to buy the buildings for the architecture reconstruction project of Lisbon (CD4). For achieving this, they used the drawing tool available on the interactive board.

#### *3.3. Core Drives Applied*

Through the Octalysis tool, it is possible to represent the Core Drives achieved in the activity and the intensity of it in a graph. The intensity is represented by the extension of the Core Drive vertex.

P1 and P2 both used Educaplay, but the graphic representation is different (Figure 3a,b).

**Figure 3.** Octalysis Framework representation of (**a**) Gamification activity applied by P1; (**b**) Gamification activity applied by P2.

As explained above, P1 used all the features of Educaplay, and P2 embedded the game created on Educaplay in a Blog. This choice eliminated most of the publicity of Educaplay and did not offer any functionality that allowed students to achieve CD 4 (*Ownership and Possession*) and CD 5 (*Social Influence and Relatedness*). It was intended to reduce possible students' distractions, as they were of special education. Even with different functionalities available on Educaplay, P1 and P2 achieved their aims.

On the other hand, P3 and P5 were history teachers, and they decided to use different tools and different approaches. However, they both light up all Core Drives in their respective graphics (Figure 4a,b).

**Figure 4.** Octalysis Framework representation of (**a**) Gamification activity applied by P3; (**b**) Gamification activity applied by P5.

It is the Figure of P3 that has more intensity in all Core Drives due to the more complex activities and the mystery introduced through the storytelling. On the other hand, P5 used Bluerabbit and Educaplay and still applied all Core Drives. The use of storytelling gave context, and the activities sequence allowed the achievement of CD1 (*Epic Meaning and Calling*) and enabled the curiosity to achieve CD7 (*Unpredictability and Curiosity*). For P5, the storytelling or narrative was a suggestion of Bluerabbit, where it is possible to define the role given to students. P5 had some difficulty completing the narrative because, in the beginning, the quests were very similar to school assignments. This had to be worked on during training. Being a participant with no game experience, preparing more game-like activities was a challenge for her.

P4 only applied two of the Core Drives (Figure 5), as he only delivered a weekly challenge without a reward system. It was not a complex activity but achieved the aim he defined. His students who completed the challenges showed more interest in a diversity of sports. One difficulty was identified: not all students participated in the activity because they had to do it in their free time.

Looking at all Octalysis' representations of the activities, most of the participants applied Left Brain Core Drive (CD6—*Scarcity and Impatience* is present in all activities, with medium intensity) and the positive emotions (CD2—*Development and Accomplishment* is present in four of the representations, with great intensity).

#### *3.4. Implementing Gamification in the Classroom*

Several technical problems were reported during the implementation of the gamified activities in school. Particularly, the lack of equipment or the poor internet quality. This was a source of frustration for students, not being able to log in to the tool or losing access during the activity.

Participants were surprised with students' difficulties logging in and their lack of knowledge on how to use the platforms. As they reported, they were not expecting to have to support students with technical explanations and to have to spend a lot of time with it. As participants commented, they were expecting that students 12 to 15 years old would easily understand how to use the platforms.

Some of the activities were planned to be done at home [P1, P4, and P5] after classes, due to the lack of computers in the classroom. However, disappointingly, only two or three students completed the homework. For this reason, both P1 and P5 had to implement the activities in the computer Lab. The competition in the classroom gave students the motivation to complete the tasks. P5 wrote in her report: "I reported the score of each student (some were quite satisfied, others not so much!). I noticed an effort by students

who got "zero". They quickly tried to carry out the activity and move to a more prominent place. An environment of some competition was created, which was quite interesting."

#### **4. Discussion**

The group of participants planned diversified activities oriented to their teaching context, with different levels of complexity, according to their needs and aims.

The results were very positive. P1 mentioned, "I was positively surprised by the receptivity of my students, particularly the most disturbing ones, who [during the activity] used a much more moderate language and attitudes [than usual]". Even P3, who usually used these kinds of activities, was surprised with students' engagement: "It was an activity [treasure hunt] where the students were very committed and not even the rain stopped them from solving all the challenges." Similar findings are mentioned by Hamari et al. [14] making evident the improvement of motivation and engagement.

The technical conditions available, such as equipment and internet access, are one of the most conditioning aspects in planning and in implementing gamified activities. As stated by Alenezi [22], time, access to resources, and technical support are some of the obstacles identified when technology is to be implemented in class.

To minimize the effort required in the preparation, they chose tools that had more features that satisfied their gamified activity. Most of the participants used one platform and its games or tools with their students because they only spent time mastering one. Similar findings are mentioned in different studies [23,24], where teachers sought to maximize the time available during planning and implementation in the classroom.

Looking at all activities, it is possible to identify competition in all of them. What did stand out from all experiences is that when students realized in real-time their ranking on the leaderboard, they tried to improve their marks, as reported by P1 and P5. It showed that competition in real-time has more effect on students. Studies analyzed by Kalogiannakis et al. [15] (p. 19) show that "competitiveness in a gamified setting positively affects students' behavior".

The Core Drives applied by participants depended, firstly, on the features available on the chosen platform, and, secondly, on the features teachers chose to offer to students. For instance, P1 and P2 chose the same platform, but they did not apply the same Core Drives. P1 applied all Core Drives possible to achieve with Educaplay, and he implemented a mechanism not available on the platform. For instance, he applied a rule stating that only the students with more points could go to a Golden Group Collection where activities were more complex. Inversely, P2 used only a few features of Educaplay, due to the difficulties of her special education students. The versatility of the platform is an important request to create different activities.

The Core Drives applied with more intensity are from the left side of the Octalysis Framework, which is related to extrinsic motivation, such as rewards and progress. They are the ones that are easier to implement, available on most of the platforms, and easier to understand how they work. As stated by Majuri et al. [2] (p. 11), "results indicate that gamification in education and learning most commonly utilizes affordances signaling achievement and progression, while social and immersion-oriented affordances are much less common".

Gaming experience also had an effect on gamification planning. P3 was the one with more experience in playing games and in using digital tools in classes. Based on his experience, he created a complex gamified activity. During the training sessions, he helped other participants to understand some concepts of game design that were approached. For him, it was easy to understand how to achieve all Core Drives. He had more time for planning because he was teaching part-time. Concluding, P3 combined important conditions and characteristics, such as time, game experience, previous use of technology, and some creativity.

Throughout this study, we identified enablers and difficulties in planning and implementing gamification, which are synthesized in Table 4.


**Table 4.** Enablers and difficulties identified.

There are enablers and difficulties related to technical issues, teachers, and students. Students were not directly included in our study. However, their reaction to the activities implemented by our participants affected decisions made throughout the process. Competition in the classroom was the mechanic that promoted a change in students' behavior. Most of the difficulties are well known through similar findings in other studies [22–24], such as technical issues and time. The enablers related to previous experience and creativity stand out due to the example of the activities created by P3.

All participants were engaged in applying gamification to motivate their students to learn in a different way. They embraced innovation, challenging their creativity, using new tools, and a new approach to engage their students. They also took some risks in applying gamification to their classes. They faced unexpected difficulties with their students in using technologies. P1 and P5 also had some problems with students who were not doing the activities. These participants found a new solution to engage students in the activities, as mentioned.

In the future, it will be important to study further the impact of variables such as time available, game experience, creativity, and previous use of technology during the planning of gamified activities. Most gamification studies focused on specific software or mechanics applied to Learning Management Systems [1,13]. However, it is possible for teachers to create their own gamified activities when provided with the necessary knowledge. Teachers have an important role in gamification [15] that it is essential to study further. Extrinsic motivation is usually used in gamification [1–3], but future studies need to focus on intrinsic motivation, such as narrative, creativity, curiosity, and social- and immersion-oriented approaches.

#### **5. Conclusions**

According to this study, the enablers associated with the planning and implementation of gamification are related to the teacher's previous experience with games and digital tools, as well as their capacity for creativity and risk-taking. The difficulties are related to the time needed for creating the gamified activities, the technical conditions of the classroom, particularly internet access, and sometimes students' difficulties in using technology. Technical issues and time available are also identified in several studies [22–24].

Some digital tools help to implement gamified activities that capture the interest of the students through extrinsic motivation. To implement more complex and enduring gamified activities in class, intrinsic motivation has to be included [2,8,25], as the Right Brain Core Drives mention in the Octalysis Framework. These are the more demanding Core Drives to implement because they require creativity, more social interaction, and mystery. These requirements are difficult to find in some digital tools available.

For teachers to apply gamification, knowledge is required, as well as game experience, creativity, and resilience. Several obstacles can occur, but it is possible to adjust the plan and see behavior changes in students, as occurred with P1. Gamification is a continuous process of motivational discovery. It is important to adapt the plan to new adjustments, creating a flow to maintain the engagement [8,16].

**Author Contributions:** Conceptualization, I.A. and A.A.C., Methodology, I.A. and A.A.C., Validation, I.A. and A.A.C., Formal analysis, I.A. and A.A.C., Investigation, I.A. and A.A.C., Resources, I.A. and A.A.C., Data curation, I.A. and A.A.C., Writing, I.A. and A.A.C., Review and editing, I.A. and A.A.C. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Institutional Review Board Statement:** Ethical review and approval were waived for this study due to submission to the IRB is not required for this type of study. The research project that led to this article followed the guidelines of the Ethical Charter of the Portuguese Society of Education Sciences. Participants in the study signed informed consent forms, granting proper authorization for the use of the collected data. Participation was voluntary, and no incentives were offered besides access to the results of the project. This research did not involve physical interventions in the study of participants.

**Informed Consent Statement:** Informed consent was obtained from all subjects involved in the study.

**Data Availability Statement:** The data presented in this study are available on request from the corresponding author.

**Acknowledgments:** Work developed within the scope of the activities of LabTE (University of Coimbra, Portugal).

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**


## *Article* **Developing and Applying a Chinese Character Learning Game App to Enhance Primary School Students' Abilities in Identifying and Using Characters**

**Yao-San Lin \*, Jie Ni Lim and Yung-Sen Wu**

Singapore Centre for Chinese Language, Nanyang Technological University, No.287, Ghim Moh Road, Singapore 279623, Singapore; jieni.lim@sccl.sg (J.N.L.); yung-sen.wu@sccl.sg (Y.-S.W.)

**\*** Correspondence: yao-san.lin@sccl.sg

**Abstract:** The Chinese language is the mother tongue that most students in Singapore need to master. However, for many local students, due to the use of English as the main language in Singapore's families and the living environment, the time and frequency of using Chinese and the exposure to Chinese characters are relatively insufficient, which leads to a high forgetting rate, confusion of the characters and the improper use of Chinese characters. This study attempts to develop an app of a Chinese character learning game for Singapore primary school students, aiming to stimulate students' interest in learning Chinese, increase their frequency of contact and use of Chinese characters, and ultimately strengthen their ability to remember, understand and use Chinese characters. By collecting the data from students' questionnaire surveys, teachers' questionnaire surveys, students' literacy tests, and classroom observations, the research team found that the designed app can enhance the interest of lower grade primary school students in learning Chinese and strengthen their ability to memorize and use Chinese characters.

**Keywords:** app development; Chinese learning; Chinese character literacy; game-based learning; learning game

#### **1. Introduction**

Chinese character literacy is the foundation of Chinese language learning and a key focus of primary education. Studies show that the difficulty of Chinese character literacy teaching in the lower grades of primary schools is mainly concentrated in the shape of the Chinese character, accounting for 54.4% of errors; in the category of common Chinese character errors, the lower primary students had up to 50% component errors. Therefore, we need to strengthen the teaching of Chinese characters in the lower grades of primary school, especially the teaching of Chinese character components. Compared with pinyin characters, Chinese characters are characterized by rich and complex glyphs. There are thousands of commonly used Chinese characters, which also constitutes the main difficulty in learning Chinese characters. However, through the analysis of the structural characteristics of Chinese characters and the rules of the use of Chinese characters, we can find that most Chinese characters are made up of two or more parts. Based on this characteristic, the "component teaching method" can be said to be a feasible method. "Component teaching" was first proposed by [1], who extracted 118 basic parts from 1000 common characters and divided them into character parts and non-character parts, according to the word structure of the parts. By mastering the pronunciation, shape, and meaning of these components, learners can further learn the combined characters. From the perspective of students' memories of Chinese characters, component structure teaching is an effective method to reduce memory load. In terms of teaching rules, the teaching of component structure follows the principle of simple to complex, step by step and cultivating solid basic skills. For the long-term purpose of Chinese character teaching, component structure teaching focuses

**Citation:** Lin, Y.-S.; Lim, J.N.; Wu, Y.-S. Developing and Applying a Chinese Character Learning Game App to Enhance Primary School Students' Abilities in Identifying and Using Characters. *Educ. Sci.* **2022**, *12*, 189. https://doi.org/10.3390/ educsci12030189

Academic Editor: Huei Tse Hou

Received: 29 January 2022 Accepted: 3 March 2022 Published: 9 March 2022

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

on the cultivation of students' abilities to correctly write Chinese characters and self-study Chinese characters [1]. Therefore, the recognition and memory of Chinese characters can be promoted by strengthening the contact and learning of the component structures.

In the information age, especially in the era of mobile Internet, game culture is becoming increasingly popular. The digital game has become a universal entertainment and education medium. Among them, digital learning games or educational games have the characteristics of games, meet certain educational purposes, and can stimulate the active initiative of the players and help them to acquire knowledge or cultivate their abilities in the process of the games. Digital games for learning are one of the research hotspots in the development of today's information society and are also considered as a major educational medium of the next generation, which will have a significant impact on teaching and learning methods [2]. The advantage of digital games is that they can effectively stimulate the "positive attitude", motivate learners to learn Chinese, and make the learning process easy and natural. The theory of "experiential learning" holds that in order to gain real knowledge, we must learn by "doing", through practical activities such as application, trial, and transformation. Digital learning games can create and simulate certain social situations and scenes, where learners can finish learning tasks and master relevant language skills joyfully, through the game experience links such as immersion, plot, acting, competition, task, action, creation, and exploration. The integration of interactive, competitive, and cooperative game models in digital learning in Chinese teaching can highlight the status of students' independent learning and achieve good learning results.

The objectives of this study:


The research questions of this study:


#### **2. Literature Reviews**

Applications (apps) in education, built or installed in mobile devices such as smartphones, tablets, etc., have been regarded as assistance for teaching and learning. To make the teaching more deliverable and to make the learning more accessible, app development takes responsibility for connecting learners and teachers with knowledge-based content.

Mobile devices including apps are widely adopted in school learning and broadly used by young learners. In school learning, game-based apps and related development are valued when innovating the educational environment; even novel tools and technologies are involved, such as augmented reality. Augmented reality technology incorporating apps, mentioned above, can engage language learners in learning content, and contribute to student learning [3]. The learning environment, with the intervention of game-based apps, is believed to bring interesting learning experiences to students that can increase motivation and improve learning outcomes. The survey research [4] found that gamified apps have been introduced in various levels of school education and shows that school learning tied to the use of game-based apps can influence students' positive attitudes towards their learning, promoting their knowledge and hands-on digital experience acquisition. Another research [5] reported the learning effects of educational apps on young learners, children from 2 to 5 years old, that the mobile apps in school learning can aid in literacy development for many subjects, such as mathematics, science; even though there is a limited understanding of the impact on young learners. For young adult learners or students, the educational apps make them more willing to join the class activities, stimulating their initiative in learning [6].

Teachers also prefer to adopt them in class to lift teaching quality and provide appsbased teaching and an evaluation method with feasibility and practicability, for mastering student learning status and adjusting teaching strategies. Furthermore, class management can be achieved more effectively and efficiently with the assistance of educational apps [7] when teachers try to manage the student's attendance and learning performance. They can access all the learning related data derived from the students more easily and give them advice or feedback about their learning outcomes more instantly. Therefore, the teacher's viewpoint and the user experience should be considered when developing apps for teaching/learning.

Language learners form the main part of educational apps' market share, so it makes sense that most of the educational apps rolling out into the market focus on language subjects. The download count and the use of language learning apps have grown greatly, meaning the popularity results in prosperity. In English learning, the introduction of educational apps brings ideas of innovation to the traditional English class, and the possibility of blended or virtual learning environments. Automated writing evaluation and guide-based writing assistance provide language teachers with more strategies in teaching [8].

For users of language learning apps, dictionary and lexical features appeared to be the most frequently used functions [9]. Language learners could be typically positive to adopting apps for learning a specific language, but their attitudes towards language learning with apps are significantly subject to what kind of apps they used [10]. Sentiment analysis is well known for its powerful performance in monitoring, analytical and alarm systems, as well as customers feedback. In language learning, sentiment analysis can bring an alternative way to master the emotional vocabulary of a sentence or paragraph, which is a difficult task for language learners. And this technology enables language learning apps easier to suggest more suitable content to learners [11]. Augmented reality technology incorporating apps, mentioned before, can engage language learners in learning content, and contribute to student learning.

At present, there are many Chinese learning games on the market based on different electronic platforms. In the Student Learning Space (SLS), launched by the Ministry of Education of Singapore in recent years, several digital games have been introduced for some Chinese language exercises, which are very popular among students [12]. There are also many Chinese character learning or game apps on the market. Researchers have conducted detailed studies and comparisons on various Chinese character literacy apps [13,14], including many that are based on Chinese as a second language. However, the current Chinese character learning game apps have the following shortcomings:


Chinese is the mother tongue language that Chinese students in Singapore need to master. However, for many local students, due to the use of English as the main language in their family and living environments, the time and frequency of using Chinese and exposure to Chinese characters are relatively insufficient, which leads to a high forgetting rate, confusion of the characters and the improper use of Chinese characters [15]. At the same time, these learning difficulties may harm students' interest in learning Chinese [16]. Mastering Chinese characters is the foundation of learning Chinese, especially in the lower grades of primary school. Mastering Chinese words and phrases is the foundation of reading and writing skills for attending middle and upper grades. Therefore, we must consolidate the ability to memorize Chinese characters in the lower grades of primary school [17]. For primary school students in the lower grades, to guide the students to have contact with and learn Chinese, we first need to stimulate their interest and make

good use of their interest in learning Chinese. Game apps are a way to attract students effectively [18–20].

This study attempts to develop an app of a Chinese character learning game for local primary school students in Singapore, aiming to stimulate students' interest in learning Chinese, increase their frequency of contact and use of Chinese characters, and ultimately strengthen their ability to remember, understand and use Chinese characters. This study aims to achieve the following objectives:


By strengthening the ability of the lower grade students to remember, understand, and use Chinese characters, they can lay a solid foundation for their ability to write in Chinese as well as to master their reading and writing skills.

#### **3. Materials and Methods**

*3.1. Conceptualization*

Based on the investigation [4,5] of existing apps related to Chinese character learning, the research team reviewed the relevant research articles on Chinese character games and discussed them in many meetings. The design concept of this project can be summarized as follows:


As Figures 1 and 2 show, the proposed design for the app development contains four main functions: Preview (我先玩), Review (我再玩), Monopoly Game (我来了) and Collaboration (一起玩). From Figure 1, the functions Preview and Review will assign quizzes to users, students of P1 and P2, and the questions bank embedded in the app will randomly promote questions according to the textbook content of P1 and P2. The Monopoly Game is motivated by the commonly known board game for kids, and we adopted a similar concept from it for the app development. When playing the Monopoly Game, students will see a familiar road map to that they have seen before. The difference is that each cell of the roadmap represents a task of CL including a glyph, a word component and a phonetic quiz, etc. All the questions are also randomly assigned to each cell and developed according to the textbook content of P1 and P2. Another selling point for the designed app is collaboration, which means that users need to finish the task only with partners. This function allows users to create a team and answer/challenge the word component test. We anticipate students' good sense of teamwork and teachers' willingness to adopt it as a learning activity in their class.

**Figure 1.** App wireframe 1 (user sign up, log in, sending result). From starting with the upper left sub-graph, the user can sign-up for the first time using the app and then shift to the login page. After entering the app in the user's account, a user interface would be displayed with the main functions: "Review", "Preview", "Play", and "Collaboration ". Besides, the main-function page also shows "Marks" indicating the score earned for the last game.

**Figure 2.** *Cont*.

**Figure 2.** *Cont*.

**Figure 2.** (**a**) App wireframe 2 (Preview); (**b**) App wireframe 3 (Review); (**c**) App wireframe 4 (Play); (**d**) App wireframe 5 (Collaboration); (**e**) App wireframe 6 (Collaboration).

#### *3.2. Mock-Up and Prototype*

The app, named "Han Zi Hunter", is mainly divided into four functions: Preview, Review, Play and Collaboration.


classmates to provide the components they need. Students who send requests will be deducted points, and students who assist classmates will be awarded points.

#### *3.3. Implementation Process*

The implementation process of this research project was divided into two main parts. The first was the development of the app. During the development process, the research team held several meetings with the developer and the teachers at the participating schools to discuss the content and functions of the app, and also involved the teachers in the pretesting process of the app development to better understand students' needs and provide a better user experience. The research team and the developer adopted the feedback provided by the teachers as a reference to improve the app and made several modifications to get to the current version (version 1.8). Since this research project is targeted at Singapore students from Primary 1 and Primary 2, the app was developed to closely match the content of Singapore Primary 1 and Primary 2 Chinese language textbooks: Chinese Language for Primary Schools (CLPS).

After the development of the app was completed, the research team also conducted a trial experiment with students from Primary 1 and Primary 2, with the support and assistance of teachers. The research team conducted classroom observations through the online meeting tool, Zoom, to observe the classroom situations in which the teachers used (experimental classes) or did not use (control classes) the developed App for teaching. Details about the classroom observation will be explained in the following section.

#### *3.4. Participants and the Scope*

According to the research proposal, a total of 4 experimental classes (100 students and 2 teachers) and 4 control classes (100 students) from 2 schools were invited to participate in the project. Data collection arrangements were delayed due to the home-based learning arrangement during the COVID-19 pandemic and school holidays were also encountered, under the tight schedules of the teachers and students. Although the research team worked as closely with the teachers as possible, the planned number of participants was not able to meet the pre-setting amount due to the reasons stated above, and we were not able to hold the observation of the control class at the second school. The actual number of participants in this study is as follows:

#### *3.5. Data Collection*

According to the data collection plan, data from interviews with teachers, students' surveys, and classroom observations should be collected. In addition, to verify the enhancement effect of the teaching activities using the game App on students' literacy, literacy tests for the experimental group and the control group should also be collected. However, due to the COVID-19 pandemic, with the active efforts and the cooperation of the research team and the teachers, some adjustments have been made to the data collection process, so the research team have managed to collect the data from the following:


#### *3.6. Classroom Observation*

As stated above, the research team could not get to the schools to conduct the classroom observations due to the strict measures of the COVID-19 pandemic, so the research team adopted an online meeting tool, Zoom, to conduct the classroom observations, to observe the classroom situations in which the teachers used (experimental classes) or did not use the "Han Zi Hunter" Chinese character learning game app (control classes) for teaching. As Figures 3–5 show, the study received great support and assistance from the teachers

under the difficult circumstances; the teachers helped to set up laptops and/or iPads from different angles so that we could better observe the lessons, especially in the experimental classes.

**Figure 3.** Control class students use little whiteboards to draw the new Chinese character they have just learned, and the teacher uses an object projector to present students' work.

**Figure 4.** Experimental class students are playing "我来了".

**Figure 5.** Experimental class students are having a discussion.

Through observation, the teaching tools used by the control classes are mainly learning portals like Blooket, little whiteboards, flashcards, and pen and paper games, while the experimental classes were focused on using the "Han Zi Hunter" app. The research team found that the students in the experimental classes were more active when using the "Han Zi Hunter" app; they took the initiative to play the games, they discussed and shared their learning progress with their classmates and teachers, and they also felt a sense of accomplishment and excitement. Some of the observations are shown below:

#### *3.7. Data Analysis Method*

Based on the above collected data, the team analyzed the data as described below. Firstly, qualitative analysis was conducted on the teacher questionnaires to understand the common points in the feedback of the teachers participating in the trial teaching, as the main reference for the game and the supporting design. Secondly, a quantitative analysis of the students' questionnaires was conducted to understand the students' views on the games, the classroom activities, and their effects on learning. In addition, the literacy test results were analyzed to see whether the literacy accuracy rate of the test group was higher than that of the control group, and there was a significant difference, to prove the effectiveness of the teaching activities using the game app. Descriptive statistics and statistical testing were used in the data analyses.

#### **4. Results**

According to the research questions mentioned above, the research team has analyzed research questions 1 and 3 here, based on the following collected data:


The research team conducted a questionnaire survey and a literacy test of P1 and P2 students in two primary schools. Under the COVID-19 pandemic regulatory limitations, a total of 77 student questionnaires and 53 student literacy tests were collected, as Table 1 shows.


**Table 1.** Total number of students who participated in the study.

The research team conducted a quantitative analysis of the questionnaire survey. We found that although more than half of the P1 and P2 students mainly speak English at home, they are interested in learning Chinese. There are as many as 71.4% of P1 students and 95.9% of P2 students claiming that they like the learning game app, especially the Monopoly Game. In addition, most of the students think that it is very interesting for teachers to use the app in class, and the questions and contents in the app are appropriate. In general, students' reactions and feedback to the learning app are very positive. Table 2 shows the details of the students' feedback.


**Table 2.** Students' questionnaire survey feedback.

In addition, as mentioned above, due to the impact of the pandemic and under the tight schedules of the teachers and students, the research team only conducted the literacy tests in the P2 of one of the schools. From the analysis results of the literacy test, we found that although the sample size of the experimental class was smaller than the control class, the average score of the experimental class was still slightly higher than the control class, and their value of standard deviation is lower than the control class. Table 3 shows the results of the literacy tests.

**Table 3.** Students' literacy tests in the P2 of one of the schools.


To better understand the application of the "Han Zi Hunter" app in classroom teaching, we also surveyed the teachers to find out their opinions on the effect of the app in classroom teaching and their opinions and suggestions on the content and design of the game.

From the teacher survey results, we found that the teachers' overall evaluation of the app was very positive. Table 4 shows some of the feedback from the teachers' surveys.

For Table 3, we conducted statistical testing to examine the difference between the control class and the experimental class. The mean difference between them equals 0.1000 and the t statistic value is 0.0955; not sufficient to show the statistical significance considering the 95% confidence level. Despite the above statistical testing inference, we balanced the

concept with Table 2 and conclude the result: the implementation of the trial class might be affected by the social distancing measures amid the pandemic.


**Table 4.** Teachers' feedback on adopting "Han Zi Hunter" app as a teaching tool in their class.

The feedback in Table 4, has been collected from the schoolteachers and classified into four categories: app content, students' interest in the app, teachers' perceptions of the app, and teachers' suggestions for the app. For the content delivered by the app, the teachers who were involved in the trial class and adopted the app in their class thought the content was appropriate and suitable for the students' current abilities. To summarize the feedback about the students' interest in the app, observed from the teachers, pupils feel it is interesting and amusing when operating tasks assigned by the app function and listening to the phonics for matching a corresponding Chinese character. They enjoyed the game in the app in their Chinese class and expected to engage in it for the next class or after school. In teachers' viewpoints and perceptions, the app can appropriately help the implementation of consolidation and review for pupils' Chinese knowledge through the user experience and appropriate content. For the app's future development and revision, the teachers interviewed also provided several suggestions for the user interface, the adaptive learning content, and component refinement.

#### **5. Conclusions and Discussion**

In the research, we developed and proposed a game-based educational app for Chinese language learning, concerning primary school learning for pupils from grade 1 and grade 2. In content construction, we associated the graphics, words, sentences, and the corresponding quizzes with the textbooks of Singapore's primary schools. It is inevitable to arrange an adequate budget for content construction, including content creation, external material licensing, and proofreading, independent of the cost of developing the app

architecture. The developer needs to be aware of this when undertaking the development of an educational app.

We conducted trial class teaching with the assistance of our proposed app, "Han Zi Hunter", and collected user experience data and feedback from four classes of two primary schools. The data were collected through the questionnaire survey for pupils, literacy tests for pupils, interviews for teachers, and class observations, and the corresponding analyses were conducted.

For the research questions mentioned in Section 1, we can conclude the following points based on the results in Section 4:


During the app development, we concluded two suggestions for future developers devoted to educational apps.

Less requirement for operating knowledge can enable the intuitive operation for users to use the app more instinctively. Since the young students of P1 and P2 are the targeted users of our proposed app, they are not equipped with enough knowledge about information technology, such as internet connection through Wi-Fi, personal data security, and identity authentication for the app login, which they might be asked to do by apps but with which they are probably unfamiliar. Therefore, all the technical issues raised could concern IT knowledge and the environment required for user operation. Apps with less requirement for operating knowledge can benefit school pupils' instinctive use. Table 2 also provides the evidence to support this. More than 70% of the pupils like to use the proposed app and the support rate even gets close to 95% for pupils of P2. That means the adopted app is easily accessible because of less requirement for operating knowledge.

Age-centered content would make the learning more formative. Educational apps can provide good assistance for class teaching if the embedded app content is compatible with the textbook content. Suppose the app content is not developed according to the textbook content, a teacher will not be able to expect pupils' learning outcomes through the app and it will be a game app only for use after school. Age-centered content can enable the app to help class teaching. In Table 3, we found that the experimental class teaching with the intervention of our proposed app results in better performance than the control class. Based on the statistical testing result, we can also conclude that the app would not make the learning outcome of class teaching worse and could probably make it better, supposed with more observations collected.

**Author Contributions:** Conceptualization, Y.-S.L.; methodology, Y.-S.L. and Y.-S.W.; software, J.N.L.; validation, Y.-S.L. and J.N.L.; writing—original draft preparation, J.N.L.; writing—review and editing, Y.-S.W.; supervision, Y.-S.L. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research was funded by the Singapore Centre for Chinese Language and the Academy of Singapore Teachers, grant number CDF-2020/03-LYS.

**Institutional Review Board Statement:** The study was conducted in accordance with the Declaration of Helsinki, and approved by the Institutional Review Board of Nanyang Technological University (protocol code IRB-2021-01-024, 24 January 2021).

**Informed Consent Statement:** Informed consent was obtained from all subjects involved in the study.

**Data Availability Statement:** The data presented in this study are available on request from the corresponding author. The data are not publicly available due to the Institutional Review Board protocol of Nanyang Technological University.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**


## *Article* **Preservice Teachers' Knowledge and Attitudes toward Digital-Game-Based Language Learning**

**Jose Belda-Medina \* and José Ramón Calvo-Ferrer**

Department of English Studies, University of Alicante, 03690 Alicante, Spain; jr.calvo@ua.es

**\*** Correspondence: jr.belda@ua.es

**Abstract:** There is a good body of literature about digital-game-based language learning (DGBL), but research has mainly focused on students as game players rather than as future educators. This paper reports on a research conducted among 154 teacher candidates at a higher-education institution in Spain regarding the adoption of digital games in education. It analyzes the participants' knowledge of and attitudes toward digital games in foreign language learning. Quantitative and qualitative data were gathered through a pre/post-test, digital game presentations, and student blog posts. The research comprised five stages associated with critical thinking skills (definition, selection, demonstration, discussion, and reflection), including a game learning module. In the first two stages, preservice teachers completed the module activities and selected different games aimed at teaching English to children in preschool and elementary education. In the last two, they illustrated, discussed, and evaluated the digital games in class following a rubric and reflected on their perception in blog posts. In this four-week research based on a mixed method and convenience sampling, quantitative and qualitative data were gathered through a pre- and post-test survey about student perceptions toward the use of video game in the classroom, class discussion, and blog posts. Statistical data analysis unveiled gender-based differences related to gameplay frequency and genre preferences. The Wilcoxon signed-rank test was used as a nonparametric statistical hypothesis test to compare the two sets of scores resulting from the same participants, and it showed a significant difference (*p* ≤ 0.05) after the treatment in two of the five dimensions in the survey about teacher candidates' attitudes toward game usage in education, namely, usefulness (U) and preference for video games (PVG). Research findings revealed preservice teachers' positive attitudes but lack of practical knowledge about the use of digital games in foreign-language learning.

**Keywords:** digital games; language learning; teacher candidates; knowledge; attitudes

#### **1. Introduction**

The use of digital games in language learning has been the object of study in several publications over the last two decades [1–5]. Some works focused on the consequences of integrating such games in education and language learning, particularly as regards student motivation [6,7] and enjoyment [8,9], while others analyzed their impact on the development of certain language skills and components such as listening and speaking [10,11], reading and writing [12], grammar [13], and vocabulary [14–16]. Conversely, few articles examined the knowledge and attitudes of preservice teachers toward the integration of digital games in an English as a Foreign Language (EFL) classroom [17–19]. This is vital, as these students and prospective teachers will soon take a leading role in language instruction in 21st century education.

The increasing popularity of digital games and their use in the language classroom reinforced the need of proper training in teacher education programs. Therefore, several authors advocate for a better-quality preparation regarding knowledge on the principles and practices of digital games in language learning [18–20].

**Citation:** Belda-Medina, J.; Calvo-Ferrer, J.R. Preservice Teachers' Knowledge and Attitudes toward Digital-Game-Based Language Learning. *Educ. Sci.* **2022**, *12*, 182. https://doi.org/10.3390/ educsci12030182

Academic Editor: Huei Tse Hou

Received: 26 January 2022 Accepted: 28 February 2022 Published: 6 March 2022

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

Two aspects seem to confirm the need for a better training of such students in their transition from digital native students, meaning individuals born after the widespread adoption of digital technology, to digital native teachers [20]. On the one hand, the proliferation of smartphones and the growing number of game-based apps elicit new pedagogical approaches in language learning, as attested by the increasing number of research papers about digital game-based language learning (DGBL) [21,22]. In this sense, various authors reported on the impact of integrating game-based apps in the foreign-language classroom, investigating its affordances such as student enjoyment and enhanced motivation, and its constraints such as limited functionality and lack of human interaction [23,24].

On the other hand, the application of game-design principles to education, particularly to second- and foreign-language learning, and the steady rise of new game formats and platforms (consoles, web-based, apps) strengthened this demand for more updated preparation [4,24–27]. In fact, this need has been more evident since the outbreak of the COVID-19 pandemic and its global impact on modern education due to online learning and social distancing [28].

This research has two objectives: first, to examine teacher candidates' knowledge on digital games in the foreign-language classroom; second, to measure their attitudes toward the use of such games in language learning. For this purpose, 154 education students grouped in teams completed a module about game-based language learning, selected a game, and illustrated it in the classroom. Then, the participants evaluated the different games and discussed the possible affordances and obstacles for their integration in language learning, and they reflected on different games and game-based learning on their blog posts.

#### **2. Background**

Digital games have outperformed other types of entertainment such as movies in terms of total number of users and economic power over the last two decades [29], particularly in the last two years, after the outbreak of the COVID-19 pandemic and its global impact [30–32]. In this paper, a digital game is defined as any kind of interactive program, either online or standalone, employed with different electronic devices such as a console, smartphone, or computer, which is primarily used for entertainment. There is a rich body of literature on the use of digital games in education from different perspectives, and foreignlanguage learning is no exception [33,34]. Studies on DGBL have multiplied in recent years, as evidenced by the appearance of new specialized journals. In these publications, several scholars evaluated the effectiveness of using such games in foreign-language education, particularly in relation to vocabulary development [3,35].

Regarding attitudes toward game-based learning, some works delved into the beliefs and determinants to adopt digital games among inservice teachers [36–38], stressing the benefits of integrating such games in the foreign-language classroom, such as increased motivation, shared enjoyment, and enhanced interaction in a student-centered model. However, a few voices were against the educational use of such games on the grounds of alleged ineffective learning structures, pedagogical inadequacies, and distracting factors [39–41]. According to Kaimara et al. [42], 'recent findings concluded that teachers were unwilling to adopt digital educational games because they were not really convinced that games are very useful for enhancing their job' (p. 827). However, there is ample evidence that digital games can be effectively used to reinforce certain cognitive skills such as problem solving, risk taking, and reasoning [43,44].

However, the majority of published studies have focused on current language learners as game players rather than as future educators. There is a little research on knowledge and attitudes toward the integration of digital games among teacher candidates in foreign-language education. Previous works in this area generally showed that prospective teachers hold positive attitudes toward the use of games, particularly after a gaming intervention [18,45]. However, some authors also expressed reservations about the teacher

candidates' practical knowledge and ability to integrate meaningfully digital games in foreign-language education, partly due to a lack of proper training [46].

In this sense, Demirbilek et al. [47] explored the relationship among four categories (current situation, usage, game features, and efficacy for lesson) and second-language instructors' perspectives. The authors concluded that 'the way how computer games are employed during the instruction, the features of the game, hardware and software infrastructure of the classroom affect the perception and attitudes of instructors and students towards the computer games' (p. 720). Likewise, Alyaz and Genc [19] investigated teacher candidates' beliefs about digital games to learn German as a foreign language, and claimed that participants found games beneficial as both teachers and learners, stressing the need to integrate DGBL into the foreign language education curriculum.

Similarly, Sardone and Devlin-Scherer [48] studied teacher-candidate perceptions and reactions to 33 digital games as a learning technique, concluding that game usage enabled participants to understand the teacher's role as a facilitator of instruction, and that it is necessary to consider 'major curricular changes requiring the incorporation of gaming pedagogy in K-12, higher education, and teacher preparation settings' (p. 65). The same authors in a subsequent study recommended the integration of a digital game module in teacher education courses as a way to foster creativity, innovation, and motivation through alternative forms of educational technology [49].

In her study on preservice EFL teachers' behaviors and perceptions about digital games, Blume [17] found significant correlation between game playing and positive beliefs on the one hand, and between game playing frequency and perceived usage of language learning strategies on the other. According to this author, having no previous learning experience in digital games can actually be better than having a negative one, and she highlighted the relative receptivity of preservice teachers toward DGBL despite this lack of experience. As a result, Blume emphasized the need of better teacher training to strengthen strategic language learning.

More recently, Kaimara et al. [42] examined the preservice teachers' perception of the potential barriers to the implementation of digital-game-based learning in the classroom, identifying as the major obstacles 'the lack of financial resources, the preference for traditional teaching methods and stereotypes about the value of digital games, the lack of ICT training, the lack of infrastructure and the lack of educational policy' (p. 838). In general, three main factors seem to shape learners' beliefs about the use of digital games in language learning, namely, previous playing experience, the perception of other significant individuals such as teachers and parents, and perceived self-efficacy [50,51].

Concerning pedagogical approaches to computer games, Munkundan et al. [52] examined the potential of incorporating such games in foreign language curricula from different learning theories and game design principles. The first generation of games were based on programed instructions, and focused on repetition and reinforcement techniques following the principles of behaviorism. However, these drill-and-practice games were criticized for promoting rote learning and lower levels of knowledge [53,54]. Therefore, the second generation of computer games were more learner-centered and aimed at developing students' creativity and cognitive skills through authentic and situated learning tasks, thus following a constructivist approach [55,56]. The third generation of electronic games were designed from a sociocultural perspective to explore other factors such as context, culture, interaction and learners' identity, where teachers become facilitators of the learning process [57,58].

On the whole, previous works illustrated that there is a positive attitude toward the use of digital games in education and language learning among teacher candidates. However, there is also a lack of educational experience and insufficient professional development, and a necessity to incorporate such games in teacher training programs [59,60]. This article aims to bridge the research gap between preservice teachers' knowledge and attitudes toward digital games in language learning as future educators.

#### **3. Objectives and Method**

#### *3.1. Objectives*

This research had two objectives: first, to examine the teacher candidates' knowledge on digital games in the foreign language classroom; second, to measure their attitudes toward the use of such games in language learning. The two research questions were as follows: (1) What knowledge do teacher candidates have about game-based language learning? (2) What are their perceptions toward game usage after selecting, illustrating and discussing different games in the classroom? In this study all forms of digital games were considered without distinguishing between serious or educational games, commercial off-the-shelf games (COTs), web-based games, and game-based apps.

#### *3.2. Context and Sampling*

Participants in this research were 154 students enrolled in undergraduate course Integrating Skills in English addressed to preschool and elementary education teacher candidates at the University of Alicante (Spain). This course ran daily for two consecutive months, and students were expected to learn how to integrate different ICTs through a collaborative learning approach. In this course, the teacher candidates had to develop two digital projects aimed at teaching English: the first was related with game-based language learning (4 weeks), while the second was associated with the integration of augmented reality (AR) in the English as a Foreign Language (EFL) classroom. This research presents the results of the first project. Bearing in mind the exploratory nature of the research, participants were selected following a convenience sampling approach based on data collection from population members who were available [61]: 154 student of the 167 officially enrolled who had completed all the tasks. As regards age and gender, 83% were female students, and 93% of them were aged 20 to 30.

#### *3.3. Procedure*

This four-week research was based on a mixed method. Quantitative and qualitative data were gathered through the use of a pre/post-test survey, game demonstrations, class discussion, and students' reflections in the form of blog posts. The project covered five stages corresponding to different critical thinking skills included in education studies [62]: defining, selecting, demonstrating, discussing, and reflecting, as shown in Figure 1. Each thinking skill was associated with several academic abilities: stating the problem and becoming familiar with the current situation in DGBL (defining); carefully choosing the most suitable game among different choices on the basis of certain criteria (selecting); explaining and illustrating the games selected through practical examples (demonstrating); debating and evaluating the affordances and limitations of each game with peers (discussing); and considering the learning gains from a critical perspective and expressing the ideas in blog posts (reflecting). For this purpose, students were randomly arranged in teams of 4–5 members in order to find, play, analyze, and select digital games that could be used to teach English to children. The project took place in 8 two-hour class sessions during four weeks.

First, a game-based learning module was created in Moodle with information about gamification and the use of digital games in language learning in the form of readings, websites and practical examples. Then, the different teams were requested to find and select digital games aimed at teaching English as a foreign language, and to think about the target students, context, and learning goals. Next, the participants illustrated the use of the selected games in class. In this project, particular relevance was given to inclusive digital games that could be addressed to different types of learners, such as those with special needs (SEN).

**Figure 1.** Stages of game-based language learning project (1 session = 2 h).

In the fourth stage, teacher candidates discussed the potential affordances and limitations in class, and they evaluated the games following an assessment rubric which covered aspects such as game design (interface, visual elements, instructions, mechanics, and rewards) and educational use (target learners, language level, learning goals, corrective feedback). In the last stage, the teams had to post some comments and game images in their blogs as multiple reflections based on class presentations and discussion.

#### *3.4. Instruments*

An online pre/post-test was administered on the first and last weeks of class. The pretest contained 35 questions divided into three sections: the first section included 5 items associated with sociodemographic data; the second section comprised 6 questions related to technology ownership and usage; the third section was formed by 23 specifically related items to the use of video games in education. This scale was based on previous research by Bourgonjon et al. [63] about students' perception of using video games in the classroom, it surveyed the different factors determining the students' preference for using video games grouped into five dimensions: personal experience (EXP), usefulness (U), ease of use (EOU), learning opportunities (LO), and preference for video games (PVG). The items included in this scale are displayed in Tablet 7 with the results.

The post-test replicated the third section so as to compare the results at the beginning and the end of the treatment, and included questions aimed at measuring students' satisfaction with the games they had selected and discussed in class. Qualitative data were obtained through class discussion and blog posts.

#### *3.5. Method*

Quantitative data were analyzed using SPSS 22.0 statistical software. The Wilcoxon signed-ranked test was employed to analyze the medians of the two measurements about students' perceptions toward video games before and after the experiment with the significance level set at 0.05. This nonparametric test was used to compare the difference between the two paired sets of ordinal data as suggested in previous research [64]. In order to identify any existing relations between the participants' gender and the different dimensions, correlational analyses were performed with the simultaneous inclusion of all measured variables. Qualitative data obtained from the students' reflection in the form of blog posts were analyzed and coded by the two researchers according to different patterns of data in the main themes.

#### **4. Results and Discussion**

Concerning technology ownership, the pretest results indicated that every student had a smartphone, 93% of them owned a laptop and/or a tablet, and 85% had access to a desktop personal computer. As regards frequency of computer usage and game play, some gender-based differences were observed, as shown in Table 1. Female students spend more time on average on the computers, while males dedicate more time to game playing. In fact, nearly 43.3% of female participants stated they never play games, as opposed to 7.4% of males. Results are in line with previous research findings indicating higher gaming time of males over females [65], and contrary to other reports stating that game play, meaning the amount of time dedicated to playing games, is gender-neutral [48]. However, these results may be limited by the context and different number of participants based on gender, as most teacher candidates were women.


**Table 1.** Computer usage and game play frequency (*N* = 154).

Concerning reasons for computer usage and based on a multiple-choice questions with multiselect answers, the top three options were academic purposes, web searching for information, and entertainment (excluding games) such as watching movies or videos, as shown in Table 2. Playing games yielded the lowest score among participants. As regards smartphone usage, communication purposes (instant messaging, video call) ranked first, followed by web searching, entertainment, online shopping and social networking, while playing games also provided the lowest score.

**Table 2.** Main reasons for computer and smartphone or tablet usage.


As for game platform, participants who spend more time playing games (3+ h per week), opted for home video game consoles, particularly PlayStation (PS), whereas those who played occasionally (<1 h per week) preferred their smartphones, as illustrated in Table 3. Regarding genres, the data confirmed previous academic and commercial reports on genre/gender game differences [66–68]. Among participants who had indicated that they did play games, male students (*n* = 25 out of 27) predominantly chose two categories, fighting and sports, whereas females' preferences (*n* = 72 out of 127) were much more diverse, with puzzle/card, role playing, simulation, strategy, and action and adventure games providing similar results. The data also confirmed previous findings that action and adventure games are popular among both genders, and that women tend to play more RPG and strategy games, while men are more into sports and fighting [67].


**Table 3.** Preferred game consoles and game genres among teacher candidates.

\*\* Excluding participants who indicated that they never play games (7.4% males and 43.3% females).

In this course, students needed to work collaboratively. The main goal was to train them in the meaningful integration of such games in foreign-language education. First, a game-based learning module was created in Moodle containing different readings, websites, and examples of digital game practices, which the participants had to consult. In the second stage, each team had to find, play, and select a digital game aimed at teaching English to children in preschool and elementary education following the rubric provided in the materials and shown in Figure 2. This was the same rubric that the students later used to evaluate all games after the presentations.


**Figure 2.** Rubric used for assessment of digital games (1 = basic, 2 = medium, 3 = high).

Next, the teams had to explain and illustrate the use of the games they selected, specifying the context (educational level and target students) and goals (language skills and components) to their peers, who had in turn a limited time to play some of these games in the classroom. For this purpose, each team had to prepare a visual presentation which was later shared in the learning module (Moodle) containing relevant information about the game, some images explaining the details included in the rubric and a link to the website so that the classmates could check out, and play whenever possible, the games in and after class. Two examples of presentations are shown in Figure 3.

Most of the recommended games were first- and second-language learning websites and apps specifically oriented to children, a relatively unexplored area for the participants. Some examples mentioned were *Pili Pop*, a game-based language app designed to improve the listening and speaking skills among children aged 5–10; *My Word Coach*, a video game aimed at improving children's vocabulary; and *Monkey puzzles,* developed by the University of Cambridge to help children read in English.

#### **Figure 3.** Digital game presentations for language learning.

A few games were mentioned by several teams such *Reader Rabbit*, an edutainment software used to teach children how to read and write in English, and *Sesame Street Games*, a website dedicated to children which contains several minigames devised to promote reading, spelling, and problem-solving skills. Some game-based apps were conceived for multilingual education, such as *Naraba World*, an app available in English, Spanish, French, and German, aimed at developing cognitive skills such as space, visual memory, and coordination in several languages.

Some games were not specifically oriented toward language learning, but the participants clarified how to use them with such purpose. Simulation games came up as the favorite genre, for example *World of Zoo,* a game that can be used to teach children about animals in English. As the teacher candidates illustrated, this game includes several rewards in the form of hearts and tokens that can be exchanged for food and tools, and an 'animal creature' feature to foster learners' creativity and interactivity, a trait of third-generation games as they had previously learned. Another example was *SimCity Edu*, a simulation game with online multiplayer elements that can be used to teach children English together with certain social skills such as friendship and respect.

Role-playing games (RPGs) were frequently mentioned, such as *Miitopia*, an RPG based on Mii characters to teach children about intercharacter relationships; *Pepi Bath Lite*, an RPG addressed to children aged 0–5 about body hygiene in English; and *Layton's Mystery Journey*, a game for children aged 10+ where players take the role of Katrielle, a young woman who opens a detective agency in London and needs to solve some mysterious cases, thus promoting learners' analytical skills.

As mentioned earlier, particular relevance was given to inclusive digital games that could be used among children with special needs (SEN). According to previous research [69,70], there are several benefits in using computer games with different types of learners, such as children with hearing or visual impairment [71–73], attention-deficit hyperactivity disorder (ADHD) [74], and autism spectrum disorder (ASD) [75,76]. However, the impact of digital gaming on special education (SEN) and its use in language learning is a relatively unexplored area, so most teacher candidates were unaware about the potential of such games.

One of the most cited inclusive games was *Otsimo*, a highly awarded app that focuses on helping autistic children in learning basic concepts through different games and develop their speaking, reading, and writing skills. Two more mentioned examples were *GraphoGame,* a Finnish game for children with dyslexia available in several languages; and *Visual Reading app,* an application created by a parent with an autistic child that is particularly addressed to children with autism or dyslexia. In this game, children can place images or videos above each word to help them make a connection. A special example was *Autcraft*, announced as the first *Minecraft* server for children with autism and their families. This platform was created by an autistic father of an autistic child, and provides a safe and respectful environment including different games.

On a more commercial side, some teams referenced popular video games such as *Minecraft* and the *Legend of Zelda* series. In fact, these options were mainly recommended by participants with a higher playtime score (over 3 h per week). The use of such commercial off-the-shelf games or COTS for language learning had been investigated. *Minecraft* is a massive multiplayer online (MMO) game with an offline version available, which is globally very popular among children. As the teacher candidates explained, in this game, young learners can choose an avatar to build things, learn words (castle, volcano, bridge, etc.), and chat with other users. In line with the research carried out in previous works [77,78], participants focused on the use of *Minecraft* for vocabulary building.

On the other hand, *Legend of Zelda* is a series of fantasy action-adventure video games based on a medieval-inspired journey where players must travel through dungeons and labyrinths to learn the origin of the Triforce and save the princess Zelda. Some teams proposed this game to be autonomously used for incidental vocabulary learning (signs, holes, fireballs, dungeons, leaf pile, stairs, etc.) alongside other educational values such as risk taking and memorization, although previous research works indicated that teacher intervention in the classroom might be necessary to strengthen the pedagogical value of this game [79].

In the fourth stage, the teacher candidates discussed the different games explained in class, and reflected on the benefits and limitations of integrating them in language learning among preschool and elementary education children. Class presentations and discussion were analyzed as qualitative data in this research. Consistent with previous studies [2,75,80], participants highlighted three potential benefits: enjoyment, enhanced motivation, and autonomous learning. However, they also predicted some challenges such as lack of or limited digital access and technological differences among children in and outside the classroom, inappropriate choice of the games depending on different factors (safety, students' educational needs), and distrust and lack of interest among current educational practitioners and supervisors. These conclusions confirmed results from previous works [5,17].

The prevailing view among future educators in the class discussion was that digital games are still conceived by a good number of professionals and parents as distracting and marginal to more formal education, and there is a widespread belief that these games may negatively impact children, such as through lack of attention and social interaction [81]. Therefore, participants highlighted the necessity to enhance the educational value and benefits of some digital games through practical examples, and to better prepare both inand preservice teachers in foreign-language education.

In the last research stage, the teacher candidates had to post their thoughts about digital games in the team blogs as illustrated in Table 4 (transcribed game reviews) and Figure 4 (blog posts).

**Figure 4.** Screenshots of two students' blog posts about digital games.


**Table 4.** Blog post reflections about the use of digital games in language learning.

Concerning students perception about the use of video games in the classroom, the pre/post-test results based on Bourjongon et al. [63] revealed a small increase in all five dimensions (experience, usefulness, ease of use, learning opportunities, and preference for video games) as shown in Table 5. Results of learning opportunities (LO) were higher compared to the other dimensions, thus confirming participants' positive attitudes toward video games, but gaming experience (EXP) yielded low scores, and results about the ease of use (EOU) only improved to a small degree.

The Wilcoxon signed-rank test was employed to analyze the difference in the means of the two related samples before and after the treatment, revealing statistically significant differences in only two of the five dimensions, namely, usefulness (U) and preference for video games (PVG), as illustrated in Table 6. In light of these data, teacher candidates seemed to be more confident about the positive impact of using video games on learning performance (*p* = 0.011), effectiveness (*p* = 0.014) and achievement (*p* = 0.045). Similarly, they expressed a stronger interest (*p* = 0.026) and enthusiasm (*p* = 0.014) about the adoption of video games in foreign-language education after the treatment.

However, no significant difference was observed in the results of the three other dimensions of experience (EXP), ease of use (EOU), and learning opportunities (LO). The result of EOU was particularly surprising, given the fact that teacher candidates had illustrated the use of digital games in the classroom, but according to their own comments, this just helped them in realizing the complexity of technological and pedagogical factors involved for the effective integration of digital games in language learning and their lack of preparation.


**Table 5.** Students' perception about the use of video games in the classroom based on five-point Likert scale (1 = strongly disagree to 5 = strongly agree).

**Table 6.** Wilcoxon signed-rank test of students' perception about the use of video games.


Reported values are two-tailed.

Correlational analyses were also performed to identify any existing relations between the participants' gender and any of the aforementioned dimensions. As it turned out, no significant data were found to identify the *gender* variable as related to experience, usefulness, ease of use, learning opportunities or preferences in either the pretest or the post-test. However, there seemed to be positive correlation between the gender of the participants and the time they stated they spend on their tablets and computers per week; specifically, males claimed to devote more time (*r* = 0.400) than females did on such devices (*p* = 0.000). This supports the notion that perceived usefulness directly and positively influences teachers' intention, while gender and age do not impact teachers' attitudes as explained in previous works [82].

Qualitative data obtained from the blog posts were analyzed and coded according to different patterns in three main themes. The first theme was related with the technical description and reasons to select each game. The three most repeated arguments were visual design, clear instructions, and pricing, which were part of the rubric, and they had been analyzed in previous research as predictors of game preference [83]. However, teacher candidates did not consider safety (data protection, ads, etc.) as a key element, despite the fact that they were selecting games addressed to children.

The second theme was related with the expected affordances or impact on languagelearning progress. In line with previous findings [84], participants mostly focused on vocabulary development as the most relevant benefit over other areas and skills. Other anticipated effects mentioned in the blog posts were enhanced motivation and enjoyment. The last theme was about the potential obstacles and limitations for the adoption of the selected games, and the most cited problems were three: lack of or poor digital resources, no accessibility of some selected games for different types of learners, and fear of no academic support for such innovative practices among current educators and school authorities due to lack of knowledge and mistrust about the benefits of using digital games in language learning.

#### **5. Conclusions and Implications**

The use of digital games in education is on the rise, and current technological advances are reshaping traditional methodologies in foreign-language learning. Research has mainly focused on students as game players rather than as future educators. However, new generations are already transitioning from digital native students to digital native teachers. This paper aimed to analyze teacher candidates' knowledge and attitudes toward the use of digital games in foreign-language education. For this purpose, 154 education students participated in this four-week research comprising five stages related with critical thinking skills (analyzing, selecting, demonstrating, discussing, and reflecting).

As the pretest results revealed, computer usage among participants was primarily dedicated to web searching for information retrieval and academic purposes, whereas tablet and smartphone usage was mainly linked with instant communication and entertainment such as watching videos and movies. Gameplay scored low compared to all the other options. Contrary to previous research findings, some gender-based differences were observed, such as a longer gameplay time and a more limited choice regarding genres, basically fighting and sports among males, as opposed to female students whose preferences were more diverse (simulation and RPG, action and adventure, puzzle and card games). The first implication is that better preparation is necessary to familiarize all future educators with different game principles, practices, and genres, and bring out their potential in education, since the integration of digital games seems to be closely related with their previous personal experiences as gamers [17].

Although participants' previous knowledge on the use of digital games for language learning was scarce, they managed to describe a wide range of games through class presentations. Some teams chose digital games specifically designed for language learning such as *Pili Pop* or *Lingokids*, while others preferred more commercial games such as *Minecraft*. Simulation games such as *World of Zoo* and *SimCity* emerged as the preferred genre among future educators. However, participants lamented being unfamiliar with inclusive digital games that could be used for children with special needs. As future educators, they valued the knowledge shared among peers about certain games such as *Otsimo* and *GraphoGame.* A second implication is that more technical and pedagogical efforts are necessary about game accessibility and students' needs in teacher training programs to prepare preservice teachers for an inclusive and diverse education in which digital games can play a positive role in the case of students with special needs [69,70].

Concerning participants' perception about the use of video games in the classroom, pre/post-test results and data analysis revealed significant differences in only two of the five dimensions, namely, usefulness (U) and preference for video games (PVG). The teacher candidates endorsed the educational value of video games and believed in their positive impact on learning productivity, performance, and achievement. They were also keen on learning more about digital games. In line with previous works [85], results showed that teacher candidates hold positive views toward video games, as evidenced by the scores of learning opportunities (LO).

However, no relevant changes were observed concerning ease of use (EOU) after the treatment. This may have been due to the insufficient time dedicated to the use of digital games in language learning and the wide range of available options, the complexity of technological and pedagogical factors involved, and their inexperience in integrating them in the classroom. As highlighted by Kaimara and Fokides [51], 'even if the motivation is a good starting point for games to be integrated into education, games should be aligned with the principles of pedagogy and educational methodology, as well' (p. 8217). As a third implication, digital games are a fast-growing sector requiring constant training and updated knowledge about emerging technologies (AR, VR) and new applications, so this may require more systemic integration in the curriculum and not only a game-based learning module.

Qualitative data based on class discussion and blog post reflections evidenced the benefits of including a digital game module, as indicated in previous works [49]. Future educators became more familiar with game-related principles and practices in language learning, and aware of the necessity to include digital games in the curriculum. However, and consistent with previous research findings [86,87], they also expressed their concern on the academic distrust about video games among some inservice teachers and parents because of the alleged negative effects such as the distraction factor and social isolation. The last implication is that preservice teachers may be reluctant to integrate digital games in their future career if they perceive there is no professional support for such innovative practices in the classroom. For this reason, special programs should be designed to highlight affordances, and discuss the risks of such integration to be specifically addressed to education stakeholders.

Lastly, these results may be limited to the specific context and participants in this research, particularly regarding gameplay experience and previous formal training, since the adoption of game-based language learning courses in the curriculum may vary depending on each setting. Further research is needed about the adoption of digital games in teacher training programs, particularly regarding confidence development and practical experience in early education, through game-based learning modules, and in the students' preparedness to include digital games for language learning from an inclusive perspective.

**Author Contributions:** Conceptualization, J.B.-M.; methodology, J.B.-M., J.R.C.-F.; validation, J.B.-M., J.R.C.-F.; formal analysis, J.B.-M.; investigation, J.R.C.-F.; resources, J.B.-M.; data curation, J.R.C.-F.; writing—original draft preparation, J.B.-M., J.R.C.-F.; writing—review and editing, J.B.-M., J.R.C.-F.; visualization, J.B.-M.; supervision, J.R.C.-F. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Informed consent was obtained from all subjects involved in the study.

**Data Availability Statement:** Data supporting the research findings are available from the authors upon request.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**


## *Article* **Design and Evaluation of a Board Game in Food and Nutrition Education**

**Feng-Kuang Chiang 1,2,\*, Shan Wang <sup>3</sup> and Zhonghua Tang 2,4**

	- Shanghai 200234, China; zhonghua.t@mail.sfao.gov.cn
	- <sup>3</sup> School of Educational Technology, Beijing Normal University, Beijing 100875, China; 201622010023@mail.bnu.edu.cn
	- <sup>4</sup> Foreign Affairs Office, Suzhou Municipal People's Government, Suzhou 215006, China
	- **\*** Correspondence: fkchiang@sjtu.edu.cn

**Abstract:** The outbreak of the COVID-19 pandemic has caused increasing concern over health care, part of which involves the role of nutrition. In this study, a nutrition educational board game, featuring quantitative computation, was designed to help students acquire nutrition knowledge and improve dietary behaviors. A group of 22 students in grade 7 of a middle school in Beijing were selected as the participants. A single-group pre- and post-test quasi-experimental design was adopted. Knowledge-Attitude-Practice (KAP) questionnaires and interviews were used to investigate students' changes in dietary knowledge, attitude, and behavior. The influence of gender and body mass index (BMI) on the teaching effectiveness was also explored. The results showed that some dimensions of knowledge, attitude, and behavior significantly increased in the students. Their changes in behavior were mainly related to some aspects that were easier to adjust. In addition, the study also found that females and students with abnormal BMI were more likely to be influenced by the educational game to make positive changes in attitude and behavior.

**Keywords:** nutrition education; board game; knowledge-attitude-practice

#### **1. Introduction**

Adolescence is a critical period of growth and mental development. Balanced nutrition intake and dietary behaviors play an important role in healthy growth and prevention of various chronic diseases. However, the incidence of obesity is increasing in recent years [1], which is mostly due to irregular diet structure, bad eating habits such as not eating breakfast, deficient intake of vegetables and coarse grains, and decreased physical activities [2].

Adolescence is also a crucial period for cultivating good eating attitudes and behaviors, which depend on mastery of good food knowledge [3]. However, there is not a special curriculum on food nutrition in primary and middle schools in China, and students can only acquire a little knowledge in science courses [4]. Primary science curriculum standards specify that students learn the basic concepts of nutrients and energy. Secondary chemistry curriculum standards specify that students learn the function of some nutrition organic compounds (carbohydrates, starch, protein, vitamins, and so on), mineral elements (calcium, iron, zinc, and so on), and poisonous compounds in their third year in middle school. Secondary biology curriculum standards specify that students learn how nutrients are digested and absorbed in the body. The knowledge in primary school is too shallow for students and they start learn deeper content too late. Additionally, although the government has provided some documents such as "guidance outline of health education in primary and secondary schools", the items are not quite explicit and a number of schools do not take them as key content in normal classes. Some schools, especially those in remote

**Citation:** Chiang, F.-K.; Wang, S.; Tang, Z. Design and Evaluation of a Board Game in Food and Nutrition Education. *Educ. Sci.* **2022**, *12*, 162. https://doi.org/10.3390/ educsci12030162

Academic Editor: Huei Tse Hou

Received: 17 December 2021 Accepted: 11 February 2022 Published: 25 February 2022

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

areas, suffer from desperate shortages of professional teachers; instead, nutrition knowledge is taught by teachers of other disciplines [5]. The above points are all reasons why schools did not provide favorable conditions for students to master good nutrition knowledge.

In recent years, many researchers have used educational games to assist traditional teaching in order to enhance students' interest in active learning and improve teaching effectiveness [6]. Chen et al. investigated the time effect of cooperative games such as cards, board games, and riddles on students' emotions of learning science and found that games helped to maintain students' positive emotions. Another study [7] designed a serious game in the area of nutritional knowledge to test its effect on children and discovered it was an adequate educational tool. The researchers in [8] explored the gamification as a teaching strategy in the pandemic years and concluded that it increased students' motivation and engagement and improved their attitudes [9].

Among various education games, the board game has a long history, fewer adverse effects, lower development costs, and better experience. In general, it has following advantages: (1) It can improve students' learning motivation [10]; (2) It is easy to combine multiple disciplinary content and improving students' interdisciplinary ability; (3) When used in multiplayers format, it can enhance students' language performance and communication ability; (4) The physical attributes of the board game can enhance students' sense of immersion and leave them a deeper impression of knowledge [11].

However, although there are many board games in the market, most lack the guidance of teaching theories and are not suitable for class teaching. Some educational games designed by researchers such as chemical pokers [12] are mainly directed at young children, which gives priority to increasing learning interest with less emphasis on subject education. In some countries such as Turkey, Australia, and New Zealand, board games have been widely developed and used in primary and middle schools with multiple themes, including not only traditional subjects but also human and science topics like planets [13], first aid [14], health education [15], chemistry [16], creative problem-solving skills [17], and so on. Evaluation studies have been conducted by many researchers. In contrast, board games are deficient in health and food topics in China and empirical studies are few.

This research attempts to employ game-based learning theory by developing an interesting game situation and interaction mechanism to intrigue students' intrinsic motivation. Specifically, it tries to integrate STEM elements, particularly the knowledge of chemistry and nutrition, with daily education through a board game with the purpose of helping students to learn food nutrition in a fun environment. Students are expected to have a positive change in attitude and behavior after they finish the board game. The degree of change may vary for students of different gender and weight, as previous literature has suggested that girls are more concerned about body image than boys [18], and there was a strong relationship observed between body image dissatisfaction and body mass index (BMI) [19]. It is assumed that students of different gender and weight will exhibit different degrees of eagerness to obtain nutrition knowledge and change dietary habits. Therefore, the research questions are as follows:


#### **2. Materials and Methods**

This study intends to integrate a board game with chemistry lectures and thus allow the adoption of a single-group pre- and post-test quasi-experimental design to find whether the students can adapt and what effect the board game will have on them in the cognitive and affective domains.

#### *2.1. Participants*

A group of 22 students (8 females and 14 males, aged 12–13, from the same class) in grade 7 of a middle school in Beijing were selected as the participants, whose profiles and body index (BMI) can be found at Table 1.

#### **Table 1.** Basic profiles of participants.


#### *2.2. Study Methods*

The activities were carried out for 6 weeks, 1–2 times per week and 30 min per time. One day before and after the program, questionnaires were used in pre- and post-test. A month after the program, an additional test was carried out to evaluate students' attitude and behavior. In addition, six students were randomly selected to join a semi-structured interview at the end of the program.

#### *2.3. Study Instruments*

#### 2.3.1. The Introduction of the Board Game

This board game has 90 checks, including food check, sports check, treasure box, supermarket, hospital, testing station, food problems tasks, and so on. Students need to move their pieces along the checks in turns by throwing the dice and collect food cards or drawing task cards in certain checks. The final goal is to acquire more healthy food according to the food pyramid and standards of basic energy metabolism.

The teaching goals are as follows:


The game set, as Figure 1 shows, is comprised of following parts: The game map;


**Figure 1.** The sketch of the board game set.

#### 2.3.2. Game Rules

It is suggested that 3–5 students play the game together. Boys set the basic energy metabolism rate as 1400 kilocalories per day and girls set it as 1200 kilocalories per day. They then throw the dice to move their pieces on the board and complete corresponding tasks. The detailed procedures can be seen from Figure 2, as explained in the following:

**Figure 2.** The sketch of game rules.

2.3.3. Game Designing Principles

First, based on the key points in the literature review, this game involves two elements: quantitative thinking and more operational knowledge. During the process of energy calculation and dietary pyramid calculation, students need to remember the energy and nutrient value of different foods, which helps them to deepen memories; when stepping into task checks, they need to solve problems on the food cards they have collected, which amounts to the utilization of the knowledge, helping them to establish linkages between theories and reality.

Second, according to the food pyramid, it is suggested to eat a higher proportion of vegetables and less meat, but there are more meat and high-fat food checks and fewer vegetable and fruit checks on the board. Therefore, students need to use strategies to acquire more healthy food cards and abandon unhelpful cards.

Third, some food task checks set punishment for going backwards, helping students to strengthen awareness of avoiding some unhealthy dietary habits, such as unreasonable dietary structure and unbalanced intake of nutrients.

Last, the final grades contain multiple dimensions, which requires students to think more and use more strategies and knowledge. "Speed grades" are designed for entertainment. "Energy grades" are intended to test whether students can consider the energy intake in an overall view. "Food pyramid grades" are designed to test whether students mastered the balanced intake of different types of food.

The professional part of the game refers to teaching materials of food nutrition [19], and the content has been checked by associate professor Chen Yanhui of food science and nutrition engineering college at China Agricultural University, so the game can be guaranteed to be scientific.

#### 2.3.4. Questionnaires

The questionnaires used in pre- and post-test have four parts: basic information, nutrition quiz, dietary attitude scales, and behavior scales.

Basic information includes name, gender, height, and weight (for the calculation of BMI).

Food nutrition knowledge comprises three parts: food and nutrient (nine questions), food pyramid and dietary structure (three questions), and food safety and package (three questions), 15 questions in total. All questions refer to the literature of An [20], Zhao [21], and national nutritionist exam questions. The proportion of different parts of knowledge points corresponds to their distribution in the game.

Dietary attitude questions are in the form of Likert scales, nine items in total, which comprise three dimensions: the awareness of the importance of healthy diet, self-efficacy of the command of nutrition knowledge, and the willingness to adjust dietary behaviors. All items are adapted from the research of Turconi [22]. The Cronbach's alpha value ranges from 0.773 to 0.8.

Dietary behavior questions contain food intake frequency scales (11 items) and Likert scales (seven items). All items are adapted from the research of Anderson [23]. The Cronbach's alpha value ranges from 0.794 to 0.818.

#### *2.4. Implementation Schemes*

As can be seen from Figure 3, students were divided into six groups (3–4 students in one group) to play the game. Each activity lasted about 30 min. In the first five activities, a tutor led students to discuss playing skills based on certain topics in the first 10 min, as can be seen from Table 2. And the difficulty of the game would be increased step by step, during which the knowledge points were delivered to students in a more interesting way.

**Figure 3.** Photos of game activities.


**Table 2.** Discussion topics.

#### *2.5. Data Analysis*

SPSS 19.0 was used for data analysis. Due to the small sample size, Wilcoxon rank test was used in the analysis of the teaching effect of the board game. The Mann–Whitney rank sum test was used to analyze the differences of learning effects among students with different gender and physical indicators.

#### **3. Results**

#### *3.1. Students' Change in Knowledge, Dietary Attitude, and Behavior*

Food nutrition knowledge test contains three parts: food and nutrient, food pyramid and dietary structure, and food safety and package, with grades of 14 points, 4 points and 5 points, 23 points in total. As can be seen from Table 3, students' total knowledge grades increased significantly after the 6-week intervention. In specific knowledge points, students had significant increases in two dimensions: food and nutrients and food safety and package. Most students had better command of nutrient deficiency, what food contained high level of nutrients, and food pyramid structure. In contrast, for some questions that needed comparison (e.g., which of the following foods has the highest calorific value?) and questions about GI that had lower prevalence in the game, students showed less improvement.

**Table 3.** Students' changes in nutrition knowledge.


Note: \* *p* < 0.05.

Dietary attitude scales contained three dimensions: the awareness of the importance of a healthy diet, self-efficacy of the command of nutrition knowledge, and the willingness to adjust dietary behaviors. As can be seen from Table 4, students significantly increased scores in importance awareness and self-efficacy, but there were no significant changes in willingness to adjust.

Figure 4 shows students' intake frequency of different types of food before and after the game activities. Breakfast, fruits, vegetables, milk, eggs, and coarse grains are recommended for higher intake frequency (every day); candy, puffed food, sugary drinks, and fast food, which have little nutritional value, are recommended for lower intake. As can be seen from the Figure 4, students already had a relatively reasonable dietary structure. Approximately 80% of students had breakfast, fruits, and vegetables more than 5 days per week. Among recommended food, eggs and coarse grains were two kinds of food with

relatively low intake frequency. More than 60% of students ate candy, puffed food, and fast food less than 2 days per week. Among not recommended food, sugary drinks had higher intake frequency.

**Table 4.** Students' change in dietary attitude.


**Figure 4.** Intake frequency of different types of food in pre- and post-test. Note: for the same kind of food, the upper stripe shows pre-test result and the lower stripe shows post-test result.

After the 6-week intervention, the number of students who ate breakfast and fruit with higher frequency had increased. For vegetables, the number of students who ate it every day had decreased, but the overall change was not large. For milk, there was a small increase in the number of students who drank it every day or with low intake frequency. The number of students who ate eggs with higher frequency increased. There was little change in the intake of coarse grains. Among food not recommended, the average intake frequency of sugary drinks and fast food had decreased. The intake frequency of puffed food and candy had only a slight fluctuation.

Dietary behavior scales contain three dimensions: balanced intake of nutrients (students make some food paring adjustments between meals to achieve balanced intake of nutrients), food intake frequency (students increase the intake of recommended food such as fruits and vegetables and reduce the intake of not recommended food), and reading food labels (students pay more attention to food labels when buying goods).

As can be seen from Table 5, students significantly increased their scores in food intake frequency and reading food labels, which corresponded to the result of Figure 4 to some extent.

163

#### *3.2. Students' Change in Delay Test*

The delay test was designed to investigate whether students' attitude and behavior would be further improved over time. In terms of attitude, there were not significant changes in the scores of the delay test compared with the post-test. However, students' scores of willingness to adjust (4.61 ± 0.64) were significantly higher than those in the pre-test (Z = 2.32, *p* < 0.05). In terms of behavior, there were not significant changes in scores of balanced intake of nutrients and food intake frequency between the delay test and the post-test. Scores of reading food labels (4.09 ± 0.75) had significant decrease compared with those in the post-test (Z = 2.71, *p* < 0.01).

**Table 5.** Students' change in dietary behavior.


Note: \* *p* < 0.05, \*\* *p* < 0.01.

#### *3.3. Difference in the Effects of Board Game on Students with Different Gender and BMI*

Due to the small sample sizes after the split of all students, there may be some error in the analysis. As can be seen from Table 6, there was no significant difference in knowledge scores between boys and girls. In terms of willingness to adjust, only girls had significant increasing scores, with little change on the level of all students. In terms of other dimension of attitude and behaviors, boys and girls had different changes. In the dimension of importance awareness and food intake frequency, only girls had significant improvement. In the dimension of self-efficacy, only boys had significant improvement.


**Table 6.** Differences in the effects of board game on students with different gender.

Note: \* *p* < 0.05.

According to BMI standards, normal BMI ranges from 18.5 to 24. A BMI of more than 24 means overweight, and a BMI of less than 18.5 means underweight. Students were divided into two groups: normal BMI group (9 students) and abnormal BMI group (13 students). As can be seen from Table 7, the abnormal BMI group had significant increasing scores in multiple dimensions, including self-efficacy, food intake frequency, and reading food labels, whereas there was little change in the normal BMI group.

In the pre-test, scores of self-efficacy in the abnormal BMI group were significantly lower than that in the normal BMI group (z = 2.31, *p* < 0.05), and scores of willingness to adjust were significantly higher (z = 1.98, *p* < 0.05).


**Table 7.** Differences in the effects of board game on students with different BMI.

Note: \* *p* < 0.05.

#### **4. Discussion and Conclusions**

#### *4.1. Discussion*

4.1.1. Effects of the Board Game on Students' Knowledge, Attitude, and Behaviors

Nutrition knowledge that teenagers need to master is relatively simple compared to other science subjects. Many nutrition education programs using game-based teaching to improve students' knowledge to varying degrees. Knowledge improvement is also related to the difficulty. In the Giocampus program [24], the research carried out a nutrition education program containing different topics with different difficulty levels in the third to fifth grades of a primary school and found there was no significant change of scores among senior students on many types of questions. That is mainly because senior questionnaires contained difficult questions (e.g., about carbohydrates and fiber). In this study, after a 6-week intervention, students' total knowledge scores had significantly improved, but when focusing on certain knowledge points, there were no significant changes in food safety and package. That is mainly because the different kinds of knowledge points are not evenly distributed, such as food poisoning and GI value, which are only present in examinations with a high level of difficulty. Students had fewer opportunities to use this knowledge, thus they saw little improvement. In addition, students are more apt to get higher grades in qualitative judgment questions (e.g., what nutrients are rich in certain food? What is nutrient deficiency?) and get lower grades in quantitative comparison questions (e.g., comparing the food calories), mainly because qualitative knowledge is easy to perceive due to the promotion mechanism. Although quantitative thinking is involved in the game, students are more easily impressed by foods with large energy differences.

In nutrition education, it is more important to improve students' dietary attitude and behaviors rather than just delivering knowledge. Therefore, it is necessary to motivate students' intrinsic motivation, promote their transformation from attitude to behavior, and guarantee the program's duration. Previous studies have found that the short duration of educational programs is a major obstacle to teaching effectiveness [25]. The gamebased learning method can give students a better sense of immersion and can be used as a long-term teaching activity. In this study, significant improvements of attitude and behaviors were only found in the formal study, but not in pre-study, which also prove that lengthening the duration of the program can help to promote students' transformation of attitude and behaviors. In addition, teachers' guidance before each activity was added to deepen students' cognition of dietary influence in the formal study. Additionally, the number of people in each group decreased from five to three, which meant the participation time of each player increased. That corresponds to some previous studies that showed reducing the number of study groups helped to enhance students' participation and made them more likely to be driven by positive teammates [26].

In general, students had significant improvement in some dimensions of behaviors. The main changes were that the intake frequency of breakfast and eggs increased, whereas

the intake frequency of fast food and sugary beverages declined. The intake frequency of vegetables and fruit was relatively high, so there was no significant change between preand post-test. In the interview, students mentioned they usually had lunch and dinner at the school canteen, so the intake of vegetables and fruit could be guaranteed without much change, which was consistent with Roszanadia's findings [27].

It can be concluded that students' behavior transformation concentrated on some types that are easy to adjust. For example, as reading food labels is easy, students' scores significantly increased in the post-test. However, they did not have significant change in the balanced intake of nutrients, because this dimension was more difficult, requiring students to have a more comprehensive understanding of their dietary structure and food characteristics. It still needs more time and motivation for further promotion, such as the enriching incentive mechanism of the game.

Some studies also found the influence of game-based teaching may continue to play a role in students' later learning and life, showing a significant improvement in the delayed test [25]. In the delay test of this study, students' scores of willingness to adjust had significant enhancement compared to that in the pre-test, because students may put ideas from the game into the reality and deepen their cognition, which was also noted in the interview. Scores of other dimensions of the delay test basically had no significant enhancement, but they also showed a long-standing effect.

4.1.2. Differences in the Effects of the Board Game on Students with Different Gender and BMI

Some studies found that factors such as gender, economic level, and physical quality would influence the teaching effect of the educational game. For example, Ogunsile and Ogundele [28] found girls got higher KAP scores than boys in an educational board game. There was a similar finding in this study—that girls got higher scores in both the pre- and post-test—though without significance. In the terms of attitude and behaviors, boys and girls had different changes. For instance, only girls had significant improvement in the willingness to adjust and food intake frequency. As junior school students have entered puberty, girls develop their secondary sexual characteristics earlier than boys and they may pay more attention to their appearance. They had learned that nutrition intake had much to do with body shape, so girls may be more easily influenced by the nutrition knowledge and therefore adjust their behaviors.

Teaching effects also varied on students with different BMI ranges. Underweight and overweight students had significant increasing scores in self-efficacy and food intake frequency, whereas students with normal BMI value had no significant change. One reason may be that students with abnormal BMI got lower scores in the pre-test, so they had larger room for improvement. Another reason may be that the guidance progress emphasized the relationship between food intake and energy transformation and the adverse consequence of fat accumulation in the game, which were more likely to call the attention of students with abnormal BMI values. Jennings [29] also found overweight students had stronger willingness to adjust their dietary behaviors, although they did not command good nutrition knowledge in a nutrition summer camp.

#### *4.2. Conclusions*

The board game is designed within the domain of chemistry, particularly nutrition. By incorporating game elements into crucial concepts of nutrition, the students are encouraged to understand the concepts and elements and explore the relationship connected within. The knowledge, awareness, and self-efficacy of the students have been significantly enhanced, although the changes happened unevenly for boys and girls and for students of different BMI. It is still not easy to translate the knowledge to dietary attitude and behaviors.

As nutritional knowledge might be deficient for children and adolescents, board games can help to impart the knowledge in an entertaining format, which provides an alternative to chemistry classes. There is limited evidence for the effectiveness of board games for nutritional education. Further research is warranted to evaluate the long-term effectiveness of board games on nutritional knowledge and dietary behavior.

#### **5. Limitations and Prospects**

Due to the small sample size of this study, the result may be not representative. Additionally, the intervention was only carried out for 6 weeks, but it was still short for transformation of attitude and behaviors.

Through the evaluation of teachers and students, this board game still has much space to develop:


**Author Contributions:** Writing—original draft preparation, S.W.; writing—review, and investigation; Z.T.; visualization and editing, F.-K.C.; project administration and supervision F.-K.C. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Institutional Review Board Statement:** This research was carried out according to ethical guidelines. No conflict of interest exists in the submission of this manuscript, and the manuscript is approved.

**Informed Consent Statement:** Informed consent was obtained from all subjects involved in the study.

**Data Availability Statement:** The data that support the findings of this study are available from the corresponding author upon reasonable request.

**Acknowledgments:** We acknowledge Beijing Academy for their support for the experiment.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**


## *Article* **Online Escape Room during COVID-19: A Qualitative Study of Social Education Degree Students' Experiences**

**Ana Manzano-León, José Manuel Aguilar-Parra \*, José M. Rodríguez-Ferrer \*, Rubén Trigueros, Rocío Collado-Soler, Cristina Méndez-Aguado, María Jesús García-Hernández and Laura Molina-Alonso**

> Department of Psychology, University of Almería, 04120 Almería, Spain; aml570@ual.es (A.M.-L.); rtr088@ual.es (R.T.); rcollado98@gmail.com (R.C.-S.); cristinamendezaguado@gmail.com (C.M.-A.); marigahe@hotmail.com (M.J.G.-H.); laura97eduso@gmail.com (L.M.-A.) **\*** Correspondence: jmaguilar@ual.es (J.M.A.-P.); joserf@cop.es (J.M.R.-F.)

**Abstract:** Confinement due to the COVID-19 pandemic has hastened an educational shift from face-to-face to online classrooms. This distanced education seeks to achieve learning goals mediated by technology as they would be achieved in the face-to-face classroom, without ignoring the psychological and social impact that COVID-19 has had on students and teachers. Faced with this situation, the use of online educational escape rooms has been proposed as a motivating strategy for students to review curriculum content in a cooperative and fun way. A qualitative investigation was carried out to explore the perceptions of university students in the Social Education degree program after the implementation of an educational escape room. Our main findings are that most students found that it allowed them to interact with their peers beyond traditional education, that it was useful for their learning and that it was a pleasant activity. However, it was also mentioned that it can be a stressful activity as being an online activity, some students may have connectivity problems. It is concluded that online escape rooms can be active and effective learning strategies for university students.

**Keywords:** escape room; educative innovation; cooperative learning; higher education

#### **1. Introduction**

During the period of health emergency and confinement caused by the COVID-19 pandemic, the educational system from early childhood education to university education has had to adapt its face-to-face methodology to an online methodology in a hasty and forced manner [1]. The transition to that virtual model was the only opportunity to continue teaching. However, it was limited by different factors, including the digital divide across students, the lack of digital resources and the scarce training of teachers in digital skills [2].

In addition, the witnessing of a global pandemic and the confinement have been a challenge for the mental health of both teachers and students due to increased feelings of uncertainty, stress, and anxiety [3]. These feelings can create unfavorable effects on learning and psychological health [4].

To try to alleviate these negative effects on the mental health of students and promote motivating teaching, it is necessary to implement active learning strategies that can be recreated in online teaching. This research explored the perceptions of university students regarding the use of a playful learning strategy, educational escape rooms, during confinement in Spain in the second quarter of the 2020–2021 academic year. Escape rooms are immersive narrative games in which participants are required to solve puzzles to escape from a room. Escape rooms are currently a popular leisure activity for young people. A global survey found that 19% of escape room players are young, under 21 years old [5], so it could be an interesting activity to adapt it and use it in college education. Escape rooms are an innovative and playful educational strategy that is increasingly incorporated into education and research. Their main benefit is that they can facilitate motivation toward learning through playful challenges.

**Citation:** Manzano-León, A.;

Aguilar-Parra, J.M.; Rodríguez-Ferrer, J.M.; Trigueros, R.; Collado-Soler, R.; Méndez-Aguado, C.; García-Hernández, M.J.; Molina-Alonso, L. Online Escape Room during COVID-19: A Qualitative Study of Social Education Degree Students' Experiences. *Educ. Sci.* **2021**, *11*, 426. https://doi.org/10.3390/ educsci11080426

Academic Editor: Huei Tse Hou

Received: 22 July 2021 Accepted: 10 August 2021 Published: 12 August 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

Escape rooms have a direct relationship with gamification due to their playful nature that favors positive behavior [6]. Different systematic and meta-analysis reviews indicate that this playful strategy has an overall positive effect on student learning, participation, and academic performance, as well as an improvement in school motivation [7–9].

Mainly in university education, research has been carried out in health sciences, particularly in nursing, showing that escape rooms can be very effective in consolidating routines, concepts, and basic procedures for professional development and at the same time, teach important skills to deal with stressful situations under time pressures [10–12]. They are also used in other areas of knowledge such as social sciences. For example, after conducting an escape room, master's students showed a statistically significant improvement in their academic performance, educational flow, and classroom climate, as well as qualitatively confirming that escape rooms can be fun and motivating for students [13]. However, escape rooms, like other playful strategies such as game-based learning or gamification, have possible disadvantages to consider, such as potentially consuming too much time, excessive competitiveness between students, too much noise can be generated or the playful content can exceed educational objectives [14].

Recent studies on their educational use during the pandemic confinement offer promising results with some limitations. For example, the study carried out by Da Silva [15] showed that the use of an interactive game-based application created a pleasant learning environment, although both the control group (that conducted their class in a traditional way) and the experimental group obtained similar results in learning outcomes. On the other hand, the use of gamified digital platforms such as Classcraft were also effective in improving student engagement during confinement, however, in the long term, students may have difficulties participating with asynchronous tools [16].

Other studies indicate that educational escape rooms under normal conditions have benefits in the motivation, engagement, and learning outcomes of the students [17–19]. For this reason, we proposed a qualitative research study that explores the reflections of university students in educational sciences on the use of escape rooms to review curricular content during the confinement of COVID-19. Our research hypotheses are: (1) the students will consider the escape room fun and motivating; (2) the escape room will have a positive impact on their learning; and (3) the escape room is going to be an effective strategy for online teaching.

#### **2. Materials and Methods**

#### *2.1. Participants*

This study implemented an educationally innovative program through escape rooms in the subject of "Socio-educational Programs in Children, Youth, Adults and Seniors" in the degree of Social Education at the University of Almería during the 2019/2020 academic year. This subject is part of the annual training itinerary for the degree in Social Education, which is divided into 5 subjects per semester. The main objective of this subject is to explore and design socio-educational programs for different groups in situations of social vulnerability.

This subject is divided into a theoretical part where the teacher traditionally exposes the curricular contents and a practical part where the students delve into the design of social programs. The objective of using an escape room is to sensitize students to gender violence in a relationship and learn about programs to prevent or alleviate gender violence, since gender violence is a topic of the subject.

This escape room was designed so that the students of this subject could participate voluntarily. The teams that participated in the escape room got 0.25 extra points in the practices of the subject.

To select the participants, a convenience sample was chosen. The inclusion criterion for the experimental group was a willingness to participate in the escape room organized by the teaching staff. A total of 56 students (41 women and 15 men, aged between 19 and 44 years) participated in the escape room and voluntarily answered the online survey of open questions.

All participants received information about the project and gave written informed consent in accordance with the Declaration of Helsinki. Prior to data collection, the students were informed about the nature of the study and were assured of their anonymity. Ethics approval was obtained from the Research Ethics Committee of the University of Almería (Ref. UALBIO 2021/01).

#### *2.2. Design and Implementation of the Educational Escape Room*

In the second semester (a four-month period) of the 2019/2020 academic year, home confinement occurred in Spain, therefore all theoretical and practical teaching was carried out online. For this reason, the escape room was designed online, with the Wix and Genially platforms, and was implemented cooperatively through the Blackboard teaching platform.

The escape room (see Figure 1) consisted of 4 rooms in a family home where each team of students determined what happened since María (a fictional character) disappeared. After several puzzles and challenges, the narrative is resolved when the students learn that María has managed to report her abusive husband and is in a safe shelter together with her young son. The escape room is available at the following link: https://diversatics.wixsite. com/escapeduso (accessed on 11 August 2021).

**Figure 1.** Escape room design.

In the class schedule, it was explained to the students that they were going to complete an escape room and what is involved. The students were randomly divided into teams of 4 to 6 players in the online work rooms and the escape room link was provided. The teacher acted as a game master for any doubts that students had throughout the escape room.

#### *2.3. Instruments*

After conducting the escape room, the students were asked to voluntarily answer a qualitative online questionnaire with open questions [20]. An ad hoc questionnaire with six open questions was designed, with the Google Forms platform, to find out the opinions of the students about their experience in the escape room. This format was used to respect anonymity so that students could freely express their opinions regarding the escape room. The general rules of the escape room were:


Table 1 summarizes the challenges for each room, mechanics, and related learning objectives.



With these playful challenges, it was intended that students reinforce this part of the subject's content (gender violence) through an attractive experience. Previous research shows how escape rooms, thanks to their challenging design and cooperative learning capacity, facilitate student learning [21,22].

#### *2.4. Data Analysis*

An online open question survey was proposed for the students to voluntarily answer once they had completed the escape room. The data analysis was carried out through content analysis based on grounded theory [23]. As the survey was online, the transcription was automatic and was introduced in the ATLAS.ti software for Mac (version 9, ATLAS.ti Scientific Software Development GmbH, Berlin, Germany). An in-depth analysis of categories with labels was carried out. Related codes were grouped into subtopics, and overlapping subtopics were grouped together (See Figure 2).

#### **3. Results**

After the analysis of the interviews were conducted with the students, the following categories were identified (see Figure 2 and Table 2).



**Table 2.** *Cont.*

#### *3.1. Fun Experience*

Escape rooms have been established as a recreational leisure mode and their use in education is beginning to be investigated due to students' interest. Based on the opinions of the students, it can be affirmed that the majority of students perceive the use of educational escape rooms as a playful and immersive experience, coinciding with previous research [13,19,24].

The students considered the different challenges to be balanced with their abilities and to be a pleasant challenge for them, thus they had fun and were motivated to get out of the escape room. This translated into a greater educational flow [25] and in turn, to greater participation and engagement [26], as mentioned by the students. For example, a student stated that "(The escape room) has been very dynamic. It makes you focused on getting out on time and solving the case, you want to know what happened to the woman and you understand the background of the character and feel relieved when you find out that she is fine". The fact that the escape room had a narrative related to the profession of the students encouraged a greater connection and participation.

Positive emotions have a direct relationship with fun and engagement [27]. The students primarily mentioned the enjoyment of completing the escape room. Previous research has found that playful strategies connect with students and can make them enjoy the teaching [28]. When a situation is pleasant and excites students, it encourages their commitment. On the contrary, some students mentioned that they felt nervous or confused during the escape room, which is an aspect for improvement when designing these strategies in the classroom. To reduce these feelings, an initial tutorial could be added on how escape rooms and clues work in each of the rooms, just like other escape room-themed board games.

#### *3.2. Learning and Teamwork*

This research aimed to explore the use of escape rooms as an educational strategy to strengthen curricular knowledge in university education. The students considered that in addition to the fun that an escape room can produce, it can also be an effective tool to raise awareness about a certain topic, in this case gender violence, and strengthen the theoretical knowledge taught in the subject.

Additionally, working in teams was well-received by the students. The cooperation and relationships between classmates were highly valued. They affirmed that they felt cohesive and enjoyed working in a group, for example: "It has been easy to coordinate as a team, it has been a long time since we met to do things together in class and I liked going back to that" and "I liked learning to play escape room, and more with my classmates, like this it was very enjoyable to learn and review what we saw in class".

#### *3.3. Possible Improvements*

During the interview, students were asked to mention what they did not like about the escape room or what could be improved. It should be noted that some of the students said that there was nothing to improve. However, any educational activity has a range of improvement. When designing educational escape rooms, it must be taken into account that it is likely that some students have not completed an escape room before or have not played a game for years, making it necessary for the explanation, prior to beginning the escape room, of what they have to do and what they can expect from a team escape room. Still, the perceptions of some students showed that they felt stress or fear of failing from not knowing how to continue without continuous cues.

Technical difficulties are also mentioned regarding hardware (problems with headphones to talk to other colleagues) and software (slow internet connection and problems loading web pages). This has been a big problem for tele-teaching, especially during the confinement period. The university educational system particularly has had problems adjusting from face-to-face teaching to 100% online teaching and some students did not have the necessary resources for its adequate implementation. The lack of resources and the digital divide have been great challenges during the confinement by the COVID-19 pandemic [29]. One of the reasons for conducting the escape room cooperatively was so that the students could help each other if there was a problem, being able to share their screen and carry out the missions as a team. However, it would be highly recommended to be able to offer technical solutions to the students on top of this to facilitate online teaching.

#### **4. Discussion and Conclusions**

The main purpose of the application of an online escape room in a university degree course was to offer an innovative learning experience different from the traditional classroom during the COVID-19 pandemic. Designing active learning experiences for students during forced online classes was and continues to be a challenge for university faculties.

The objective of this research was to analyze the perception of the students of the Social Education degree program on the use of educational escape rooms during the 2019–2020 academic year. For this, an escape room was designed related to its content on gender violence in the subject of "Socio-educational Programs in Childhood, Youth, Adults and Seniors", based on a case of gender violence during confinement due to COVID-19 restrictions.

Firstly, it sought to provide new learning experiences that are motivating for students, with the aim of producing positive feelings about the subject to increase their participation, beyond connecting to the online class. This escape room was also made to simulate a real case that they could have as social education professionals, so they have interactive resources that could be useful for their professional future. Previous studies mention the importance of teaching practical content to university students and the benefits of simulations [30], as well as the playful and motivating nature of educational escape rooms [31]. Our results confirmed that escape rooms in university education can awaken the curiosity, participation, and motivation of students. Coinciding with our results, the motivation achieved in the escape rooms has been one of the main advantages found in most of the research on this strategy [32–34].

In addition, the cooperative character of the game allows students to work on other essential competencies in their professional future, communication, and teamwork. The students widely emphasized that they enjoyed working as a team with other classmates. Facilitating activities where students must cooperate favors their social skills such as communication, empathy, respect, and negotiation [35]. Our results coincide with previous studies that indicate escape rooms can be valid tools to promote teamwork, favoring

interaction between students in small groups, and working on the curricular content in a playful and cooperative way [36].

In general, the students perceived this activity as pleasant and educational, except for a small percentage who suggested that they felt nervous during the activity. In future applications of the escape room, we will try to offer a better initial orientation and offer more feedback during the activity so that participants do not feel incapable of solving the puzzles. This limitation of the tool has been found in other investigations, for example, after the application of an escape room in the degree program of Social Education, the majority of students gave a positive assessment to the activity, however a minority of participants reported having suffered stress and frustration [37].

This study has limitations. First, as it was not an interview, it was not possible to delve into the students' responses that could offer a greater perspective and reflection on the use of escape rooms in the classroom. Second, being a qualitative study and having a relatively small sample, the results are not generalizable.

Future research could study the quantitative impact of escape rooms on academic performance, motivation, and the acquisition of social skills through longitudinal experimental studies with a control group. In addition, its use in combination with other active learning strategies, such as cooperative learning and project-based learning, could be investigated.

This research was concluded by evaluating the use of educational escape rooms specifically our escape room for the degree program of Social Education—to strengthen the knowledge of university education students and to value them as a motivating strategy, which favors cooperative learning and encourages participation and engagement, even in such adverse circumstances as confinement due to COVID-19.

**Author Contributions:** Conceptualization, A.M.-L. and J.M.R.-F.; methodology, J.M.A.-P.; formal analysis, L.M.-A. and M.J.G.-H.; investigation, M.J.G.-H.; data curation, J.M.A.-P.; writing—original draft preparation, A.M.-L., R.C.-S. and C.M.-A.; writing—review and editing, A.M.-L.; visualization, R.C.-S. and C.M.-A.; supervision, R.T. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research has been funded, in part, by the Ministry of Universities through the University Teacher Training Program (FPU2019). The support of the Ministry does not imply acceptance of its contents, which is the sole responsibility of the authors.

**Institutional Review Board Statement:** The study was conducted according to the guidelines of the Declaration of Helsinki and approved by the Institutional Ethics Committee of University of Almería (UALBIO 2021/01).

**Informed Consent Statement:** Informed consent was obtained from all subjects involved in the study.

**Data Availability Statement:** The data that support the findings of this study are available from the corresponding author upon reasonable request.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**


### *Systematic Review* **Gamification in Physical Education: A Systematic Review**

**Víctor Arufe-Giráldez 1,\*, Alberto Sanmiguel-Rodríguez 2, Oliver Ramos-Álvarez 1,3 and Rubén Navarro-Patón <sup>4</sup>**


**Abstract:** Background: In the last 10 years, gamification has entered the educational field incrementally. The subject of Physical Education has been one of the scenarios where multiple gamified learning environments were carried out. The objective of this work was to evaluate and analyze the scientific evidence of the pedagogical proposals and didactic experiences that have used gamification in the Physical Education classroom in Kindergarten, Elementary School and Middle, Junior and High School. Methods: A systematic review has been carried out following the recommendations set by the PRISMA Declaration. A total of five international databases were used: Web of Science (WoS), Scopus, Sport Discus, ERIC and Psycinfo. The descriptors "gamification", "gamify" and "Physical Education" were used, limiting the search to December 2021. Several inclusion and exclusion criteria have been established, selecting only empirical research articles. Results: The search yielded a total of 177 eligible articles, and finally, 17 scientific articles that addressed the effects of gamification in Physical Education were selected. No gamified didactic experiences have been found in Early Childhood Education, but they have been found in Elementary School (7 experiences) and Middle, Junior and High School Education (10 experiences). Most of the studies have confirmed an improvement in motivation and commitment toward physical exercise in students; only one study has confirmed improvements in academic performance. The diversity of the applied protocols and the different evaluation instruments used by the researchers prevent a meta-analysis of the data. Some studies that have used a hybrid pedagogical model are recorded, combining gamification with other pedagogical models, and confirmed positive effects on different variables such as intrinsic motivation or autonomy in learning. Conclusion: The results of this review suggest the need to continue evaluating the effects of applying gamification, as an active methodology, in the Physical Education classroom.

**Keywords:** gamification; physical education; pedagogical models; active methodologies; motivation

#### **1. Introduction**

A recent literature review on what quality Physical Education is like concludes, among other aspects, that it should use active methodologies [1].

Active methodologies are those that seek greater learning by students, enhancing their sociability and teamwork, meaningful learning and critical thinking, and learning interactivity [2].

The student becomes an active part in the process of building knowledge, following the principles of neuroeducation: experimentation, curiosity, emotion, motivation and attention [3]. On the other hand, the teacher assumes the role of a guide or facilitator of learning. In recent years, active methodologies have captured the attention of many researchers and adopted an exponential growth evolution [4]. In addition, multiple benefits are confirmed, in its implementation, not only in face-to-face education and in different areas of knowledge and educational stages [5–9] but also in non-attendance [10]. Among

**Citation:** Arufe-Giráldez, V.; Sanmiguel-Rodríguez, A.; Ramos-Álvarez, O.; Navarro-Patón, R. Gamification in Physical Education: A Systematic Review. *Educ. Sci.* **2022**, *12*, 540. https:// doi.org/10.3390/educsci12080540

Academic Editor: Huei Tse Hou

Received: 18 July 2022 Accepted: 8 August 2022 Published: 10 August 2022

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

the so-called active methodologies is gamification, in addition to Inverted Learning, Project-Based Learning and Game-Based Learning, among others.

Educational gamification is a technique that consists of transferring the mechanics of games and video games to the educational field, with the aim of seeking behavior modification. Thus, it creates attractive and interesting didactic experiences to increase the motivation of the students, their commitment and learning of the contents of the subject or the enjoyment of the pedagogical tasks themselves, always using the motivational elements of the games [11].

Well-planned and correctly implemented gamification in the classroom generates in students a certain sense of control and assumption of responsibility in the teaching–learning process. Being able to also contribute effectively so that students focus and enjoy more in the construction of learning than in studying just to pass the exam [12].

The latest literature reviews carried out by different authors on gamification in education confirm an annual increase in the number of publications on this subject [13–17] and a focus on three aspects: the use of video games in educational settings, the effects of technology on learning and the study of flipped classroom experiences [18]. One of these reviews [16] used the scientific mapping method, visualizing that gamification has been implemented in different areas, but highlighted the educational field. This has attracted the largest number of investigations related to improving the motivation, commitment and performance of students participating in gamified experiences, as well as the study of gamification combined with other technologies, such as social networks, virtual and augmented reality or mobile applications.

It should also be noted that the application of gamification in the Physical Education (PE) classroom is compatible with other active methodologies and/or PE pedagogical models that have shown positive effects on student learning, such as cooperative learning, service learning, sports education, adventure education or the movement-oriented practice model, among others, addressed by multiple authors [19–24]. Even some researchers [25,26] confirm greater advantages in the hybrid implementation of pedagogical models over isolated implementation, justifying that the former can promote results in many different domains, overcoming the limitations of individual pedagogical models.

Due to all these antecedents, the objective of this work was to carry out a systematic review of the empirical research articles that have addressed the effects of the creation of gamified learning environments (GLE) in the PE classroom in Kindergarten, Elementary School and Middle, Junior and High School. This article seeks to compile and synthesize, in a single document, all the results of original research related to gamification in PE that were published until 2021 (inclusive) in peer-reviewed scientific journals indexed in five leading international databases.

#### **2. Materials and Methods**

A systematic review has been carried out on the creation of GLE in PE in the different educational stages. Given the heterogeneity of the nomenclature of the educational stages in the educational systems of countries around the world, in this study, it has been decided to establish 3 school educational stages based on the age range of the subjects of the research sample, thus establishing the following: Kindergarten (0–6 years), Elementary School (6–12 years) and Middle, Junior and High School (12–18 years). Systematic review has been chosen because it is a type of research through which researchers summarize evidence in a certain field of knowledge or topic, using a rigorous process (to minimize biases) through which the studies are identified, evaluated and synthesized in order to respond to the research objective and establish the main conclusions about the analyzed documents [27].

#### *2.1. Protocol and Registration*

The recommendations of the PRISMA Declaration [28] were used to carry out this systematic review. A total of 27 items indicated by said declaration were fulfilled.

#### *2.2. Eligibility and Risk Criteria*

With the aim of evaluating the possible risks of biasing the information, the following inclusion criteria have been developed (Table 1), applied in the search phase of the manuscripts and in the phase of preparing the results.


**Table 1.** Publication selection process and inclusion and exclusion criteria.

#### *2.3. Information Sources*

To ensure the correct choice of information sources, it was decided to include 5 international databases in the search, arguing the following reasons:


#### *2.4. Search*

The search was carried out in January 2022. The scientific articles that resulted from the combination of the following descriptors were included: ["Gamification" OR "Gamify"] AND "Physical Education"; selecting as search fields: title, keywords, abstract or subject. The search deadline was set to 31 December 2021. Subsequently, all the references that were extracted were uploaded to the Proquest© Refworks bibliographic manager, where the filtering was carried out to find duplicates, and the registered articles were filtered.

Five phases of the systematic review were established. In the first, the databases in which the search for bibliographic references on gamification and PE would be carried out were determined and the terms to be searched and the combination of Boolean operators were agreed upon. In the second phase, the articles resulting from the application of the first one were selected, incorporating the inclusion criteria 1.a., 1.b., 1.c., 1.d and 1.e and obtaining a total of 177 eligible papers. In the third phase, the articles were transferred to the Refworks bibliographic manager for the purification of the files and elimination of duplications, leaving a total of 87 articles after applying the exclusion criterion 2.e.

In the fourth phase, all the papers were read carefully and the exclusion criteria 2.a., 2.b., 2.c. and 2.d were applied, resulting in a final sample of 17 articles. Among the articles that were rejected are gamified didactic proposals that were not carried out, gamified didactic experiences in PE but in the university stage, didactic experiences in PE but outside the school teaching environment (not in PE class) and others causes (Figure 1).

**Figure 1.** Flowchart of the review on gamification in Physical Education.

#### *2.5. Study Selection*

Once the filtering was performed, each of the references was carefully read, analyzing the title, abstract and full text. In the case of articles whose full text was not open access, the authors were contacted to request a copy of the manuscript. Contact with the authors was established through their institutional email or through the international research platform Researchgate. All studies that did not meet any of the inclusion criteria were excluded. A total of 70 scientific articles were eliminated. The selection of the studies was carried out by three researchers independently, subsequently agreeing on their selection based on the inclusion and exclusion criteria previously established in Table 1.

#### *2.6. Data Extraction Process and Listing of These*

All the data of the articles were incorporated into an Excel sheet, breaking down the information into different categories. The data dump process to the Excel file was carried out by two researchers and a third researcher acted as reviewer/auditor.

A total of 5 broad categories of analysis were established, with a total of 30 subcategories listed in Table 2.

**Table 2.** Categories and subcategories established for the analysis of the articles of the systematic review. Note: Category 1. Publication Data; Category 2. Research Design; Category 3. Characteristics of the Sample; Category 4. Characteristics of the Gamified Learning Environment; and Category 5. Objectives and Results.


#### **3. Results**

To present the results of the articles that have been incorporated in this review on the creation of PGL in PE, it has been chosen to group the works according to the educational stage where gamification was applied. Initially, and in order to contextualize all the research, a table (Table 3) is presented with the articles found in each of the educational stages. Subsequently, a narrative description of the main contributions and/or findings of each of the scientific articles is made, accompanied by a summary table with some of the categories analyzed. A meta-analysis could not be performed due to the lack of uniformity in the protocols of the analyzed studies and the impossibility of calculating the effect size.


**Table 3.** Number of articles that addressed research in Physical Education and gamification.

Note: The study by Fernández-Río et al. [23] mainly addresses the Elementary School stage but also incorporates a small sample with ages from Middle, Junior and High School.

Given that the names of the different educational stages may vary from one country to another, depending on the type of educational system it has, the age group to which each stage refers has been included so that the data from the studies can be extrapolated to the educational context of different countries.

#### *3.1. Gamification in Physical Education in Kindergarten (0–6 Years)*

At this stage, no research articles have been found that address the creation of a GLE in PE.

#### *3.2. Gamification in Physical Education in Elementary School (6–12 Years)*

In the field of Elementary School, a total of seven scientific articles have been found (Table 4). The vast majority of them investigate the creation of a GLE with the aim of improving motivation and commitment toward the practice of PE on the part of students. A study analyzed the possible impact of gamification on the academic performance of students, finding an improvement in this. The study by Parra-González et al. [33] found that Elementary and Middle, Junior and High School students who had a gamified learning environment obtained better scores in the student–student relationship, autonomy, collaboration and resolution dimensions than pre-university students.

The total sample size for studies with a mixed design (quantitative and qualitative) was *n* = 516 and *n* = 142 for quantitative. Three investigations had a control group and an experimental group, four with pretest and post-test variables measurements and three studies presented only an experimental group. As for the most present elements of gamification, there are points and badges, and a single article talks about the narrative. A great diversity of the evaluation instruments used and the means to evaluate the objectives are also detected, from discussion groups and portfolios to ad hoc questionnaires or validated questionnaires, among other instruments. The duration of the GLE ranges from 1 month to a full school year.



gamification

 in PE in Elementary

 School.

**Table 4.** Analysis of the articles that provide scientific evidence on


**Table 4.** *Cont.* of the same variables after a short, medium or long period of time. The

Parra-González

 [33] study is a work that also addresses Middle, Junior and High School.

3.2.1. Improved Commitment and/or Motivation to Practice PE and/or Physical Activity

The article developed by Bellamy [29] focuses on a Primary Education school in Carmarthenshine, Wales, which partnered with an e-commerce company to develop and improve PE and the school gym through a health and fitness gamified program. In this program, they provide rewards using a badge system to enhance the development of children's motor skills.

Another investigation with a quasi-experimental design carried out in six children with sensory integration dysfunction assesses the efficacy of a sensory and motor stimulation program based on the use of video games that promote motor skills [30].

The authors, Fernandez-Río Javier et al. [23], present an interesting quantitative and qualitative study where they work with a sample of 290 students of Primary and Secondary Education, creating a GLE based on the theme of superheroes during 30 sessions distributed over 15 weeks. PE teachers from four educational centers are involved in the gamification and there is an increase in the intrinsic motivation of the students. Four of the five elements identified as central to promoting meaningful experiences in PE and sport are present in this work [44], these being social interaction, fun, challenge and learning, lacking the analysis of motor competence. They also highlight that the teachers expressed that the gamified process involved a large workload. Parra-González et al. [33] highlight that there is a greater motivation for students to use gamification in Primary Education compared to the last years of Secondary and Baccalaureate. The article by Serrano Durá et al. [34] found that girls assimilated the contents better with the gamified intervention than with the traditional methodology and, in general, both sexes had greater motivation and effort with the gamification.

#### 3.2.2. Improved Academic Performance and Motivation and Enjoyment of PE

Exergames and gamification have been present in a work carried out by Spanish authors [31] and that had a scientific design carrying out a natural experimental investigation with a control and experimental group and, pretest–post-test, measuring the motivation of the Primary Education students and their predisposition for learning dance activities. The control group received a total of nine sessions with a traditional methodology and the experimental group enjoyed the experience of a video game linked to dance and the use of the gamification platform called Classdojo. The results confirm a higher level of motivation and academic performance in the experimental group, who also had a better disposition for learning these contents. Some of these authors [32] also addressed a qualitative analysis of the fusion of gamification and used exergames to teach content related to dance in Primary Education, verifying that the attitudes shown by the teachers and students were very positive, but their expectations about its future use were not conclusive. Correct compatibility with the study plan was also verified, and in general, the students and teachers perceived more enjoyment, motivation, a taste for dance, creative inspiration and autonomous learning.

#### *3.3. Gamification in Physical Education in Middle, Junior and High School (12–18 Years)*

At this stage, 10 articles were selected that met the inclusion and exclusion criteria (Table 5). All studies have addressed gamification as a means to improve student motivation and greater involvement in PE class. In addition, several studies have been observed that have analyzed the effects of gamification combined with other pedagogical models, thus opting for a hybrid pedagogical model. Some of the models with which gamification has been combined are: flipped learning, cooperative learning or the pedagogical model of personal and social responsibility.


#### *Educ. Sci.* **2022**, *12*, 540

**Table 5.** Analysis of the articles that provide scientific evidence on

gamification

 in PE in Middle, Junior and High School.


**Table 5.** *Cont.*


**Table 5.** *Cont.*

190

Note: The

Parra-González

 study [33] is a work that also addresses Elementary School.

The largest sample size recorded was 414 students and the smallest was 30 students. Five investigations had a control group and an experimental group, five studies had only an experimental group and there were seven with pretest and post-test variables measurements. Regarding the most present elements of gamification, there are points, levels, badges, leaderboards, challenges and rewards, and in a single article, esthetics are discussed. The vast majority of researchers used questionnaires for data collection, these being of various kinds. The duration of the GLE ranges from one month to one year, the most frequent period of time.

#### *Improved Commitment and/or Motivation to Practice PE and/or Physical Activity*

Some authors have approached gamification as an alternative to traditional PE teaching models based on textbooks and the standardization of learning. Thus, Quintero-González, Jiménez-Jiménez and Area Moreira [38] present a gamification experience for the Secondary Education classroom in which students are invited to overcome a series of challenges in a GLE with a futuristic theme and demonstrating an outstanding use of Information and Communication Technologies (ICTs) with a focus on a transmedia narrative and cooperative learning. Through this environment, different competences and objectives of the PE subject have worked and confirm an increase in student motivation and a greater effective involvement in class work.

In Secondary Education, there is also another gamified didactic experience [36] with a duration of 3 months and through which about 100 students of the 2nd Year of Secondary Education improved their motivation toward learning the contents of the subject after experiencing a gamified Didactic Unit. The evaluation instrument was a questionnaire created ad hoc by the study authors and a qualitative socio-critical methodology was used. Another similar study [37] confirmed an increase in the commitment to learning in the group that underwent a gamification project. Its authors conclude that a strategy for learning healthy habits and practicing physical activity through gamification could improve student motivation. Reducing, through gamification, the prevalence of overweight and obesity in adolescents has also been one of the challenges for some researchers. Thus, a study [38] was carried out in 65 Brazilian adolescents with overweight and/or obesity. The objective was to implement a program of 12 sessions of 50 min of active video games (AVG). The authors used a randomized intervention with a control group that only played these video games and another experimental group that underwent challenge-based gamification in order to stimulate a greater amount of physical activity. The results confirm an increase in the time of adherence to the physical exercise program. This study is pioneering in addressing this concern for the health of students within the school setting.

Other authors [39] have carried out an investigation with a sample of 64 secondary students, confirming an increase in satisfaction and enjoyment, as well as an increase in intrinsic motivation and a predisposition toward learning after having used a hybrid teaching model that combined gamification with flipped learning. The authors of this study report on the need to make the scientific community aware of the potential of combining active methodologies, both face-to-face and digital, in the teaching and learning process in the field of PE, in order to raise awareness in the field teaching group of the benefits reported after its application. In the study of Real, M. et al. [43], the results suggest that the use of gamification improves student motivational variables such as: support for autonomy, support for social relations, autonomy, intrinsic motivation, identified motivation and external motivation.

In another study [40] with a hybrid methodological approach, the model of personal and social responsibility was combined with gamification in a sample of 28 girls and 27 boys from Compulsory Secondary Education with a mean age of 14.29 years. The intervention took place over 10 sessions distributed over a quarter. The authors highlight, as the main finding, the prevalence of the transfer of autonomy and responsibility in the teacher's behaviors to the participants, which generated a more self-determined motivation among the students, thus improving the levels of autonomy, responsibility

and motivation. In a similar study by Melero-Cañas [41] with the hybridization of the social and personal responsibility model and gamification, it was confirmed that the gamified hybrid intervention produced improvements in cardiorespiratory fitness, agility and speed-agility, in addition to reducing the time of sedentary lifestyle during the week and weekend. The students in the control group obtained a greater increase in Body Mass Index. These same authors, in another similar study [42], confirmed improvements in the experimental group in cognitive performance but not in academic performance. However, in the work of Arufe-Giráldez et al. [45], a higher academic performance was observed in university students in a gamified intervention during an academic year. This indicates that more studies are necessary to confirm a possible higher academic performance with gamified techniques.

The study by Parra-González et al. [33] is a study that compared the effects of a gamified Didactic Unit versus a flipped classroom Didactic Unit. As the main findings, the authors highlight that both active methodologies favor different psychological and psychosocial variables of the students, highlighting that the flipped classroom is more successful in pre-university stages with older adolescents and gamification in students of Primary Education and first years of Secondary.

#### **4. Discussion**

The objective of this systematic review was to analyze all the scientific literature published on gamification and PE in the different educational stages, specifically to evaluate and analyze the effects produced by the GLE in the PE classroom in Kindergarten, Elementary School and Middle, Junior and High School.

Gamification is presented in the educational field as a technique that can have different positive effects on students, from improvements in their social behavior to increases in levels of motivation or academic performance [18].

The vast majority of studies have focused on studying the motivation of students toward PE or learning the contents of the subject, confirming an increase in this. It can be argued that the use of rewards or punishments through points (health, experience or damage points) in the creation of a GLE can have a double motivational aspect, increasing motivation in some students and not affecting, or even decreasing, motivation in others [46]. In our review, all studies have confirmed improvements in student motivation; however, not all studies used a randomized controlled design with a control and experimental group and a pretest and post-test. In an investigation that addressed the effects of a GLE in the university classroom with a randomized controlled design, an increase in external regulation was recorded only in the experimental group. Furthermore, this group achieved significantly better academic performance. The findings of this study suggest that gamified implementation is beneficial for academic performance in college, although intrinsic motivation does not change. Furthermore, the nature of rewards or punishments, as a characteristic of this pedagogical approach, could play an important role in the expected results, because external regulation increased significantly after the intervention [47].

Escaravajal-Rodríguez and Martín-Acosta [48] conducted a literature review related to gamification in PE using the databases Dialnet, EBSCOhost and Web of Science and the academic search engine Google Scholar. A total of 19 works were selected and confirmed that 42.1% corresponded to didactic experiences and 31.6% to didactic proposals. Most of the works dealt with gamification in the Secondary Education stage (52.2%) followed by the university level with 26.1%. The authors conclude that gamification presents positive results and that teachers use it more and more.

In another study [49] that addressed a review of the literature on gamification and PE, a greater number of articles published on this subject were found in the Elementary School stage, followed by the stage of Middle, Junior and High School and Higher Education. These results are different from those found in our review. The reason may be that these researchers took into account all types of work, most of the articles being didactic experiences (69.2%) followed by research articles (23.1%) and reflections on gamification and PE (7.7%), similarly noting the scarcity of articles that investigate the effects of this technique on the different elements or variables related to learning. In our work, we have only selected scientific articles that provide scientific evidence, using research protocols with greater or lesser scientific rigor. It should be noted that only 4 of the 17 studies analyzed followed a controlled study design, one being random and the rest non-random.

Another aspect to discuss is that no scientific evidence has been found for the use of gamification in PE in Kindergarten. This may be due to the fact that at this stage, some authors have confirmed that in itself the student's motivation is high and perhaps the creation of a GLE is not so necessary to improve the intrinsic and extrinsic motivation of students [50,51]. However, in a study [52] that compared the levels of physical activity of children aged 4–6 years in the Early Childhood Education classroom, during the school day and according to the methodology used, it was found that they spend most of their time in class in a sedentary way, while in the experimental group that used active methodologies, they registered the highest amount and intensity of physical activity. Although children in Early Childhood Education have high levels of motivation, some studies confirm that 90% of Early Childhood Education centers only teach 1 or 2 sessions of Physical Education a week, often being taught by professionals who are not experts in Physical Education [53]. Therefore, the contribution of more scientific evidence at this stage could be interesting.

Another reason that can justify the scarcity of scientific production related to gamification in PE in the stage of Kindergarten, Elementary School and Middle, Junior and High School may be the lack of training in active methodologies in the faculties of teacher training. In a study [51] carried out in a sample of 220 PE teachers in Primary and Secondary Education, it is concluded that active methodologies are applied by a small number of PE teachers in their lessons, while a combination of methodologies predominates. In addition, teachers highlight the lack of training in active methodologies, despite the fact that the learning they try to encourage is in line with those associated with active methodologies. Another qualitative study [54] carried out in PE teachers in Secondary Education suggests that although teachers recognize the benefits of active teaching that endows students with autonomy, traditional, reproductive and directive teaching styles prevail in their professional practices. These results are in line with other research [55] carried out in a sample of 205 PE teachers in Early Childhood and Primary Education that confirms that traditional methods are mostly accepted in all stages of professional experience, being higher in teachers with a range of 6 to 11 years of experience, with teachers opting more for individualizing, cognitive and creative styles than female teachers, although in both cases they use traditional styles. However, in a recent work [56] that investigated the perception of a sample of more than 300 university students (future teachers of PE in Secondary Education) in relation to the use of active methodologies by its teachers, it was detected that they perceived that their teachers make use of different organizational modalities, methodological strategies and evaluation systems that favor the use of active methodologies.

Discussing the hybridization of pedagogical models and the incorporation of gamification in the teaching–learning processes, although there are few works in the literature that address this, the evidence points to possible improvements in the motivation of the students and in the learning of the contents of PE using a hybrid pedagogical model. Thus, in a study [57] not included in this review due to not meeting one of the inclusion criteria, the effects of gamification combined with the pedagogical model flipped learning in the matter of Natural Sciences, using a pretest–post-test design. The findings obtained showed that the application of this hybridization increased the motivation of the students, as well as their autonomy and self-regulation when facing the contents of the course. In another recent study [25], the authors proposed an educational intervention in Secondary Education students using a hybrid model that combines cooperative learning, adventure education and gamification; on this occasion, the study does not present scientific evidence and is published only as a didactic proposal, showing the reader educational guidelines for its implementation.

It should be noted that the link between video games and PE class and gamification is also present in the literature on gamification, having found a total of three works that addressed the use of active video games or exergames and their positive effects [30–32]. A study carried out in a sample of 47 university students, future PE teachers in Primary Education on a gamified session based on the adaptation of the Fortnite video game, confirms that said didactic proposal improved the motivation of students toward sports practice and their adherence to it, favoring collaborative teamwork and the promotion of values [58]. This last work is focused on the stage of Primary and Secondary Education, but it was not included because the experience was carried out in university students.

It should be noted that in the gamification proposals of the 17 scientific documents selected for this review, not all the elements recommended by the authors were present when establishing gamified learning environments.

There are multiple existing models that can explain the processes involved in gamification. Some models are not typical of the educational field, such as Chou's explanatory model, the Octalysis Framework [59], or the Kaleidoscope of Effective Gamification [60], both more ascribed to the business field. However, others do, such as the taxonomy of Toda et al. [61] and the model called Edu-Game [62], a system that facilitates the mechanics of creating a gamified learning environment, paralleling the elements of the educational curriculum and the game. Or finally, the model based on four blocks [63], presented here in Figure 2, which are limited to the educational field and can be a reference model for all the research on gamification in all educational stages. This model is based on four large blocks of elements, educational, motivational, game and prior knowledge elements, and explains how each of the large elements and their corresponding sub-elements should be planned in order to have a certain coherence and synergy between all of them to guarantee the success of the gamification.

**Figure 2.** Creating a Gamified Learning Environment (GLE) [63].

In order to achieve the greatest effects of gamification, it is recommended that pedagogical proposals are gamified in PE use and to correctly plan the use of these elements. In agreement with other authors [15], gamification has become a research focus with enormous potential but more work with appropriate designs is necessary as the available studies have their limitations and many of them are not scientifically sound enough.

Finally, it is necessary to discuss the relevance of Physical Education to promote healthy lifestyle habits and avoid physical inactivity in infancy, childhood and adolescence. Some authors have addressed multiple systematic reviews on the importance of working on student health in Physical Education [64,65]. Teachers must not only choose what health content to teach or work on but also how to teach it, with what methodology they can cause a change in the student's lifestyle, favoring the practice of physical activity beyond

school, and thus avoiding the appearance of multiple diseases associated with a sedentary lifestyle [66].

Abundant scientific evidence confirms multiple benefits of practicing physical activity in school educational stages [67–69], including obtaining benefits in cognitive competence [70,71]. Gamification can be presented as a means to cause positive changes in the behavior of students by increasing the levels of physical activity and the emotional state, as well as favoring the attitude and motivation toward the practice of physical activity [72].

#### **5. Conclusions**

The analysis of the systematic review carried out on the use of gamification in PE reveals a low scientific production in the stages of Primary and Secondary Education and an absence of this in Early Childhood Education. The vast majority of the studies that addressed gamification in the PE classroom confirmed improvements in the motivation of students toward learning the contents of the subject and an increased commitment to the practice of PE/physical activity. Only one study analyzed the positive impact on academic performance, and another study did not find any effect on academic performance, but it did on cognitive performance.

The diversity of the research protocols and instruments used to evaluate the different variables studied stands out. On the other hand, a very low number of studies that used randomized controlled designs and a certain lack of planning of all the elements involved in a gamified system are also confirmed. All this suggests the need to continue investigating the possible positive effects that the implementation of gamified pedagogical proposals may have in the PE classroom, whether incorporated in isolation or using a hybrid model in combination with other active methodologies, such as the flipped classroom, the model of personal and social responsibility or cooperative learning.

#### **6. Limitations and Proposals**

This systematic review has several limitations, among them the difficulty in finding, in some scientific articles, clear and concise information on the protocol used in the re-search and in the creation of the gamified learning environment, although most of the authors were contacted personally to collect more information about their studies, this contact was not successful with some authors, obtaining only the information published in the article itself.

For future studies, it is recommended to use common protocols to be able to perform a meta-analysis with the data of various investigations. It is also necessary that more gamified pedagogical proposals be addressed using all the necessary elements for the creation of a gamified learning environment in order to accurately measure the effectiveness of this technique in the PE classroom.

The use of a control and experimental group, and a pretest and post-test, guarantees greater scientific solidity. On the other hand, using the same test to measure how the students' commitment, their motivation or another variable changed will allow a metaanalysis to be carried out and to create a greater scientific base on educational gamification. For example, in relation to the measurement of the physical condition or psychomotricity of students, it could be interesting to use standardized test batteries such as Eurofit or the MABC-2, respectively.

**Author Contributions:** Conceptualization, V.A.-G., A.S.-R. and O.R.-Á.; methodology, V.A.-G., A.S.-R., O.R.-Á. and R.N.-P.; validation, V.A.-G., A.S.-R., O.R.-Á. and R.N.-P.; formal analysis, V.A.-G. and R.N.-P.; investigation, V.A.-G., A.S.-R., O.R.-Á. and R.N.-P. resources, V.A.-G., A.S.-R., O.R.-Á. and R.N.-P. data curation, V.A.-G., A.S.-R., O.R.-Á. and R.N.-P. writing—original draft preparation, V.A.-G., A.S.-R., O.R.-Á. and R.N.-P.; writing—review and editing, V.A.-G., A.S.-R., O.R.-Á. and R.N.-P. project administration, V.A.-G. All authors contributed to the manuscript writing. All authors have read and approved the final version of the manuscript, and agree with the order of presentation of the authors.

**Funding:** This research received no external funding.

**Institutional Review Board Statement:** The study was conducted in accordance with the PRISMA Declaration.

**Informed Consent Statement:** Not applicable.

**Data Availability Statement:** Not applicable.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**


## *Systematic Review* **Models of Instructional Design in Gamification: A Systematic Review of the Literature**

**Alberto González-Fernández 1,\*, Francisco-Ignacio Revuelta-Domínguez <sup>2</sup> and María Rosa Fernández-Sánchez <sup>2</sup>**


**Abstract:** Gamification allows for the implementation of experiences that simulate the design of (video) games, giving individuals the opportunity to be the protagonists in them. Its inclusion in the educational environment responds to the need to adapt teaching–learning processes to the characteristics of homo videoludens, placing value once again on the role of playful action in the personal development of individuals. The interest that has arisen in studying the implications of gamification processes in the different educational stages, in order to determine their impact and suitability, has led to an increase in scientific publications. With the intention of studying the presence and implications of gamification in teacher training as a methodological principle implemented in the teaching–learning process, both in its initial and permanent stages, this systematic review of the literature identifies those instructional design models applied in the field of gamification, as well as its educational significance. Thus, the need to introduce gamified practices in the field of teacher training is observed, providing an experiential learning that allows teachers to apply this methodology in a relevant way in their professional development, based on their own experience.

**Keywords:** gamification; teacher training; gamification design frameworks; systematic review

#### **1. Introduction**

The proliferation of educational experiences that propose the implementation of active methodologies in teaching–learning processes has provoked the academic interest of the research community. In relation to gamification, there are numerous practices that introduce, in one way or another, elements of (video) games in educational contexts to increase student motivation and involvement. The interest in studying the implications of gamification processes in the different educational stages, in order to determine their impact and suitability, has led to an increase in scientific publications in recent years. As such, the difficulties that at present still persist in clearly defining what gamification is, a term that is often confused with (video) game-based learning, when associated with its root game or with the concept of fun learning, has promoted a massification of studies in this field.

Thus, knowing the state of the literature in relation to gamification becomes a complicated task, a challenge that requires a major investment of time that, on many occasions, is not productive. Therefore, it is necessary to resort to research methodologies that allow the collection of relevant evidence in relation to a specific topic.

To this end, prior to conducting a systematic review of the literature (SRL or SR), it is necessary to determine those scientific productions that implement this methodology as a means of synthesizing the available evidence on the area of interest covered by this article, serving as a precedent for conducting a new SR that updates the results found or delves into other areas of interest related to gamification and its instructional design.

**Citation:** González-Fernández, A.; Revuelta-Domínguez, F.-I.; Fernández-Sánchez, M.R. Models of Instructional Design in Gamification: A Systematic Review of the Literature. *Educ. Sci.* **2022**, *12*, 44. https://doi.org/10.3390/ educsci12010044

Academic Editor: Huei Tse Hou

Received: 10 November 2021 Accepted: 6 January 2022 Published: 13 January 2022

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

After a thorough reading of 20 systematic reviews, the areas of knowledge for the selected SLRs relate to teaching in the fields of health, science, and English as a second language; at the level of higher education and primary education; with other educational models such as e-learning; and with other methodologies such as flipped learning or peer assessment activities.

In an initial study, Alomari et al. [1] analyzed 40 publications related to the promotion of learning in university students after the implementation of gamification techniques between 2016 and 2018. The presence of a series of common strategies is determined and grouped under the acronym PBL (points, badges, and leaderboards) with a presence of 75%, 65%, and 63%, respectively. As evidenced by the authors, these gamification elements allow for the emergence of a controlled competitive environment that leads to an increase in student motivation and participation.

Along the same lines, Rauschenberger et al. [2] developed a systematic review of the term gamification in the field of learning environments, extracting the relationship between dynamics present in the 10 studies analyzed: emotions and progress (relationships, narrative, choices, and restrictions); mechanics: rewards, (opportunities, resource acquisition, and victory states), feedback, and challenges (cooperation, competition, and transactions); and gamification components: badges, avatars, points, rewards, missions, etc.

Elsewhere, Bozkurt and Durak [3] performed a meta-analysis of 208 studies on gamification published between 2008 and 2016. Through it, they highlight those methodologies most commonly used in gamification research. They also perform a lexical analysis to determine the relationship and reiteration of words in the titles and abstracts of the articles that make up the sample. Gamification is the most used term, related to education, learning, students, training, etc. In a second cluster, the terms game, engagement, social, elements, motivation, experience, behavior, effectiveness, etc., appear. Finally, gamification is related to words such as design, technology, software, online, tools, etc. The study by Zainuddin et al. [4] also addresses issues related to platforms and apps found in the scientific literature on gamification (ClassDojo, ClassBadges, Kahoot!, Duolingo, etc.).

Of interest is the review conducted by Cordero-Brito and Mena [5], representing the evolution of gamification and its influence in the social domain. For this purpose, they analyze a total of 136 articles published between the years 2011 and 2016. The authors establish the temporal trend in publications on gamification, with a considerable increase in recent years. They also identify the most representative model of instructional design, called MDA: mechanics, dynamics, and aesthetics. Finally, the authors establish a list of gamification components and tools and their impact on the motivation of individuals.

Similarly, Mora et al. [6] analyze gamification design frameworks through their background and scope of application (education, business, and healthcare), the suitability of such models for student engagement in the university environment, as well as the elements included in the design process. To this end, they review a sample of 40 studies published between 2011 and 2015, highlighting the need to turn design into an iterative, user-centered process with a technological presence. They also point out that the studies analyzed include the 6D model, MDA, as well as Werbach's or DMC. Based on the results, they propose three differentiated design approaches for gamification systems: user-centered, game-centered, and technology-centered.

Other selected systematic reviews restrict the area of analysis to studies related to gamification and English as a second language acquisition [7], health care and medical studies [8,9], education and science [10], as well as experiences related to virtual education [11], flipped classroom [12], and peer assessment activities [13]. It is necessary to highlight the deep analysis performed by Kalogiannakis et al. [10], pointing out educational level, contents related to science curriculum, educational context, underlying learning models or theories, methods, results, gamification elements, and assessment tools of each of the analyzed articles.

Regarding the different educational stages, systematic reviews such as that of Fadhli et al. [14] are proposed. Such reviews analyze studies published between 2014 and 2018 on the effectiveness of gamification in the acquisition of conceptual, procedural, and attitudinal content in students aged 6 to 10 years, corresponding to primary education. Meanwhile, Pegalajar Palomino [15] identifies the main findings in the scientific literature, from 20 studies published between 2010 and 2019, on the perception of university students towards the implementation of gamification strategies in their teaching–learning processes. Subhash and Cudney [16] also focus their review on the university setting, attending to the areas of knowledge (computer science, business, science, pedagogy, etc.), the countries of production (Spain in first place, followed by the United States and Germany), the gamification elements employed (badges, feedback, collaboration, levels, narrative, etc.), and the benefits of gamification (motivation, attitude, engagement, enjoyment, etc.) found in 41 articles between 2012 and 2017.

Finally, Navarro-Mateos et al. [17] propose a review of the term gamification in Spanish education. This would allow for knowledge regarding the impact of gamification in the different educational stages through the analysis of 15 studies published until 2020. In addition, it is pointed out that, in general, the main objective of these interventions is to influence student motivation, as well as to improve the learning experience and academic performance.

Based on the study of these previous systematic reviews, it has become necessary to propose an integrative perspective that encompasses the most relevant approaches to the aspects analyzed above. Therefore, based on the proposals made by Zainuddin et al. [4], Kalogiannakis et al. [10], and Navarro-Mateos et al. [17], who study, among other aspects, instructional design models, (video) game mechanics involved in gamification systems, digital resources related to this methodology, and potential effects on the teaching–learning process, this SLR aims to analyze the situation of gamification in the field of teacher training, both in its initial or university stage and in continuing education.

#### **2. Materials and Methods**

Systematic reviews, according to Manterola et al. [18], "are studies whose population comes from already published case articles; that is, they are studies of studies" (p. 150). Thus, SLR makes it possible to concentrate knowledge of a specific area, giving it meaning through the results obtained in different studies, in order to identify prospective research priorities. According to Ferreria González et al. [19], "they constitute an essential tool for synthesizing the available scientific information, increasing the validity of the conclusions of individual studies and identifying areas of uncertainty where research is needed" (p. 688).

Conducting an SLR involves the specification of a series of research steps to guide the review process towards a specific area of study. To this end, it is necessary to establish, as a starting point, a series of issues or research questions that determine the object of interest or focal element of the review.

Using the aforementioned systematic reviews as references, the need arises to update the data provided in these publications, as well as to specify and orient the scope of the SLR study in order to approach other research priorities that have not yet been addressed. To this end, a structured process is followed through a pre-established design, which provides validity, quality, and rigor. With the intention of providing objective criteria for the publication of systematic reviews, the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) statement, published in 2009 (recently updated in 2020), presents a series of recommendations and guidelines for their preparation. Through a checklist of 27 items [20], it establishes an optimal planning process that simplifies the researcher's task in conducting an SLR. Thus, the systematic review process applied in this study consists of different phases [21]:

• Phase 1: Research questions (RQ). They are organized around three areas: (a) conceptual framework, to analyze the relationships between the keywords identified in the literature (RQ1); (b) documentary characteristics, to identify geographical location and research methodologies used (RQ2,RQ3); and (c) pedagogical dimension (RQ4,RQ5), to recognize the methodological strategies for the inclusion of gamified practices in

the field of teacher training and their relationship with other active methodologies, as well as instructional design models applied in the studies analyzed (Table 1).


**Figure 1.** Literature selection process flowchart.


**Table 1.** Areas, questions, and initial coding criteria.

#### **3. Results**

Based on the sample obtained as a result of the methodological process established for this SLR, we outline details of the aspects related to the analysis of the information obtained from the 28 scientific publications related to the implementation of gamified practices in the field of teacher training.

To this end, the aim is to answer the research questions (RQ1-RQ5) posed above and thus to obtain an accurate picture of the state of the situation in relation to scientific production on gamification.

#### *3.1. What Is the Conceptual Relationship Surrounding the Term Gamification?*

In order to analyze the network of concepts as an answer to question 1, it has been necessary to highlight the terminological relationships existing among the publications that make up the review sample. In this way, it was possible to identify a series of clusters or categories generated by the co-occurrence of the key words in the studies, as can be seen in Figure 2.

**Figure 2.** Co-occurrence map by keywords (frequency = 1).

Through VOSviewer, a series of clusters have been identified that make it possible to highlight those areas of interest in the studies reviewed. Obtained from the co-occurrence of keywords established by the authors of the publications, four different and interconnected main categories can be observed. The red cluster encompasses aspects related to teaching– learning processes, such as "continuous assessment," "educational innovation," "digital competence," or "cooperative learning".

Similarly, the blue cluster, from a more generic perspective, connects elements linked to teacher training. The purple cluster is related to the design process of gamification systems through (video) game elements and mechanics. Finally, the green cluster identifies, in a more segregated way, methodological aspects related to "distance education," "active methodologies," or "motivation".

Likewise, through the LitMaps tool, a relationship is established between the articles that make up the review sample that present DOIs, as well as their temporal distribution, as shown in Figure 3. Thus, connections can be observed between Villalustre Martínez and del Moral Pérez [23] and Castañeda Vázquez et al. [24], as well as between Kopcha et al. [25] and Falcó Boudet and Huertas Talón [26]. It also allows the identification of the most cited reference authors in the field of gamification [27–32].

**Figure 3.** List of authors and citations between articles.

#### *3.2. What Is the Geographical Distribution of the Publications?*

The selection of countries in the publications that make up the established sample are specified according to the nationality of the reference author of each article. Thus, from RQ2, it is possible to determine the geographical distribution of those studies related to gamified experiences in the field of teacher training, allowing us to observe the interest aroused by this topic from a global perspective.

Spain is the country with the highest number of scientific productions related to gamification and teacher training, both in its initial and permanent stages, with 16 articles out of the 28 that make up the sample. Next, 5 publications each are attributed to the United States and Brazil. Finally, Italy and Turkey complete the geographical distribution with 1 article in each country.

In relation to the language of the scientific productions, Spanish is the language most used in the selected sample, with a total of 14 articles (50%). English is the second most common language, with 11 publications (39.29%). Finally, in coherence with the geographical distribution detailed above and with the search process carried out in the three databases consulted, Portuguese is another of the languages present in the articles reviewed, specifically in 3 (10.71%).

#### *3.3. What Research Methodologies Are Used in the Selected Studies and What Is the Sample Size?*

Through RQ4, which refers to the research methodologies used in the selected studies, as well as the sample size of the publications, it is possible to determine the most relevant methodological strategies in research processes in the field of gamification.

Thus, it is possible to observe that the qualitative approach is the most frequent among studies related to gamified educational practices (53.57%). In second place, the quantitative approach (39.29%), followed finally by mixed research (7.14%).

Likewise, it is possible to establish a relationship between the methodological approaches identified and the research methods applied in the articles that make up the review sample. In relation to qualitative techniques, in Figure 4 can be observed that 12 of the publications use the case study; in 1 publication, formative research [33]; in another, the phenomenological study [34]; and finally, in 1 of the publications narrative research is applied [35]. Regarding quantitative techniques, 4 articles are identified that implement questionnaires as a research instrument, 2 that apply program evaluation, 2 of a quasiexperimental type, 1 study of a descriptive nature [36], 1 descriptive–interpretative [37], and 1 case study [23]. Finally, in relation to mixed techniques, 1 case study [38] and 1 design-based research [39] were identified.

**Figure 4.** Research methods identified in the studies.

In summary, the most frequent research method in the publications that make up the sample is the case study (50%). In second place, we see research based on questionnaires (14.29%). With the same percentage (7.14%) are quasi-experimental studies and program evaluation. Finally, methodologies such as formative research, descriptive studies, descriptive-interpretative, design-based research, phenomenological studies, and narrative research appear with the same level of presence (3.57%).

#### *3.4. What Instructional Design Models Applied to Gamification Systems?*

The specification of the elements, strategies, and resources that make up the gamification systems implemented in the experiences analyzed responds to a series of instructional design criteria established by the various models or frameworks that allow for the development of gamified educational interventions. Based on RQ4, a series of instructional design models have been identified among the publications that make up the review sample, which in turn, make it possible to determine the level of depth or insertion of the approach taken.

As can be seen, the instructional design model with the greatest relevance among the publications reviewed is the PBL or points, badges, and leaderboards strategy with 35.71%, including variants such as PBL+K and PL. In second place, we see the MDA or mechanics, dynamics & aesthetics architecture, with 28.57% presence. This is followed by Pyramid DMC or dynamics, mechanics and components with 14.28%. With the same percentages (7.14%) are the 6D approach and Learning Tangram. Finally, the Bonk and Dennen model, as well as the social gamification approach, also with the same proportion (3.57%). It should also be noted that the PBL and MDA instructional design models are present in more than 50% of the sample (Figure 5).

**Figure 5.** List of instructional design models identified in the sample.

From the descriptions of the gamification systems implemented in the various educational practices analyzed, the depth of the designs carried out can also be evidenced. For this purpose, the elements that make up the gamifications proposed in the publications that make up the sample have been recorded, as well as their pretensions and their temporality, in order to determine their penetration capacity.

From this analysis, it is observed that 53.57% of the gamification systems present a superficial configuration or thin layer level gamification. 46.43% correspond to deep level gamification designs. In relation to the identified models, it is possible to evidence the relevance of elements that make up the gamified experience, being determinant to assess its effectiveness in relation to design. The PBL strategy is the most representative among surface gamification systems, while the DMA and DMC models are more related to deep gamification designs (Figure 6).

#### *3.5. What Are the Effects of Gamified Practices in the Teaching–learning Process?*

To determine the impact of gamified educational interventions detailed in publications that make up the review sample, we begin with RQ5 to identify those results most highlighted by their authors. To this end, it was necessary to establish a series of categories that would allow grouping such evidence, presenting a direct relationship between (1) student motivation, mainly related to intrinsic motivation; (2) educational commitment, related to engagement and involvement with respect to the educational intervention; (3) student participation in activities related to the educational process; (4) attitudes towards their own learning; (5) communication; (6) perception of knowledge acquired and competencies developed; (7) academic results; and finally, (8) updating of knowledge, in relation to continuing teacher training. Similarly, seven of the studies included in the sample present results regarding the perceptions or opinions of the participating students themselves, so they have not been taken into account in the previous categorization.

As can be seen in Figure 7, engagement or educational commitment is one of the main implications of gamification in educational practice identified in the sample, with 29.41%. Next, the impact of gamification processes on student motivation is evident (26.47%) through an increase in motivation. Based on the practice of Pérez-López et al. [40], gamification, as a methodological strategy, creates an improvement in student motivation and an increase in their involvement. With the same percentage (11.77%), there are results related to student participation and attitudes, as a consequence of the previous elements. Likewise, an improvement in the students' perception of the knowledge acquired has been identified (8.82%) [41], followed by aspects related to the updating of pedagogical, technological, and conceptual knowledge, as a consequence of ongoing teacher training (5.88%) [42]. Finally, with the same percentage (2.94%), results related to the improvement of communication and academic results of participating students were observed.

**Figure 6.** Instructional design models and their relationship with gamification type.

**Figure 7.** Categories established to analyze the effects of gamified practices.

#### **4. Discussion and Conclusions**

Through the systematic review process developed, it has become possible to know the impact of gamified practices in the field of teacher training, both in its initial and permanent stage, through implementation strategies of elements of this methodology in teaching–learning processes. In this way, we have provided answers to five research questions concerning the conceptual framework that has been configured around the concept of gamification; the documentary characteristics of the articles that make up the review sample in relation to gamification practices, the thematic areas from which they are studied, and the research methodologies used; and finally, attending to the pedagogical dimension of the studies, the identification of various instructional design models, and a list of gamification elements and technological resources related to this methodology.

The most relevant results of this systematic review are found, on the one hand, in the identification of the conceptual network that emerges from the analysis of the research that makes up the review sample. The key concepts linked to gamification processes are evidenced, as well as the relationship established between them. Through this representation of the state of knowledge, a terminological framework can be established that allows an in-depth exploration of those unknown areas that require a reflective dissection. In this sense, there is little reference to specific models of instructional design for the configuration of gamification systems, identifying, on the contrary, certain elements typical of this methodology.

Furthermore, this systematic review provides a detailed description of the documentary characteristics of the studies, allowing recognition of research processes related to gamified practices, as well as the instructional design models identified in the review process. Thus, a relationship between qualitative research approaches, mainly through case studies, and the field of gamification applied to teacher training has been evidenced. This is contrary to Zainuddin et al. [4], who observes a greater presence of quantitative studies in practices that implement a gamified methodology; Ekici [12], who identifies a predominance of mixed methods; or Bozkurt and Durak [3], who observe a majority of theoretical or conceptual–descriptive type articles. However, from the present systematic review, other research methods in trend in the field of education and gamification, such as design-based research [39], can also be appreciated.

Finally, this study has made it possible to identify the main instructional design models for gamification systems. For this purpose, it has been necessary to establish a relationship with elements implemented in the practices proposed, since in many cases the model involved in the design of the gamified practice has not been explicitly established. In this sense, coinciding with the study by Navarro-Mateos et al. [17], there is a general lack of knowledge of the process of gamification systems or specific models of instructional design by teachers, causing the introduction of gamification elements without a specific criterion or without a configuration that has a specific purpose.

The systematic review has shown the prevalence of PBL, i.e., gamification practices that introduce, in isolation, three components: points, badges and leaderboards. Although other studies [17] dismissed those gamification proposals based on PBL, considering that gamification "is a more abstract, complex and strategic process that aims to go beyond the use of points, badges and rankings" (p. 512), the reality is that it represents one of the most widely used gamification models in the field of instructional design [1,10,12,43,44]. However, other more complex models have been identified that require a more reflective and elaborate design process, resulting in deep gamification systems, such as the MDA architecture, coinciding with the study conducted by Bozkurt and Durak [3], the Elements Pyramid and the 6D approach.

In relation to the educational implications of gamification in the teaching–learning processes, through the proposals analyzed in the articles that make up the sample, a direct relationship between this methodology and increase in motivation, commitment, participation, and attitudes of the participating students has been evidenced. Conclusions that can also be observed in other studies [5,10,12,17,44], which identify a series of implications of

gamification at all educational levels, are those such as improved academic performance and increased student engagement and motivation. Pegalajar Palomino [15] states that, "at the cognitive level, it is worth noting how the practice of gamified learning experiences allows an improvement in the academic performance of students, helping them to maximize learning" (p. 178).

The results of this systematic review of the literature allow us to conclude that an adequate educational approach to gamification requires a deep knowledge of the implications derived from the implementation of this methodology. To this end, it is necessary to assess the importance of instructional design models that allow an adequate development of gamified practices. The interconnection of elements that make up a system of these characteristics requires a process of reflection, planning, and arrangement of its components, avoiding improvisation and arbitrariness.

Educational implications, which aim to go beyond the improvement of students' academic performance, pursue an increase in motivation, commitment, and positive attitude towards the teaching–learning process itself, through the entertainment and uniqueness provided by gamified practices. It also becomes necessary to implement experiences in the field of teacher training, both in its initial and permanent stage, providing experiential learning that allows teachers to introduce, in their professional development, this methodology in a relevant way, based on their own experience.

Some limitations apply to this review. This study focused only on experimental academic research regarding gamification experiences in the teacher training field, both in its initial and continuous stage, published in academic journals. Consequently, the number of articles reviewed is limited, evincing the need of developing more gamification experiences in this scope of action. The issues limit the generalizability of the review results. However, this study allows to determine some implications of gamification as an active methodology, bringing together several experiences in the teacher training field, studying the gamification instructional design models and the gamification elements most used.

**Supplementary Materials:** The following supporting information can be downloaded at: Selection https://www.mdpi.com/article/10.3390/educsci12010044/s1.

**Author Contributions:** Conceptualization, A.G.-F.; methodology, A.G.-F. and F.-I.R.-D.; software, A.G.-F.; validation, A.G.-F. and F.-I.R.-D.; formal analysis, A.G.-F. and F.-I.R.-D.; investigation, A.G.-F., F.-I.R.-D. and M.R.F.-S.; resources, A.G.-F.; data curation, F.-I.R.-D.; writing—original draft preparation, M.R.F.-S.; writing—review and editing, M.R.F.-S.; visualization, F.-I.R.-D.; supervision, F.-I.R.-D. and M.R.F.-S.; project administration, A.G.-F.; funding acquisition, A.G.-F. and F.-I.R.-D. All authors have read and agreed to the published version of the manuscript.

**Funding:** This publication has been made possible thanks to funding granted by the Consejería de Economía, Ciencia y Agenda Digital de la Junta de Extremadura and by the European Regional Development Fund of the European Union through the reference grant GR18071.A.G-F. is supported by a predoctoral contract granted by Ministry of Education, Culture and Sport of Spain (FPU15/05851).

**Institutional Review Board Statement:** Not applicable.

**Data Availability Statement:** Not applicable. The data for this study originated from secondary desk research.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**


MDPI St. Alban-Anlage 66 4052 Basel Switzerland Tel. +41 61 683 77 34 Fax +41 61 302 89 18 www.mdpi.com

*Education Sciences* Editorial Office E-mail: education@mdpi.com www.mdpi.com/journal/education

Academic Open Access Publishing

www.mdpi.com ISBN 978-3-0365-7515-5