Next Article in Journal
Development of an Integrated Model to Assess the Impact of Agricultural Practices and Land Use on Agricultural Production in Morocco under Climate Stress over the Next Twenty Years
Previous Article in Journal
Soft Is Better: Determinants of Preferences for Non-Motorized Forms of Transportation in Urban Tourism Destinations
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Review

Gamification in E-Learning and Sustainability: A Theoretical Framework

by
Renata Pereira Oliveira
*,
Cristina Gomes de Souza
,
Augusto da Cunha Reis
and
Wallice Medeiros de Souza
Production Engineering Departament, Federal Center for Technological Education Celso Suckow da Fonseca, Rio de Janeiro 20271-110, Brazil
*
Author to whom correspondence should be addressed.
Sustainability 2021, 13(21), 11945; https://doi.org/10.3390/su132111945
Submission received: 27 September 2021 / Revised: 22 October 2021 / Accepted: 23 October 2021 / Published: 28 October 2021
(This article belongs to the Section Sustainable Education and Approaches)

Abstract

:
Progress in communication technologies and social isolation caused by the COVID-19 pandemic have supported the acceptance of e-learning. In the e-learning context, gamification has been identified as one of the most promising trends. Many researchers believe in the game elements’ capacity to drive learning, skills acquisition, and changes for more sustainable behavior. However, the literature on the subject is dispersed, addressing behavioral attitudes and elements in an isolated and fragmented way. This study aims to present a framework relating game elements to behavioral attitudes to promote sustainability and ensure quality of learning. The methodology is based on a systematic literature review using the PRISMA protocol, covering the content analysis of 130 articles indexed in the Web of Science database. For the framework construction, the following were identified: behavioral attitudes stimulated by gamification; main game elements and how they relate to each other. Based on this, a framework called 7GOALS (Gamification-Oriented Active Learning Steps) associated with the PDCA (Plan, Do, Check, Act) was established. The proposed structure is multidisciplinary and can be used in any knowledge field that uses gamification. With this, themes such as sustainability can be widely disseminated and leverage changes towards more sustainable behaviors that adhere to the real world.

1. Introduction

The 21st-century information technology revolution has affected the economic, social, and educational spheres of society. In the education sector, the most distinguishable change was the integration of computers and information and communication technologies (ICT) in the teaching and learning processes to help active education and achieve goals [1].
The proliferation of enabling learning technologies worldwide and the boom in information has caused e-learning to become more accessible, being pointed out as a trend in the educational field [2,3,4]. Recently, due to the social isolation caused by the COVID-19 pandemic, educators have been being forced to find alternatives for teaching by adopting and disseminating new technologies aimed at online education, which reinforces this trend.
New pedagogies, practices, and theories are emerging in the educational technology, such as gamification [5,6,7]. Gamification refers to applying game design aspects in non-gaming contexts to increase engagement, motivation and improve task performance [8,9,10,11,12,13]. It is used as an innovative way to engage citizens in sustainability practices directed at climate change, water scarcity, or extreme weather events, etc. in their local communities [14].
Game elements allow the players to participate in a pseudoconflict to encourage experimental learning in an interactive way and a safe and risk-free environment [15,16,17,18,19,20]. It is produced to educate the player by using attractive and pleasurable educational scenarios [21,22,23,24].
Furthermore, games are present in young learners’ lives [25]. That is why the new generation, known as the “generation of players”, “digital natives”, “network generation”, or “generation Z”, widely accept games [10,26,27,28,29]. This new generation learns differently from their predecessors and has effective “e-lives”, connecting and sharing information instantly through blogs, online shops, social networks, online games, or webchats [30].
Although it is recent, the concept of gamification is widely adopted by researchers from different domains and research areas [31] such as mathematics, computing, social sciences, engineering, health, environment, sustainability, and others [32,33,34,35]. In the educational field, the benefits of gamification have become more apparent due to the need to provide the student with a unique and unusual experience [36]. This experience is essential, especially in courses that strive to emphasize the multidisciplinary nature of disciplines such as sustainability and address issues that raise awareness of sustainable processes in industries [37].
Research focused on game design began in 2003. However, the application of gamification in the educational sector and especially in the e-learning environment increased exponentially from 2014 [27,38,39]. Many researchers strongly believe in gamification to drive learning, interaction, collaboration, skill acquisition, understanding content, and changing attitude or behavior. Compared to traditional practices, they support effectiveness in engagement, motivation, stimulating curiosity, autonomy, and satisfaction [5,6,34,38,40,41,42,43,44,45,46,47,48].
On the other hand, some researchers believe that gamification should be used as a complementary practice despite being a valuable teaching and learning strategy and not replace the current educational methods [13,49,50,51,52]. Moreover, scholars point out that gamification depends on the context and, even more, on users [3,10,53,54,55,56].
In this scenario, the following research question arises: How can the game elements be organized logically to support the gamified system planning to effectively promote more sustainable behavior and guarantee the quality of the learning process?
A systematic literature review was carried out with a content analysis of 130 articles indexed in the Web of Science database to answer this question. Based on the research findings, a framework called 7GOALS (Gamification-Oriented Active Learning Steps) were established relating behavioral attitudes and gamification elements to the PDCA cycle stages (Plan, Do, Check, Act) to provide continuous improvement of the learning process by gamification and expand the application of themes such as sustainability.
To the best of the authors’ knowledge, this is the first study that brings together the main game elements, the PDCA cycle, and behavioral attitudes to promote learning. As a result, researchers on this topic, teachers, and education professionals will have a summarized and structured portrait of the relationship between the main elements and behavioral attitudes leading to improvement by learning gamification.

2. Materials and Methods

This study is a systematic literature review based on the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) protocol [57,58,59]. The search occurred on the Web of Science database, which is widely recognized by the academic community. Several preliminary searches were performed in the topic field, analyzing titles, abstracts, authors’ keywords, and KeyWords Plus. The investigation broadly included the following query: game* AND e-learn* AND (education OR teaching). There is a gap in the academic literature that includes articles on gamification in e-learning to induce sustainable behaviors.
Therefore, the research method adopted here consisted of two steps: the first established the systematic review of the literature and the second stage involved the content analysis of articles [60]. The search retrieved 578 documents published until 2019, and it covered all research areas. After filtering only the documents classified as Article and Review, 166 articles remained. This study was also limited to documents with full text available and published in English or Spanish, resulting in an additional 10 and 4 documents discarded, respectively. After a full reading of the articles, 130 papers were assessed as eligible to meet the proposed objectives of this study (Figure 1).
After conducting a content analysis, the selected articles’ data were clustered to analyze how behavioral attitudes and gamification elements are related in the literature to provide changes for more sustainable behavior. In this stage, the data were computed quantitatively through meta-analyses and in an inclusive way, allowing an article to encompass various elements and/or attitudes. The tables organized the data and conditional formatting to identify possible relations. The Venn diagram, commonly used in mathematics, was also used to symbolize the associations.
Based on this analysis, the gamification elements that effectively promote behavioral attitudes, sustainable environments and guarantee the quality of the learning process were structured in a sequence of steps and classified as 7GOALS (Gamification-Oriented Active Learning Steps). This study phase was based on adapted concepts of the WCM methodology (World Class Manufacturing) and the PDCA cycle.
The WCM methodology presents a set of management principles, procedures, and techniques to reduce losses and waste and improve production processes [61,62]. Although developed and consolidated in manufacturing environments, this methodology can also be applied in other areas. The WCM methodology uses seven measures (“Seven Problem-Solving Steps”), which allow its evolution step by step [63].
In the “Seven Problem-Solving Steps” approach, the first step identifies the problem phenomenon; the second step studies the system and classifies the problems according to their impact factor; the third defines the objectives and ways to find a solution to it; the fourth analyzes the root cause; the fifth step aims to apply countermeasures with the implementation of actions to overcome the root causes; the sixth step checks the results; and the seventh expands these results so that there is a continuous improvement [63].
The PDCA cycle was also used as a complementary tool, and it corresponds to a method commonly employed for continuous improvement by covering four stages: Plan, Do, Check, Act [64]. These steps are an approach widely used in distinct fields, including e-learning [65,66]. It is one of the most popular quality management system models worldwide. This study was adopted as a method to support action planning to achieve the learning objectives proposed [67]. It is a management tool applied to ensure company’s products and processes continuous improvement [68]. It leads to systematic actions that speed up obtaining better results and guaranteeing organizations’ growth.
Thus, phase P (Plan) defines what you wish to do, establishes objectives and activities to achieve it. Phase D (Do) is characterized by executing what was planned to implement the necessary actions. Phase C (Check) involves monitoring the process and verifying that the objectives have been achieved. Finally, phase A (Act) refers to seeking continuous improvement until reaching the excellent standard or the expected results [69].
A framework is presented associating 7GOALS with the PDCA cycle to ensure continuous improvement of the learning process and generate changes for more sustainable behavior.

3. Results

In response to the research question, this section presents a framework called 7GOALS (Gamification-Oriented Active Learning Steps) with the main game elements organized in a sequence of steps to positively affect users’ emotional feelings. This structure is multidisciplinary and can be used in any knowledge field, such as in sustainability, and it is associated with the PDCA for continuous improvement of the learning process.

3.1. Mapping: Behavioral Attitudes and Elements of Gamification

3.1.1. Behavioral Attitudes Associated with Gamification

According to the literature, gamification induces three behavioral attitudes in students: motivation, engagement, and satisfaction (Table 1).
Motivation, combined with human affection and emotions, is one of the most relevant behavioral attitudes in learning, especially in online courses, as cited by 45 articles. It can be extrinsic or intrinsic. Extrinsic motivation refers to doing something in search of a result, such as getting a good grade. However, intrinsic motivation refers to doing something because it is appealing or pleasant, and it is related to three aspects: autonomy, relationship, and competence [8,33,89,101].
Some factors encourage motivation: competition, challenge, imagination, collaboration, goals, cooperation, reward, and recognition, many of which are present in games [12,28,39,87,91,97,100].
The game also stimulates motivation because of its impact on cognitive development, affective skills, emotional and social states of the players [33,76,93,96]. According to [16,90], add that games allow slightly reducing the theory’s abstraction by encouraging a greater motivation. In addition to motivation, educational games improve interest and concentration, improving students’ learning [94].
Engagement is another behavioral attitude identified as a valuable indicator of academic performance and positively correlates with student learning outcomes [103,106], as seen in 26 articles. Engagement happens when someone’s attention is completely focused on a specific activity. Thus, the virtual games industry indicates that engagement is a tool to keep the player’s attention on the game [110]. Engagement can also be explained as the time and effort that students put into educational activities and the prerequisite for effective educational practices. It is made up of three elements: affection, cognition, and behavior [103].
Behavioral engagement consists of the degree of involvement, imagination, attention, enthusiasm, and willingness to learn students demonstrate during the learning process. Cognitive engagement can be portrayed as the psychological or mental investment and the student’s effort to understand, learn, or acquire accurate knowledge and skills [9].
Strength of educational games derives from their potential to induce student motivation and engagement during the activities performance [30,74,92,104,105]. In [108], it is also highlighted that games can vary their difficulty level, keeping the participant at a high level of engagement. In turn, in [75], it is declared that the games’ learning anxiety might increase the students’ attention as they devote more time to the learning process and task fulfillment, thus obtaining more knowledge retention. The interaction of users with social factors also influences engagement [11,79].
Another crucial behavioral attitude is student satisfaction, which positively affects learning and brings a sense of pleasure [75,80], according to 21 articles. Satisfaction is related to student performance and is an indicator of educational quality as well as an important factor in measuring success [9,10,11,12,13,72,78] or the effectiveness of the e-learning process [98,115].
Studies show that the perception of ease of use and, mainly, playfulness (or usefulness) directly affects the satisfaction and continuous use of a gamified system [80,83]. Social influences generate a sense of competition and improve results, thus yielding various emotional skills such as self-satisfaction, self-esteem, self-efficacy, and pride [79,83,88,113,116]. Still, activities that satisfy the motivational needs for capacity, autonomy, and skills positively impact users’ satisfaction [33].

3.1.2. Relationship between the Behavioral Attitudes Associated with Gamification

Although the behavioral attitudes presented in the previous section are said to be the main ones, there are no studies in the academic literature that associate these attitudes. This work contributes to mitigating the gap.
Therefore, given what is presented in the literature individually, it is possible to analyze the relationships between the three attitudes pairwise. From the Venn diagram, it is possible to verify that of the 63 articles that mention motivation, engagement, and satisfaction as the main behavioral attitudes to promote the gamified learning process, five simultaneously attribute these attitudes while 39 articles, that is, 61.9% of the total, mention them separately. Still, 24 articles indicate at least two of these attitudes, corresponding to 38% of the total. One more attitude, satisfaction, is little mentioned in isolation (9.5%), which shows that students need to be engaged and motivated to be satisfied with the learning process (Figure 2).
Finally, it is possible to conclude that motivation, engagement, and satisfaction improve student interest in the learning process and performance results and should be highly encouraged in gamified systems.

3.1.3. The Main Elements Used to Stimulate the Learning Cycle

Activities based on active learning and gamification can be an alternative to influence more positive attitudes among students and create a more friendly environment for teaching [117]. Besides, the active learning provided by gamification is among the most effective learning strategies as it provides the students with better understanding, retention of information, and greater use of the classes [12,96,98,111].
It is believed that identification and use of game elements in this process can boost learning in several segments, domains, and research areas [31]. The main elements identified in the systematic review are presented by steps to follow (Table 2).
Step 1:
User-centricity and personalized experiences
It is necessary to analyze the psychological profile of the student. Theory and practice should be centered on the individual and not on technology, guided by the learner’s cognitive and socioemotional profile and how it impacts the learning technology. In personalization, user-centered design is of increasing importance and requires clear recognition. The user-centered design allows learning experiences to be psychologically accessible and not just physically. It also reflects the student’s cognitive knowledge and socioemotional profile besides how unique psychological attributes relate to the environment and the pedagogical framework. Still, it allows considering that each student has different needs that must be acknowledged [12,27,54,75,105,119,120,121,123,124].
Personalized or adaptive learning experiences for each student are an essential characteristic of e-learning systems. When referring to personal information, systems can provide customized content for each student or direct them to a specific path [122]. Besides, it must focus on a target audience and be age-oriented. To be successful, correctly getting the age of future players is crucial [34,118].
Step 2:
Challenging and clear missions
Games need to be challenging, involving an appropriate amount of difficulty and complexity, having multiple objectives, and some amount of randomness [30,33]. Players must overcome challenges along the way of the game to achieve specific and clear goals [103].
The challenge refers to the level of difficulty of the activities. It is an element present in many case studies, which states that increasing difficulty levels for students help them develop knowledge progressively [5,30,76,79,99,125,127]. The challenges must be adequately engaging so that students are motivated to solve them. However, it is vital to ensure that the challenges are not too complicated nor generate boredom or anxiety. The challenge should be considered when defining the game’s goals and rules to guarantee that they are both feasible and encouraging [55]. Moreover, it must start at a lower level before increasing the difficulty to keep the interest and the flow going. If there is a moderate difficulty due to in-game mechanics variations, these mechanics should be manageable [118].
Furthermore, the tasks to be performed must be clearly explained by involving players in an active learning process that helps them master the game’s mechanics and overcome challenges [82,93,103,118,126].
Step 3:
Narrative and fantasy
The game’s context, which is expressed by its narrative, graphics, and music, can stimulate the players’ interest if adequately used [93]. The staging and storyline provide scenarios that help the players’ decision-making [53]. A strong narrative with multiple arcs is a sequence of events based on storytelling, exciting characters that students can interact with, and a realistic scenario supported by challenging resources and activities can also influence learning outcomes [103].
The narrative consists of some essential aspects such as event, character, configuration, arrangement, and time. The results show that with these elements, students can convince themselves with the game’s stories through fantasy, which leads to higher levels of learning and motivation to complete tasks [11,39,72,74,79,81,84,106,110]. However, the emphasized fantasy must be carefully chosen to attract the target audience and create intrinsically motivating environments.
It also allows for a fantastic metaphor and a dramatization dynamic underlying a problem-based learning strategy [95,128]. Still, it makes it possible to create more authentic and realistic environments that contribute to better identifying behaviors and greater involvement of participants [102].
Step 4:
Repetitive Loops and Freedom to Fail
The repetitive loops of trying, failing, tuning, improving, and trying again in a game are called productive negativity and contribute to game design. A user tries a challenge with their current knowledge and may fail (negative). They need then to reformulate their understanding (productive) to achieve the game’s goal and get a result [71].
In the form of resentment, disappointment, and irritation, this productive negativity exposes students to something new. It allows them to reach a different perspective of awareness through experience, which promotes learning. Disappointment, failure, and frustration can play a prominent role in motivating and learning in the game. Failure is necessary for this experience, so much so that winning without previous losses creates a feeling of discontent with the victory. In general, games cause negativity while encouraging task-solving, thus ensuring the advancement of knowledge and skills through an engaging environment [71].
Although some degree of anxiety caused by failure is required, it is not interesting when it turns into disappointment. Therefore, task sequences must be carefully designed to suit the players’ knowledge and profile, including low penalties for failures to allow multiple attempts and the willingness to get it right [27,55,93].
Still, the literature suggests that several repetition practices are part of the learning process and are necessary to achieve the desired quality [39,71,87,93,98,109,129].
Step 5:
Competition and social engagement
Competition is a crucial element of games and is constantly used to improve academic performance [9,13]. It must be considered in the game, either against yourself or other users [30,55]. The desire to win the game results in competition and ambition, making the students think and learn quickly to achieve success, providing extra motivation that makes it possible to achieve better results [30,131]. Many scholars believe that the use of competition can improve the robustness of the game’s design [32,55,85,95,98,104,110,116,126,130,131].
The competition allows students to share ideas for help or shift the blame in failure cases. Besides, students like to learn from the unexpected and excitement, waiting for what will come next asking what the topic will be and who will win. That is, challenges and visibility of tasks by other users can boost competition [30,79,92].
Social influence can also stimulate the competition to reach a higher status on the leaderboard and enable numerous emotional skills such as self-satisfaction, self-esteem, and pride [9,79]. Leaderboards, which usually contain names, nicknames, and/or avatars, serve as sources of motivation because students want their effort to be recognized publicly. It helps to compare their progress with other colleagues [10,93].
However, too much emphasis on competition can also hurt some users as they may become obsessed with trying to improve their points and, therefore, lose focus on learning [39,55].
Step 6:
Feedback and reflection
Providing feedback and allowing knowledge acquisition through game experience are also essential factors in a positive learning experience [55,75,99,124,134,136]. Feedback prevents students from getting lost or confused about what to do and how to do a task or mission. Furthermore, it helps in tracking progress, performance, improving understanding, and encouraging engagement [70,77,92,106,111,112,133,137,138]. Feedback can also help students experiment and learn from the game [3,135].
In complex games, it is essential to provide specific and individual feedback provided by a facilitator or a tutor for the evolution of tasks [55]. Although the teacher does not act synchronously during the game, he/she plays the role of the mentor by clarifying the rules, training, sending information, updating the content, and providing feedback on the game’s progress [13].
Many students consider immediate feedback as a gamified environment [50,75,138]. According to the learner’s pace and time, direct, individualized, detailed, and continuous feedback is considered to be of great importance to increase engagement and improve understanding [30,70,73,82].
Feedback can also increase through levels by determining specific points and allowing content to be unlocked progressively through levels. The same logic works for tasks as a task answered correctly allows the user to proceed to the next step. This visual progression of content allows feedback and increases focus on the learning process [10,132]. Furthermore, dialog boxes clarifying the appropriate responses, explaining why a response is correct, or simply indicating that the choice was not the most suitable in an educational and nonpunitive way are important sources of feedback [103].
Step 7:
Reward and social credibility
Games try to activate the students’ feelings with reward systems that immediately recognize the success of players [93]. The reward-scoring systems, plotting with animations, badges, trophies, and leaderboards are often employed in educational games to stimulate the player and encouragingly present the progress of activities. When a reward is involved, the players are generally more willing to win, and thus their motivation is increased and eventually encourages participation. Furthermore, it provides teachers with an incentive to introduce technologies into the classroom and offer learning feedback [5,33,56,71,72,75,79,93].
The rewards can be through virtual or real prizes based on the time spent in the system after completing tasks or on other elements. Commitment, motivation, engagement, or satisfaction are the main benefits of this element. The awards/badges and honors are directly related to the visual status and social credibility of players, and that is why they are considered highly motivating, fun, and encouraging [10].
The use of badges (medals) is a typical method of reward. They are visual displays of the users’ progress that can function as an extrinsic reward, leading people to perform specific activities to receive them. They can also stimulate social comparison, that is, compare their achievements with the accomplishments of others and serve as a conductor of behavior [8,85]. The possibility of posting the progress on the social media timeline to share with friends can increase motivation, user interaction with the game and facilitate self-directed learning [83].
However, care must be taken to balance intrinsic and extrinsic motivation in reward systems. Learning can be most effective when one is intrinsically motivated, for example, by one’s curiosity or desire to solve a challenge for one’s satisfaction; or extrinsically, to obtain a grade or other earnings [8,33,55,93].
The elements listed here are essential to promote the learning process. The logic of the proposed learning cycle will not change if other elements are added following the steps shown in Table 2. This addition can make the process even more user-friendly.

3.1.4. Relationship between Gamification Elements

Although game elements are the basic building blocks of gamification, there is no consensus on their use. Academic literature on the topic is dispersed, isolated, and fragmented, as shown in Table 3. Within this scenario, this article focuses on the dispersed elements, that is, to understand the relationship between the elements of the main game to boost the students’ behavioral attitudes and then organize them in a didactic way to contribute to a better learning process.
Table 3 shows the main game elements identified in this work and compares how the studies mention these elements together. It can be seen that the authors make little association of the “User-centricity and personalized experiences” element with the other elements. The “Challenging and clear missions” element and “Repetitive loops and freedom to fail” appear with representation in all the other elements, confirming their importance in gamified systems. In addition, “Narrative and fantasy” is more related to “Competition and social engagement” since systems that allow for greater immersion in the game’s narrative positively impact the competition and the desire to obtain better results to share in social networks. Furthermore, although “Feedback and reflection” in quantitative terms is the most representative element, it still appears in isolation, with few citations compared to other elements. Finally, it is possible to note that the “Reward and social credibility” element is related to “Challenging and clear missions” and “Competition and social engagement”, which is expected since for students to receive rewards, they need to have clear game rules to stimulate competition.
Therefore, this work aimed to reduce the gap in the literature regarding the association of these elements and present how the aspects of gamification can be organized in a sequence of steps to promote behavioral attitudes and ensure the quality of the learning process.

3.2. A Theoretical Framework

The step-by-step idea based on the “Seven Problem-Solving Steps” approach of the WCM methodology was used and adapted to present, in a sequential manner, the evolution of concepts and the system planning process to achieve the learning allowed by gamification in e-learning by increasing motivation, engagement, and satisfaction of students. These elements were classified here using the acronym 7GOALS—Gamification-Oriented Active Learning Steps.
The 7GOALS were identified in the literature in a dispersed and isolated way and here they were gathered and structured to facilitate the design of gamified environments in e-learning. With the support of the PDCA, the elements were combined in phases of planning, execution, control, and action to cluster the systematic activities necessary to create such systems, ensure that all stages are considered and meet the fundamental requirements for the perception of value in students’ interactions with the gamified learning process.
The association of the PDCA with the 7GOALS steps is shown by the framework presented in Figure 3. Thus, through increasing cycles of evolution aimed at student learning, it is possible to promote the improvement and expansion of gamification in this process.
Step 1 (user-centered and personalized experiences), Step 2 (challenging and clear missions), and Step 3 (narrative and fantasy) are part of the system’s planning process (P—Plan). The aim was to define the student’s profile, cognitive skills, target audience, age group, level of difficulty of the activities and establish the challenges by clearly explaining their objectives and drawing them through narratives with a fantasy metaphor to attract the initial audience. This step is vital to planning a gamified system, that is, to get to know the public for whom it is intended, define the challenges that these students expect, and understand the necessary narrative for the immersion to be complete.
Then, Step 4 (repetitive loops and freedom to fail) and Step 5 (competition and social engagement) are associated with the execution of tasks by students (Phase D—Do). Thus, repeating activities several times with the permission to fail prepares students for proficiency in skills/knowledge and competition through social influence and leaderboards. It is here that the student will be able to perform the tasks, repeat them as many times as necessary, not be severely punished for failures, in addition to competing with other students collaboratively or individually with the possibility of sharing the results through social networks.
Then, Step 6 (feedback and reflection) allows you to monitor the learning process, verify that the objectives have been achieved, and reflect on the necessary improvements (Phase C—Check). At this time, students will be able to have signs of the progress of tasks and direction regarding what needs to be done to complete them. As seen in the systematic literature review, several studies consider this element as one of the main elements to learning effectively.
Finally, Step 7 (reward and social credibility) corresponds to the search for the expansion of knowledge caused by the stimulus of rewards and the social credibility that points, medals, and trophies, among others, represent (Phase A (Act)). It is a way of celebrating learning, correct the flaws found in the previous step, and maintaining that constant cycle.

4. Discussion

Although recent, gamification in e-learning has been widely discussed in the literature in several areas, such as mathematics, social sciences, engineering, health, environment, sustainability, and others [32,33,34,35]. The focus of gamification in the learning process has always been on increasing motivation, engagement, and satisfaction to promote behavioral change and increase interest in learning [10,12,18,27,93], whether to stimulate sustainable environments or in the context of general education [53,87].
For example, it is possible to observe that gamification has already been applied in business (to engage employees and increase customers’ loyalty), education (to foster the engagement of students), and environmental sustainability (to inspire and provoke engagement in environmentally positive behaviors) [79]. In the context of education and sustainable environments, scholars have conducted studies to confirm further the advantages of the learner-centered scenario design and game-based learning approach in the knowledge transfer in the topic of sustainability [53]. Some studies integrate agribusiness with environmental and social issues to boost sustainability through gamification applied to producers/professionals/pupils/students on a journey of discovery about agribusiness in all its complexity [128]. Others highlight the players’ quest to achieve the success, whether by fulfilling the game’s mission or developing a sustainable society [97]. Another one mentions the importance of using technology throughout the school working together with companies to produce sustainable, durable, affordable, and safe devices for students in the classroom [5]. In this way, game-based learning and adaptive learning are reasonable solutions to motivate the student, address the knowledge facing a large and heterogeneous public, reduce the costs of teaching collaborating with the financial aspects of education, and share resources and infrastructure for a sustainable society [87].
Learning based on game elements is one of the main trends currently in e-learning [3,55,56]. This learning format can be designed to collaboratively solve complex, real-world problems, make the learning process more efficient, get predictive modeling and real-time visualization, and retain more knowledge than the traditional methods [81,125]. In [108], games were used in virtual-world scenarios to take risks in a safe place for real-world activities. In [93], the students who completed the gamified experiment obtained better grades for practical tasks and overall scores; the results and effectiveness of learning compared to the traditional models is also presented in [102]. Learning to deal with complex and constantly evolving environments is also a result of this learning format [32].
Thus, gamified systems that encourage these behaviors and mix the lightness of playfulness with serious content can be used to disseminate changes towards more sustainable behaviors, considering psychological and social aspects that influence students in the learning process.
In addition, they can be powerful vehicles for assessing skills and behaviors such as persistence, flexibility, creativity, critical thinking, and cognitive development, positively impacting the students’ academic performance [9,33,75,80,93,103,106]. In this way, if the game is focused only on teaching, forgetting the playfulness part, you can create a tool that, despite being motivational, does not succeed in the results. On the other hand, if it is focused only on playfulness without paying attention to the motivational part, it is possible to create a product that does not serve as an awareness tool and does not fulfill its objective. Therefore, games must be developed in a format that should be as fun as possible and as a motivational tool that addresses the teaching theme without the player feeling that he is studying [34].
These duality, playfulness, and seriousness of the teaching process need to be contemplated in gamified systems. The 7GOALS is proposed to help with this, bringing together the main building blocks of gamification. The correct game elements to leverage the learning process and behavioral perspectives should be addressed. Thus, 7GOALS merge the playfulness and seriousness of teaching and support the planning of gamified systems to stimulate sustainable education in both knowledge and behavioral attitudes. First, it establishes elements that seek to know the user centrality and rules required to achieve the learning objectives and then include elements of fantasy, narrative, freedom of failure, feedback, and recognition to provide fun and awareness of learning simultaneously.
In this way, the leading game elements that encourage behavioral attitudes are organized in a logical sequence to guarantee the quality of the learning process. The steps are as follows: user centrality and experience personalization through challenges and clear missions, use of narratives and fantasies for better immersion in the game’s history, allowing repetitive loops with the possibility of failures, encouragement by competition and social engagement to achieve success, providing feedback and reflection to progress and performance, and, finally, rewards and social credibility to celebrate achievements and increase motivation. The PDCA cycle was used to expand the learning process through gamification in e-learning, with its four stages (planning, execution, monitoring, and improvement actions). These steps were associated with 7GOALS and the students’ behavioral attitudes with the development of a framework.
This will help to enable greater applicability of gamified systems for teaching sustainability, explore the finite nature of natural resources through the immersion in games, and demonstrate the importance of collective action at the community, national, and global levels.
The encouragement of more sustainable behaviors for this purpose through the game elements presented here and attitudes such as satisfaction, engagement, and motivation can contribute to students’ awareness and proximity to the real world that requires care and attention. In addition, the proposed format can be playfully used in sustainable education, approaching the new generations that are adept at this format and at the same time bringing a discussion that needs to be exhaustively contemplated in educational environments for greater environmental awareness and correspondence to the sustainable goals of the millennium (United Nations Sustainable Development Goals [139]). It is also possible to be used in the three objectives that make up the sustainability tripod, that is society [140], economy [141] and environment [142]. Furthermore, it can be an excellent resource to bring up sustainability issues in business environments that are not sustainable and help to intraduce the topic to those who have not had access to it yet.

5. Conclusions

This work aimed to present a framework proposal that relates behavioral attitudes and gamification elements to the stages of the PDCA (Plan, Do, Check, Act) to continuously improve the learning process over time. Although gamification is possibly the methodological tool that has stood out the most for implementation in the educational field and disciplines such as sustainability, the literature still presents the game’s elements in an isolated, dispersed, and fragmented way. Therefore, this work aimed to reduce the gap in the literature since to the best of the authors’ knowledge, this is the first study that brings together the main game elements, the PDCA cycle, and behavioral attitudes to stimulate change for more sustainable behavioral attitudes and ensure the quality of the learning process. This work can be helpful for researchers, teachers, and scholars who intend to use gamified environments in e-learning, such as for sustainability across several fields.

Author Contributions

Conceptualization, R.P.O., C.G.d.S., A.d.C.R. and W.M.d.S.; Methodology, R.P.O., C.G.d.S., A.d.C.R. and W.M.d.S.; Investigation, R.P.O., C.G.d.S., A.d.C.R. and W.M.d.S.; Writing—original draft preparation, R.P.O., C.G.d.S., A.d.C.R. and W.M.d.S.; Validation, R.P.O., C.G.d.S., A.d.C.R. and W.M.d.S.; Writing—review and editing, R.P.O., C.G.d.S., A.d.C.R. and W.M.d.S. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Not applicable.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Arouri, Y.M.; Odat, B. The Extent of Using E-Learning Among Teachers of Higher Basic Stage in Jordan. Int. J. Inf. Commun. Technol. Educ. 2020, 16, 57–69. [Google Scholar] [CrossRef]
  2. Liu, G.-Z.; Wu, N.-W.; Chen, Y.-W. Identifying emerging trends for implementing learning technology in special education: A state-of-the-art review of selected articles published in 2008–2012. Res. Dev. Disabil. 2013, 34, 3618–3628. [Google Scholar] [CrossRef]
  3. Sood, S.K.; Singh, K.D. An Optical-Fog assisted EEG-based virtual reality framework for enhancing E-learning through educational games. Comput. Appl. Eng. Educ. 2018, 26, 1565–1576. [Google Scholar] [CrossRef]
  4. Moubayed, A.; Injadat, M.; Shami, A.; Lutfiyya, H. Student Engagement Level in an e-Learning Environment: Clustering Using K-means. Am. J. Distance Educ. 2020, 34, 137–156. [Google Scholar] [CrossRef]
  5. Chew, S.W.; Cheng, I.-L.; Chen, N.-S. Exploring challenges faced by different stakeholders while implementing educational technology in classrooms through expert interviews. J. Comput. Educ. 2018, 5, 175–197. [Google Scholar] [CrossRef]
  6. Silva, F.B. Implicações da Gamificação no Projeto de Plataforma de Educação Online: Um Estudo de Caso [Tese (Doutorado)]; Universidade Federal de Minas Gerais: Belo Horizonte, Brazil, 2018. [Google Scholar]
  7. Ramos, D.K.; Da Silva, G.A. Formação de professores a distância para o uso de jogos digitais na escola: Mudanças na prática pedagógica? RIED 2019, 22, 265–285. [Google Scholar] [CrossRef]
  8. Kyewski, E.; Krämer, N.C. To gamify or not to gamify? An experimental field study of the influence of badges on motivation, activity, and performance in an online learning course. Comput. Educ. 2018, 118, 25–37. [Google Scholar] [CrossRef]
  9. Bouchrika, I.; Harrati, N.; Wanick, V.; Wills, G. Exploring the impact of gamification on student engagement and involvement with e-learning systems. Interact. Learn. Environ. 2019, 1–14. [Google Scholar] [CrossRef]
  10. Garcia--Cabot, A.; Garcia--Lopez, E.; Caro--Alvaro, S.; Gutierrez--Martinez, J.; De--Marcos, L. Measuring the effects on learning performance and engagement with a gamified social platform in an MSc program. Comput. Appl. Eng. Educ. 2019, 28, 207–223. [Google Scholar] [CrossRef]
  11. Romero-Rodriguez, L.M.; Ramirez-Montoya, M.-S.; Gonzalez, J.R.V. Gamification in MOOCs: Engagement Application Test in Energy Sustainability Courses. IEEE Access 2019, 7, 32093–32101. [Google Scholar] [CrossRef]
  12. Terras, M.M.; Boyle, E.A. Integrating games as a means to develop e--learning: Insights from a psychological perspective. Br. J. Educ. Technol. 2019, 50, 1049–1059. [Google Scholar] [CrossRef]
  13. Lorenzo--Alvarez, R.; Rudolphi-Solero, T.; Ruiz-Gómez, M.J.; Sendra--Portero, F. Game--Based Learning in Virtual Worlds: A Multiuser Online Game for Medical Undergraduate Radiology Education within Second Life. Anat. Sci. Educ. 2019, 13, 602–617. [Google Scholar] [CrossRef]
  14. Koroleva, K.; Novak, J. How to Engage with Sustainability Issues We Rarely Experience? A Gamification Model for Collective Awareness Platforms in Water-Related Sustainability. Sustainability 2020, 12, 712. [Google Scholar] [CrossRef] [Green Version]
  15. Rubin-Vaughan, A.; Pepler, D.; Brown, S.; Craig, W. Quest for the Golden Rule: An effective social skills promotion and bullying prevention program. Comput. Educ. 2011, 56, 166–175. [Google Scholar] [CrossRef]
  16. Callaghan, M.J.; McCusker, K.; Losada, J.L.; Harkin, J.; Wilson, S. Using Game-Based Learning in Virtual Worlds to Teach Electronic and Electrical Engineering. IEEE Trans. Ind. Inform. 2013, 9, 575–584. [Google Scholar] [CrossRef]
  17. Cura-González, I.; López-Rodríguez, J.A.; Sanz-Cuesta, T.; Rodríguez-Barrientos, R.; Martín-Fernández, J.; Ariza-Cardiel, G.; Polentinos-Castro, E.; Román-Crespo, B.; Escortell-Mayor, E.; Rico-Blázquez, M.; et al. Effectiveness of a strategy that uses educational games to implement clinical practice guidelines among Spanish residents of family and community medicine (e-EDUCAGUIA project): A clinical trial by clusters. Implement Sci. 2015, 11, 71. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  18. Lemcke, S.; Haedge, K.; Zender, R.; Lucke, U. RouteMe: A multilevel pervasive game on mobile ad hoc routing. Pers. Ubiquitous Comput. 2015, 19, 537–549. [Google Scholar] [CrossRef]
  19. Easterday, M.W.; Aleven, V.; Scheines, R.; Carver, S.M. Using Tutors to Improve Educational Games: A Cognitive Game for Policy Argument. J. Learn. Sci. 2016, 26, 226–276. [Google Scholar] [CrossRef]
  20. Hernández-Lara, A.B.; Serradell-López, E.; Fitó-Bertran, À. Students’ perception of the impact of competences on learning: An analysis with business simulations. Comput. Hum. Behav. 2019, 101, 311–319. [Google Scholar] [CrossRef]
  21. Kenigsberg, P.-A.; Aquino, J.-P.; Bérard, A.; Gzil, F.; Andrieu, S.; Banerjee, S.; Brémond, F.; Buee, L.; Cohen-Mansfield, J.; Mangialasche, F.; et al. Dementia beyond 2025: Knowledge and uncertainties. Dementia 2015, 15, 6–21. [Google Scholar] [CrossRef] [Green Version]
  22. Newbery-Jones, C. Ethical experiments with the D-pad: Exploring the potential of video games as a phenomenological tool for experiential legal education. Law Teach. 2016, 50, 61–81. [Google Scholar] [CrossRef]
  23. Cárdenas-Robledo, L.A.; Peña-Ayala, A. Ubiquitous learning: A systematic review. Telemat. Inform. 2018, 35, 1097–1132. [Google Scholar] [CrossRef]
  24. Wisittanawat, P.; Gresalfi, M.S. The “tricky business” of genre blending: Tensions between frames of school mathematics and video game play. J. Learn. Sci. 2020, 30, 240–278. [Google Scholar] [CrossRef]
  25. Clark, D.; Tanner-Smith, E.; Hostetler, A.; Fradkin, A.; Polikov, V. Substantial Integration of Typical Educational Games Into Extended Curricula. J. Learn. Sci. 2017, 27, 265–318. [Google Scholar] [CrossRef]
  26. Bekebrede, G.; Warmelink, H.; Mayer, I. Reviewing the need for gaming in education to accommodate the net generation. Comput. Educ. 2011, 57, 1521–1529. [Google Scholar] [CrossRef]
  27. Simões, J.; Redondo, R.D.; Vilas, A.F. A social gamification framework for a K-6 learning platform. Comput. Hum. Behav. 2013, 29, 345–353. [Google Scholar] [CrossRef]
  28. Martín-SanJosé, J.-F.; Juan, M.-C.; Gil-Gómez, J.-A.; Rando, N. Flexible learning itinerary vs. linear learning itinerary. Sci. Comput. Program. 2014, 88, 3–21. [Google Scholar] [CrossRef]
  29. Nyembezi, N.; Cishe, N.; Mantlana, D. Teachers’ Adoption of Information and Communication Technology in Foundation Phase Teaching: A Study of Selected Schools in the Eastern Cape in South Africa. Int. J. Educ. Sci. 2016, 12, 57–65. [Google Scholar] [CrossRef]
  30. Karaaslan, H.; Kilic, N.; Yalçin, G.G.; Gullu, A. Students’ Reflections on Vocabulary Learning through Synchronous and Asynchronous Games and Activities. Turk. Online J. Distance Educ. 2018, 19, 53–70. [Google Scholar] [CrossRef]
  31. Alshammari, M.T. Design and Learning Effectiveness Evaluation of Gamification in e-Learning Systems. Int. J. Adv. Comput. Sci. Appl. 2019, 10. [Google Scholar] [CrossRef]
  32. Wautelet, Y.; Kolp, M. e-SPM: An online software project management game. Int. J. Eng. Educ. 2012, 28, 11. [Google Scholar]
  33. De-Marcos, L.; García-Cabo, A.; López, E.G. Towards the social gamification of e-learning: A practical experiment. Int. J. Eng. Educ. 2017, 33, 66–73. [Google Scholar]
  34. Romero-Hernandez, A.; Riojo, M.G.; Diaz-Faes-Perez, C.; Manero-Iglesias, B. The Courtesy of Spain: Theater for the New Generations. IEEE Rev. Iberoam. Tecnol. Aprendiz. 2018, 13, 102–110. [Google Scholar] [CrossRef]
  35. Manzano-León, A.; Camacho-Lazarraga, P.; Guerrero, M.; Guerrero-Puerta, L.; Aguilar-Parra, J.; Trigueros, R.; Alias, A. Between Level Up and Game Over: A Systematic Literature Review of Gamification in Education. Sustainability 2021, 13, 2247. [Google Scholar] [CrossRef]
  36. Klock, A.C.T.; Gasparini, I.; Pimenta, M.S. User-Centered Gamification for E-Learning Systems: A Quantitative and Qualitative Analysis of its Application. Interact. Comput. 2019, 31, 425–445. [Google Scholar] [CrossRef]
  37. Paravizo, E.; Chaim, O.C.; Braatz, D.; Muschard, B.; Rozenfeld, H. Exploring gamification to support manufacturing education on industry 4.0 as an enabler for innovation and sustainability. Procedia Manuf. 2018, 21, 438–445. [Google Scholar] [CrossRef]
  38. Torres-Toukoumidis, A.; Ramírez-Montoya, M.S.; Romero-Rodríguez, L.M. Assessment and evaluation of games-based learning (gbl) in e-learning contexts. Educ. Knowl. Soc. 2019, 19, 109–128. [Google Scholar] [CrossRef]
  39. Ebner, M.; Holzinger, A. Successful implementation of user-centered game based learning in higher education: An example from civil engineering. Comput. Educ. 2007, 49, 873–890. [Google Scholar] [CrossRef]
  40. Bieliková, M.; Divéky, M.; Jurnečka, P.; Kajan, R.; Omelina, L. Automatic generation of adaptive, educational and multimedia computer games. Signal Image Video Process. 2008, 2, 371–384. [Google Scholar] [CrossRef] [Green Version]
  41. Wang, T.-H. Web-based quiz-game-like formative assessment: Development and evaluation. Comput. Educ. 2008, 51, 1247–1263. [Google Scholar] [CrossRef]
  42. Miyagawa, T.; Yamagishi, Y.; Mizuno, S. Walk-rally support system using two-dimensional codes and mobile phones. Interact. Learn. Environ. 2013, 21, 199–210. [Google Scholar] [CrossRef]
  43. DelSignore, L.A.; Wolbrink, T.A.; Zurakowski, D.; Burns, J.P.; Hoy, M.; Larsen, D.P.; Johnson, H.; Getchius, T. Test-Enhanced E-Learning Strategies in Postgraduate Medical Education: A Randomized Cohort Study. J. Med. Internet Res. 2016, 18, e299. [Google Scholar] [CrossRef] [PubMed]
  44. Khenissi, M.A.; Essalmi, F.; Jemni, M.; Chang, T.-W.; Chen, N.-S. Unobtrusive monitoring of learners’ interactions with educational games for measuring their working memory capacity. Br. J. Educ. Technol. 2017, 48, 224–245. [Google Scholar] [CrossRef]
  45. Stevenson, C.D.; Porter, K.; Stevenson, K.T. Effects of a Game-Based e-Learning Module on Undergraduate Food Science Students’ Planned Behaviors Concerning Good Manufacturing Practices. J. Food Sci. Educ. 2018, 17, 111–117. [Google Scholar] [CrossRef] [Green Version]
  46. Tolks, D.; Lampert, C.; Dadaczynski, K.; Maslon, E.; Paulus, P.; Sailer, M. Spielerische Ansätze in Prävention und Gesundheitsförderung: Serious Games und Gamification. Bundesgesundheitsbl Gesundh. Gesundh. 2020, 63, 698–707. [Google Scholar] [CrossRef] [PubMed]
  47. Alshammari, M.T. Evaluation of Gamification in E-Learning Systems for Elementary School Students. TEM J. 2020, 9, 806–813. [Google Scholar] [CrossRef]
  48. Santos-Villalba, M.J.; Leiva Olivencia, J.J.; Navas-Parejo, M.R.; Benítez-Márquez, M.D. Higher Education Students’ Assessments towards Gamification and Sustainability: A Case Study. Sustainability 2020, 12, 8513. [Google Scholar] [CrossRef]
  49. Zhong, R.Y.; Huang, G.Q. RFID-enabled learning supply chain: A smart pedagogical environment for TELD. Int. J. Eng. Educ. 2014, 30, 471–482. [Google Scholar]
  50. Boada, I.; Benitez, A.R.; Garcia-Gonzalez, J.M.; Olivet, J.; Carreras, V.; Sbert, M. Using a serious game to complement CPR instruction in a nurse faculty. Comput. Methods Programs Biomed. 2015, 122, 282–291. [Google Scholar] [CrossRef]
  51. Akaslan, D.; Law, E.L.-C. A model for flipping electrical engineering with e-learning using a multidimensional approach. Turk. J. Electr. Eng. Comput. Sci. 2016, 24, 3419–3431. [Google Scholar] [CrossRef]
  52. Johnsen, H.M.; Fossum, M.; Vivekananda-Schmidt, P.; Fruhling, A.; Slettebø, A. Nursing students’ perceptions of a video-based serious game’s educational value: A pilot study. Nurse Educ. Today 2018, 62, 62–68. [Google Scholar] [CrossRef] [PubMed]
  53. Chan, C.-S.; Chan, Y.-H.; Fong, T.H.A. Game-based e-learning for urban tourism education through an online scenario game. Int. Res. Geogr. Environ. Educ. 2019, 29, 283–300. [Google Scholar] [CrossRef]
  54. Balogun, W.G. Using Electronic Tools and Resources to Meet the Challenges of Anatomy Education in Sub-Saharan Africa. Anat. Sci. Educ. 2018, 12, 97–104. [Google Scholar] [CrossRef] [PubMed]
  55. Doney, I. Research into effective gamification features to inform e-learning design. Res. Learn. Technol. 2019, 27. [Google Scholar] [CrossRef] [Green Version]
  56. Bernik, A.; VusiÄ, D.; Milkovic, M. Evaluation of Gender Differences Based on Knowledge Adaptation in the Field of Gamification and Computer Science. IJET 2019, 14, 220–228. [Google Scholar] [CrossRef]
  57. Moher, D.; Liberati, A.; Tetzlaff, J.; Altman, D.G. The PRISMA Group Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLoS Med. 2009, 6, e1000097. [Google Scholar] [CrossRef] [Green Version]
  58. Vieira, P.S. Supply Chain Risk Management in Healthcare: Identification, Assessment and Mitigation; Programa de Pós-Graduação em Engenharia de Produção e Sistemas, Federal Center for Technological Education of Rio de Janeiro: Rio de Janeiro, Brazil, 2019. [Google Scholar]
  59. Santos, A.C.S.G.; da Cunha Reis, A.; de Souza, C.G.; dos Santos, I.L.; Ferreira, L.A.F. The first evidence about conceptual vs. analytical lean healthcare research studies. J. Health Organ. Manag. 2020, 34, 789–806. [Google Scholar] [CrossRef]
  60. Da Cunha Reis, A.; Oliveira, R.P.; de Carvalho Castro, A. Logística hospitalar: Uma síntese do estado da arte. 2018, 15, 205–231. Revista Gestão Desenvolvimento 2018, 15, 205–231. [Google Scholar] [CrossRef] [Green Version]
  61. Silva, M.M.; De Gusmão, A.P.; De Andrade, C.T.A.; Silva, W. The integration of VFT and FITradeoff multicriteria method for the selection of WCM projects. In Proceedings of the 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC), Bari, Italy, 6–9 October 2019; pp. 1513–1517. [Google Scholar] [CrossRef]
  62. Arsovski, S.; Đokić, I.; Pešić-Đokić, S. Quality in world class manufacturing. Int. J. Qual. Res. 2011, 5, 309–316. [Google Scholar]
  63. Svrzić, U.; Danon, G. Solving problems in wood parquet production by using ‘7 steps’ of World Class Manufacturing Methodology (WCM). J. Appl. Eng. Sci. 2014, 12, 113–120. [Google Scholar] [CrossRef] [Green Version]
  64. Novicka, A.; Papcun, P.; Zolotova, I. Mapping of machine faults using tools of World Class Manufacturing. In Proceedings of the 2016 IEEE 14th International Symposium on Applied Machine Intelligence and Informatics (SAMI), Herlany, Slovakia, 21–23 January 2016; pp. 223–227. [Google Scholar] [CrossRef]
  65. Vyt, A. Smartly using PDCA in quality of distance teaching. In The Envisioning Report for Empowering Universities, 3rd ed.; European Association of Distance Teaching Universities: Maastricht, The Netherlands, 2019. [Google Scholar]
  66. Hasan, Z.; Mallik, A.; Tsou, J.-C. Learning method design for engineering students to be prepared for Industry 4.0: A Kaizen approach. High. Educ. Ski. Work. Learn. 2020, 11, 182–198. [Google Scholar] [CrossRef]
  67. Pastrana, R.; Manabat, A. An Outcomes-Based Education (Obe) Approach & Typology-Based Quality Assurance (Qa) System: A Proposed Framework And Transition Strategy For Philippine Higher Education Institution’s (HEI) Shift Towards International Standards. Balk. Reg. Conf. Eng. Bus. Educ. 2014, 1, 639–644. [Google Scholar] [CrossRef] [Green Version]
  68. Chen, Y.; Zheng, J.; Wu, D.; Zhang, Y.; Lin, Y. Application of the PDCA cycle for standardized nursing management in a COVID-19 intensive care unit. Ann. Palliat. Med. 2020, 9, 1198–1205. [Google Scholar] [CrossRef]
  69. Pacheco, A.P.R.; Salles, B.W.; Garcia, M.A.; Possamai, A. O Ciclo PDCA na Gestão do Conhecimento: Uma Abordagem Sistêmica; Universidade Federal de Santa Catarina: Florianópolis, Brazil, 2012; Volume 2. [Google Scholar]
  70. Martinengo, L.; Yeo, N.J.Y.; Tang, Z.Q.; Markandran, K.D.; Kyaw, B.M.; Car, L.T. Digital Education for the Management of Chronic Wounds in Health Care Professionals: Protocol for a Systematic Review by the Digital Health Education Collaboration. JMIR Res. Protoc. 2019, 8, e12488. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  71. Gauthier, A.; Jenkinson, J. Designing productively negative experiences with serious game mechanics: Qualitative analysis of game-play and game design in a randomized trial. Comput. Educ. 2018, 127, 66–89. [Google Scholar] [CrossRef] [Green Version]
  72. Pérez-Manzano, A.; Almela-Baeza, J. Gamification and transmedia for scientific promotion and for encouraging scientific careers in adolescents. Comunicar 2018, 26, 93–103. [Google Scholar] [CrossRef] [Green Version]
  73. Olszewski, A.E.; Daniel, D.A.; Stein, D.R.; McCulloch, M.; Su, S.W.; Hames, D.L.; Wolbrink, T.A. Teaching Pediatric Peritoneal Dialysis Globally through Virtual Simulation. Clin. J. Am. Soc. Nephrol. 2018, 13, 900–906. [Google Scholar] [CrossRef] [PubMed]
  74. Jackson, L.C.; O’Mara, J.; Moss, J.; Jackson, A.C. A Critical Review of the Effectiveness of Narrative-Driven Digital Educational Games. Int. J. Game-Based Learn. 2018, 8, 32–49. [Google Scholar] [CrossRef]
  75. Ge, Z.-G. The impact of a forfeit-or-prize gamified teaching on e-learners’ learning performance. Comput. Educ. 2018, 126, 143–152. [Google Scholar] [CrossRef]
  76. Ortiz Benavides, F.L.; Piña Lopez, C.E. Estrategia tecno-didáctica para la solución de problemas de genética en estudiantes de educación a distancia. REurEDC 2018, 15, 1–19. [Google Scholar] [CrossRef] [Green Version]
  77. Kooiman, B.J.; Sheehan, D.P.; Wesolek, M.; Retegui, E. Moving online physical education from oxymoron to efficacy. Sport Educ. Soc. 2017, 22, 230–246. [Google Scholar] [CrossRef]
  78. Torres-Toukoumidis, A.; Romero-Rodríguez, L.M.; Perez-Rodriguez, A.; Björk, S. Modelo Teórico Integrado de Gamificación en Ambientes E-Learning (E-MIGA). 2017, 29, 129–145. Revista Complutense Educación 2017, 29, 129–145. [Google Scholar] [CrossRef] [Green Version]
  79. Sardi, L.; Idri, A.; Fernández-Alemán, J.L. A systematic review of gamification in e-Health. J. Biomed. Inform. 2017, 71, 31–48. [Google Scholar] [CrossRef] [PubMed]
  80. Lin, H.-C.; Chiu, Y.-H.; Chen, Y.J.; Wuang, Y.-P.; Chen, C.-P.; Wang, C.-C.; Huang, C.-L.; Wu, T.-M.; Ho, W.-H. Continued use of an interactive computer game-based visual perception learning system in children with developmental delay. Int. J. Med. Inform. 2017, 107, 76–87. [Google Scholar] [CrossRef] [PubMed]
  81. De-Marcos, L.; García-López, E.; García-Cabot, A.; Medina-Merodio, J.-A.; Domínguez, A.; Martínez-Herráiz, J.-J.; Diez-Folledo, T. Social network analysis of a gamified e-learning course: Small-world phenomenon and network metrics as predictors of academic performance. Comput. Hum. Behav. 2016, 60, 312–321. [Google Scholar] [CrossRef]
  82. Fernández, C.; Esteban, G.; Conde, M.A.; García, F. Improving motivation in a haptic teaching/learning framework. Int. J. Eng. Educ. 2016, 32, 553–562. [Google Scholar]
  83. Li, T.M.H.; Chau, M.; Sung, W.-K.; Lee, A.J.; Wong, P.W.C.; Yip, P.S.F. Design and evaluation of a Facebook game for self-directed e-learning. Knowl. Manag. E-Learn. Int. J. 2016, 8, 464–480. [Google Scholar] [CrossRef]
  84. Stubbé, H.; Badri, A.; Telford, R.; Van Der Hulst, A.; Van Joolingen, W. E-Learning Sudan, Formal Learning for Out-of-School Children. EJEL 2016, 14, 136–149. [Google Scholar]
  85. Tenório, T.; Silva, A.R.; Tenório, A. A influência da gamificação na Educação a Distância com base nas percepções de pesquisadores brasileiros. Rev. EDaPECI 2016, 16, 320–335. [Google Scholar] [CrossRef]
  86. Wang, C.-S.; Li, Y.-C.; Tzeng, Y.-R. How to replicate the cognitive process in computer game-based learning units. Inf. Technol. People 2015, 28, 327–343. [Google Scholar] [CrossRef]
  87. Luthon, F.; Larroque, B. LaboREM—A Remote Laboratory for Game-Like Training in Electronics. IEEE Trans. Learn. Technol. 2014, 8, 311–321. [Google Scholar] [CrossRef] [Green Version]
  88. Labus, A.; Despotović-Zrakić, M.; Radenkovic, M.; Bogdanović, Z. Enhancing formal e-learning with edutainment on social networks. J. Comput. Assist. Learn. 2015, 31, 592–605. [Google Scholar] [CrossRef]
  89. Olsson, M.; Mozelius, P.; Collin, J. Visualisation and Gamification of e-Learning and Programming Education. EJEL 2015, 13, 441–454. [Google Scholar]
  90. Faíña, A.; López-Rodríguez, J.; Varela-Candamio, L. Using game theory in computer engineering education though case stady methodology: Kodak vs. Polariod in the market for instant cameras. Int. J. Eng. Educ. 2014, 30, 252–262. [Google Scholar]
  91. Lin, C.F.; Yeh, Y.-C.; Hung, Y.H.; Chang, R.-I. Data mining for providing a personalized learning path in creativity: An application of decision trees. Comput. Educ. 2013, 68, 199–210. [Google Scholar] [CrossRef]
  92. Jackson, G.T.; McNamara, D.S. Motivation and performance in a game-based intelligent tutoring system. J. Educ. Psychol. 2013, 105, 1036–1049. [Google Scholar] [CrossRef] [Green Version]
  93. Domínguez, A.; Saenz-De-Navarrete, J.; De-Marcos, L.; Fernández-Sanz, L.; Pagés, C.; Martínez-Herráiz, J.-J. Gamifying learning experiences: Practical implications and outcomes. Comput. Educ. 2013, 63, 380–392. [Google Scholar] [CrossRef]
  94. Chen, H.R.; Huang, J.W. Learning effects of RFID-based game-guided learning in libraries. Int. J. Mob. Commun. 2012, 10, 351–365. [Google Scholar] [CrossRef]
  95. Sancho, P.; Torrente, J.; Fernández-Manjón, B. MareMonstrum: A Contribution to Empirical Research about How the Use of MUVEs May Improve Students’ Motivation. J. Univ. Comp. Sci. 2012, 18, 2576–2598. [Google Scholar]
  96. Kanthan, R.; Senger, J.-L. The impact of specially designed digital games-based learning in undergraduate pathology and medical education. Arch. Pathol. Lab. Med. 2011, 135, 135–142. [Google Scholar] [CrossRef]
  97. Brom, C.; Šisler, V.; Slavík, R. Implementing digital game-based learning in schools: Augmented learning environment of ‘Europe 2045’. Multimed. Syst. 2010, 16, 23–41. [Google Scholar] [CrossRef]
  98. Regueras, L.M.; Verdu, E.; Munoz, M.F.; Perez, M.A.; De Castro, J.P.; Verdu, M.J. Effects of Competitive E-Learning Tools on Higher Education Students: A Case Study. IEEE Trans. Educ. 2009, 52, 279–285. [Google Scholar] [CrossRef]
  99. Vassileva, J. Toward Social Learning Environments. IEEE Trans. Learn. Technol. 2008, 1, 199–214. [Google Scholar] [CrossRef]
  100. Munz, U.; Schumm, P.; Wiesebrock, A.; Allgöwer, F. Motivation and Learning Progress Through Educational Games. IEEE Trans. Ind. Electron. 2007, 54, 3141–3144. [Google Scholar] [CrossRef]
  101. Martens, R.; Gulikers, J.; Bastiaens, T. The impact of intrinsic motivation on e-learning in authentic computer tasks. J. Comput. Assist. Learn. 2004, 20, 368–376. [Google Scholar] [CrossRef]
  102. Mason, J.; Loader, K. Using a Serious Game to Train Violence Risk Assessment and Management Skills. Simul. Gaming 2019, 50, 124–135. [Google Scholar] [CrossRef]
  103. North-Samardzic, A.; De Witt, M. Designing a Human Resource Management Simulation to Engage Students. J. Manag. Educ. 2019, 43, 359–395. [Google Scholar] [CrossRef]
  104. Bond, S.E.; Crowther, S.P.; Adhikari, S.; Chubaty, A.J.; Yu, P.; Borchard, J.P.; Boutlis, C.S.; Yeo, W.W.; Miyakis, S.; Castro-Sánchez, E.; et al. Design and Implementation of a Novel Web-Based E-Learning Tool for Education of Health Professionals on the Antibiotic Vancomycin. J. Med. Internet Res. 2017, 19, e93. [Google Scholar] [CrossRef] [Green Version]
  105. Callaghan, N. Investigating the role of Minecraft in educational learning environments. Educ. Media Int. 2016, 53, 244–260. [Google Scholar] [CrossRef]
  106. Ibáñez, M.-B.; Di-Serio, A.; Delgado-Kloos, C. Gamification for Engaging Computer Science Students in Learning Activities: A Case Study. IEEE Trans. Learn. Technol. 2014, 7, 291–301. [Google Scholar] [CrossRef]
  107. Creutzfeldt, J.; Hedman, L.; Felländer-Tsai, L. Effects of pre-training using serious game technology on CPR performance—An exploratory quasi-experimental transfer study. Scand. J. Trauma Resusc. Emerg. Med. 2012, 20, 79. [Google Scholar] [CrossRef] [Green Version]
  108. Creutzfeldt, J.; Hedman, L.; Heinrichs, L.; Youngblood, P.; Felländer-Tsai, L. Cardiopulmonary Resuscitation Training in High School Using Avatars in Virtual Worlds: An International Feasibility Study. J. Med. Internet Res. 2013, 15, e9. [Google Scholar] [CrossRef] [PubMed]
  109. Moreno-Ger, P.; Torrente, J.; Bustamante, J.; Fernández-Galaz, C.; Fernández-Manjón, B.; Comas-Rengifo, M.D. Application of a low-cost web-based simulation to improve students’ practical skills in medical education. Int. J. Med. Inform. 2010, 79, 459–467. [Google Scholar] [CrossRef] [PubMed]
  110. Franceschi, K.; Lee, R.M.; Zanakis, S.H.; Hinds, D. Engaging Group E-Learning in Virtual Worlds. J. Manag. Inf. Syst. 2009, 26, 73–100. [Google Scholar] [CrossRef] [Green Version]
  111. Choudhury, B.; Gouldsborough, I.; Gabriel, S. Use of interactive sessions and e-learning in teaching anatomy to first-year optometry students. Anat. Sci. Educ. 2009, 3, 39–45. [Google Scholar] [CrossRef]
  112. Pistoljevic, N.; Hulusic, V. Educational e-book for children with and without developmental disorders. J. Comput. Educ. 2019, 6, 117–141. [Google Scholar] [CrossRef] [Green Version]
  113. Khenissi, M.A.; Essalmi, F.; Jemni, M.; Graf, S.; Chen, N.-S. Relationship between learning styles and genres of games. Comput. Educ. 2016, 101, 1–14. [Google Scholar] [CrossRef]
  114. Fitó-Bertran, À.; Hernández-Lara, A.B.; Serradell-Lopez, E. The effect of competences on learning results an educational experience with a business simulator. Comput. Hum. Behav. 2015, 51, 910–914. [Google Scholar] [CrossRef]
  115. Zafar, S.; Safdar, S.; Zafar, A.N. Evaluation of use of e-Learning in undergraduate radiology education: A review. Eur. J. Radiol. 2014, 83, 2277–2287. [Google Scholar] [CrossRef] [PubMed]
  116. Martín-SanJosé, J.F.M.S.; Juan, M.-C.; Seguí, I.; García-García, I. The effects of computer-based games and collaboration in large groups vs. collaboration in pairs or traditional methods. Comput. Educ. 2015, 87, 42–54. [Google Scholar] [CrossRef] [Green Version]
  117. Zepeda-Hernandez, S.; Abascal-Mena, R.; Lopez-Ornelas, E. Integration of gamification and active learning in the classroom. REVISTA RA XIMHAI 2016, 12, 315–325. [Google Scholar]
  118. Tan, J.W.; Zary, N. Diagnostic Markers of User Experience, Play, and Learning for Digital Serious Games: A Conceptual Framework Study. JMIR Serious Games 2019, 7, e14620. [Google Scholar] [CrossRef]
  119. Hernández-Lara, A.B.; Serradell-Lopez, E.; Fitó Bertran, M.À. Do business games foster skills? A cross-cultural study from learners’ views. Intang. Cap. 2018, 14, 315–331. [Google Scholar] [CrossRef]
  120. Marchetti, E.; Valente, A. Interactivity and multimodality in language learning: The untapped potential of audiobooks. Univers. Access Inf. Soc. 2018, 17, 257–274. [Google Scholar] [CrossRef]
  121. Slootmaker, A. Developing scenario-based serious games for complex cognitive skills acquisition: Design, development and evaluation of the EMERGO platform. J. Univ. Comp. Sci. 2014, 20, 561–582. [Google Scholar]
  122. Hwang, G.J.; Sung, H.Y.; Hung, C.M.; Huang, I. A Learning Style Perspective to Investigate the Necessity of Developing Adaptive Learning Systems. Educ. Tech. Soc. 2013, 16, 188–197. [Google Scholar]
  123. Siddiqui, A.; Khan, M.; Akhtar, S. Supply chain simulator: A scenario-based educational tool to enhance student learning. Comput. Educ. 2008, 51, 252–261. [Google Scholar] [CrossRef]
  124. Wall, J.; Ahmed, V. Use of a simulation game in delivering blended lifelong learning in the construction industry—Opportunities and Challenges. Comput. Educ. 2008, 50, 1383–1393. [Google Scholar] [CrossRef]
  125. Slimani, A.; Elouaai, F.; Elaachak, L.; Yedri, O.B.; Bouhorma, M.; Sbert, M. Learning Analytics Through Serious Games: Data Mining Algorithms for Performance Measurement and Improvement Purposes. Int. J. Emerg. Technol. Learn. 2018, 13, 46–64. [Google Scholar] [CrossRef] [Green Version]
  126. Fu, F.-L.; Su, R.-C.; Yu, S.-C. EGameFlow: A scale to measure learners’ enjoyment of e-learning games. Comput. Educ. 2009, 52, 101–112. [Google Scholar] [CrossRef]
  127. Retalis, S. Creating Adaptive e-Learning Board Games for School Settings Using the ELG Environment. J. Univ. Comp. Sci. 2008, 14, 2897–2908. [Google Scholar]
  128. Bryceson, K.P. The development of VAG—A 3D virtual agribusiness environment and strategy game. Electron. Commer. Res. 2009, 9, 27–47. [Google Scholar] [CrossRef]
  129. Schmidt, M.W.; Friedrich, M.; Kowalewski, K.-F.; De La Garza, J.; Bruckner, T.; Müller-Stich, B.-P.; Nickel, F. Learning from the surgeon’s real perspective—First-person view versus laparoscopic view in e-learning for training of surgical skills? Study protocol for a randomized controlled trial. Int. J. Surg. Protoc. 2017, 3, 7–13. [Google Scholar] [CrossRef]
  130. Atanasijević-Kunc, M.; Logar, V.; Karba, R.; Papic, M.; Kos, A. Remote Multivariable Control Design Using a Competition Game. IEEE Trans. Educ. 2011, 54, 97–103. [Google Scholar] [CrossRef]
  131. Fernandez, J.; Marin, R.; Wirz, R. Online Competitions: An Open Space to Improve the Learning Process. IEEE Trans. Ind. Electron. 2007, 54, 3086–3093. [Google Scholar] [CrossRef]
  132. Prochazkova, K.; Novotný, P.; Hancarova, M.; Prchalova, D.; Sedlacek, Z. Teaching a difficult topic using a problem-based concept resembling a computer game: Development and evaluation of an e-learning application for medical molecular genetics. BMC Med. Educ. 2019, 19, 1–8. [Google Scholar] [CrossRef]
  133. Schwade, F.; Schubert, P. The ERP Challenge. Int. J. Hum. Cap. Inf. Technol. Prof. 2018, 9, 53–69. [Google Scholar] [CrossRef] [Green Version]
  134. Spiridonova, L.I. Characteristics and advantages of using an e-learning system for training of pre-school and primary school teachers. J. e-Learn. Knowl. Soc. 2016, 12. [Google Scholar] [CrossRef]
  135. Rasila, A.; Malinen, J.; Tiitu, H. On automatic assessment and conceptual understanding. Teach. Math. Appl. Int. J. IMA 2015, 34, 149–159. [Google Scholar] [CrossRef] [Green Version]
  136. Duque, G.; Demontiero, O.; Whereat, S.; Gunawardene, P.; Leung, O.; Webster, P.; Sardinha, L.; Boersma, D.; Sharma, A. Evaluation of a blended learning model in geriatric medicine: A successful learning experience for medical students. Australas. J. Ageing 2013, 32, 103–109. [Google Scholar] [CrossRef]
  137. Magro, A.; Swarz, J.; Ousley, A. CancerSPACE: An Interactive E-learning Tool Aimed to Improve Cancer Screening Rates. J. Comput. Commun. 2010, 15, 482–499. [Google Scholar] [CrossRef] [Green Version]
  138. Foss, B.A.; Eikaas, T.I. Game Play in Engineering EducationÐ Concept and Experimental Results. Int. J. Eng. Educ. 2006, 22, 10. [Google Scholar]
  139. Mabkhot, M.M.; Ferreira, P.; Maffei, A.; Podržaj, P.; Mądziel, M.; Antonelli, D.; Lanzetta, M.; Barata, J.; Boffa, E.; Finžgar, M.; et al. Mapping Industry 4.0 Enabling Technologies into United Nations Sustainability Development Goals. Sustainability 2021, 13, 2560. [Google Scholar] [CrossRef]
  140. Senna, P.; Pinha, D.; Ahluwalia, R.; Guimarães, J.C.; Severo, E.; Reis, A.D.C. A three-stage stochastic optimization model for the Brazilian biodiesel supply chain. Production 2016, 26, 501–515. [Google Scholar] [CrossRef] [Green Version]
  141. Dos Santos, A.L.T.; Reis, A.D.C. Tendências Teóricas do Gerenciamento de Riscos em Compras Públicas Sustentáveis: Uma Análise Bibliométrica A Partir das Bases Scopus e Web of Science. Rev. Gestão Soc. Ambient. 2021, 15, e2733. [Google Scholar] [CrossRef]
  142. Acioli, C.; Scavarda, A.; Reis, A. Applying Industry 4.0 Technologies in the COVID-19 Sustainable Chains. Int. J. Product. Perform. Manag. 2021. Available online: https://www.emerald.com/insight/1741-0401.html (accessed on 14 October 2021).
Figure 1. PRISMA flow.
Figure 1. PRISMA flow.
Sustainability 13 11945 g001
Figure 2. The main behavioral attitudes for the gamified learning process.
Figure 2. The main behavioral attitudes for the gamified learning process.
Sustainability 13 11945 g002
Figure 3. 7GOALS and PDCA: a theoretical framework.
Figure 3. 7GOALS and PDCA: a theoretical framework.
Sustainability 13 11945 g003
Table 1. Behavioral attitudes addressed in the literature.
Table 1. Behavioral attitudes addressed in the literature.
AttitudeTotalReferences
Motivation45 articlesGarcia-Cabot et al., 2019 [10]; Hernández-Lara et al., 2019 [20]; Martinengo et al., 2019 [70]; Doney, 2019 [55]; Terras and Boyle, 2019 [12]; Romero-Rodríguez et al., 2019 [11]; Gauthier and Jenkinson, 2018 [71]; Kyewski and Krämer, 2018 [8]; Pérez-Manzano and Almela-Baeza, 2018 [72]; Olszewski et al., 2018 [73]; Jackson et al., 2018 [74]; Ge, 2018 [75]; Ortiz Benavides and Piña Lopez, 2018 [76]; Romero-Hernandez et al., 2018 [34]; Karaaslan et al., 2018 [30]; Kooiman et al., 2017 [77]; Torres-Toukoumidis et al., 2017 [78]; Sardi, Idri, and Fernández-Alemán, 2017 [79]; Lin et al., 2017 [80]; De-Marcos et al., 2016 [81], 2017 [33]; Fernández et al., 2016 [82]; Li et al., 2016 [83]; Stubbé et al., 2016 [84]; Tenório, Silva, and Tenório, 2016 [85]; Wang, Li, and Tseng, 2015 [86]; Luthon and Larroque, 2015 [87]; Labus et al., 2015 [88]; Olsson et al., 2015 [89]; Faíña, López-Rodríguez, and Varela-Candamio, 2014 [90]; Martín-SanJosé et al., 2014 [28]; Simões, Redondo, and Vilas, 2013 [27]; Lin et al., 2013 [91]; Jackson and McNamara, 2013 [92]; Callaghan et al., 2013 [16]; Domínguez et al., 2013 [93]; Chen and Huang, 2012 [94]; Sancho, Torrente, and Fernández-Manjón, 2012 [95]; Kanthan and Senger, 2011 [96]; Brom, Šisler, and Slavík, 2010 [97]; Regueras et al., 2009 [98]; Vassileva, 2008 [99]; Ebner and Holzinger, 2007 [39]; Munz et al., 2007 [100]; Martens, Gulikers, and Bastiaens, 2004 [101]
Engagement26 articlesGarcia-Cabot et al., 2019 [10]; Bouchrika et al., 2019 [9]; Terras and Boyle, 2019 [12]; Doney, 2019 [55]; Mason and Loader, 2019 [102]; North-Samardzic and de Witt, 2019 [103]; Romero-Rodríguez et al., 2019 [11]; Gauthier and Jenkinson, 2018 [71]; Jackson et al., 2018 [74]; Karaaslan et al., 2018 [30]; Ge, 2018 [75]; Bond et al., 2017 [104]; Sardi, Idri, and Fernández-Alemán, 2017 [79]; Callaghan, 2016 [105]; Ibáñez, Di-Serio, and Delgado-Kloos, 2014 [106]; Callaghan et al., 2013 [16]; Jackson and McNamara, 2013 [92]; Domínguez et al., 2013 [93]; Simões, Redondo, and Vilas, 2013 [27]; Creutzfeldt et al., 2012 [107], 2013 [108]; Rubin-Vaughan et al., 2011 [15]; Moreno-Ger et al., 2010 [109]; Franceschi et al., 2009 [110]; Choudhury, Gouldsborough, and Gabriel, 2009 [111]; Regueras et al., 2009 [98]
Satisfaction21 articlesChan, Chan, and Fong, 2019 [53]; Pistoljevic and Hulusic, 2019 [112]; Terras and Boyle, 2019 [12]; Ge, 2018 [75]; De-Marcos et al., 2017 [33]; Lin et al., 2017 [80]; Sardi, Idri, and Fernández-Alemán, 2017 [79]; Tenório, Silva, and Tenório, 2016 [85]; Li et al., 2016 [83]; Khenissi et al., 2016 [113]; Nyembezi, Cishe, and Mantlana, 2016 [29]; Labus et al., 2015 [88]; Fitó-Bertran, Hernández-Lara, and López, 2015 [114]; Ibáñez, Di-Serio, and Delgado-Kloos, 2014 [106]; Zafar, Safdar, and Zafar, 2014 [115]; Martín-SanJosé et al., 2014 [28]; Jackson and McNamara, 2013 [92]; Sancho, Torrente, and Fernández-Manjón, 2012 [95]; Kanthan and Senger, 2011 [96]; Moreno-Ger et al., 2010 [109]; Regueras et al., 2009 [98]
Table 2. The main gamification elements used to stimulate the learning cycle.
Table 2. The main gamification elements used to stimulate the learning cycle.
StepDescriptionTotalReferences
1User-centricity and personalized experiences13 articlesTan and Zary, 2019 [118]; Terras and Boyle, 2019 [12]; Gbolahan Balogun, 2019 [54]; Hernández-Lara et al., 2019 [20]; Hernández-Lara, Serradell-Lopez, and Fitó-Bertran, 2018 [119]; Marchetti and Valente, 2018 [120]; Ge, 2018 [75]; Callaghan, 2016 [105]; Slootmaker, 2014 [121]; Hwang et al., 2013 [122]; Simões, Redondo, and Vilas, 2013 [27]; Siddiqui, Khan, and Akhtar, 2008 [123]; Wall and Ahmed, 2008 [124]
2Challenging and clear missions14 articlesDoney, 2019 [55]; Tan and Zary, 2019 [118]; North-Samardzic, and de Witt, 2019 [103]; Chew et al., 2018 [5]; Karaaslan et al., 2018 [30]; Ortiz Benavides and Piña Lopez, 2018 [76]; Slimani et al., 2018 [125]; De-Marcos et al., 2017 [33]; Sardi, Idri, and Fernández-Alemán, 2017 [79]; Fernández et al., 2016 [82]; Domínguez et al., 2013 [93]; Fu et al., 2009 [126]; Retalis, 2008 [127]; Vassileva, 2008 [99]
3Narrative and fantasy15 articlesChan, Chan, and Fong, 2019 [53]; Mason and Loader, 2019 [102]; Romero-Rodríguez et al., 2019 [11]; North-Samardzic and de Witt, 2019 [103]; Jackson et al., 2018 [74]; Pérez-Manzano and Almela-Baeza, 2018 [72]; Sardi, Idri, and Fernández-Alemán, 2017 [79]; Stubbé et al., 2016 [84]; De-Marcos et al., 2016 [81]; Ibáñez, Di-Serio, and Delgado-Kloos, 2014 [106]; Domínguez et al., 2013 [93]; Sancho, Torrente, and Fernández-Manjón, 2012 [95]; Bryceson, 2009 [128]; Franceschi et al., 2009 [110]; Ebner and Holzinger, 2007 [39]
4Repetitive loops and freedom to fail9 articlesDoney, 2019 [55]; Gauthier and Jenkinson, 2018 [71]; Schmidt et al., 2017 [129]; Luthon and Larroque, 2015 [87]; Domínguez et al., 2013 [93]; Simões, Redondo, and Vilas, 2013 [27]; Moreno-Ger et al., 2010 [109]; Regueras et al., 2009 [98]; Ebner and Holzinger, 2007 [39]
5Competition and social engagement19 articlesGarcia-Cabot et al., 2019 [10]; Lorenzo-Alvarez et al., 2019 [13]; Doney, 2019 [55]; Bouchrika et al., 2019 [9]; Karaaslan et al., 2018 [30]; Bond et al., 2017 [104]; Sardi, Idri, and Fernández-Alemán, 2017 [79]; Tenório, Silva, and Tenório, 2016 [85]; Martín-SanJosé et al., 2015 [116]; Domínguez et al., 2013 [93]; Jackson and McNamara, 2013 [92]; Sancho, Torrente, and Fernández-Manjón, 2012 [95]; Atanasijevic-Kunc et al., 2011 [130]; Regueras et al., 2009 [98]; Franceschi et al., 2009 [110]; Fu et al., 2009 [126]; Wautelet et al., 2012 [32]; Ebner and Holzinger, 2007 [39]; Fernandez, Marin, and Wirz, 2007 [131]
6Feedback and reflection25 articlesGarcia-Cabot et al., 2019 [10]; Lorenzo-Alvarez et al., 2020 [13]; Doney, 2019 [55]; Martinengo et al., 2019 [70]; North-Samardzic and de Witt, 2019 [103]; Pistoljevic and Hulusic, 2019 [112]; Prochazkova et al., 2019 [132]; Schwade and Schubert, 2018 [133]; Olszewski et al., 2018 [73]; Ge, 2018 [75]; Sood and Singh, 2018 [3]; Karaaslan et al., 2018 [30]; Kooiman et al., 2017 [77]; Spiridonova, 2016 [134]; Fernández et al., 2016 [82]; Boada et al., 2015 [50]; Rasila et al., 2015 [135]; Ibáñez, Di-Serio, and Delgado-Kloos, 2014 [106]; Jackson and McNamara, 2013 [92]; Duque et al., 2013 [136]; Magro et al., 2010 [137]; Choudhury, Gouldsborough, and Gabriel, 2009 [111]; Vassileva, 2008 [99]; Wall and Ahmed, 2008 [124]; Foss and Eikaas, 2006 [138]
7Reward and social credibility13 articlesGarcia-Cabot et al., 2019 [10]; Bernik Bernik, Vusić, and Milkovic, 2019 [56]; Doney, 2019 [55]; Chew et al., 2018 [5]; Gauthier and Jenkinson, 2018 [71]; Ge, 2018 [75]; Kyewski and Krämer, 2018 [8]; Pérez-Manzano and Almela-Baeza, 2018 [72]; De-Marcos et al., 2017 [33]; Sardi, Idri, and Fernández-Alemán, 2017 [79]; Tenório, Silva, and Tenório, 2016 [85]; Li et al., 2016 [83]; Domínguez et al., 2013 [93]
Table 3. Pairwise relationship between gamification elements.
Table 3. Pairwise relationship between gamification elements.
Elements of Gamification User-Centricity and Personalized ExperiencesChallenging and Clear missionsNarrative and FantasyRepetitive Loops and Freedom to FailCompetition and Social EngagementFeedback and ReflectionReward and Social Credibility
User-centricity and personalized experiences13101021
100%8%0%8%0%15%8%
Challenging and clear missions11432555
7%100%21%14%36%36%36%
Narrative and fantasy03152523
0%20%100%13%33%13%20%
Repetitive loops and freedom to fail1229412
11%22%22%100%44%11%22%
Competition and social engagement05541955
0%26%26%21%100%26%26%
Feedback and reflection25215253
8%20%8%4%20%100%12%
Reward and social credibility15325313
8%38%23%15%38%23%100%
Color legendWeak Sustainability 13 11945 i001 Sustainability 13 11945 i002Strong
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Oliveira, R.P.; Souza, C.G.d.; Reis, A.d.C.; Souza, W.M.d. Gamification in E-Learning and Sustainability: A Theoretical Framework. Sustainability 2021, 13, 11945. https://doi.org/10.3390/su132111945

AMA Style

Oliveira RP, Souza CGd, Reis AdC, Souza WMd. Gamification in E-Learning and Sustainability: A Theoretical Framework. Sustainability. 2021; 13(21):11945. https://doi.org/10.3390/su132111945

Chicago/Turabian Style

Oliveira, Renata Pereira, Cristina Gomes de Souza, Augusto da Cunha Reis, and Wallice Medeiros de Souza. 2021. "Gamification in E-Learning and Sustainability: A Theoretical Framework" Sustainability 13, no. 21: 11945. https://doi.org/10.3390/su132111945

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop