Viewing Gamification Design Limitations and Weaknesses through a Pandemic Lens

Abstract

Design challenges and limitations of gamification were examined using the COVID-19 pandemic as a lens. Online or remote environments were also examined. These environments highlight the literature gap in evidence-based design recommendations and studies that isolate gamification from other pedagogical interventions or methodologies. The literature recognizes the differences between actual games and gamification. Gamification focuses and relies on entertainment to boost academic achievement. This focus on entertainment and its implications to motivation, both intrinsic and extrinsic, are examined. This reliance on entertainment creates unrealistic expectations. In fact, gamification expectations may be conflated with game expectations—especially in an educational setting.

1. Introduction

Gamification is the inclusion of game elements in an activity that is not game related. For example, in their implementation of a sign language learning application, Ref. [1] included the awarding of points and badges, a progress tracker, separate levels, time limits, and random challenges. With the inclusion of these recognizable game mechanics, the application did not cease to be instructional. The gamification was meant to add entertainment to the instruction.

To the extent possible, this paper examines gamification’s design challenges and limitations, using the COVID-19 pandemic as a lens and a backdrop. Due to the COVID pandemic’s recentness, there is still scant data and literature on gamification interventions during the pandemic. Accordingly, this paper examines gamification interventions in online or remote environments, along with interventions that took place during the COVID 19-pandemic. An important distinction between the two environments is that COVID 19-pandemic learners did not choose their instructional modality. They were compelled by circumstances.

While the literature reports design challenges of gamification, there is a gap in the literature of evidence-based design recommendations. Moreover, an examination of the few gamification studies during the pandemic revealed a gap in studies that isolate gamification from other pedagogical interventions or methodologies. By situating gamification in remote and online environments caused by the worldwide pandemic, this review focuses in on these gaps. Using this sharp focus, recommendations for improvement are noted.

2. Methodology

Articles for review were chosen from two databases—EBSCOHost and ProQuest. Key words used were Gamification, COVID 19, Pandemic and Anxiety. The articles were randomly chosen for a conceptual review and then brought into conversation with a standard text on instructional design: Brown and Green, “The essentials of instructional design: Connecting fundamental principles with process and practice”.

3. Literature Review

For the past year, due to COVID-19-inspired restrictions, public and private education underwent a dramatic change. Educational institutions, to maintain social distancing requirements, were forced to rapidly switch from the traditional in-person instructional modality to either a remote or an online modality [2,3,4,5]. This meant an unprepared and resistant cohort were forced to embrace this remote modality. Instructors unfamiliar with the details and intricacies of remote instruction were expected, with little training, to deliver their classes remotely. Learners, who also received little training in remote delivery technology, were forced to participate and expected to maintain the same achievement level as they achieved in their prior face-to-face and on-campus environment. This compulsory change in instructional modality coupled with inadequate training meant that students experienced varying degrees of anxiety levels.

For example, in a survey of 82 undergraduate students at Wingate University in North Carolina, Ref. [5] reported that 76% “held some level of anxiety towards rapidly shifting to finishing a semester online” (p. 256). The survey also offered participants the opportunity to expand on their responses. Most of these expanded responses indicated that “online would negatively affect their learning, grades, and also be very different than in-class learning.” (p. 260).

Ref. [2] also studied student attitudes towards forced online learning during the COVID-19 pandemic. They examined a sample of 88 students from a private university in Northern Cyprus. They reported that one reason for the anxiety experienced by learners was a lack of knowledge of how digital environments operate. Ref. [2] described this as a lack of “digital citizenship” (p. 4). Learner apprehension due to limited knowledge makes sense. If individuals are apprehensive when they have a desire to participate in a novel experience for which they lack training, then all the more so should their anxiety increase when their participation is forced.

4. Game Design vs. Gamification Design

Game is a noun and gamify is a verb. The distinction between the two is more than grammatical. Gamification happens when an instructor includes game elements in a non-game activity or assignment [6,7,8]. (In fact, Ref. [7] described educational games as “full-fledged games”, and gamification as “only a lightweight application” (p. 29) where the instructor introduces game elements into the learning environment.

Academics and practitioners do not always appreciate this sharp dividing line between the two. For example, Ref. [9] examined whether gamification could help in the identification of COVID-19 misinformation. However, they described the gamification technique as “novel and freely available five-minute choice-based browser game similar in design to other ‘fake news’ games” (p. 3). In other words, whatever [9] examined, they believed it had the appearance and even the feel of a game. This means they may not have been examining gamification.

Ref. [10] expanded on the notion of a game being more complete than a gamified activity. In a review of gamification design frameworks, Ref. [10] observed that game design should include an understanding of player choices and outcomes, rulemaking and rule breaking, and the synergy between game rules and game play. Moreover, the design of a game should recognize and support the epiphenomenon of the social interaction and meanings created. On the other hand, they recognized that gamification’s purpose is “far removed from traditional game-design objectives” (p. 519). This difference in objective implies a different design process.

In fact, unlike full-fledged educational games, gamification applications are lacking in evidenced-based design [7,8,10,11]. In their systematic review of gamification design frameworks, Ref. [10] found only 5% related to learning. Ref. [11] observed that gamification’s main alure to educational practitioners is entertainment value. In other words, gamification’s entertainment component is currently the sole design solution to content learners find boring.

Ref. [12] observed that gamification proponents focus on including reward components in activities. Ref. [12] suggested that this overemphasis on reward components could be corrected. However, his correction pushed gamification’s trajectory into full-fledged game creation. In fact, his design solutions and examples included adding narratives, multiple quests with multiple pathways, and even describing competing groups of students as “guilds” (p. 58). It is difficult to conclude that his solution is something other than creating a game.

Gamification seems to work best in conjunction with other pedagogical interventions. For example, Ref. [6] studied “Gamified Authentic Mobile Enquiry in Society (GAMES)” (p. 277). GAMES was a “tablet-based GPS-supported mobile application” (p 280). In total, 559 grade-10 Hong Kong classes used GAMES with an enquiry-based learning (EBL) outdoor activity. The activity was also highly student centered. Students planned their own routes and exercised a large degree of control over their own time. Ref. [6] compared the gamified activity with a control group. The control group did not include any game-mechanics, but rather paper-based worksheets. The gamified group showed better results than the control group. However, in their discussion, Ref. [6] reported that the GAMES application made the outdoor activity much easier to complete. In other words, it is possible that any technological intervention in the EBL activity could have produced similar positive results.

The lack of design in existing gamification interventions may be attributable to the discipline not reaching its full maturity and potential. Indeed, Ref. [13] observed that the meaning of gamification is still in flux, and that its meaning may be instructor- or activity-dependent. It makes sense that busy instructors will gamify by overemphasizing the reward components of games. However, if learners lack motivation to engage in an instructional activity, the mere addition of a reward lacking tangible value, like a badge or game currency, provides neither motivation nor engagement.

5. The Impact of the Lack of Gamification Design on Learner Motivation

Poor motivation has been identified as a major opportunity for improvement in learner content retention—especially when learning occurs remotely [6,7,11,14]. Gamification’s entertainment component has been proposed as one solution to this motivational challenge. However, gamification’s lack of design limits its effectiveness. Indeed, Ref. [8] agreeing with [11], observed that gamification interventions lack design and instead rely on the presumed “self-evident nature” (p. 597) of entertainment to provide learner motivation.

Motivation can be either extrinsic or intrinsic [7,14,15]. Reference [15] described intrinsic motivation as “learners enjoy the instruction for its own sake and take pleasure in the (instructional) activity” (p. 75). On the other hand, learners who are extrinsically motivated “anticipate some reward beyond the instruction itself” (p. 76). For example, Ref. [7] examined a gamified traffic speed reduction effort. A camera photographs and records the speed of the drivers. While scofflaws are still issued citations, law-abiders are entered into a lottery. Game-players (drivers) receive a reward of a potential lottery winning for learning the instruction (driving the speed limit). Individuals whose behavior indicates they did not correctly learn the lesson (scofflaws) receive a double punishment: They are cited and do not earn a chance in the lottery. Ref. [7] concluded the motivation was extrinsic in this case.

In fact, Ref. [7] observed that the “goal of gamification is always something other than gameplay itself” (p. 33). The instructor’s goal in gamifying an activity may be for his or her students to be (intrinsically) motivated to learn the underlying content. However, with the inclusion of the gamified elements, the students’ motivations are now, at least in part, to master the gamification elements. This means that gamification always creates extrinsic motivation in the learner. This presents an instructional designer with a pedagogical challenge. Instructional designers should provide a balanced motivational approach. Students can be incentivized to win badges, but content retention cannot be ignored.

6. Gamification during the Pandemic

The COVID-19 pandemic carried with it more than just a disease. The measures taken by society to control the pandemic—isolation and masks—led to increased anxiety [16]. At the pandemic’s start, the change from in-person learning to a remote modality was a form of isolation that increased student anxiety [16,17] further observed factors such as “campus closures, loss of resources, housing and jobs, access to personal relationships formed with professors and friends, and academic uncertainty” (p. 2) which led to increased anxiety among students.

It makes sense that the more gamification an activity includes, the more extrinsic motivation it provides to the learners. In other words, if students enjoy the game elements too much, it is possible that the content will become relatively unimportant to them. Moreover, if the gamification activity overemphasizes competition, this can result in an increase in a learner’s already heightened anxiety. Although this increased anxiety may not manifest itself during normal periods, during a pandemic, it may become all too apparent.

At present, the ramifications of gamification’s limitations during a pandemic-like environment can only be hypothesized. Few peer-reviewed studies on gamification interventions during the COVID-19 pandemic were found by this researcher. Given the amount of time needed to perform a study and the amount of time needed to review it, this absence of research is not surprising. Moreover, in the studies found by this researcher, either the concept is not used in a testable manner, or like in the study by Jong et al., gamification gets credit when there are other pedagogical factors at play. Arguably, these other pedagogical factors obscure or mitigate gamification’s limitations.

For example, Ref. [18] situated gamification as one small component in a larger program designed to engage learners in computer science classes. This larger program—TechTeach—included “flipped classrooms, bring your own device (BYOD), gamification, training of soft-skills, and quizzes and surveys to increase the student’s engagement” (p. 1). Indeed, Ref. [18] described TechTeach as a “global solution able to group several approaches/methods in one” (p. 5). Given the diversity of pedagogy and interventions, it is difficult to see how any results can be attributed to any one approach or method.

Moreover, Ref. [18] described gamification inconsistently. For example, TechTeach requires the instructor to evaluate a student’s contribution to group projects. Having the instructor involved in the assessment of group projects makes pedagogical sense. However, if instructor grading is part of a gamification intervention, then Ref. [18] has expanded its meaning in a way that makes it indistinguishable from normal instructional behavior. Ref. [18] also included a (one-time only) dropping of a grade in the gamification methodology. Arguably, dropping a grade is a form of grade-inflation and not gamification.

Observing that learning biology vocabulary is “not highly motivating” (p. 3516), Ref. [19] gamified this vocabulary learning with a treasure hunt. The hunt was facilitated with a web application called QuoVidi. Once the names of the plants and animals in the curricula were learned, QuoVidi created competing groups of students tasked at finding these plants and animals in their natural environments. The students uploaded pictures of the finds to QuoVidi where the results could be validated.

Although not described as such, the treasure hunt activity had a constructivist design. In a constructivist activity knowledge is constructed by learners who use numerous resources, including their instructor [20]. The structure of groups competing in an outdoor treasure hunt meant that the learning experience was not only gamified, but also constructivist. The game elements were collaboration and competition. Teams competed against each other, and team members collaborated with each other. However, since the students were required to search for and find the plants and animals in their own environment, this means they were also engaged in a constructivist activity. Arguably, it would be difficult—if not impossible—to attribute outcomes to either the gamification intervention or the constructivist nature of the activity.

7. Discussion

Notwithstanding the distinction between games and gamification, Refs. [6,7,12] contended that results from studies on educational games may apply to gamification. Indeed, Ref. [6] described gamification as “leveraging the idea of gaming in education” (p. 278). It makes superficial sense that the design work and effort already put into educational games should be used by gamification instructional designers.

Refs. [20,21] contended that effective full-fledged educational games are designed using constructivist principles. Ref. [21] studied 12 design principles that should underly a constructivist educational game. For example, the “Probing Principle” and the “Multiple Routes Principle” (p. 290). The probing principle encourages players to formulate hypothesis and test if they lead to solutions. The Multiple Routes Principle (MRP) requires that the game should allow players more than one method of progress. This MRP supports and encourages decision making by the players.

This design complexity forces holism to be an epiphenomenon of the design. Constructivism supports learners in their creation of new meanings out of existing meanings [15]. Individuals who learn constructively “generate mental models… to make sense of experiences” [15]. If an effective game is designed constructively and is supported by multiple design principles, it makes sense that these principles work holistically. It would be hard to construct a mental model from a haphazardly designed game.

If a game’s design is holistic, then gamification designers cannot just remove one element of the game and bring the whole design along with it. For example, Rubik’s Cube is a game that offers a player a limited number of path choices—turning one of the six sides 90 degrees left or right. When a player turns the correct number of moves in the correct direction, he or she is rewarded with a desired, and desirable, pattern. Indeed, Rubik’s Cube is supported by many of the design principles investigated by [21]:

Since there are a limited number of moves that can be made in a limited number of directions, the game offers ease of navigation. A player who keeps track of these limited moves can, by reversing these moves, practice continuously. In a classroom setting, players can easily communicate with other players to learn how to create different patterns.

However, transferring just one of Rubik’s Cube’s game elements, such as pattern creation, to a nongame activity leaves behind Rubik’s Cube’s underlying design. For example, if a difficult math application with confusing navigation was gamified by rewarding correct answers with a pretty pattern, the reward alone does not mitigate either the difficult content or the confusing navigation. In other words, a game’s design, especially a holistic design, is not automatically included in the game’s components.

Gamification, with or without design, typically occurs in learning environments where the learners have chosen their modality. For the past 18 months, instruction, and therefore exposure to gamification, occurred either remotely or online. Many of these exposed learners preferred a traditional classroom experience. Moreover, the exposure occurred while most learners were stressed and anxious due both to their unfamiliarity with the instructional delivery, and also uncertainty about their near-future health outcomes.

It makes sense that gamification’s entertainment component overshadows its design limitations during normal periods. However, it also makes sense that during the COVID-19 pandemic, gamification’s design limitations contributed to students’ anxieties and uncertainties about their new learning modality. Moreover, as we enter this post-pandemic period, gamification’s limitations may deserve a spotlight.

8. Recommendations

Since design transfer from holistically designed games to gamified activities or assignments may not be achievable, instructional designers tasked with gamifying assignments or activities during a pandemic—or any situtation that requires leaners to use a remote or online modality learning—should first consider design principles and game elements which relate to anxiety mitigation. Moreover, instructional designers should not just default to game elements that focus on entertainment or non-tangible rewards.

In fact, no instructional design should be built on a one-size-fits-all philosophy. In educational environments, students arrive with varying levels of academic preparedness and metacognitive awareness [22]. In non-pandemic situations, these different levels of preparedness are best addressed by some form of differentiated instruction [22]. All the more so should differentiated instruction be part of the design during a pandemic.

Ref. [7] proposed design changes aligned with differentiated instruction. These design changes deemphasized game-winning elements. Instead, the game elements functioned as a guide to learners and help them set individual goals. However, Ref. [7] did not propose a clear path to achieve this gamification style of differentiated learning.

A more detailed example of a gamification design aligned with differentiated instruction can be found in [20]. She referenced Piaget’s theoretical work that used games for educational development. Games with good design offer more scaffolding in lower levels. This scaffolding is then removed [20] “gradually” (p. 36) as learners advance through the game levels. Since scaffolding and levels are game elements, this design best practice can be included when these game elements are made of part of class activities and assignments. In other words, gamification designers do not need to restrict themselves to game elements focused on entertainment or rewards. The abundance of design guidance from the literature on games and interventions allows instructional designers choices that are currently unused.

Ref. [6] examined a three-component design approach for gamifying learning. These three components are cognitive, emotional, and social. The cognitive component requires the aggregate of learning content be divided into tasks. Aligned with each task are a set of rules, that when followed, lead students to the successful completion of the task. Learners should have the option of breaking down tasks into subtasks to “lower the cognitive load” [6] p. 278. Moreover, to accommodate differentiated learning, students should have the option of completing tasks in a non-linear fashion. The emotional component should provide a system of “rewards” [6] p. 279 for task completion and “penalties” [6] p. 279 for failure to complete the task. However, these penalties should be “mild and low stakes” [6] p. 279. This penalty minimization should both mitigate anxiety of failing and motivate learners to continue with the assignment even after failing a task. To ensure the social component, the tasks should “facilitate social interaction among students” [6] p. 279. This component can be collaborative or competitive and will provide learners with new identities in the gamified environment.

A” back to the basics” design approach may also prove profitable in gamification design. Gamification is an intervention. Gamification design can and should look to established principles of intervention design. For example, good design should distinguish between goals and objectives [15]. As opposed to goals, objectives are “much more specific about how and to what degree the instruction will affect the learners” [15] p. 91. Accordingly, good gamification design should ensure that introducing game elements does not overshadow the learners’ understanding of this distinction.

A user-centered design is usually part of any conversation that involves learning in a digital environment [15]. This means that the gamification design should fully incorporate the learners’ characteristics. For example, Ref. [7] observed that medical students tend to more competitive than other students. A gamification design which takes this characteristic into account should incorporate more competition into an activity designed for medical students. Moreover, the completed gamification intervention should be tested. In a user-centered design, the testing is performed from the perspective of the user. In other words, gamification designers should observe learners as they use the intervention. The designer (or instructor) should verify that the instructions, navigation, and even any multi-media enhancements are well understood.

9. Conclusions

Gamification involves choosing game elements and including these elements in a nongame activity. This choosing and including are performed by either the instructor or an external designer. In either case, this choosing and including are done with the intention of creating a better nongame activity. In other words, a gamified activity is not a game, not even a pastiche, but rather a patchwork.

The effectiveness of gamified activities may depend on the environment in which they are used and the individual who created the activity. If the designer is also the instructor, then he or she should be familiar enough with gamification’s limitations to effectively design the activity for his or her class. If an external designer is creating the activity, then communication between the instructor and the designer is a necessity for a bespoke activity.

The benefits of gamification may be mitigated if they are used in a modality that the students have not chosen. Moreover, since gamified activities are not games, but only include elements of games, they do not enjoy a holistic design. This lack may negatively effect user experience of the activity and the outcome.