Constructive Alignment in Game Design for Learning Activities in Higher Education

Abstract

The Bologna process has led European universities to review quality assurance in Higher Education. Active learning, competency-based education as well as outcome-based teaching and learning have been encouraged as means to maintain quality assurance in Higher Education. The aims of this paper are (a) to introduce the context of European Higher Education quality assurance challenges and review existing initiatives in game design in Higher Education, and (b) analyze the game design process in the Game Based Learning course of the MSc SmartEdTech program through the prism of constructive alignment. The analysis of student deliverables, in the form of Game Design Documents, shows improvement on how students approach educational game design but also points out some aspects for improvement on the course structure. The study also considers opportunities and limits of game design for learning in Higher Education in the specific context of online education.

1. Introduction

Within the last few years, an evolution of the teaching practices in Higher Education (HE) has been observed, moving from transmissive approaches towards a more active learning approach, in which students are highly engaged in the learning process. However, active learning involves a wide range of learning activities which range from simulations to dialogic teaching and includes a wide range of Technology Enhanced Learning (TEL) applications, including digital game-based learning (DGBL). This study focuses on DGBL as a means of engaging master level students in a co-creativity activity around the development of proof of concepts for serious games.

Outcome-based teaching and evaluation (OBTE) aims at designing and facilitating learning activities based on a set of intended learning outcomes (ILOs). OBTE proposes learners with learning activities, the outcomes of which require them to engage in learning activities defined by teachers and to develop relevant ILOs [1]. The constructive alignment between learning objectives and outcomes of a course requires the consideration of the appropriate elements of the learning activities proposed to students, in relation to both outcomes and learning objectives. The way that students are assessed in learning activities is also a point of focus in the scope of constructive alignment. Constructive alignment relies on the coherence between assessment, teaching strategies and intended learning outcomes in an educational activity, course or program (Figure 1). For Biggs [2], assessment is considered by teachers as the last step of learning activities, but from a student perspective, assessment is often the first considered element of a learning activity in a course.

In order to make sure that both students’ perspective, focusing firstly on assessment, and teachers’ perspective, which starts by defining the learning objectives, are properly taken into account in HE learning activities, a coherence between the three main aspects of learning activities is needed, including task assessment, learning and teaching experiences (and therefore content and methods) as well as course learning outcomes. A lack of constructive alignment could hinder the learning experience perceived by students but also redirect their attention and efforts towards the elements that, in their mind, can improve their performance during the activity’s assessment, without considering the learning objectives of the course. Surface learning could appear when “students target only the essentials of a course and use rote memorization to remember facts rather than understanding important concepts” [3]. In order to engage students in deep learning approaches, the constructive alignment between the learning objectives, the learning activities and the assessment should be reviewed and considered in ways so that students do not skip the learning process and activities in order to focus on the essential elements that will help them pass the activity’s assessment process.

The non-alignment of learning activities, learning outcomes and assessment could also hinder student engagement in the opposite way, by overloading them, hence distracting them from completing the different activities. Since human cognitive capacity is limited [4], carefully designed learning activities are needed in order to ensure that student efforts are focused on meaningful tasks, related to the proposed learning objectives. On large scale undergraduate courses, the differences regarding the constructive alignment can be observed and perceived if we consider the different teachers assigned to the different groups in charge of a course. In the context of an introductory chemistry course, Benko and colleagues [5] improved learning experience through the enhancement of learning requirements and learning activities development among the different teachers of their course. Team teaching approaches can help improve the constructive alignment of Higher Education courses, in which the perception of learning objectives, learning activities and assessment requires them to adjust to the appropriate exigence level of students.

The type of learning activities selected to engage Higher Education students in a course is important not only in terms of cognitive engagement in itself [6], but also in relation to the alignment between learning objectives. Despite students’ potential interest in Digital Game Based Learning (DGBL), if a game activity is not part of the core aspects of the learning objectives and the assessment, students may superficially address it if they do not consider the serious game to be relevant and coherent [6]. Sanchez and colleagues [7] proposed a customizable learning platform, used also to create geolocalized games, providing educators with the capabilities of adjusting the game mechanics of their educational games based on the learning context of their course as well as the needs of their students. On top of this, student agency can have a significant impact on their motivation and engagement in digital game-based learning activities. Even if DGBL activities aim at providing rich, effective and engaging learning experiences, a very open, and non-guided game structure may have adverse effects, decreasing student engagement and motivation for the activity [8].

2. Game Design for Learning in Higher Education

The interest in introducing and using educational games in the context of higher education is continuously increasing [9]. Games, as well as simulations, are expected to have an impact on a pedagogical shift from lecture-based to student centered instruction [10]. Game design is part of the active learning pedagogies that could be proposed in Higher Education. Game design aims to cognitively engage the students in a learning outcome of a certain complexity. Being confronted to complexity, students require to develop a complex problem-solving activity [11]. Two prominent aspects of academic research in the field of game design in Higher Education focus on the analysis and design of educational games and the effect of introducing them to higher education contexts, but also the impact of the participation of students and teaching staff in designing educational games in higher education.

There have been several studies regarding designing for the introduction of games in higher education contexts, focusing on a variety of fields, among which Science, Technology, Engineering and Mathematics (STEM) [12], entrepreneurial skills [13], 21st century skills [14], storytelling [15], health education [16], history and cultural heritage [17] and social sciences [18]. Such studies suggest that the use of educational games in higher education contexts has a positive impact on knowledge acquisition [19], and attitude towards educational game adoption [20]. However, research also indicates that the introduction of games in higher education contexts is accompanied by misconceptions and uncertainty regarding learning activities, feedback, assessment and the role of teachers in the learning process [21,22] suggest that educators need to increase focus on project-based collaborative learning, an aspect that stands out from other studies too [10,23]. On the other hand, educators feel overwhelmed by the various design options that exist [21], while the need for tools and approaches, which create links between students’ previous knowledge and present reflection, is increasing [24]. As a result, there have been several approaches to mapping an educational game design lifecycle [16] as well as analyzing the connections between game mechanics and learning mechanics [25,26,27,28] or exploring the facilitation of learning experiences in gaming contexts [29,30]. On the other hand, the impact of students’ and educators’ participation in designing educational games is a topic of increasing research interest. The abundance of game design tools [31] as well as the growing “modding” possibilities provided by games themselves [22] have positively contributed to the expansion of this field. The participatory nature of educational games [32,33] is on several occasions used to create learner-centered learning experiences, where both teachers and students are part of the design process [34]. Kalmpourtzis [32] ran participatory educational game design activities, where kindergarten students could create educational games around pre-algebraic patterning for their peers, showing that student participation in making games had a positive impact on their mathematical problem posing skills. This potentially more passive role of the teacher requires the creation of the necessary game mechanics [21] and processes [35] in order to account for this teaching configuration. A consequence of the continuous interest for creation or co-creation of educational games as part of the learning process, is the proposal of a set of technical infrastructures [36] as well as online, content independent resources [37,38], where educational game designers can combine game and technical system elements along with the learning topics they are working on.

3. Frameworks for Game Design in Educational Settings

There have been several examples of facilitating learning by making games in higher education and relevant studies include higher education contexts around geography and geology [7], journalism [31], health education [26] and programming [39]. However, research on this field shows that different studies examine and approach the structure of game design for learning through different frameworks and theories.

On the one hand, there are frameworks that structurally examine educational game design. For instance, the 4-Dimensional Framework, proposed by de Freitas and Jarvis [40], identifies four dimensions (learner specifics, pedagogy, representation and context) that can help with the analysis of educational games. The Learning Mechanics–Game Mechanics (LM-GM) framework [26] deconstructs and analyzes the different components of educational games through the prisms of facilitating both gaming and learning experiences. The LM-GM finds connections between different learning and gaming elements and goes even further by connecting the use of different learning and mechanics to Bloom’s ordered thinking skills analysis [41]. The Game Object Model (GOM) [25], provides another form of structural analysis that can be used in educational game design analysis by presenting different types of abstract or concrete components, which are referred to as interfaces, aiming at facilitating the connection between the pedagogical and gaming dimensions of an educational game.

On the other hand, other frameworks focus more on process rather than structure, or address both of those aspects. The LM-GM framework for instance presents also an iterative design process, where an educational game starts from an initial concept, the idea is refined, implemented, tested and revised based on the assessment outcomes. The framework presents a trans-disciplinary development methodology, considering the development steps that are required for the creation of an educational game. Intervention Mapping [42], a tool focusing on project planning, presents an iterative process of six steps aiming to generate design requirements, implement and evaluate them, creating effective learning experiences. The aspects of co-creation and user-centered design play a key role in several studies, where the participation of both students [43] or adults [39]. Based on an iterative and participatory design approach, the Game Design Strategies Analysis Framework (GDSA). Kalmpourtzis [32] examines the application and development of game design strategies throughout the course of organized game design interventions, aiming to help students develop their game design skills.

4. Constructive Alignment under the Lens of Game Design

Constructive alignment between game-based learning as a teaching strategy and the learning outcomes and objectives cannot be achieved if the teaching strategy is not coherent with the aspects of game mechanics and evaluation. As a result, the development of a correct alignment between these aspects, in order to ensure this coherence is required. Constructive alignment should help remove the cognitive overload that may appear when learning objectives are not related and connected with the game mechanics proposed [44]. However, the constructive alignment between game mechanics and learning mechanics in game design for educational serious games is one of the biggest challenges in educational game design.

Game based learning design is a complex activity, requiring the reflection upon the game’s playful elements but also its learning objectives. Learning Mechanics (LM) is defined by Arnab and colleagues [26] as “the dynamic operation of learning, that we typically model relying on learning theories and pedagogical principles”. In the scope of this paper, Game Mechanics (GM) is also defined as a set of rules, loops of interaction (objectives, challenges, rewards) and retroactions which aim to allow an engaging and playful experience. Mechanics invite players to come up with an interaction strategy to achieve the objectives of a game. If game mechanics are oriented towards a potential interaction between players and the game, in digital game-based learning (DGBL) they are oriented towards the learning mechanics [45]. These two aspects affect the game rules that describe interactions as related to the learning process and its consequences on the game, especially on the formative or evaluative retroaction. Each of these concepts will be introduced in more detail in the sections further below.

Game mechanics identify the mechanical elements used in a serious game to ensure that it’s playful. As described by Suttie and colleagues [46], being aware of the game mechanics behind a game can also help evaluate the learning strategies that are used in it. However, constructive alignment in game-based learning should not only consider the alignment of learning mechanics and game mechanics; an additional challenge in developing the constructive alignment in game based learning is the alignment, simultaneously for the different stages, but also congruent, between the game objectives and the learning goals. Game objectives are the outcomes that have to be achieved by players during their interactions (i.e.; put all the apples in a basket within a limited amount of time). Some digital educational games align learning and game goals. In other games, the learning goals are developed by the game mechanics or, inversely, the game mechanics are impacted by the learning goals [21]. In the case of the serious game Forget-Me-Knot, that aims at raising awareness on Alzheimer’s disease, the goal of the game is to go over memories inside a bedroom. These interactions aim at increasing learners’ awareness around Alzheimer’s disease by playing the role of a person who suffers from it.

5. Method

5.1. Context of the Game Design Activity

The MSc SmartEdTech of the University of the Cote d’Azur is an online Higher Education program, all of the courses for which are fully offered online. A set of comprehensive materials, including different forms of audiovisual content is available to students, who can also contact the teaching personnel of the course anytime during the academic year for questions or clarifications. Additionally, once per semester, students are invited to join a week-long event, called an “intensive week”, during which teachers from different courses prepare activities regarding the whole program curriculum.

In the scope of the MSc SmartEdTech program, a course has been designed, in which students are engaged in learning game design through developing a Game Design Document (GDD). The alignment between the course objectives, the course outcomes and the assessment are coherently designed in order to ensure that the students can follow the course within a constructive alignment perspective. The main objective of the "Game Based Learning. From design to the learner experience" course, is to develop competencies for the analysis, design, pedagogical integration and evaluation of serious games and digital game-based learning activities in an educational setting. The Game Based Learning course is a hybrid from a pedagogical and methodological point of view: Students learn about the process of developing a Game Design Document based on a 10-step process aiming at developing a constructive alignment within the different perspectives of the game design process.

The ten steps of the reflexive design procedure aim at developing a constructive alignment on game design (Figure 2). The process starts with the pedagogical reflection that begins with the definition of the learning goals, then deals with the game and learning mechanics, prototyping and ends with the assessment of the game experience and learning. More specifically, this process is broken down into the following steps:

• Learning goals: defining the learning goals of the proposed activity
• Analyzing context and learner needs and pedagogical integration: understanding the individual needs of one’s learners as well as the context around which the game is to be designed
• Game mechanics: proposing the game mechanics for the envisioned game
• Learning mechanics: proposing the learning mechanics for the envisioned game
• Game universe and game narrative: establishing the narrative elements of one’s game, considering the game and learning mechanics of the envisioned game
• Prototyping: working on a prototype version of the envisioned game
• Development: developing a playable version of the envisioned game
• Economic and distribution models: reflecting upon and setting the monetizing models, as well as finding distribution channels for one’s game
• Assessment and learning transfer: finding and proposing tools and methods for assessment of one’s game regarding transfer of knowledge and skills acquired while playing the game
• Assessment of play experience and usability: finding and proposing tools and methods for assessment of one’s game regarding play experience and usability

Working on an iterative game design process, the course’s students are invited to develop a Game Design Document (GDD) based on a topic of their choice. The game design document addresses all aspects mentioned above, trying to create a connection between GM, LM and evaluation of learning objectives.

5.2. Constructive Alignment as an Assessment Tool for the Course’s Effectiveness

Taking into account that the 10-step process of the course entails the understanding of one’s learning context, the definition of learning and game objectives as well as the proposal of a game’s learning and game mechanics, constructive alignment presents an interesting potential in showing the potential development of students’ skills to work on both learning and game objectives and creating coherent connections between them; to propose coherent and interconnected learning and game mechanics; and to propose the proper assessment tools, based on one’s learning mechanics.

In the scope of this paper, we conducted a qualitative analysis of students’ final deliverables, aiming to obtain feedback on the course’s effectiveness through the development of the constructive alignment of students proposed educational games. In the scope of this study, we focused on the final student assignments, in form of GDDs, based on the principles of constructive alignment while considering:

• the degree of coherence between the learning objectives (LO) and the game objectives (GO);
• between the learning mechanics (LM) and game mechanics (GM), and finally;
• between the learning mechanics (LM) and the assessment.

For each of these three components, we rated the degree of coherence from 0 (any evidence of the coherence) to 5 (highly coherent). Two individual raters examined and analyzed the deliverables at different instances.

6. Results

The analysis of the constructive alignment of the six GDDs developed by the MSc SmartEdTech students shows different degrees of constructive alignment. The proposed games have not yet been developed and the final submitted deliverable by students remains their GDD. In the scope of this study, we analyze three different GDDs to evaluate their degree of coherence in the constructive alignment (Figure 3).

ETIQRAFT, coming with a GDD with the highest constructive alignment, is an app-based game whose main learning objective (LO) is to promote positive social behavior among players by giving them real-life challenges to choose and learn from (GO). When players practice making positive choices and get rewarded in the game (GM), they are expected to develop a positive behavior in the real world as well (LO). This GDD aims to combine player engagement through experimentation, along with immersion and simulation (GM). Players can visualize the impact of their virtual world choices if they happen in the real world, thus achieving the learning objectives set forth in this game, providing a highly coherent constructive alignment with formative assessment of the game. The game progresses when players make choices in each scenario within the given time limit. In this GDD real-life challenges are considered within a playful context, which creates a coherence between the life objective of playing a game for overcoming challenging situations and the game mechanics. The constructive alignment is developed through a narrative, learning mechanics and learning objectives alignment through each scenario.

The GDD of the game Do it like a girl also achieves a high degree of constructive alignment. This serious game’s main goal is to inspire and motivate girls to pursue a career related with STEAM disciplines (LO). Through this game, girls can learn about iconic role models that made or are making a difference in the world through their work on STEAM disciplines (GO). Girls will also have the possibility to discover which kind of tasks could be demanded in some STEAM careers, and they will have the opportunity to ‘live’ the experience of being experts on some STEAM jobs, through VR technology (LM-GM). The game proposes ‘GirlPower’ badges earned by cumulative points that promote the engagement in the VR experiences to discover STEAM careers. However, the way the badges rewards the discovery of the STEAM careers is not clearly articulated. In this game, constructive alignment is reduced in the way feedback is provided.

The GDD of the game Happy Resources Creed assumes that unhappy people are much less efficient than happy ones. Unhappiness is being linked to depression, absenteeism and sometimes suicide, which leads to smaller productivity, causing an economical miss-to-win. The game aims at creating conditions where people are viewed as authentic resources, that need to be managed in an appropriate manner (LO). To make this happen, the game requires that players consider people’s individual backgrounds by providing a personalized environment that can help them efficiently evolve and gets the best behavior out of them (GO). The way the background of players is considered is not coherently modeled in relation to the learning objectives, which misaligns the learning objectives and the game ones. Players earn points according to the choices they make and the timing they have while making them (LM-GM). The way points reward players’ choices in relation to their unhappiness remains unclear in the GDD, which does not ensure the required coherence between neither the game mechanics, the learning mechanics nor the assessment.

7. Discussion and Future Work

In the scope of this paper, we laid out the structure and configuration of the game-based learning course of the MSc SmartEdTech program, as well as presented the potential limitations that stem from the online and distant nature of the course. We then examined the notion of constructive alignment as an assessment tool for the effectiveness of the course, considering that constructive alignment addresses the coherence, interconnection and impact of different important components of the educational game design process.

Through the previous analysis, we can observe that GDDs, in the scope of this online course, do not always ensure a correct degree of constructive alignment between the learning objectives (LO), the game objectives (GO), the learning mechanics (LM), the game mechanics (GM) and assessment methods. When these different game aspects are not correctly aligned, they can hinder learning objectives and overload player-learners, through the incoherence between game mechanics and learning mechanics.

In the three cases that were examined in the scope of this paper, two GDDs showed already a developed degree of constructive alignment for all three examined aspects at the end of the course, while in one case constructive alignment was not rated high.

In the context of the GDD of Happy Resources Creed, participants are required to make choices which will influence their level of happiness. However, the model, and hence the outcomes, of (un)happiness consequences of different player choices is not properly articulated, which could potentially lead to an inappropriate understanding of decision making and well-being in the game. The arbitrariness of the relation between learning objectives, game mechanics and learning mechanics could have negative consequences in the context of a serious game which is expected to help players learn or change behaviors based on appropriate models of the domain knowledge of the game. While entertainment games do not pretend to be based on serious knowledge and help achieve certain learning objectives, educational serious games have the responsibility to be developed on established knowledge, but also to develop a constructive alignment between the learning objectives, the game objectives, the learning mechanics, the game mechanics and the assessment.

Through the three case studies, we have been able to observe different degrees of constructive alignment in educational serious games design. The lack of constructive alignment that appears in some game designs makes us consider the interest of introducing the concept of constructive alignment on next editions of the course but also to consider further research to examine the influence of a more explicit instructional design support to raise awareness on this key aspect of game design and observe the potential improvement on constructive alignment when this concept is stressed when learning how to game design an educational serious game.

Constructive alignment is an important aspect in order to establish and create learning experiences in which students focus their engagement in activities correctly aligned with the objectives. We can discuss the importance of constructive alignment in relation to the cognitive load framework of Sweller [48]. Games designed for learning purposes can introduce an extraneous cognitive load (not connected with learning objectives), related to the way information or tasks are presented to learners. By focusing on constructive alignment, game designers could consider ways to reduce extraneous cognitive load for a better integration of game in Higher Education.

8. Limitations of the Study

Since the analysis of this study has been based on a limited amount of GDDs, examination of more such documents could provide a greater overview on the impact of the course on how students design their educational games, through the prism of constructive alignment. The increase in data should potentially be followed by a more extensive triangulation procedure, during the analysis process, in order to support the validity of the results. Additionally, further research on the use of constructive alignment as an assessment tool in the field of game design and game-based learning could provide further empirical data on its use in the context of those fields.