Next Article in Journal
The Activity of Natural Radionuclides Th-232, Ra-226, K-40, and Na-22, and Anthropogenic Cs-137, in the Water, Sediment, and Common Carp Produced in Purified Wastewater from a Slaughterhouse
Next Article in Special Issue
Learning through Challenges and Enigmas: Educational Escape Room as a Predictive Experience of Motivation in University Students
Previous Article in Journal
Reliability Analysis of Seismic Slope Incorporating Interactions among Multiple Sliding Blocks Using Imbalance Thrust Force Method in Primary Sliding Direction
Previous Article in Special Issue
Effects of Gamification on Students’ English Language Proficiency: A Meta-Analysis on Research in South Korea
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Sustainability
Volume 15
Issue 16
10.3390/su151612351
sustainability-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Review

SCHEMA: A Process for the Creation and Evaluation of Serious Games—A Systematic Review towards Sustainability

by
Alba Merino-Cajaraville
*,
Salvador Reyes-de-Cózar
and
Pablo Navazo-Ostúa
Department of Communication and Education, Universidad Loyola Andalucía, 41704 Dos Hermanas, Spain
*
Author to whom correspondence should be addressed.
Sustainability 2023, 15(16), 12351; https://doi.org/10.3390/su151612351
Submission received: 25 June 2023 / Revised: 28 July 2023 / Accepted: 7 August 2023 / Published: 14 August 2023
(This article belongs to the Special Issue Gamification in Sustainable Education)
Browse Figures
Versions Notes

Abstract

:
The Internet and technology have flooded all the activities of our daily lives, making digitalisation a reality. Education is no stranger to this reality and is beginning to incorporate new learning methodologies in the classroom. Methodologies such as game-based learning are presented as appropriate solutions for digital generations, leading the video game industry to produce in a less controlled way. Therefore, this research aims to systematise the scientific production of serious games by studying methodological models and processes for both the creation and evaluation of educational video games and identifying common patterns and differential elements. To this end, a systematic literature review of existing models was carried out under the PRISMA protocol. An initial sample of 13,692 articles was used to arrive at the 15 studies included in this review, following the eligibility criteria of the PICO model. The results show that, although there is a certain unanimity in the methodologies of the different authors, aspects, such as the iteration of the process or the inclusion of teachers in the creation of the video game, must be sufficiently considered. This review led to the creation of SCHEMA, a methodological model for designing serious games that incorporates these deficiencies.

1. Introduction

In recent years, dominated by confinement and teleworking, the rise of digitalisation has been a social enabler at all levels. Thanks to the incorporation of technology in the different spheres of daily life, it has been possible to avoid the collapse of societies and break down physical barriers to reach any place on the planet [1]. Information and Communication Technologies (ICTs) have played a prominent role as an effective and easily accessible tool. They stood out as a robust solution during the pandemic, lending itself a global medium [2], especially in education. In the current context of technological transformation, it is essential to incorporate tools that serve as a bridge to connect educational needs with digital citizens. Therefore, to maintain motivation for learning, it is crucial to introduce media with which the new generations feel comfortable using their native language. Education 4.0 is a new educational paradigm that aims to respond to the needs and potential of the Fourth Industrial Revolution [3].
Thus, applying new technologies in education is, today, a decisive reality that connects digital citizenship and new social discourses. Turning to these narratives makes it possible to become a tool for dialogue with the new generations and dilute educational boundaries [4]. The use of technologies in the learning process takes advantage of their virtues in order to improve the quality of education, reduce inequality at a global level, and favour inclusion in the education of children and young people since advantages such as ubiquity or easy access to these tools make it possible, contributing to the reduction of inequalities embodied in Goal 10 of the 2030 Agenda [5].

1.1. Theoretical Framework

1.1.1. Serious Games: An Educational Tool for Digital Generations

In this technological scenario in which the aim is to adapt pedagogical approaches to 21st-century generations, one of the fields of research that is emerging in the educational field, and attracting the attention of the scientific community, is that of digital narratives [6,7,8,9]. From a social and cultural perspective, researchers are devoting efforts to understanding the impact of new formats that connect with the interests of the public, such as the transmedia nature of content [10], the expansion of streaming [11], the phenomenon of social media networks [12], or the rise of video games [13,14]. Such is the success of the latter that the education system is beginning to incorporate it into its classrooms as a tool for developing intrapersonal skills [15] as well as using it as a vehicle for curricular content. The use of educational video games (also known as serious games) in the classroom has been increasing, thus becoming a good tool for teachers in terms of methodologies such as game-based learning or gamification [16]. The latter, framed in the educational field, is understood as the mechanism that applies game elements (rules, points, challenges) to non-game activities to create engagement in the learning process [17]. This methodology has been gaining interest in educational contexts because of its potential to energise processes that would otherwise be less engaging. A gamified environment can transfer the motivational elements of games to learning activities, thus engaging learners in the task and transforming dull classroom environments into intelligent environments [18].
In this point, serious games (SGs) combine pedagogy and interactivity to foster intrinsic motivation and a positive user experience [19]. Likewise, these narratives are a fun mechanism, placing the student at the centre of the teaching process [20], leading their learning. Today’s learners are less attracted to conventional educational methods and are looking for learning to be an exciting and engaging experience [21].
Not only in terms of pedagogy, but interactive narratives also bring to light numerous advantages in several aspects. Economically, more and more professional projects are emerging with significant financial investments and thousands of jobs. In addition to these promising data, some institutions claim that this industry already has a higher turnover than music and cinema combined [22]. Likewise, from a political point of view, European countries, such as Sweden, include video games in their educational programmes. In other areas, such as Canada, they have fiscal policies that encourage their development [22]. From a social perspective, they become a cultural product capable of questioning or proposing realities and transporting society’s problems to fiction [23].
In this booming framework, and at a time when education is looking for new and inclusive languages, educational video games (also known as serious games) are postulated as a suitable alternative to contribute to the progress of education towards new methodologies, becoming an effective medium [24] for the transmission of educational content. Game-based learning has numerous advantages regarding the format itself, aligning with the course of the younger generations and improving students’ engagement in the teaching process [25]. Authors such as Abidin et al. [26] claim that serious games positively impact performance and increase students’ motivation to learn. Since serious games balance playful and educational aspects, they constitute a privileged learning method for digital natives [27]. Video games have become a growing solution that positively influences the learning experience for students born in the digital age [28].

1.1.2. The Need to Find a Complete Model for Designing, Creating, and Evaluating Educational Video Games

Still, while preserving the advantages of this format, it is essential to remember that the ultimate goal of using this tool is knowledge acquisition, as there are occasions when it is difficult to find a balance between playfulness and pedagogy [29,30].
As researchers widely recognise the potential of educational video games, classroom interventions are being designed and developed using serious games as a knowledge transmission tool in subjects such as critical thinking [31], sexual health [32], safety [33], rehabilitation [34], or mathematics [35]. However, this growing application of educational video games in learning awakens the urgent need to investigate their application in the pedagogical context, as well as the design and development process of these [36], to ensure that it is an effective means to cover the pedagogical objectives proposed by educators.
Along these lines, it is crucial to control and establish taxonomies regulating the booming educational video game sector [37]. In the literature, some researchers try to solve this situation by creating models, guidelines, and methodologies for the standardisation of serious games. On the one hand, there are theoretical frameworks to guide designers in the process of creating educational video games [38]. Research is also available in the literature that presents recommendations and advice for developers focusing on the creation of the game or its subsequent implementation [39]. Likewise, several authors present processes or methodologies that detail the development of a video game, from its initial stage to its use, but more focused on the engineering and IT part of the game [40]. Despite models in the latter category, each of the methodologies focuses on different points of the process, which shows that there are gaps in the science and, above all, a consensus and unanimity among them. This raises the need to find a complete and versatile methodology or process that is easy to implement when designing a serious game.
Given this lack of unity, it is interesting to study what the expert authors in the field propose and which dimensions are most worked on in each of their models. On the one hand, we find guidelines that focus on the field, such as Gao et al. [41], whose model focuses on promoting reframing, or that of Yuxuan et al. [42], who focuses on Engineering and Computing. In parallel, there are frameworks focused on the serious game format, such as the study by Egea-Vivancos and Arias-Ferrer [43] which focuses on the evaluation of immersive virtual reality video games or the study by Koivisto et al. [44], which is based on simulation games. On the other hand, some authors focus on the specific design requirements of a game dimension, such as player enjoyment [45], rules and scenarios [46], or narrative [47]. Concerning use, several models are confined to a single stage of the process. In their research, Carvalho et al. [48] develop a model for conceptual design, sticking only to a first phase. Others, such as Ledezma and Simini [49], limit to the design stage, while the evaluation of educational video games is the stage that is least addressed in the literature and which, as some authors point out, needs to be investigated [50]. Another of the characteristics that researchers underline as necessary in this type of video game is the balance between entertaining and pedagogical content. While preserving the entertainment advantages of this format, it is essential to remember that the ultimate goal of using this tool is the acquisition of knowledge, as there are times when it is difficult to find a balance between fun and learning [29,30], with the former taking precedence. Faced with this lack of order and unity in the literature, it is necessary to look at these variables to find an integrating thread between them to incorporate them in a unified and representative way in a global model.
In addition, as in many emerging industries, the latent economic potential leads to an overproduction that is difficult to control. In this way, assessing the real impact of this cultural and educational product is complicated, sometimes generating misinformation or creating resource banks with little academic rigour or doubtful evaluation of their impact. Under these considerations, and given the rise of serious games as a new tool within game-based learning, the considerable increase in the number of users who use them, and the significant growth in their social and educational impact, it would be advisable to guarantee a sustainable production modality, following the guidelines of Goal 12 of the 2030 Agenda [5], establishing limits to this mass production so that the material is useful and of higher quality, making the economic, human, and scientific investment that is deposited profitable.

2. Materials and Methods

This research aims to identify common patterns and differential elements in the existing methodological models of serious games focused on the design, development, and evaluation of these educational video games in the last decade.
This general objective is specified in the following specific objectives:
  • To systematise the scientific production around serious games by studying methodological models and processes to create and evaluate educational video games.
  • To explore the stages proposed in the existing literature obtained from the studies analysed.
  • To investigate the methodological models’ underlying characteristics to understand their context and see if they incorporate the necessary components for a consistent and complete process.
  • To find similar patterns in the methodologies reviewed to outline a standard, versatile, straightforward process.
  • To determine if there are complete and adequate models that contemplate the design process of a serious game from all its variables to rescue and value this quality content to be used in future developments, promoting the use and sustainability of already available resources.

2.1. Procedure

The scientific methodology followed for this research is carried out in two stages. In the first phase, a systematic review of the existing literature on methodologies and processes for designing, developing, and evaluating serious games is carried out. This systematic review is carried out following the criteria recommended in the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) statement to ensure the validity and accuracy of the process. Eligibility, inclusion and exclusion criteria, and search strategies, are defined through the structure of the PICO model. In the second phase, a qualitative analysis of the selected studies is carried out to identify which steps are followed when designing and developing serious games. These processes are detailed below.

2.1.1. Systematic Literature Review

For the development of this study, a systematic review of the existing scientific literature on methodologies and processes available for the conceptualisation, design, and evaluation of educational video games was carried out. The recommendations and indications of the PRISMA statement [51] were followed in order to carry out this review correctly and to guarantee its validity and rigorousness. The process followed in carrying out the review is detailed below, explaining the different phases of the protocol.
The initial search began in February 2023 using the combination of the terms “serious games” and “educational video games”. Two databases, WoS and Scopus, have been chosen as the most representative in the field of Social Sciences since the object of study of this research is framed in education and communication. In a second phase, the search was expanded using the Boolean operators AND and OR in combination with the descriptors ‘model’, ‘education’, ‘pedagogical’, and ‘framework’, among others. These searches showed a large number of scientific productions, and thanks to this initial phase, it was possible to obtain an overview of the object of study, the existing models in the literature on serious games, and, therefore, the relevance of conducting a systematic review.
The final systematic search was completed in March 2023 using the WoS and Scopus databases, using a search interval of 10 years (from 2014 to 2023 inclusive). The final combination of terms used was as follows: (Model OR guidelines OR principles OR dimensions) AND (Development OR design OR evaluation) AND ((educational AND games) OR (serious AND games) OR (applied AND games) OR (learning AND games) OR (pedagogical AND games)). Finally, this search yielded 13,692 results, of which 3985 were obtained from Scopus and 9707 from WoS. The inclusion and exclusion criteria used to filter the results following the PICO format are shown below (Table 1).
After identifying the 13,692 results found in WoS and Scopus, we discarded duplicate articles in the two databases (n = 2957), giving us 10,735 results. Next, we started the screening phase based on the abovementioned inclusion and exclusion criteria. According to these criteria, after reading the title, n = 10,626 were discarded as articles irrelevant to our object of study. Most of this research described attributes of games in isolation but not processes for developing them. Many others focused on commercial rather than pedagogical video games, and others focused on designing classroom interventions rather than a general process for developing a serious game. After the screening, n = 105 records were selected for eligibility assessment. After reading the abstract, n = 4 that were unavailable for reading and n = 40 that did not meet the study’s inclusion criteria were eliminated, leaving a sample of n = 61 records. Finally, after reading the full text of the selected articles, n = 46 were discarded because they did not meet some of the above criteria not detected in the previous filtering phases. Thus, n = 15 articles were included in the systematic review. The summary of the process can be seen in Figure 1.

2.1.2. Qualitative Analysis

The qualitative analysis of the information extracted from the articles is carried out by identifying the most representative attributes of the research. Firstly, we describe the methodologies and processes proposed by the different authors, their phases and stages, and their most relevant elements, as well as whether or not they have a practical validation of the model. Subsequently, a table is included with the main dimensions extracted from the publications included in the review, such as the scope of application for which the methodology is created, the video game format, the type of process developed, or the relationship between the pedagogical and ludic components of the same. Finally, this section concludes with the proposal, which emerges from the literature reviewed, of a four-step process for designing, developing, and evaluating educational video games.
The qualitative analysis procedure consists of four phases. First, an in-depth reading of the article is carried out. Then, the phases are extracted by focusing on the methodology proposed and developed by the research. Thirdly, we check whether the process proposed by the authors is supported by practical validation. To conclude the qualitative analysis, the model’s main characteristics are classified according to the dimensions. Figure 2 shows graphically the four phases of qualitative analysis.

3. Results

Concerning the specific objective of systematising the scientific production of serious games through the study of methodological models and processes aimed at the creation and evaluation of educational video games and identifying the most representative characteristics, the articles were analysed and organised by nomenclature used in the study, author and year, and methodology. These are presented (Table 2).
Likewise, as shown in Figure 3, research on methodologies for designing and developing educational video games shows a balanced distribution over the last ten years. However, a peak in the scientific production of these models was observed in 2018, when maximums were reached. Between 2016 and 2021, 80% of the research in this field is concentrated, with 2023 being the year with the lowest production of the years analysed.

3.1. Methodologies for the Design and Development of Serious Games: Articles Included in the Review

3.1.1. Article 01

Firstly, ref. [53] develops a conceptual model for digital educational games, focusing on learning about sustainability in the residential environment. Using the Design Science Research Methodology (DSRM), the authors propose a process framework for creating, applying, and evaluating serious games.
In the first stage, the creation stage, the model proposes a dual approach, pedagogical and playful. On the one hand, the use of Bloom’s taxonomy and Feuerstein’s mediated learning criteria at an educational level and the incorporation of critical elements of the game, such as mechanics, aesthetics, story/narrative, and technology at the level of entertainment. This bidirectional approach aims to increase the didactic knowledge of game developers so that they create a mechanic favourable to the mediation of player learning concerning specific content. In the application stage, a structure for the teaching and learning process is established, following Feuerstein’s mediated learning criteria, focusing on mediating content learning during pedagogical practice. Finally, it concludes with an evaluation of two ways. On the one hand, descriptions and narratives through participant observation and, on the other hand, the application of two questionnaires (pre- and post-test) to determine the player’s reaction to their experience with the game, motivation, and the achievement of the pedagogical objectives. This study concludes with a pilot test to check the validity of the process in the Computer Laboratory of the Faculty of Florianópolis, which presented positive preliminary data on the suitability of the conceptual model presented, giving freedom for future work to incorporate other learning theories.

3.1.2. Article 02

In ref. [54], the authors present an incremental design process with four levels: educational content design, entertainment content design, relating educational and entertainment content, and user modelling. In the first level, educational content design, they present the following activities: knowledge areas design, educational goals design, educational tasks and activities design, and educational model design. The second level, entertainment content design, is divided into four fundamental tasks: game basic design, video game challenges design, game stages and levels design, and game model design. Thirdly, the level of the relationship between educational and entertainment content establishes the links between the two previous levels. Finally, user modelling is built around four perspectives: general, educational, game, and interaction. With the design and development of this process, the authors aim to facilitate the specification and design of educational and recreational content and to ensure a balance between educational and recreational components. In support of the proposed methodology, they present the pilot of an authoring tool that helps teachers and designers in the video game creation process.

3.1.3. Article 03

On the other hand, ref. [55] shows a framework called FRACH to conceive, design, and evaluate immersive and collaborative educational video games focused on cultural heritage. This model provides a design structure with steps to follow throughout the process, from the initial design phase to the evaluation phase of the game. The authors emphasise this framework as an iterative process in which each phase can be carried out several times, even in a different order or intertwined. The preliminary phase is a prelude to this process, although indispensable. Its objective is to shape the description of the serious game to be produced, its learning objective, or the audience, among others. Once this phase has been completed, the process begins in the conceptual phase to design the significant elements of the video game, such as the narrative, the challenges, the interactions, or the micro-learnings of the scenes. All of the above is implemented in the development phase, and the game is modelled, tested, and corrected for errors. In the final stage of the process, the evaluation phase, validation and user testing are carried out. To measure the effectiveness of FRACH, the authors implemented it in the design and development of a serious game called HippocraticaCivitasGame, in which players could visit the baths of a historical site.

3.1.4. Article 04

Ref. [56] introduces a generic model for designing educational video games, focusing on discovering whether there is a universal framework for designing a serious game or whether it is inevitable to use several different methods that address different aspects. After reviewing ten design methodologies, the author constructs DICE, a four-stage model (define, imagine, create, and evaluate) to design serious games. Once again, it is an iterative process among its stages, including the definition stage, which aims to improve the game cyclically. In the “define” phase, the educational objectives, the information to be transmitted, and the skills to be developed are specified. Subsequently, in the “imagine” phase, the game is designed to lay the theoretical foundations implemented in the “create” phase. Furthermore, finally, the prototype is evaluated with the target audience on a project-specific basis in the “evaluate” phase.

3.1.5. Article 05

Another vision is presented by [47], the eLuna framework. This model focuses on narrative and is based on co-design, allowing educators to collaborate with developers in the multidisciplinary design of educational video games. This methodological co-design starts with a first preparation phase in which educators define the curricular content, learning objectives, student data, or the learning situation. Subsequently, through the phases of co-design and co-specification, this model proposes a methodological process based on the visual language of icons, colours, and descriptions, allowing a partial and flexible design between the agents working on creating the serious game. In the final stage of development, the project jointly generated in the previous phase is used as the basis for developing an educational video game. To illustrate the use of this framework, Idun’s Apples, a serious game co-designed, co-specified, and implemented in a prototype using the method and visual language proposed by eLuna, is presented.

3.1.6. Article 06

Ref. [57] develops a model for the design of educational spiral games based on observation and the final product. This process starts with defining the learning objectives and goals, the purpose of the game, and the needs to be covered. All this information enters the spiral cycle composed of the design stage, where the concept and fine-tuning of the game are carried out, followed by the prototyping stage, and finally, concluding with the evaluation of the game and the identification of minor errors to be modified. This type of model allows the results of the designed game to be perfected at each iteration of the spiral. As the development progresses, the scale of the adjustments decreases. The process is finished when the development team considers the prototype acceptable. As in previous cases, the authors put into practice the implementation of the model in the design and development of SEO War, a game for students who are not familiar with the subject and whose objective is to achieve the learning of the main concepts of Search Engine Marketing (SEO).

3.1.7. Article 07

On the other hand, in ref. [58], the authors propose a roadmap with practical actions for creating and developing games based on the principles of Design Thinking. Six phases are presented, starting with the involvement stage, where a diagnosis is made on the information extracted from the literature on gamification, active methodologies, and design thinking. In the inspiration stage, we seek to understand and define the factors that help achieve the objective pursued with the game’s development by analysing existing models that serve as a reference. This stage is followed by ideation, a co-creative process to obtain consistent ideas about the model to be embodied in the game. Fourthly, it is time to integrate the ideas in a combination of elements between the large containers of gamification, active methodologies, or design thinking, developing the prototype. Once completed, it is implemented through testing and validation to finish with a final stage of interaction in which the previous results are incorporated into the model developed to correct errors and improve the result.

3.1.8. Article 08

Ref. [59] proposes an integrative, user-friendly, and versatile methodological model that can be used in virtual and face-to-face environments to design serious games. After carrying out a literature review of models and methodologies, the findings show eight general categories that will be the basis of the integrative methodological model: establishing the objectives of the game, characterising the target audience, proposing solutions and type of game, literature study, incorporating game elements, design or prototype testing, evaluating the game, and other additional elements. According to these categories, the model is divided into three main phases: pre-design, design, and testing. In the first phase, also known as the planning phase, the objectives and the audience are defined. The design stage, in turn, is divided into two sub-phases: the writing phase, which details the elements of the game and other components such as emotions, aesthetics, and rewards, and the construction phase, in which the game’s prototype is made. Finally, in the testing phase, the prototype is applied to the sample described above, and the game is refined. This last stage can be considered iterative insofar as it makes adjustments and modifications to aspects described in previous phases, although not continuously.

3.1.9. Article 09

Ref. [60] introduces iPlus, a methodology for designing serious games with a participatory, flexible, and user-centred design approach. The process starts with the identification phase, where the design process begins with the establishment of needs and requirements by the owner. In the second stage, the pedagogical objectives phase, expert educators define the pedagogical goals in a participatory and consensual manner. The Game Design Document (GDD) is created in the third stage, the ludic game script phase, where the game’s story, characters, or scenarios are established. This stage is followed by the gameplay phase, to specify the functions or actions that the player will interact with during the game and the genre of the video game. Finally, the refine phase tries to filter the serious game by eliminating some elements, correcting bugs, or removing what is hardly applicable to the game. To complete this pre-established methodology, the authors present a metamodel where the basic concepts and the relationships between them are defined. Some cases of serious game studies already developed with the iPlus methodology are also analysed.

3.1.10. Article 10

Another research, ref. [61], proposes PROGame, an iterative process for developing serious games, which aims to ensure that these educational products are developed and validated following a coherent and systematic method that leads to quality serious games. In this study, the authors propose a process that begins with a previous stage called “project initiation”, which includes the general description of the project, including the operational objectives, restrictions, stakeholders involved, and the therapy selection (as it focuses on motor rehabilitation). As in previous models, this phase is not part of this iterative process. The first stage, “planning and control”, is about structuring, scheduling, and localising all tasks of the future process. In the second stage, the “modelling” stage, all the above is embodied in the design of the graphical user interface, the requirements, and the system components. In the “construction” stage, it is put into practice and executed through software to conclude with “evaluation”, which is the stage that closes the cycle and whose objective is to play the video game in order to find imperfections and correct the detected errors before the game is made available to rehabilitation patients. As a practical validation, the authors develop a serious game for improving balance and postural control in adults with cerebral palsy.

3.1.11. Article 11

Ref. [62] proposes, on the other hand, a conceptual model for the design of serious games that combines the requirements of the Game Design Document (GDD) and pedagogical design, which, as the authors state, had not been previously related and discussed in the literature. The proposed process is structured around four stages: analysis, design, development, and evaluation. The first phase determines the user profile, skills acquired, pedagogical objectives, estimated budget, and game idea/concept. Subsequently, with all this defined, the mechanics, scenarios, environment, or learning system to be used are designed in the second stage. The development stage is the phase in which the tasks are planned, the software is coded, and the game is prototyped. The last phase comprises the evaluation, where the initially proposed objectives are validated, the game is implemented and tested, and the contents are updated. When everything is finished, the game is launched to the users. The article concludes with a case study that gives practical validity to the proposed model with the development of Athynos, a game for children with learning difficulties to improve motor and cognitive skills.

3.1.12. Article 12

For its part, ref. [63] shows a development model for educational video games adapted from the fundamentals of game-based learning. This framework is structured in four stages, the first two related to the four bases of learning (affective, motivation, cognitive, and social). The first phase of this model is analysis, the main objective of which is ideating educational content. In this first stage, the affective foundations of game-based learning are understood as attitudes, the motivation foundations as interest, the cognitive foundations as information, and the social foundations as relatedness. In the second phase, the production phase, the game prototype is developed, and at this stage, the foundations are translated as follows: affective, as the storytelling and the visual aesthetics; motivation, as the incentive, mechanic, and interaction; cognitive, as the context, representation, or scaffolding; and social, as the social interaction or learning culture. In the testing phase, feedback and evaluation by the players are expected, concluding with the release phase, with the game’s final version. If minor adjustments are needed, the testing stage can be moved back to the production stage. To conclude the article, the model was validated using the relativism approach and used to develop several prototype games for universities, national companies, and the military.

3.1.13. Article 13

Another study, ref. [64], proposes a model for developing educational video games to improve the attention of students with attention deficit hyperactivity disorder (ADHD). The general outline of this model is divided into three phases. The first of these, which sets out the requirements and functionalities, is in turn divided into two sub-phases: enquiry, which includes the hyperactivity and attention problems, participants or the team that must participate in the development of the game, and modelling, where the data, the scenario, the behaviours, are detailed. Phase two comprises the design, in which the software attributes, the game architecture, the interface design, or the definition of patterns are detailed. Finally, it concludes with an evaluation phase of tasks such as determining metrics, testing, and error correction. As a result, it is established that pedagogical and therapeutic aspects can be considered by the designers of educational games in the analysis and design phase of the product and that the integration of augmented reality as an emerging technology for the development of a tool to support therapeutic and teaching-learning processes for children with ADHD is interesting.

3.1.14. Article 14

Ref. [19] introduces a novel model as it presents a multi-layered methodology, which provides an approach to analyse and evaluate the design of serious games as a collaborative and iterative process between the designer, the educator, and the player. This methodology comprises five layers: learning, story, gameplay, experimentation, and debriefing. The learning layer defines the tasks and subtasks, the didactic strategy and its intention or the structure of the game, as well as the characteristics of the player and the style of play (cooperative, competitive, individual). Subsequently, the story corresponds to the player’s narrative, i.e., the events of the game, the quests, or the environment. The gameplay will determine everything related to the behaviour flow, such as the avatar or the controls. On the other hand, experimentation encompasses all the interactivity of the serious game. The final layer is the debriefing layer, which compares the objectives set, results achieved, play times, or efficiency. The authors emphasise the help that this methodology can offer both game designers and education professionals in evaluating and having guidelines for joint design with other actors. Finally, the model is used to examine and evaluate a prototype created for sales training.

3.1.15. Article 15

To conclude this review, ref. [65] develops MECONESIS (methodology for conceiving serious games for deaf and hard-of-hearing children). This model comprises four phases: analysis, pre-production, production, and post-production. The analysis is where issues related to usability, technology, gameplay, or the role of the different user profiles, learning styles, and pedagogical objectives are detailed. In pre-production, the design patterns and guidelines to be followed, the narrative, the game architecture, and even design prototypes are set out. Production involves the development of the functional prototype, which, in the post-production phase, receives an analysis of the results focused on the pedagogical objectives established in the first stage. The interest of this model lies in its support of multidisciplinary communication between all the actors who may intervene in the creation and development of the educational video game. This methodology, in turn, has been applied in a case study for children with hearing impairment in the USAER programme in Mexico, where the children’s experience of using the ABC-Spanish game in learning to read and write is evaluated.

3.2. Main Dimensions Drawn from the Literature

The table below contains the most representative dimensions of each of the articles. These dimensions are the focus or base on which each model designed is centred, the field in which it is applied, the use for which it is intended, the methodology used to develop the article, the iteration of its phases, the format for which they are intended, the approach concerning the recreational and educational aspects, and the practical validation (Table 3).
These seven dimensions are drawn from the articles included in the review. The focus/base refers to the approach of the process or methodology developed on a particular aspect. Scope describes the subject area of the methodology, which may be general or subject-specific, such as learning or health. The use dimension classifies the different articles according to the purpose for which they are created. It can be for the design, development, or evaluation of serious games or several of them. The phases iterations refer to the methodologies characteristic that allows reviewing possible errors to correct them during the process and not only in the final stage. The format describes the type of game for which the methodology is created. This dimension is relevant because some formats, such as virtual reality, require different technical specifications. The approach measures the balance between the two main components of an educational video game: the playful and the pedagogical. Finally, practical validation is presented as a dichotomous question summarising whether this methodology has any validation to give consistency and rigour to the process developed.

3.3. A Process for the Creation and Evaluation of Serious Games: SCHEMA

The following is a proposal for a model that serves as a basic scheme for devising, constructing, and correcting serious games in line with the objectives of this study. Models have been reviewed in the literature, and their most representative dimensions have been extracted. The results have been analysed to find common patterns, simplify the process, and propose a complete model. Reviewing other articles to create that process gives a holistic and complete view and a more comprehensive knowledge of what already exists. This model, called SCHEMA (Specification–Content Hierarchy–Engineering–Monitoring and Assessment), can be used by designers, developers, and educators. The role of the teacher is considered fundamental in this process to balance the playful and pedagogical components of the created SG. This proposal is structured around four steps, three of which are iterative. The first one is left out of the iteration since the bases of the project (purpose of the game, target audience) are only established once, in the beginning, and are not modified regularly. It is also important to emphasise that these are steps and not phases or stages, as this nomenclature can be misleading because they are considered successive stages when time does not allow for repeated revision (Figure 4).
The first step is Specification, the starting point of the educational video game to be created. At this point, all the bases that will lay the foundations for the characteristics that determine the type of game are established. The needs for creating this serious game are identified [47,60], the target audience [55,59], its objectives [47,59], whether there are intermediaries in the process [61], and the expected skills to be acquired with the game [56,62]. At this point, it is also important to highlight specific aspects of the concrete case, e.g., if the audience has special needs or the focus of the video game [61,64]. This step should be developed jointly between the educators, who have the goal to be conveyed, and the designers and developers in charge of implementing it.
The second step is Content Hierarchy, which will shape all the material to be included in the educational video game, both in terms of pedagogical content and all the game’s ludic considerations, hierarchies, and relationships. At the level of entertainment, the mechanics [59,62], aesthetics [53,63], narrative [53,55,65], challenges [54], levels [54], scenarios [19,64], game events [19,56], dialogues [56], characters [19,60], rewards [59], and interactions [63] are defined. It is also in this step where it must be explicitly determined which format will be used (Augmented Reality, Virtual Reality, computer game, or simulation) to correctly adapt all the previous elements. On the other hand, didactically, the tasks and activities to be developed during the game [54,61], the pedagogical objectives to be achieved [54,60], the micro-learnings [55], and even the learning style [65] are shaped. In order to achieve a well-integrated and appropriate game, this step must be a co-creative process between educators and designers [58], ensuring that all entertainment components have associated pedagogical objectives or, failing that, have set small tasks. This collaborative and parallel work between designers and developers is paramount to assemble and prioritise the learning strategy between the playful layers of the videogame.
The third step is Engineering, where the serious game is implemented. The previous design is put into practice, and the game is developed [55,58]; the genre [60] and the graphical interface [61,64] are finalised. The software used will also be detailed here [62]. It is essential in this step to remember the educational component; therefore, during the development, always keep in mind the relation between both contents [54].
The fourth step is Monitoring and Assessment, which is the point where the aim is to monitor and evaluate the game in order to correct possible errors [64]. First, the prototype is validated [55,58], the game is tested with the sample, and feedback is obtained [63] to detect problems and solve them [56,62], returning to previous phases if necessary, during a refinement process [59]. It is essential at this point to check that the pedagogical objectives [65], micro-learnings, and skills have been appropriately achieved [61]. At this point, it is again vital to count on the teachers, even inviting them to test the game and detect possible deficiencies from the point of view of transmitting content and achieving the established pedagogical objectives. Once the game is finished, it will be launched [63,65], permanently preserving constant maintenance [62]. Table 4 shows these four steps and their definition, together with the references found in the literature and the educational and ludic balance.
The table above summarises the extraction of the main steps of SCHEMA through the phases found in the literature. Firstly, the nomenclature given to the different steps is listed. It then includes a brief description of what is included in that step, which complements what is described under the heading. Next are the references in the literature that are included in each phase, all identified with the reference of the article and with the original name of the phases of the methodologies developed in these articles. Finally, the phases of the literature are classified according to their approach, which can be playful, pedagogical, or a balance between the two.
SCHEMA stems from a systematic literature review and attempts to address some of the mismatches in other models in previous articles. This process introduces minor improvements only present in some research in the literature. It highlights the role of the teacher as a fundamental agent in creating and developing an educational video game. This idea still needs to be well established in most studies analysed. It is precisely thanks to the incorporation of the teacher in the process that it is possible to balance playfulness and pedagogy in the serious game. While the designers maintain attractive dynamics and mechanics for the player, the teachers take care of this interactive narrative’s pedagogical objectives and learning potential, one of the most significant weaknesses in the review and is already highlighted in the scientific literature.

4. Discussion

This study presents a literature review to systematise the available literature on the object of study, achieving the objectives initially set out. According to De Freitas [66], there are multiple benefits for students when the learning process takes place through serious games, although, to date, they are still not very widespread in the classroom [3]. Finding a more standardised approach to improve the rigour of serious games [67] and defining guidelines for their creation is not only positive but also necessary to reverse this situation.

4.1. The Dimensions Mined from Literature

When designing an educational video game, it is crucial to find a basis for the design, either using a more general model or one focused on a specific branch of knowledge. Of the 15 studies included in this review, three of them focus on the field of health [61,64,65], two on learning [47,62], one on marketing [57], and another on creative techniques such as Design Thinking [58]. At the same time, the remaining eight (53%) are conceived from a general perspective. These results suggest the possibility of expanding the literature on methodologies and processes specialising in subjects such as art, mathematics, engineering, or sport.
On the other hand, in terms of their use, 80% of the methodologies are created with the aim of being used to design serious games, and 73% of them are for the development process. However, the figure for those whose purpose is evaluation is far behind, only 20%. These data only deepen the need to develop evaluation frameworks for these interactive narratives, a deficiency that the literature had already highlighted [68,69].
Likewise, the format used by educational video games is also important since, when developing an SG, it is paramount to know which device it will be played on. Regarding design, the guidelines for creating scenes for a computer game are not the same as for a Virtual Reality game. In the literature, several interventions are carried out with a serious mobile game [70,71] or even Virtual Reality [72,73,74]. However, among the fifteen processes rescued in this literature review, only two focus on a specific format: computer game [53] and Augmented Reality [64]. This reflects the convenience of investing efforts in developing methodologies for designing, developing, and evaluating educational video games focused on a specific format, such as the mobile version or Virtual Reality.
In addition to the above, designing a serious game is a complex process in which it can be challenging to find a balance between the “serious” content of the curriculum and the playfulness of the game [75]. Based on the results obtained from the studies analysed, slightly more than half (53%) present a methodology incorporating a balance of pedagogical and entertainment elements, very close to those mostly or entirely made up of purely recreational elements (40%). Only one of them [57] has a more significant share of educational aspects in its process, which underlines the concern of several authors that it is not achieving this balance between the two sides of the game [29,30].
Another critical point assessed from the rescued methodologies is the iteration of their models. Two out of three articles have published a model that does not consider the design, development, and evaluation of serious games as a process that is iterated, corrected, and updated. Only five of them [19,55,56,57,61] did specify that their methodologies are designed to be iterative and allow for flexibility in the process. As noted above, designing and developing an educational game is a complex task, which implies a risk of failure [76]. Therefore, iteration should be an essential requirement.
On the other hand, most of the articles accompany the process or methodology developed with a practical validation that gives consistency and veracity to the proposal. Only four [56,58,59,64] of the fifteen studies do not implement the model in a practical case.

4.2. Methodologies Weaknesses to Be Improved in Future Research

Several studies emphasise the importance of educators as a fundamental part of the process of ideation, creation, and implementation of educational video games [77]. Concepts such as co-design [47] or co-creativity [58] begin to emerge from some of the articles, another refers directly to a collaborative process between agents [65], and another does not incorporate teachers in the modelling of the process itself, but with the application of an authoring tool [54]. The other fourteen studies do not explicitly include educators in the process, limiting it only to designers or developers, even though the literature highlights their importance in the process [78].
Finally, a radiography of the different phases of the models reveals some inconsistencies. The first stage, in which the objectives, needs, target audience, or skills to be acquired, among others, are established, is non-existent in some of the methodologies studied [53,54,57,63,65]. In these cases, the authors start directly in the next phase, that of the theoretical design of the contents and skeleton of the game, which is present in all the studies in the review. Likewise, the development and construction of the game are found in 100% of the processes but not the evaluation and error correction stage. Articles such as [54] or [47] do not contemplate a specific step for these purposes, which launches an incomplete and lacking methodology regarding the evaluation of educational video games.
However, in line with the objective of determining whether there are complete and adequate models that contemplate the design process of a serious game from all its variables, there is no article that meets the most critical considerations in order to rescue and value this quality content to be used in future developments, promoting the use and sustainability of already available resources. Although there are some very close to achieving them, such as the FRACH framework [55] or the PROGame [61], they lack the co-design and co-creativity aspect and are entirely playful. Others, such as MECONESIS [65], which does call itself collaborative, lack iteration, as do iPlus [60] or [62], which, in addition to this, only consider their process as a methodology for the design of serious games.

4.3. SCHEMA: A First Step towards Improving Serious Games Methodologies

Thanks to this systematic review, the SCHEMA model emerged, which attempts to address the shortcomings identified in other methodologies, in line with the aim of this study to outline a standard, versatile, and straightforward model. Using a systematic review methodology in developing SCHEMA as a process has allowed one to experience a more holistic view of the object of study. In addition, knowing the existence of models in the literature enables us to identify common patterns, simplify them, and solve slight imbalances with a new model, aiming to improve science.
Therefore, one of the strengths of SCHEMA is that it tries to improve some irrelevant or non-existent components in other models. This process contemplates both the ideation and design of the SG, as well as its development and construction stage and its subsequent evaluation, monitoring, and error correction. In its conception and definition, the importance of the role of the teacher during the process is emphasised, working together with the designers and developers to achieve a balance between the playful and the pedagogical from the initial phases of the creation of the educational video game. SCHEMA also emphasises the need to detail the format of the game from the beginning, always in line with the learning strategy, as well as highlighting the relevance of iteration during the development of its phases.
In short, and in agreement with Zhonggen [30] on the need to investigate educational video games’ design and development process, common patterns and differential elements have been identified in the existing methodological models of serious games of the last decade. Assessing the impact of this cultural and educational product and establishing taxonomies that regulate the booming sector of serious games [31] was the purpose of this research to lay a theoretical foundation for building future educational video games. With this study, SCHEMA becomes a valuable tool for this purpose, proposing a simple, common, and balanced process for the future creation of classroom educational games. In this way, it is possible to offer guidelines in the process of creating this attractive and relevant tool for the education of digital generations, in addition to taking a step forward towards a sustainable science that ensures quality production that makes a return on human, scientific, and financial investment [5].

5. Conclusions

In the last decade, educational video games have gradually established themselves in the classroom as a learning methodology that connects with today’s social discourses. Nevertheless, this new trend has pushed the video game industry towards a production that is not very controlled regarding the bases on which to build.
This research concludes with five main ideas, some related to the shortcomings detected in the methodologies and processes studied, and others as future lines of research to be explored, as well as a proposed model for the design, development, and evaluation of serious games.
Firstly, the systematic review carried out emphasises the need, already pointed out by some authors [29,30], to develop models that are committed to a balance between the playful and pedagogical components within these digital narratives, which, after all, is a matter of using a novel format to achieve specific learning outcomes, which are the primary purpose. In the same way, although some studies are timidly beginning to incorporate teachers in the processes of creation and construction of the serious game, it still needs to be established as a habitual dynamic. Precisely, this co-creative process would allow us to achieve the parity between entertainment and learning more effectively. Finally, the lack of iteration in most of the reviewed methodologies is noteworthy, a feature of utmost importance, since, as some researchers point out, the design and development of educational games have complications [76]. Therefore, this iteration, in order to rectify and correct errors, is essential.
This research presents SCHEMA, a model whose purpose is to serve as a base scheme for devising, building, and correcting serious games, in line with the objectives of this study. This model can be used by designers or developers, as well as by educators. In fact, the role of the teacher is considered fundamental in this process to achieve a balance between the playful and pedagogical components of the SG that is created. This proposal is structured around four steps, three of which are iterative (Specification–Content Hierarchy–Engineering–Monitoring and Assessment).
Secondly and finally, future lines of research are proposed in relation to the object of study. On the one hand, to design specific models for a specific format, such as Virtual Reality or educational video games for cell phones, since they have very different considerations and only a few processes with this approach have been found in the literature. Moreover, on the other hand, to develop methodologies to create serious games focused on areas other than health, marketing, or education, such as sports, art, or engineering.

Author Contributions

Conceptualisation, A.M.-C. and S.R.-d.-C.; methodology, A.M.-C. and S.R.-d.-C.; validation, A.M.-C. and S.R.-d.-C.; formal analysis, A.M.-C. and S.R.-d.-C.; investigation, A.M.-C. and S.R.-d.-C.; resources, A.M.-C.; writing—original draft preparation, A.M.-C.; writing—review and editing, A.M.-C. and S.R.-d.-C.; visualisation, A.M.-C.; supervision, A.M.-C., S.R.-d.-C. and P.N.-O.; project administration, S.R.-d.-C. and P.N.-O. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

The data presented in this study are available on request from the corresponding author.

Acknowledgments

This research is carried out under the Programme Erasmus+, with application number 626155-EPP-1-2020-2-LT-EPPKA3-PI-POLICY, titled “Teaching to Be: Supporting teachers’ professional growth and well-being in the field of social and emotional learning”.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Herrero, B. Innovación Social, Tecnología y ODS. Fórmula Magistral Para Un Mundo Mejor En La Era Del COVID-19. Rev. Iberoam. Econ. Solidar. Innovación Socioecol. 2021, 4, 29–64. [Google Scholar] [CrossRef]
  2. Escobar, J.F.; Mira, Y.M. La Globalización y La Importancia de Las TIC En El Desarrollo Social. Rev. Reflex. Saberes 2019, 11, 2–9. [Google Scholar]
  3. Almeida, F.; Simoes, J. The Role of Serious Games, Gamification and Industry 4.0 Tools in the Education 4.0 Paradigm. Contemp. Educ. Technol. 2019, 10, 120–136. [Google Scholar] [CrossRef]
  4. García, L. Necesidad de Una Educación Digital En Un Mundo Digital. RIED. Rev. Iberoam. Educ. A Distancia 2019, 22, 9–22. [Google Scholar] [CrossRef]
  5. United Nations. Sustainable Development Goals. Available online: https://www.un.org/sustainabledevelopment/sustainable-development-goals/ (accessed on 3 May 2023).
  6. Enciso, E.M.; Arias, J.; Arias, A. El Papel de Las Narrativas Digitales Como Nueva Estrategia Educativa: Resultados Desde Un Análisis Bibliométrico. Kepes 2016, 13, 197–231. [Google Scholar] [CrossRef]
  7. Acosta, A. Innovación, Tecnologías y Educación: Las Narrativas Digitales Como Estrategias Didácticas. Kill. Soc. Rev. Investig. Científica 2018, 2, 31–38. [Google Scholar] [CrossRef]
  8. López, A. Las Narrativas Digitales En Educación Infantil: Una Experiencia de Investigación e Innovación Con Booktrailer, Cuentos Interactivos Digitales y Realidad Aumentada. Diablotexto Digit. 2018, 3, 111–131. [Google Scholar] [CrossRef] [Green Version]
  9. Moreira-Chóez, J.S. Narrativas Digitales Como Didáctica Educativa. PoloConoc. Rev. Científico-Prof. 2021, 6, 846–859. [Google Scholar]
  10. Scolari, C.A. Adolescentes, Medios de Comunicación y Culturas Colaborativas. Aprovechando las Competencias Transmedia de los Jóvenes En El Aula; EC|H2020|Research and Innovation Actions: Barcelona, Spain, 2018. [Google Scholar]
  11. Echauri, G.; Figueras-Maz, M. El Acercamiento de Estudiantes Universitarios Internacionales a Plataformas de Streaming. Virtualis Rev. Cult. Digit. 2021, 12, 121–136. [Google Scholar]
  12. Moran, G.; Muzellec, L.; Johnson, D. Message Content Features and Social Media Engagement: Evidence from the Media Industry. J. Prod. Brand Manag. 2019, 29, 533–545. [Google Scholar] [CrossRef]
  13. Reyes-de-Cózar, S.; Ramírez-Moreno, C.; Barroso-Tristán, J.M. A Qualitative Analysis of the Educational Value of Commercial Video Games. Educ. Sci. 2022, 12, 584. [Google Scholar] [CrossRef]
  14. Sosa, C.D.; Capafons, J.I.; Pérez-Correa, S.M.; Hernández-Zamora, G.L.; Capafons, J.I.; Prieto, P. El Uso de Las Nuevas Tecnologías (Internet, Redes Sociales, Videojuegos) en Jóvenes: Un Estudio Con Población Canaria. Rev. Española Drogodepend. 2019, 44, 26–42. [Google Scholar]
  15. Felicia, P. Using Educational Games in the Classroom: Guidelines for Successful Learning Outcomes; Herzt, B., Pinzi, V., Sefen, M., Eds.; European Schoolnet & Interactive Software Federation of Europe (ISFE): Brussels, Belgium, 2020; ISBN 9789492913951. [Google Scholar]
  16. Ulrich, F.; Helms, N.H. Creating Evaluation Profiles for Games Designed to Be Fun: An Interpretive Framework for Serious Game Mechanics. Simul. Gaming 2017, 48, 695–714. [Google Scholar] [CrossRef] [Green Version]
  17. Mosalanejad, L.; Abdollahifard, S. Gamification in Psychiatry: Design and Development of Native Model and the Innovate Strategy in Medical Education as a Funny and Exciting Learning. Pak. J. Med. Health Sci. 2018, 12, 1820–1828. [Google Scholar]
  18. Kamalodeen, V.J.; Ramsawak-Jodha, N.; Figaro-Henry, S.; Jaggernauth, S.J.; Dedovets, Z. Designing Gamification for Geometry in Elementary Schools: Insights from the Designers. Smart Learn. Environ. 2021, 8, 36. [Google Scholar] [CrossRef]
  19. Slimani, A.; Yedri, O.B.; Elouaai, F.; Bouhorma, M. Towards a Design Approach for Serious Games. Int. J. Knowl. Learn. 2016, 11, 58–81. [Google Scholar]
  20. Camilleri, M.A.; Camilleri, A.C. Student Centred Learning Through Serious Games. In Proceedings of the 13th Annual International Technology, Education and Development Conference, Valencia, Spain, 11–13 March 2018; International Academy of Technology, Education and Development (IATED): Valencia, Spain, 2019. [Google Scholar]
  21. Lampropoulos, G.; Anastasiadis, T.; Siakas, K. Digital Game-Based Learning in Education: Significance of Motivating, Engaging and Interactive Learning Environments. In Proceedings of the 24th International Conference on Software Process Improvement-Research into Education and Training (INSPIRE 2019), Southampton, UK, 25–26 May 2019; pp. 117–127. [Google Scholar]
  22. Asociación Española del Videojuego (AEVI). La Industria del Videojuego en España En 2021; AEVI: Madrid, Spain, 2021. [Google Scholar]
  23. Mieres, R. Los Videojuegos Como Producto Cultural. Available online: http://bit.ly/3Rp0pLN (accessed on 20 June 2023).
  24. Oo, A.M.; Vallabhajosyula, R. Perceived Effectiveness of an Innovative Mobile-based Serious Game on the Improvement of Soft Skills in Minimally Invasive Surgical Training. Asian J. Endosc. Surg. 2023, 16, 41–49. [Google Scholar] [CrossRef]
  25. Berisford, C.J.; Blackburn, L.; Ollett, J.M.; Tonner, T.B.; Yuen, C.S.H.; Walton, R.; Olayinka, O. Can Gamification Help to Teach Cybersecurity? In Proceedings of the 2022 20th International Conference on Information Technology Based Higher Education and Training (ITHET), Antalya, Turkey, 7–9 November 2022; pp. 1–9. [Google Scholar]
  26. Abidin, S.R.Z.; Fadzilah, S.; Sahari, N. Low-Fidelity Prototype Design for Serious Game for Slow-Reading Students. Int. J. Adv. Comput. Sci. Appl. 2019, 10, 270–276. [Google Scholar] [CrossRef] [Green Version]
  27. El Arroum, F.-Z.; Hanoune, M.; Zidoun, Y. Serious Game Design: Presenting a New Generic Creative Reflection Framework. Int. J. Emerg. Technol. Learn. (IJET) 2020, 15, 247–255. [Google Scholar] [CrossRef]
  28. Li, J.; Spek, E.V.D.; Hu, J.; Feijs, L. Extracting Design Guidelines for Augmented Reality Serious Games for Children. IEEE Access 2022, 10, 66660–66671. [Google Scholar] [CrossRef]
  29. Ravyse, W.S.; Seugnet Blignaut, A.; Leendertz, V.; Woolner, A. Success Factors for Serious Games to Enhance Learning: A Systematic Review. Virtual Real 2017, 21, 31–58. [Google Scholar] [CrossRef]
  30. Jiang, F.; Shangguan, D. Researching and Designing Educational Games on the Basis of “Self-Regulated Learning Theory”. Front. Psychol. 2022, 13, 996403. [Google Scholar] [CrossRef] [PubMed]
  31. Katsaounidou, A.; Vrysis, L.; Kotsakis, R.; Dimoulas, C.; Veglis, A. MAthE the Game: A Serious Game for Education and Training in News Verification. Educ. Sci. 2019, 9, 155. [Google Scholar] [CrossRef] [Green Version]
  32. Shegog, R.; Brown, K.; Bull, S.; Christensen, J.L.; Hieftje, K.; Jozkowski, K.N.; Ybarra, M.L. Serious Games for Sexual Health. Games Health J. 2015, 4, 69–77. [Google Scholar] [CrossRef]
  33. Lovreglio, R.; Gonzalez, V.; Feng, Z.; Amor, R.; Spearpoint, M.; Thomas, J.; Trotter, M.; Sacks, R. Prototyping Virtual Reality Serious Games for Building Earthquake Preparedness: The Auckland City Hospital Case Study. Adv. Eng. Inform. 2018, 38, 670–682. [Google Scholar] [CrossRef] [Green Version]
  34. Bonnechère, B. Serious Games in Physical Rehabilitation: From Theory to Practice; Springer International Publishing: Cham, Switzerland, 2018; ISBN 978-3-319-66121-6. [Google Scholar]
  35. Fraga-Varela, F.; Vila-Couñago, E.; Martínez-Piñeiro, E. The Impact of Serious Games in Mathematics Fluency: A Study in Primary Education. Comunicar 2021, 29, 125–135. [Google Scholar] [CrossRef]
  36. Zhonggen, Y. A Meta-Analysis of Use of Serious Games in Education over a Decade. Int. J. Comput. Games Technol. 2019, 2019, 4797032. [Google Scholar] [CrossRef] [Green Version]
  37. Prieto-de-Lope, R.; Medina-Medina, N. A Comprehensive Taxonomy for Serious Games. J. Educ. Comput. Res. 2017, 55, 629–672. [Google Scholar] [CrossRef]
  38. Barianos, A.K.; Papadakis, A.; Vidakis, N. Content Manager for Serious Games: Theoretical Framework and Digital Platform. Adv. Mob. Learn. Educ. Res. 2022, 2, 251–262. [Google Scholar] [CrossRef]
  39. Tsekleves, E.; Cosmas, J.; Aggoun, A. Benefits, Barriers and Guideline Recommendations for the Implementation of Serious Games in Education for Stakeholders and Policymakers. Br. J. Educ. Technol. 2016, 47, 164–183. [Google Scholar] [CrossRef]
  40. Silva, F.G.M. Practical Methodology for the Design of Educational Serious Games. Information 2019, 11, 14. [Google Scholar] [CrossRef] [Green Version]
  41. Gao, N.; Xie, T.; Liu, G. A Learning Engagement Model of Educational Games Based on Virtual Reality. In Proceedings of the 2018 International Joint Conference on Information, Media and Engineering (ICIME), Osaka, Japan, 12–14 December 2018; IEEE: New York, NY, USA, 2018; pp. 1–5. [Google Scholar]
  42. Yuxuan, C.; Souza, R.C.G.; Contessoto, A.G.; Amorim, A.R. Guidelines for the Development of Educational Games to Motivate the Learning of Theoretical Concepts in Engineering and Computing Courses. Comput. Appl. Eng. Educ. 2021, 29, 1312–1323. [Google Scholar] [CrossRef]
  43. Egea-Vivancos, A.; Arias-Ferrer, L. Principles for the Design of a History and Heritage Game Based on the Evaluation of Immersive Virtual Reality Video Games. E-Learn. Digit. Media 2021, 18, 383–402. [Google Scholar] [CrossRef]
  44. Koivisto, J.-M.; Haavisto, E.; Niemi, H.; Haho, P.; Nylund, S.; Multisilta, J. Design Principles for Simulation Games for Learning Clinical Reasoning: A Design-Based Research Approach. Nurse Educ. Today 2018, 60, 114–120. [Google Scholar] [CrossRef]
  45. Ferreira de Almeida, J.L.; Machado, L. dos S. Design Requirements for Educational Serious Games with Focus on Player Enjoyment. Entertain. Comput. 2021, 38, 100413. [Google Scholar] [CrossRef]
  46. Zarraonandia, T.; Diaz, P.; Aedo, I.; Ruiz, M.R. Designing Educational Games through a Conceptual Model Based on Rules and Scenarios. Multimed. Tools Appl. 2015, 74, 4535–4559. [Google Scholar] [CrossRef]
  47. Breien, F.; Wasson, B. ELuna: A Co-Design Framework for Narrative Digital Game-Based Learning That Support STEAM. Front. Educ. 2022, 6, 775746. [Google Scholar] [CrossRef]
  48. Carvalho, M.B.; Bellotti, F.; Berta, R.; De Gloria, A.; Sedano, C.I.; Hauge, J.B.; Hu, J.; Rauterberg, M. An Activity Theory-Based Model for Serious Games Analysis and Conceptual Design. Comput. Educ. 2015, 87, 166–181. [Google Scholar] [CrossRef]
  49. Ledezma, M.R.; Simini, F. Serious Game Design by Unified Block Interactions to Support Educational Transformations. Int. J. Comput. 2021, 20, 487–493. [Google Scholar] [CrossRef]
  50. Serrano-Laguna, Á.; Manero, B.; Freire, M.; Fernández-Manjón, B. A Methodology for Assessing the Effectiveness of Serious Games and for Inferring Player Learning Outcomes. Multimed. Tools Appl. 2018, 77, 2849–2871. [Google Scholar] [CrossRef] [Green Version]
  51. Hutton, B.; Catalá-López, F.; Moher, D. La Extensión de la Declaración PRISMA Para Revisiones Sistemáticas Que Incorporan Metaanálisis en Red: PRISMA-NMA. Med. Clin. 2016, 147, 262–266. [Google Scholar] [CrossRef]
  52. Reyes-de-Cózar, S.; Pérez-Escolar, M.; Navazo-Ostúa, P. Digital Competencies for New Journalistic Work in Media Outlets: A Systematic Review. Media Commun. 2022, 10, 27–42. [Google Scholar] [CrossRef]
  53. Jappur, R.F.; Forcellini, F.A.; Spanhol, F.J. Modelo conceitual para jogos educativos digitais. AtoZ Novas Práticas Informação Conhecimento 2014, 3, 116–127. [Google Scholar] [CrossRef]
  54. Padilla-Zea, N.; Medina-Medina, N.; Gutiérrez-Vela, F.L.; López-Arcos, J.R.; Paderewski, P.; González-González, C.S. A design process for balanced educational video games with collaborative activities. DYNA 2015, 82, 223–232. [Google Scholar] [CrossRef]
  55. Andreoli, R.; Corolla, A.; Faggiano, A.; Malandrino, D.; Pirozzi, D.; Ranaldi, M.; Santangelo, G.; Scarano, V. A Framework to Design, Develop, and Evaluate Immersive and Collaborative Serious Games in Cultural Heritage. J. Comput. Cult. Herit. 2018, 11, 1–22. [Google Scholar] [CrossRef]
  56. Djaouti, D. DICE: A Generic Model for the Design Process of Serious Games. Int. J. Game-Based Learn. 2020, 10, 39–53. [Google Scholar] [CrossRef]
  57. Lui, R.W.C.; Au, C.H. Establishing an educational game development model: From the experience of teaching search engine optimization. Int. J. Game-Based Learn. 2018, 8, 52–73. [Google Scholar] [CrossRef]
  58. Fernandes, F.; Mateus, A.; Leonor, S.; Sequeira, M.; Gaio, R. Game-thinking: A roadmap to a Design Thinking-based model for Game Development education. Rev. Lusófona Educ. 2018, 40, 169–182. [Google Scholar] [CrossRef] [Green Version]
  59. Londoño, L.M.; Rojas, M.D. Determinación de criterios generales para el diseño de juegos serios: Modelo metodológico integrador. Inf. Tecnol. 2021, 32, 123–132. [Google Scholar] [CrossRef]
  60. Carrión-Toro, M.; Santorum, M.; Acosta-Vargas, P.; Aguilar, J.; Pérez, M. iPlus a User-Centered Methodology for Serious Games Design. Appl. Sci. 2020, 10, 9007. [Google Scholar] [CrossRef]
  61. Amengual Alcover, E.; Jaume-i-Capó, A.; Moyà-Alcover, B. PRO-Game: A process framework for serious game development for motor rehabilitation therapy. PLoS ONE 2018, 13, e0197383. [Google Scholar] [CrossRef] [Green Version]
  62. Avila-Pesantez, D.; Delgadillo, R.; Rivera, L.A. Proposal of a Conceptual Model for Serious Games Design: A Case Study in Children with Learning Disabilities. IEEE Access 2019, 7, 161017–161033. [Google Scholar] [CrossRef]
  63. Roedavan, R.; Pudjoatmodjo, B.; Siradj, Y.; Salam, S.; Hardianti, B.D. Serious Game Development Model Based on the Game-Based Learning Foundation. J. ICT Res. Appl. 2021, 15, 291–305. [Google Scholar] [CrossRef]
  64. Taipe, M.A. Modelo de serious game para mejorar la atención en niños con trastorno por déficit de atención e hiperactividad (TDAH). Rev. Ibérica Sist. Tecnol. Inform. 2019, 17, 936–946. [Google Scholar]
  65. Cano, S.; Munoz Arteaga, J.; Collazos, C.A.; Gonzalez, C.S.; Zapata, S. Toward a methodology for serious games design for children with auditory impairments. IEEE Lat. Am. Trans. 2016, 14, 2511–2521. [Google Scholar] [CrossRef]
  66. De Freitas, S. Are Games Effective Learning Tools? A Review of Educational Games. Educ. Technol. Soc. 2018, 21, 74–84. [Google Scholar]
  67. Gentry, S.V.; Gauthier, A.; L’Estrade Ehrstrom, B.; Wortley, D.; Lilienthal, A.; Tudor Car, L.; Dauwels-Okutsu, S.; Nikolaou, C.K.; Zary, N.; Campbell, J.; et al. Serious Gaming and Gamification Education in Health Professions: Systematic Review. J. Med. Internet Res. 2019, 21, e12994. [Google Scholar] [CrossRef] [Green Version]
  68. De Klerk, S.; Kato, P.M. The Future Value of Serious Games for Assessment: Where Do We Go Now? J. Appl. Test. Technol. 2017, 18, 32–37. [Google Scholar]
  69. Caballero-Hernández, J.A.; Palomo-Duarte, M.; Dodero, J.M.; Person, T. Evaluación de Competencias En Serious Games Mediante Analítica de Aprendizaje Con Process Mining. In Proceedings of the Aprendizaje, Innovación y Cooperación Como Impulsores del Cambio Metodológico, Madrid, Spain, 9–11 October 2019; Servicio de Publicaciones Universidad: Zaragoza, Spain, 2019; pp. 198–203. [Google Scholar]
  70. Schickler, M.; Pryss, R.; Reichert, M.; Schobel, J.; Langguth, B.; Schlee, W. Using Mobile Serious Games in the Context of Chronic Disorders: A Mobile Game Concept for the Treatment of Tinnitus. In Proceedings of the 2016 IEEE 29th International Symposium on Computer-Based Medical Systems (CBMS), Belfast and Dublin, Ireland, 20–24 June 2016; IEEE: Belfast/Dublin, Ireland; pp. 343–348. [Google Scholar]
  71. Jaramillo-Alcázar, A.; Luján-Mora, S. Mobile Serious Games. In Proceedings of the 5th International Conference on Technological Ecosystems for Enhancing Multiculturality, Cádiz, Spain, 18–20 October 2017; ACM: New York, NY, USA, 2017; pp. 1–6. [Google Scholar]
  72. Babu, S.K.; Krishna, S.; Unnikrishnan, R.; Bhavani, R.R. Virtual Reality Learning Environments for Vocational Education: A Comparison Study with Conventional Instructional Media on Knowledge Retention. In Proceedings of the 2018 IEEE 18th International Conference on Advanced Learning Technologies (ICALT), Mumbai, India, 9–13 July 2018; IEEE: Mumbai, India, 2018; pp. 385–389. [Google Scholar]
  73. Bruno, F.; Barbieri, L.; Lagudi, A.; Cozza, M.; Cozza, A.; Peluso, R.; Muzzupappa, M. Virtual Dives into the Underwater Archaeological Treasures of South Italy. Virtual Real 2018, 22, 91–102. [Google Scholar] [CrossRef]
  74. Adjorlu, A.; Serafin, S. Head-Mounted Display-Based Virtual Reality as a Tool to Teach Money Skills to Adolescents Diagnosed with Autism Spectrum Disorder; Springer: Berlin/Heidelberg, Germany, 2019; Volume 265, pp. 450–461. [Google Scholar]
  75. Gros, B. Game Dimensions and Pedagogical Dimension in Serious Games. In Handbook of Research on Serious Games for Educational Applications; IGI Global: Hershey, PA, USA, 2017; pp. 402–417. [Google Scholar]
  76. Viudes-Carbonell, S.J.; Gallego-Durán, F.J.; Llorens-Largo, F.; Molina-Carmona, R. Towards an Iterative Design for Serious Games. Sustainability 2021, 13, 3290. [Google Scholar] [CrossRef]
  77. Kangas, M.; Koskinen, A.; Krokfors, L. A Qualitative Literature Review of Educational Games in the Classroom: The Teacher’s Pedagogical Activities. Teach. Teach. 2016, 23, 451–470. [Google Scholar] [CrossRef]
  78. Lameras, P.; Arnab, S.; Dunwell, I.; Stewart, C.; Clarke, S.; Petridis, P. Essential Features of Serious Games Design in Higher Education: Linking Learning Attributes to Game Mechanics. Br. J. Educ. Technol. 2017, 48, 972–994. [Google Scholar] [CrossRef] [Green Version]
Sustainability 15 12351 g001
Figure 1. Planning, identification, and eligibility process workflow. Source: authors.
Figure 1. Planning, identification, and eligibility process workflow. Source: authors.
Sustainability 15 12351 g001
Sustainability 15 12351 g002
Figure 2. Qualitative analysis procedure. Source: authors.
Figure 2. Qualitative analysis procedure. Source: authors.
Sustainability 15 12351 g002
Sustainability 15 12351 g003
Figure 3. Frequency of publication of studies by year. Source: authors.
Figure 3. Frequency of publication of studies by year. Source: authors.
Sustainability 15 12351 g003
Sustainability 15 12351 g004
Figure 4. Process of creating an SG using SCHEMA. Source: authors.
Figure 4. Process of creating an SG using SCHEMA. Source: authors.
Sustainability 15 12351 g004
Table 1. Inclusion and exclusion criteria according to PICO model. Source: authors.
Table 1. Inclusion and exclusion criteria according to PICO model. Source: authors.
Inclusion CriteriaExclusion Criteria
ParticipantsAnyNone
InterventionAnyNone
Context 1Published between 2014–2023
Articles
Final stage		Others
Books, chapters, proceedings,
and others
In press and others
OutcomesFocused on processes or
methodologies to design,
develop and/or evaluate SG
Educational video games 		Focus on guidelines,
recommendations, principles,
or models
Commercial video games
1 The Comparator section of the usual PICO format was changed to Context, following [52].
Table 2. Main characteristics of the publications included in the review. Source: Authors.
Table 2. Main characteristics of the publications included in the review. Source: Authors.
Nomenclature
in the Study		Authors (Year)Methodology
[53]Jappur, Forcellini y Spanhol (2014)Literature review
[54]Padilla-Zea et al. (2015)Literature review
[55]Andreoli et al. (2017)Not specified
[56]Djaouti (2020)Literature review
[47]Breien y Wasson (2022)Not specified
[57]Lui y Au (2018)Literature review
[58]Fernandes et al. (2018)Literature review
[59]Londoño y Rojas (2021)Literature review
[60]Carrión-Toro et al. (2020)Not specified
[61]Amengual, Jaume-i-Capo y Moyà-Alcover (2018)Not specified
[62]Avila-Pesantez, Delgadillo y Rivera (2019)Not specified
[63]Roedavan et al. (2021)Not specified
[64]Taipe (2019)Literature review
[19]Slimani et al. (2016)Interviews with experts in game design (non-Delphi)
[65]Cano et al. (2016)Literature review
Table 3. Main dimensions extracted from the publications included in the review. Source: authors.
Table 3. Main dimensions extracted from the publications included in the review. Source: authors.
IDFocus/BaseScopeUsePhases
Iteration		FormatApproachPractical
Validation
[53]Learning about the culture of sustainability in the
residential and home
environment		GeneralDesign
Development		No Computer gamePlayful and pedagogical balanceYes
[54]Collaborative aspectGeneralDesign
Development
Evaluation		NoNot
specified		Playful and pedagogical balanceYes
[55]Collaborative and cultural
aspect		GeneralDesign
Development
Evaluation		YesNot
specified		PlayfulYes
[56]NoneGeneralDesignYesNot
specified		Mostly playfulNo
[47]Learning based on
digital and narrative games
Co-design (teachers and
designers)		STEAM
(Learning)		Design
Development		NoNot
specified		Playful and pedagogical balanceYes
[57]Teaching SEO (Search
Engine Optimisation)		Marketing
(SEO)		Design
Development		YesNot
specified		Mostly
pedagogical		Yes
[58]Design ThinkingDesign
Thinking		DevelopmentNoNot
specified		Playful and pedagogical balanceNo
[59]NoneGeneralDesign
Development		Not
at all		Not
specified		PlayfulNo
[60]User-centred designGeneralDesignNoNot
specified		Mostly playfulYes
[61]Motor rehabilitationTherapy
(Health)		DevelopmentYesNot
specified		PlayfulYes
[62]Learning disabilitiesLearningDesignNoNot
specified		Playful and pedagogical balanceYes
[63]Game-based learning
foundation		GeneralDevelopmentNot
at all		Not
specified		Playful and pedagogical balanceYes
[64]Improving attention
of children with ADHD		ADHD
(Health)		Design
Development		NoAugmented RealityMostly playfulNo
[19]Collaborative design
Multi-layered model		GeneralDesign
Evaluation		YesNot
specified		Playful and pedagogical balanceYes
[65]Children with
disabilities
hearing impairment		Disabilities
(Health)		Design
Development		NoNot
specified		Playful and pedagogical balanceYes
Table 4. SCHEMA: a process for the creation and evaluation of serious games. Source: authors.
Table 4. SCHEMA: a process for the creation and evaluation of serious games. Source: authors.
StepsDefinitionIn LiteraturePlayful vs. Pedagogical
SpecificationThis is the initial point at which
the need for creating a serious game, the target audience, and the objectives to
be achieved are identified		[55] Preliminary phase
[59] Pre-design phase
[60] Identification phase
[61] Project initiation 		Playful
[56] Define
[47] Preparation
[64] Enquiry
[19] Learning layer		Pedagogical
[58] Involvement
[62] Analysis 		Playful and pedagogical
balance
Content
Hierarchy		The aim will be to give shape
to all the material to be included
in the educational video game, both educational-pedagogical
content, as well as all the
considerations at the level of
mechanics, dynamics,
and aesthetics of the game		[54] Entertaining content design
[55] Conceptual phase
[56] Imagine
[59] Design phase (writing sub-phase)
[60] Ludic game script phase
[61] Planning and control
   Modelling
[62] Design
[19] Gameplay layer
[65] Pre-production		Playful
[54] Educational content design
[58] Ideation
[60] Pedagogical objectives phase
[63] Analysis
[64] Modelling		Pedagogical
[53] Creation stage
[47] Co-design and co-specification
[57] Conceptual design
   Game design
[58] Inspiration
[62] Analysis
[19] Story layer
[65] Analysis		Playful and pedagogical
balance
EngineeringThe serious game is
implemented, the previous
design is put into practice and
the game is created		[54] User modelling
[55] Development phase
[56] Create
[47] Development
[57] Prototyping
[59] Design phase (construction sub-phase)
[60] Gameplay phase
[61] Construction
[62] Development
[63] Production
[64] Design
[19] Experimentation layer
[65] Production		Playful
[53] Application stage Pedagogical
[54] Relating educational and entertaining content
[58] Integration 		Playful and pedagogical
balance
Monitoring and AssessmentThe aim is to correct possible
errors. The game is tested with
the sample to detect problems
and fix them.		[55] Evaluation phase
[56] Evaluate
[57] Evaluation
   Identification
[58] Implementation
   Interaction
[59] Testing phase
[60] Refine phase
[61] Evaluation
[63] Testing
[64] Evaluation		Playful
[63] Release
[65] Post-production		Pedagogical
[53] Evaluation stage
[62] Evaluation
[19] Debriefing layer 		Playful and pedagogical
balance
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Merino-Cajaraville, A.; Reyes-de-Cózar, S.; Navazo-Ostúa, P. SCHEMA: A Process for the Creation and Evaluation of Serious Games—A Systematic Review towards Sustainability. Sustainability 2023, 15, 12351. https://doi.org/10.3390/su151612351

AMA Style

Merino-Cajaraville A, Reyes-de-Cózar S, Navazo-Ostúa P. SCHEMA: A Process for the Creation and Evaluation of Serious Games—A Systematic Review towards Sustainability. Sustainability. 2023; 15(16):12351. https://doi.org/10.3390/su151612351

Chicago/Turabian Style

Merino-Cajaraville, Alba, Salvador Reyes-de-Cózar, and Pablo Navazo-Ostúa. 2023. "SCHEMA: A Process for the Creation and Evaluation of Serious Games—A Systematic Review towards Sustainability" Sustainability 15, no. 16: 12351. https://doi.org/10.3390/su151612351

APA Style

Merino-Cajaraville, A., Reyes-de-Cózar, S., & Navazo-Ostúa, P. (2023). SCHEMA: A Process for the Creation and Evaluation of Serious Games—A Systematic Review towards Sustainability. Sustainability, 15(16), 12351. https://doi.org/10.3390/su151612351

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop