Outstanding Videogames on Water: A Quality Assessment Review Based on Evidence of Narrative, Gameplay and Educational Criteria

Abstract

Videogames have become educational, communicative and social tools among the young, favouring the acquisition of skills, abilities and values, encompassing an endless number of themes, and helping them to experience and to face, in the first person, a great diversity of environmental situations and ecology problems. Thus, the present article aims: (a) to evaluate a sample of 20 educational videogames about water, making use of some empirical criteria of quality; and (b) to design, validate and apply an integrated quality indicator of educational videogames on water, based on the aspects of narrative, gameplay and education, which allows us to obtain a ranking. The findings reflect a ranking of games allowing us to suggest that the nature of the game (simulation, adventures, platforms or questions) does not determine the quality of the game, although generally simulations and adventure games are placed in a range of medium- or high-quality, as well as those games that pursue objectives related to the design and management of a territory in a sustainable way. The paper provides teachers with quality criteria based on narrative and gameplay that complement and enrich the pedagogical dimension.

1. Introduction

At present, we are immersed in the Anthropocene, the new geological era of the planet, characterized by the influence of the human species on Earth [1,2,3,4]. This influence is exerting great pressure on water and aquatic ecosystems, resulting in water pollution, loss of aquatic biodiversity, overexploitation of aquifers, and ultimately putting the survival of the species and human welfare at risk. The recent Intergovernmental Panel on Climate Change (IPCC) reports [5] point out that the most serious consequences of climate change (CC) are problems related to water resources. For this reason, the United Nations has just proclaimed the International Decade for Action “Water for Sustainable Development” [6]. This plan seeks to promote international cooperation partnerships that allow an efficient use of water, reduce tensions and disputes between the territories as a result of water deficits, and at the same time contribute to the achievement of the objectives of the 2030 Agenda for Sustainable Development, among which water is a priority goal. Agenda 2030 includes several objectives related to the sustainability of water and aquatic ecosystems, as well as recognizing youth as a key actor in achieving these objectives. In this regard, it specifies that youth have the ability to participate widely in raising awareness and promoting the Sustainable Development Goals (SDGs) both in their communities and through social networks and other digital communication tools, as they are capable of providing innovative approaches and solutions for their implementation [7].

To face this bleak panorama, citizens have united around the movement “Get wet for the water, Get involved for your community”, whose objective is to mobilize society to act in defense of water and aquatic ecosystems through actions that favor the maintenance of ecosystem services, human welfare and the recognition of water as a human right, with the slogan: “Do you dare to participate in this social movement in your community? We are waiting for you. Act!”

Accordingly, it is well known that young people spend a great part of their leisure and free time using social networks and videogames, besides online games having a lot of acceptance among this public [8]. Videogame electronic games which use computer technology and allows a player to interact with the machine in real time promote motivation for learning, as well as cognitive potentialities [9,10,11]. In addition, videogames favour personal autonomy, social and cultural commitment [12]. In McGonigal’s words [13]: “Videogames can make us better people and help us to change the world”. Some other studies on the effects of video games on young people conclude: “The usefulness of video games to complete some of the skills acquired in the formal education system, favour the formation of identities and knowledge of social rules, as well as supporting the development of organizational skills or the development of some critical capacities” [14,15,16,17,18].

New generations of “digital natives” [19] and “digital wisdom” [20] need educational tools that cover learning needs, new to previous generations, and videogames as a powerful educational resource [21]. In terms of the formal educational context, the use of videogames (called “online serious games” when they are played on the Internet) by teachers in the classroom has experienced significant growth, since its use as a curricular element or resource has allowed students to achieve greater motivation towards knowledge, making the classroom a dynamic, participatory and attractive space [22]. “Serious games, gamification and virtual reality can be seen as a response of policy analysis to a growing social need to get ‘engaged and entertained’, also in public policy making” [10] and the sustainability of water.

Recent research on games and learning points to the need to carry out shared studies that determine which games are better or worse when contributing to student learning [23]. This research shows a classification of a video game on water according to its narrative, gameplay and educational potential. This classification can help teachers to choose those excellent games and differentiate them from others of lower quality. The possibility of integrating these didactic resources as an ordinary element of the curriculum and analysing their pedagogical impact opens interesting ways to educate future generations of young people about sustainability.

2. Background

The need to promote more attention to sustainability issues, games and youth people learning and demands to develop more empirical research in the field of social sciences and especially focusing on the education for sustainability has become an explicit agenda of the academic literature [24,25]. In this agenda, the field of videogames in their different genres [26,27], is an explicitly recognized hot topic of research [28,29,30]. In this sense, the IPCC Working Group III has more recently tried to deal with socio-environmental research demands and to move beyond the topics of hard sciences; to this end, it proposes focusing social research on reducing epistemic uncertainty surrounding the perceptions and social responses to environmental topics such as water and climate change. The report identifies more than 20 future research topics in this line, including those located more specifically in the fields of behavioural sciences, education and communication [5]. Among these lines, the field of videogames stands out as one of the 20 lines of the agenda, being a topic with its own entity with the need to promote evaluative research in this regard: “Evaluate the effectiveness of experiential methods such as simulations, games and films to improve the public’s understanding and perception of processes of CC” [5]. The view of social scientists on these debates on education for sustainability is essential, since they must provide answers based on evidence of relevant empirical research that meets the new evaluation standards of IPCC initiated in 2015 and that will extend to 2022 within those debates [24].

Playing computer games is linked to a variety of impacts and perceptual, cognitive, behavioral, affective and motivational results; with the most frequent results and impacts being the acquisition of knowledge and understanding of the content and the affective and motivational results [23]. Videogames that are intended to convey ideas, values and influence the thoughts and actions of players in real-life contexts have been called serious games [31]. These games, also called “games of change” [32,33] or “social impact games” [17], have experienced a rapid rise in the last decade, due to the popularity of videogames as audiovisual technology for the media, non-governmental organizations (NGOs), politicians, activists, teachers, professionals and the art sector, because they can encompass multiple learning objectives, encompass diverse areas and target different age groups [34]. Attending to their gameplay characteristics, serious games exist in a variety of genders and formats such as simulations, simulating aspects of a real or fictional reality; quizzes, answering questions with the goal of obtaining points- platforms-navigating one’s environment while avoiding obstacles; or adventures, solving puzzles by interacting with people or the environment in a non-confrontational way [23,27,28,29,30]. The choice of the type of game influences the success of digital educational games, where success is defined as a significant gain of knowledge in combination with the positive experience of the player. In this regard, a study made a game genre map, where five types of games were defined: mini-game, action, adventure, resources and role play. This research concluded that those role-plays that combine the elements of the adventure, action and resource genres are the most appropriate for educational learning. However, it must focus on what attributes of the different types of games genre are most appropriate depending on the type of learning to be achieved [30].

Our research evaluates electronic games on water sustainability that use computer technology and allow a player to interact with the machine in real time to promote the motivation for learning as well as cognitive potentialities.

Videogames have great potential to favor scientific education among young people, awakening the motivation to learn concepts and scientific processes through new challenges adapted to the particular needs and interests of each one. They can allow learners to visualize, explore, and formulate scientific explanations for scientific phenomena that would otherwise be impossible to observe and manipulate. Increasing learning time, focusing instruction toward individual learning needs and opportunities, and providing ongoing formative feedback have been shown to support learning generally and science learning specifically [35].

While it seems that games do enhance student motivation, are engaging and can be associated with behavioral change, more active design studies are needed to ensure that the best interests of the learner are met in different contexts [28]. As with other educational tools, it is important to consider how games are integrated into the student’s learning experience to influence outcomes related to cognition, effect, and behavior [30].

Meta-analysis research on sustainability videogames like that conducted by Soekarjo and Van Oostendorp [36] suggest that limited empirical evidence is currently available to prove the effectiveness of games in attitude change: first of all, evidence of successful change in attitude after playing a persuasive game can only be found in five of 60 papers reviewed. Secondly, using a pre-test–post-test design, they tested whether change in attitude was different for people playing the persuasive game EnerCities compared to a control condition where participants read a document with highly similar information, finding no significant differences.

Reeves et al. [37] built a social game about energy use in a virtual home and, in a field test, smart meter data showed a significant decrease in electricity usage when comparing 30-day periods before and after playing. A comparative study examines the influence of 2020 Energy among north American and Spanish teenagers 12 age old (n = 108); using a pre-test–post-test research design in which participants are divided into two conditions: the experimental condition (playing) and the control condition (not playing). Results show that there have not been statistically significant differences after playing the game, although when looking closer at each factor, some positive consequences can be found. the game seems to have had a bigger impact on the American experimental group than on the Spanish group regarding positive attitude towards energy-saving behaviors, and that moving by bicycle is the activity that increased the most after playing the game, both among Spanish and North American students [38].Learning based on videogames has great advantages in the training processes by allowing students to be active and direct their process; in this type of learning we obtain immediate feedback, we learn through problem solving, and the students focus only on their learning [39]. “A well-designed serious game environment provides a feedback mechanism that allows the player to reflect on his or her actions and adopt different approaches or strategies. The internalisation of actions and reactions stimulates learning, often resulting of an increase in self-learning and knowledge retention” [40]. Therefore, serious games can play a fundamental role in the promotion and awareness of the sustainability of water and aquatic ecosystems among youth.

If videogames aspire to be validated as useful and constructive tools to foster learning, social change, or anticipated understanding of social issues, the quality of their design must be evaluated. In this sense, some research has been carried out with the aim of establishing evaluation tools that allow the identification of criteria for serious games in various dimensions [41,42,43,44], as well as the study of the characteristics of good commercial videogames [45,46] and assessment about educational aspects of commercial videogames in the teaching-learning processes [47,48,49,50,51,52]. “Research on how videogames actually contribute to, or even influence, policy making and management is scarce, perhaps because it requires an evaluation type of research that is quite difficult to set up. It would need to build on a comparative analysis of a rich and varied set of cases with such innovative approaches” [10]. Just a few of these studies have been concerned, therefore, with evaluating the quality of videogames from perspectives beyond the pedagogical dimension, integrating quality criteria based on evidence of other dimensions such as narrative or gameplay. Other research, concludes that it is important to focus on the development of interdisciplinary research that can address the different variables involved in the processes of design, development and evaluation of educational videogames [28,53].

Many teachers demand from researchers and game designers the criteria that allow them to differentiate between good and bad games from different characteristics and quality dimensions. Research on the subject is an emerging field that requires an approach that is not exclusively pedagogical, as the gameplay aspects and the narrative potential to build stories must also be considered as influential variables in the characterization and selection of quality games. In relation to the theme of water and aquatic ecosystems, there are scarcely references to the evaluation of videogames in general, and the evaluation of videogames produced in languages other than English.

Research has revealed the need for providing a greater discursive complexity to the theme of water in the different initiatives, policy, practices and educational resources that from environmental education are carried out with the aim of favouring the conservation of water resources and aquatic ecosystems [54,55,56,57,58,59,60]. Therefore, it seems extremely important to know how the issue of water is being treated from the window of virtual games, and to investigate if, as in other initiatives, it is necessary to take a step in the construction of a discursive approach towards a complexity of the same kind, besides identifying which are the characteristics that gather the online games that are approaching water [10]. Some authors have done different studies on: (1) classifications of serious games according to different criteria [27,61,62,63]; (2) use of serious games as a tool for teaching and evaluation of generic competences in higher education, by designing levels, indicators and descriptors [64,65] and (3) how online games can influence individual lifestyles of the players [60]. However, no study has yet been carried out that addresses the need to establish indicators to assess the quality of educational videogames in order to establish different quality scenarios in which to classify such games [66] taking into account integrated dimensions in a multimodal way.

Research findings suggest that the games used for teaching sustainable development have generally increased players’ understanding of issues around sustainability and have enhanced their knowledge of sustainable development strategies. Use of videogames for teaching sustainable development is an interesting way to acquire the managerial skills required to effect change and to develop and increase attitudes, knowledge and awareness in a sustainable perspective [67]. Games present great opportunities as tools of edutainment (educational entertainment) for teaching and training, with positive effects on learning outcomes [68].

With this objective, on the one hand, we considered indispensable to evaluate educational videogames on water produced today, from a holistic and multidisciplinary perspective which encompasses both aspects of narrative (discursive construction around water under the paradigm of complexity); gameplay (set of properties that describe the player’s experience with a given game system); and pedagogy or education (referred to competencies, skills and learning). On the other hand, it also seemed of paramount importance, to carry out the design and application of a quality indicator that allows users to know which games are better in relation to three fundamental aspects such as narrative, gameplay, and education. In this way, it will be possible to obtain a ranking in games, as well as to detect strengths and weaknesses in each of the aspects to be evaluated.

3. Materials and Methods

3.1. Mixed Method Research

The evaluation of the sample of selected videogames was carried out using a mixed methodology research, qualitative and quantitative [69,70], based on the evaluation of pairs and the classification of the games according to a series of scores assigned to its educational, narrative, and gameplay quality. The dimensions emphasize factors external to the technological and architectural quality of the game design, although they integrate key elements of the video game’s internal character such as the story play and narrative structure. Based on a Delphi consultation that consists in a systematic, iteractive and group process aimed at obtaining expert opinions and analyzing consensus after several rounds of consultation. A matrix of quality criteria for the evaluation of videogames has been validated after the round of consultations. This matrix was applied to each of the games by three independent evaluators who discussed and agreed on the discrepancies. Afterwards, a hierarchical classification was carried out based on which it was possible to establish a ranking of excellence by quartiles. This methodology is similar to the methodology used in other procedures of honorable mention and recognition of merit, considered as modalities of subjective peer evaluation [71].

A very popular example of this type of model is applied by the Hollywood filmaker industry to assign the annual awards in their Oscar Awards to the best films, depending on how different aspects of the film quality are valued such as costumes, music, photography, revelation actor, better director, etc. Although, for the film industry, the most objective index is the box office, in terms of collection and number of viewers who watch and pay for films, for video games one could think of similarity in the number of buyers, downloads or users who play and make use of it. As in the cinema, these quantitative indices that turn a film into a “box office” success are filtered and conditioned by other ex ante evaluation systems of prior reputation, subject to peer consensus evaluation. The publicity that the nominated films receive in the case of the Hollywood Oscar Award contributes to increasing its reputation based on the judgment of experts (before being even awarded) and is complemented by the ratings of the users in web and media communication, comments, criticisms and “likes” received in social networks. In this way, experts and users are those who assign merit to some relevant dimension of the artistic quality of that work. Advertising contemporary social media marketing also contributes to all this. Videogames assessment also imitate these evaluation procedures used in the cinematographic world and grants more or less fame and publicity to the games through awards and prizes such as those granted, for example, by the International Education Games Competitions (IEGC, 6th edition, 2018) [72], the Best of Swiss Web [73]; and the Continental Media Competition-Prix Europe [74].

3.2. Videogame Excellence and Metrics for Quality Evaluation

We understand the “qualty concept” as “a measure of excellence” or a “state of being free from defects, deficiencies and significant variations” [75]. The classic reference standard on quality, ISO/IE 9126, 2001 defines quality as “the totality of features and characteristics of a product, process or service that bears its ability to satisfy stated or implied needs” [76]. This excellence “is brought about by strict and consistent commitment to certain standards that achieve uniformity of a product (videogames) in order to satisfy specific customer or user requirements” [75].

The ISO standard has more recently developed specific proposals applied to the world of video games that use heuristics and metrics for the evaluation of quality; and have been applied to the production and use of video games in different ways. This diversity of metrics is justified from the need to objectify relevant aspects of video games according to the perspective of creators, sellers and users. The scopes of these metrics have focused initially on dimensions such as design, production and marketing and more recently on aspects such as education, gameplay and usability. These more recent dimensions put an emphasis on the assessment of more qualitative aspects such as satisfaction, learning, efficiency, reliability or efienciency.

The standard, ISO/IEC 25010-3/2009 [77], addresses both internal aspects, specific to software quality based on criteria of navigability, functionality, portability, flexibility, reliability etc., as well as other external criteria such as usability and related quality of use (Figure 1) with satisfaction, motivation, learning, immersion, attractiveness, effectiveness, efficiency, etc.

The classic ex-ante videogame evaluation metrics [78,79] focus on internal evaluation processes centered on the design (game mechanic, game engine) and on the architecture of the game (game interface, narrative game and and plot development). Other types of more recent external metrics have focused on aspects external to the design and architecture of the videogame [80,81,82].

Some recent trends suggest the use of metrics based on the analysis of educational facets and playability attributes integrated with usability and quality of use (Figure 2), based on integrated metrics or multicriteria [83,84].

3.3. Sample Study

Videogames that are available on the Internet are also called online games. In order to select the sample of our study, we focused on those educational videogames on water that were online. We used the following web search: “videogame” or “online game” and “water”; “water cycle”; “aquatic ecosystems”. Different search engines were used in different languages, such us: Google, Yahoo, Ask. On the other hand, it was also taken into account that they fulfilled the following requirements: that they are hosted on a web platform and are free to access (thus favoring social inclusion and reaching a greater number of players); that they have communicative and educational objectives; that the focus of the storyline is water or related aspects such as cycle of water, water management, ecosystem aquatic, the human right to water; that they are aimed at an adolescent audience (+12); and that they are produced in several languages, thus covering different countries and recipients.

The sample of games varies in the possible strategies to carry out, the languages, as well as in the duration of the game, thus favoring the evaluation of different types of games and valuing their pedagogical contribution. A sampling of maximum variability in all these variables has been considered. The sample consists of 20 games (

Table 1. Study simple of games.

Game	Creator Language Website	Description
	National Geography	Simulation game with the objective of designing a green city, managing the resources in an ecological way, applying environmental criteria to urban planning and promoting the creation of ecological jobs. You decide where to place each building, factory and power plant. Deciding whether to build a park or a nuclear power plant is a hard choice, but now you are the mayor and it is all in your hands. As a mayor you have to decide between providing power and keeping a clean and sustainable city.
	Languages: English
Plant it Green	https://www.nationalgeographic.org/game/plan-it-green-generation-station/
	ABC Catchment Australia SA	Simulation game related to the management of the River Basin and a sustainable city. It is an online game where you’re in charge of the whole catchment. You get to decide what activities you undertake—whether to plant crops, log forests, build factories or set up national parks. The aim is to avoid environmental problems and provide food and wealth for the population.
	Languages: English
Catchment	http://www.catchmentdetox.net.au/
	United Kingdom Government	Simulation game related to the productive management of a territory and its environmental quality. You are the territory manager, you decide where to grow, where to plant, and where to build, from the upper basin to the mouth. The objective of the game is to make a sustainable management of the territory.
	Languages: English
Pipe Dreams	http://www.pipedreams-online.co.uk/
	Water consortium of Guipúzcoa	Game related to questions about the new water culture, personified by means of the bird: the Martin Fisherman. This kingfisher is the symbol of the new culture of water. There has been a great drought and now it is in danger. If you want to help Martintxo and his world, select the correct answer.
	Languages: Spanish and Euskera
Kingfisher	http://www.gipuzkoakour.eus/Martintxo/juego.asp?idioma=E
	Hydrographic Confederacy of the Ebro River	Simulation-skill game related to the management of the Ebro basin. The reservoir’s technical scientist teaches us how to manage the reservoir volume of the dam. The objective is to avoid a flood and properly manage the water reserves of the reservoir.
	Languages: Spanish
SAIH Ebro	http://www.chebro.es:81/educativo/juego/index.html
	International Crisis Group and Reebok Human rights Foundation	Simulation game based on the narrative of a displaced refugee, overcome the forces that threaten the survival of the camp, such as water management and sanitation. It is a web-based, viral video game that provides a window into the experience of the 2.5 million refugees in the Darfur region of Sudan. It is designed to raise awareness of the genocide taking place in Darfur and empower college students to help stop the crisis.
	Languages: English and Spanish
Darfur is Dying	http://www.gamesforchange.org/game/darfur-is-dying/
	Cloud Institute	Simulation game on sustainable fisheries management and ecosystem conservation. You have 10 days to catch as many fish as you can. The money you make from these fish will need to support your family for the next month. The object of the game is to have as many fish as possible at the end of the game.
	Languages: English
Fish Game	https://cloudinstitute.org/fish-game/
	Game, Learning and Society GSL	Adventure game with the purpose of recovering eutrophic lake through different measures of ecological conservation and citizen awareness. The protagonists of the game warn about the eutrophication of the aquatic ecosystem to citizens, trying to find solutions for the restoration of it. The objective is to implement both individual and collective awareness actions for the conservation of aquatic ecosystems.
	Languages: English
Citizen Science	http://www.sciencegamecenter.org/games/citizen-science
	Zaragoza Local Government	Platform game whose objective is to help Fluvi to save the rivers of the contamination, to take care of the ground waters and to watch the operation of the factories. My name is Fluvi and I am a small creature of water. My mission is to get water to everyone. The objective of the game is to pass each of the phases, avoiding the monsters and picking up trash in your path.
	Languages: Spanish
Fluvi	http://www.expozaragoza2008.es/juegofluvi/
	Mary wharmby	Adventure game about the water crisis, focused on improving water management and promoting awareness with this problem. Water is the new oil. The Riverbed is a first-person interactive experience: part game, part story. The fictional murder-mystery set in a land devastated by water scarcity. The Riverbed is designed to raise awareness of this looming crisis and help players to better understand the dynamics at work in water scarcity situations. Issues like upstream/downstream, sustainability, conservation and the security dilemma are explored in a fun and engaging way.
	Languages: English
Riverbed	http://www.gamesforchange.org/game/the-riverbed-an-eco-noir-mystery/
	Government of Galicia	Simulation game to manage a territory in a sustainable way. Decisions must be made about how to manage a territory in areas such as water, energy, protected natural spaces, among other topics. The objective of the game is to manage a territory in a sustainable manner.
	Languages: English, Spanish and Galego
Climantica	http://xogo.climantica.org/
	The National Oceanic and Atmospheric Administration, Alabama	Adventure game whose objective is to raise awareness about the problem of pollution of aquatic ecosystems. An otter and a girl will try to raise awareness about the problems of water pollution. To do this, you must pass several tests and ensure that the water is clean for human consumption and your home: the river.
	Languages: English
Where the River Meets the Sea	http://games.noaa.gov/oscar/game/welcome.html
	United Nations	Simulation game whose objective is to plan and build a safe city in the face of natural disasters. You decide where and how you build a city, keeping in mind that the area is affected by floods and tsunamis. The objective of the game is to know how to build a safe city in the face of natural catastrophes.
	Languages: English, Chinese, French, Spanish and Russian
Stop Disaster	http://www.stopdisastersgame.org/es/home.html
	UNICEF	Adventure game whose aim is to ensure the survival of the inhabitants of a village by accessing drinking water. The protagonists of a village try to get drinking water and build infrastructure for an adequate sanitation of wastewater.
	Languages: English, French and Spanish
Water Alert	http://www.enredate.org/eng/juegoseng/water_alert
	Swedish Government	Simulation game whose objective is to manage a city in a sustainable way in the face of the various economic, social and environmental challenges and conflicts. Create an attractive city for citizens and business by improving health, comfort, safety and quality of life for you and future generations in harmony with nature, balancing economical, social and environmental effects of your decisions. The goal is to create your sustainable city.
	Languages: English
Simbiocity	http://www.btslearning.com/app/eBS/symbiocity/index.asp
	UK Government	Simulation game based on political decision-making and citizen awareness around flood management. The intention of the game is to know how to adequately manage the territory of a city in front of the floods. The objective will be to avoid floods in the city through direct actions of water management and aquatic ecosystems.
	Languages: English
Floodsim	http://playgen.com/play/floodsim/
	Hydrographic Confederacy of the Guadiana River	Adventure game about the cycle of water, aquatic ecosystems and the Guadian river. The main characterthe will show the use of water in the home and in the city, with actions for its improvement. The objective of the game is to raise awareness among the population about good water management practices in the home and in the city.
	Languages: Spanish
Dive in the Guadiana	http://www.chguadiana.es/corps/chguadiana/url/swf/cdsumergete/index.html
	Water Education for Teachers Foundation	Game of questions about the water cycle, aquatic ecosystems, water footprint and good habits at home. The game tries to discover the role that water plays in our lives through different tests and topics such as: the water cycle, the use of water in the home, aquatic ecosystems, among others.
	Languages: English
Project Wet	http://www.discoverwater.org/
	International Red Cross Association	Adventure game about the importance of water in our quality of life and conservation of the planet. The main character with the help of a drop of water discovers the important role that water plays in people’s lives. It will also face episodes of drought and, therefore, make the public aware of the importance of efficient water management in the city.
	Languages: Spanish
Moving through the Water	http://www.cruzroja.es/juego_del_agua/
	Meteorology State Agency, Government of Spain-AEMET	Game of questions about the water cycle and efficient habits in its use. The intention of the game is to know more about the role that water plays on Earth through different tests.
	Languages: Spanish
Water Cycle	http://www.aemet.es/documentos_d/conocermas/recursos_educativos/juegos/n1/agua/agua_espanol.html

), which are detailed below: “Water alert” (Unicef); “Stop disaster” (United Nations); “Simbiocity” (Government of Sweden); “Floodsim” (Government of England); “Kingfisher” (Water Consortium of Guipúzcoa); “SAIH Ebro” (Hidrographyc Confedarcy of the Ebro river); “Plant it Green” (National Geography); “Darfur is dying” (International Crisis Group and Reebok Humanrights Foundation); “Fish game” (Cloud Institute); “Catchment” (ABC Catchment Australia SA); “Pipe dreams” (Government of Englad); “Riverbed” (Mary wharmby); “Citizen Science” (Game, Learning and Society GSL); “Fluvi” (Council of Zaragoza); “Climántica” (Regional Government of Galicia); “Where the River Meets the Sea” (National Oceanic and Atmospheric Administration); “Dive in the Guadiana” (Hidrographyc Confederacy of the Guadiana River); “Project Wet” (Water Education for Teacher Foundation); “Moving through the water” (Red Cross); Water cycle (Meteorology Statal Agency, Government of Spain, AEMET).

The sample of games reveals that most water games are located in a section of the website of the institution or author, specifically 65% of games versus 35%, that are located on an independent website, and none of them have access to the game through any mobile application. In relation to the language, 10 games are exclusively in English (50%), 7 of them in Spanish—along with other official languages like Galician (Climatic) and in Euskera (“Kingfisher”)—(35%) and 3 of them in several languages such as English, Spanish, Chinese, Arabic and French (15%). On the other hand, the study indicates a wide variety of types of producers ranging from national (20%), regional (10%) and provincial (10%) governments, as well as intergovernmental institutions, mainly the United Nations (Water Alert, Darfur is Dying, Stop Disaster); to educational institutions (10%), scientific (5%), university (10%), companies (10%), media (5%) and NGOs (5%). With regard to the typology of the game according to its nature, it was found that most of them correspond to games of simulation (35%), followed by games of questions (15%) and games of action and adventure (15%). There are also mini-games, which include skill games, questions and strategy (10%), platform games (Fluvi) and games of strategy (“Fish Game”). Finally, the majority communicative proposal is to provide general information on the subject (71%), followed by the objective of providing information on causes and consequences (60%), as well as favouring reflection, critical thinking and the development of ideas and solutions (50%), and lastly, encourage the change of attitudes and behaviours (35%).

3.4. Evaluation Instrument

In order to evaluate the sample of videogames about water and have a better understanding of their communicative and educational nature, we used the validated criteria identified by [85]. The study applies the Delphi method, a structured and interactive process to collect opinions that establish consensus based on the experiences and judgments of experts, to develop a set of indicators to analyse communicative and educative features of online games on climate change related issues (including water). A total of 13 experts were chosen for their theoretical and practical knowledge, motivation to participate in the study, and feasibility of contact; their areas of expertise are communication, education, games and climate change. The technique consisted of three consulting rounds: in the first round, an open question was sent to the experts to assess the relevance and usefulness of preliminary dimensions (categories of evaluation) and criteria (indicators of evaluation), identified through an extensive literature review and pre-selected for their potential to provide useful information from a narrative and ludological point of view, inspired by the “Social Discourse of Videogames Analysis Model” by Pérez-Latorre [86], which integrates both analysis perspectives. In this round, the five dimensions were identified: identification, narrative, contents, gameplay and educational. The second consulting round consisted of a questionnaire in which experts assessed in ordinal terms (high, medium, low) the relevance and usefulness of the evaluation criteria. The criteria that obtained a consensus of 90% (high + medium) were selected. In the third and last round, a final assessment of these criteria that did not pass the second phase took place, confirming whether or not they should remain outside of the final set of criteria (

Table 2. Dimensions and evaluation criteria [adapted from reference].

Evaluation Criteria of IDENTIFICATION DIMENSION
✓ Game title. ✓ URL: Link to the website; and availability of mobile app. ✓ Language/s.		✓ Type of creator: author behind the creation of the game and type of institution. ✓ Communicative purpose: communicative intentions and objectives of the game. ✓ Brief description: summary according to the genre, objectives and back story.
Evaluation Criteria of NARRATIVE DIMENSION	Evaluation Criteria of SCIENCES CONTENTS DIMENSION	Evaluation Criteria of GAMEPLAY DIMENSION	Evaluation Criteria of EDUCATIONAL DIMENSION
✓ Relevance of narrative: narrative elements can acquire importance or be irrelevant. ✓ Global storyline: the story in its entirety, the logical or causal succession of the events. ✓ Character depiction and role: characteristics and qualities of the character/avatar. ✓ Representation of the environment: the world in which the character/player develops. ✓ Dimension/space/scale: general context and scale of the scenarios ✓ Dimension/time: period in time that the story spans.	✓ Term used: terminology used to describe the phenomenon being studied. ✓ Existence of false concepts and misconceptions. ✓ Explicit use of scientific concepts: definition of climate change terms. ✓ Explicit use of information sources: the sources of information and data are cited. ✓ Convergence with other media or social networks: links to social networks are included. ✓ Message framework: themes, causes/consequences/actions, style and tone.	✓ Number of players and type of use: individual or multi-player. ✓ Player type: players’ profile depending on their interests. ✓ Degree of interactivity: user intervention in the content. ✓ Length of playing: time employed to play the game. ✓ Game mission. ✓ Game dynamics and mechanics: structure, rules and basic elements. ✓ Feedback system: message that the player receives in light of certain actions. ✓ Reward system: actions that incentivize and the rewards themselves. ✓ Availability of game instructions and possibility of saving the game.	✓ Competences: knowledge and attitudes that students can reach. ✓ Abilities: mental operations that students can reach. ✓ Problem resolution conditions: type of reasoning to solve problems. ✓ Need for previous knowledge ✓ Learning curve: level of learning difficulty. ✓ Possibility of group work ✓ Accessibility: availability of the game for students with functional diversity. ✓ Interdisciplinarity: combination of two or more academic disciplines. ✓ Availability of educational guidelines: document or link with educational information.

).

3.5. Peer Review Evaluation

The qualitative evaluation procedure that we have developed has had the evaluations carried out by three specialists with different training and professional trajectory (teacher, gameplay expert and game research expert). These three qualified evaluators are experts in scientific communication (journalist with experience and motivation in the use of video games), environmental education (researcher specialized in the topic and with extensive research experience), and secondary school teacher (with experience in the integration of serious games in the science classroom). These evaluators have independently assessed the different facets of quality of each of the games and agreed upon the dimensions of quality, focusing on the analysis of the scientific-educational content, the educational potential, the playfulness and entertainment value, the plot of stories and implicit narratives that give structure and serve as a thread to the story that recreates the video game. Independently, each evaluator has assigned scores and assessed the quality of the dimensions and criteria on a scale of three values (high, medium, low). When consensus has not been reached among the evaluators, a debate has been held on the extreme ratings and adopted a consensus assessment.

The 20 games evaluated have been classified according to their excellence based on the calculation of scores in quartiles. The procedure used in this final phase to evaluate videogames imitates, in a certain way, the heuristic used to differentiate and order scientific publications from their impact index by quartiles [87]; although our methodology incorporates other previous elements of more qualitative assessment focused on the evaluation of pairs that judge different quality dimensions of videogames.

In this work, we propose as dimensions to evaluate in pairs a series of elements for assessing the quality of the game in terms of different dimensions such as a ludic experience experienced by the player, or other types of complementary perspectives of its usability and its formative impact on the player himself or his pedagogical potential; at the time of being used as an educational resource to address a certain curriculum content. The three dimensions in which we focus our attention are the gameplay, the narrative potential and the intrinsic educational value.

We classify the games based on a scalar score given to a set of criteria that integrate each of the dimensions of playability, narratological and educational potential (

Table 3. Matrix with levels of specification for high, medium and low quality according to indicators and categories.

Level 1. Excellent Games of High Quality in Narrative, Gameplay and Education
Narrative	Gameplay	Education
Indicator Standard	Indicator Standard	Indicator Standard
✓ Nº of elements: includes 5 or more elements in which water fulfills a specific role or function (hydraulic, hydrological, cultural, social, ecological, etc.) ✓ Spatial level: detects 3 levels (micro, meso, macro) ✓ Grade of organization: detects various complex networks ✓ Grade of evolution: evolution over time with changes and uncertainties ✓ Language: uses a broad language with different looks ✓ Story: presents a story with emotional impact and inspiring characters ✓ Objective: favors the change of attitudes and behavior ✓ Topic: focused on socio-ecosystem elements ✓ Causes: human causes ✓ Actions: promotes changes in attitudes and behaviors, both individual and collective	✓ Player profile: 2 or more profiles ✓ Level of demand: the player is pushed to the limit of his abilities to reach the goal, having to pass several levels in the game ✓ Feedback: the player receives feedback immediately after making a decision in the game, either positive or negative ✓ Dynamics: use more than two dynamics (eg: decision making, object collection, memory retention, aiming, etc.) ✓ Rewards: rewards and praise are awarded when making a correct decision.	✓ Competences: more than 2 competences ✓ Skills: the player is able to evaluate, plan and produce, getting to create something new (evaluate and create) ✓ Learning: is based on participatory learning
Level 2. Good Games of Medium-High Quality in Narrative, Gameplay and Education
Narrative	Gameplay	Education
Indicator Standard	Indicator Standard	Indicator Standard
✓ Number of elements: includes 3 or 4 elements ✓ Spatial level: detects two levels ✓ Grade of organization: greater number of linear relationships and some complex relationships ✓ Grade of evolution: intermediate situation ✓ Language: language in transition ✓ Story: presents a story without emotional impact or inspiring characters ✓ Objective: a linear relationship of cause and consequences dazzles ✓ Topics: focuses on ecosystem elements ✓ Causes: human causas and natural causes ✓ Actions: changes in attitudes and behaviors only collective	✓ Player profile: 2 profiles ✓ Level of demand: the game does not demand enough effort and is limited to only one level ✓ Feedback: the player receives feedback only at the end of a game or mission ✓ Dynamics: uses two dynamics ✓ Rewards: praises is given but no rewards when making a correct decision	✓ Competences: 2 competences ✓ Skills: the player is able to apply learned knowledge, break it down into parts and think about how they relate to its global structure (apply and analyze) ✓ Learning: it is based on an interactive learning
Level 3. Games of Lower in Narrative, Gameplay and Educational Quality
Narratology	Gameplay	Education
Indicator Standard	Indicator Standard	Indicator Standard
✓ Number of elements: includes 1 or 2 elements ✓ Spatial level: detects a level ✓ Grade of organization: some linear cause-effect relationships ✓ Grade of evolution: static look, without changes in time ✓ Language: language riddled with topics, myths, misconceptions, using an alarmist and sensationalist tone ✓ Story: does not present history ✓ Objective: general knowledge ✓ Topic: focuses on hydraulic and hydrological elements ✓ Causes: does not present ✓ Actions: changes in attitudes and individual behaviors	✓ Player profile: only one profile ✓ Level of demand: the level of difficulty does not correspond to the cognitive level of the player, becoming too easy or too difficult ✓ Feedback: the player does not receive feedback after making a decision in the game ✓ Dynamics: a single dynamic ✓ Rewards: they are not granted	✓ Competences: promotes only one ✓ Skills: the player brings to the memory relevant information and is able to interpret meanings (remember and understand) ✓ Learning: theoretical learning.

). The sum of these scores allows the assignment of values to the different dimensions and in turn an integrated global score was obtained resulting from the sum of the different dimensions. This procedure will allow indexing the games according to each one of the considered dimensions and an integrated multilevel classification that agglutinates them in a global index.

The indexation and its classification in quartiles is an operative way of ranking games based on this global index and its different dimensions depending on the quartile in which they are located with respect to the score obtained by the rest of the sample elements. Those games evaluated in their different dimensions by independent evaluators are qualified according to their excellence and from these scores a range of quartiles is established. The excellent games located in the Q1 are those that differ from the others because they have been highly valued by experts with high scores. The lowest quality games are placed in lower ranges of medium or low quality as they exceed a score threshold established from the normalized scores.

3.6. Quality Indicators

The indicators have been constructed based on a series of categories that refer to three fundamental aspects such as: narrative, gameplay and education. The category model consists of a series of thematic items (TI) associated to a scoring system (SP) of 1 to 3 points, which allows a ranking of quality about videogames on water to be obtained. In relation to the narrative aspects, it has mainly been developed incorporating the paradigm of complexity in the construction of knowledge around water, always under the look of the new water culture and ecosystem services, understanding that these currents of thought make possible the change of paradigm towards a complexiation of the content in this case that concerns us, water. Regarding the gameplay aspect, the following items have been selected: the profile of the player, the level of exigency, the feedback, and the rewards. Finally, in relation to the educational aspect, the basic competences defined in the compulsory educational systems have been taken into account; as well as the abilities that appear in the taxonomy of Anderson et al. [22] and the digital adaptation of Churches [88]. They propose a pyramid in ascending order: to get to the stage of “create”, it must pass through “evaluate”, “analyse”, “apply”, “understand” and “remember”. Regarding the curricular integration of the games, according to the typology of teaching methodologies it is observed that the games that encourage active/interactive learning seem to get higher quality scores than those that have more traditional/passive methods. Finally, it also includes the type of learning that the game fosters (Table 3). The system of categories conformed by their indicators with a certain score would reflect different scenarios of quality of the games (low, medium, high), and would allow us to outline quality intervals in which a sample of games can be placed.

4. Results

Below are developed each of the aspects taken into account in order to characterize the sample of videogames on water, which have been arranged in a general ranking of three categories: excellent games (Q1), medium-quality games (Q2 and Q3) and low-quality games (Q4) (

Table 4. Global integrated ranking of video games about water according to quantitative evaluation of experts: game classification and position by quartile.

Excellent games of high quality in narrative, gameplay and education Position Q1 in the global integrated ranking
Simbiocity		Citizen Science		Climántica		Catchment		Stop Disaster
Good games of medium-high quality in narrative, gameplay and education Position Q2 in the global integrated ranking
Plant it Green	Where the River Meets the Sea		Moving under Water		Fishgam		Riverbed		Floodsim
Good games of medium-low quality in narrative, gameplay and education Position Q3 in the global integrated ranking
Darfur is Dying			Water Alert		Dive in the Guadiana		Project Wet
Low quality games in narrative, gameplay and education Position Q4 in the Global Integrated Ranking
Pipe Dreams		SAIH Ebro		Fluvi		Kingfisher		Water Cycle

The cut-off point to include a game as excellent in the integrated ranking is given by the corresponding normalization in each dimension and in the overall score.

). The excellence is identified with their position in the corresponding quartile according to the assessment.

We obtain the integrated ranking of games dimensions (narrative, gameplay and education), sorted from highest to lowest, placing each of the games in different quality intervals. If we observe the results we have:

(a)

Q1, green colour (excellent quality): games are placed in a low-quality scenario, whose score would range from 42 to 62 points;

(b)

Q2, yellow colour (medium–high quality) and Q3, orange colour (medium–low quality): games are placed in a scenario of medium quality that would go between 63 and 83 points;

(c)

Q4, red colour (low quality): games would be placed in a scenario of high quality, with a score above 84 points.

Table 5. Ranking of videogames by dimensios and integrate indicator.

Dimensions	Narrative Quartile (Scores)	Gameplay Quartile (Scores)	Education Quartile (Scores)	Integrate Ranking Quartile (Scores)
Games
1. Simbiocity	Q1(50) Excellent narrative quality	Q1(19) Excellent gameplay quality	Q1(30) Excellent educational quality	Q1(99) Excellent global quality
2. Citizen Science	Q1(50) Excellent narrative quality	Q1(18) Excellent gameplay quality	Q1(30) Excellent educational quality	Q1(98) Excellent global quality
3. Climantica	Q1(42) Excellent narrative quality	Q1(20) Excellent gameplay quality	Q1(30) Excellent educational quality	Q1(92) Excellent global quality
4. Catchment	Q1(43) Excellent narrative quality	Q1(18) Excellent gameplay quality	Q1(30) Excellent educational quality	Q1(91) Excellent global quality
5. Stop Disaster
	Q2(38) Medium–high narrative quality	Q3(16) Medium–low gameplay quality	Q1(30) Excellent educational quality	Q1(84) Excellent global quality
6. Plant it Green	Q2(37) Medium–high narrative quality	Q1(20) Excellent gameplay quality	Q2(26) Medium–high educational quality	Q2(83) Medium–high global quality
7. Where the River Meets the Sea	Q2(41) Medium–high narrative quality	Q3(16) Medium–low gameplay quality	Q3(16) Medium–low educational quality	Q2(73) Medium–high global quality
8. Moving through the Water	Q2(40) Medium–high narrative quality	Q2(17) Medium–high gameplay quality	Q3(16) Medium–low educational quality	Q2(73) Medium–high global quality
9. Fish Game	Q3(36) Medium–low narrative gameplay quality	Q4(11) Low gameplay quality	Q2(26) Medium–high educational quality	Q2(73) Medium–high global quality
10. Riverbed	Q1(42) Excellent narrative quality	Q4(10) Low gameplay quality	Q2(18) Medium–high educational quality	Q2(70) Medium–high global quality
11. Floodsim	Q3(34) Medium–low narrative gameplay quality	Q1(18) Excellent gameplay quality	Q2(18) Medium–high educational quality	Q2(70) Medium–high global quality
12. Darfur is Dying	Q3(36) Medium–low narrative gameplay quality	Q3(14) Medium–low gameplay quality	Q2(18) Medium–high educational quality	Q3(68) Medium–low global quality
13. Water Alert	Q3(34) Medium–low narrative gameplay quality	Q1(20) Excellent gameplay quality	Q3(14) Medium–low educational quality	Q3(68) Medium–low global quality
14. Dive in the Guadiana	Q2(38) Medium–high narrative quality	Q3(16) Medium–low gameplay quality	Q3(14) Medium–low educational quality	Q3(68) Medium–low global quality
15. Project Wet	Q3(34) Medium–low narrative gameplay quality	Q1(18) Excellent gameplay quality	Q4(12) Low educational quality	Q3(64) Medium–low global quality
16. Pipe Dreams	Q4(31) Low narrative quality	Q2(17) Medium–high gameplay quality	Q3(14) Medium–low educational quality	Q4(62) Low global quality
17. SAIH Ebro river	Q4(25) Low narrative quality	Q3(14) Medium–low gameplay quality	Q2(18) Medium–high educational quality	Q4(57) Low global quality
18. Fluvi	Q4(24) Low narrative quality	Q3(15) Medium–low gameplay quality	Q3(16) Medium–low educational quality	Q4(55) Low global quality
19. Kingfisher	Q4(28) Low narrative quality	Q4(13) Low gameplay quality	Q4(12) Low educational quality	Q4(53) Low global quality
20. The Water Cycle	Q4(24) Low narrative quality	Q4(13) Low gameplay quality	Q3(14) Medium–low educational quality	Q4(51) Low global quality

Narratology [Q₁(X ≥ 42), Q₂(X ≥ 37), Q₃ (X ≥ 32), Q₄ (X ≤ 31)]; Gameplay [Q₁(X ≥ 18), Q₂ (X ≥ 17), Q₃ (X ≥ 14), Q₄(X ≤ 13)]; Education [Q₁(X ≥ 29), Q₂(X ≥ 18), Q₃(X ≥ 14), Q₄(X ≤ 13)]; Global Ranking [Q₁(X ≥ 84), Q₂(X ≥ 70), Q₃(X ≥ 63), Q₄(X ≤ 62)].

presents a detailed ranking of the 20 games in terms of narrative, gameplay and education.

The first three columns represent the sorting by categories and the last the general ranking from the summation score of the ratings in games of high, medium or low quality. If we look at column 4 (global ranking) we can observe that 5 of the games (in green) are rated as excellent, since they are positioned in quartile 1 (punctuated with ratings above 84 points); in quartile 2, the next 6 games are placed (in yellow) with ratings between 70 and 83 points; below 63 points are the games of low average quality (quartile 3, orange color); finally, with less than 62 points, are the games of lower narrative, educational and educational quality (quartile 4, red color). If we consider the three dimensions of evaluation in Table 5 (columns 1, narrative; column 2, gameplay; column 3, education) and their equivalence, in order to be classified in green as Q1, we observe that the ranking remains stable and that the order is consistent in relation to those games considered excellent (games 1 to 5), which are positioned in Q1 homogeneously for the three dimensions evaluated. We can observe the exception of game 5, which in the gameplay dimension occupies position Q2 (yellow) because it is considered of high average narratological quality by the narratives (38 points), and position Q3 (orange) because it has a low–mid range assessment (16 points); maintaining its position Q1 in the educational dimension (30 points) and Q1 in the global ranking (84 points). The stability in the arrangement of games according to the excellence in the criteria depends on the dimensions in relation to its normalized position: from game 6 onwards alternation is observed in the rankings according to the position occupied in each of the dimensions. For example, game 10 of the ranking can be considered globally of medium-high quality, although from the gameplay point of view it is a game of low quality, it is excellent narrative and of medium-high quality from the educational point of view.

Videogames that are located in a high quality scenario, in terms of narrative, gameplay and education, refer to simulation games (“Simbiocity”, “Climatic”, “Catchment”), as well as an adventure game (“Citizen Science”). In this sense, these games are primarily intended to create, design and manage (whether a territory, a basin or a city) in a sustainable manner including different ecological, social and economic aspects over time, whose main tool is the decision making between different variables and models (energy, agriculture, urban planning, waste management, conservation of natural spaces, etc.). Also, as a high-quality game, we encounter “Citizen Science”, an adventure game focused on the fight against eutrophication of a lake, encompassing both ecological conservation measures and citizen awareness, being able to incorporate numerous and diverse elements from a narrative view, as well as different profiles (citizenship, environmentalists, scientists, users of the lake, among others) feedback and dynamics (understanding of the phenomenon, measurement and data collection, conversation with local agents, etc.) in relation to the gameplay. Finally, in relation to the educational aspect, these high-quality games promote different competences (interaction with the physical, social and civic environment, cultural and artistic, etc.), and different skills (understanding, analyzing, evaluating), favouring participatory learning.

A correlational analysis of the three dimensions evaluated (

Table 6. Pearson correlations by dimensions.

		Narrative	Gameplay	Education
Narrative	Pearson correlation	1	0.364	0.653 **
	Sig. (bilateral)		0.114	0.002
	N		20	20
Gameplay	Pearson correlation		1	0.301
	Sig. (bilateral)			0.197
	N			20
Education	Pearson correlation			1
	Sig. (bilateral)
	N

** (Level of significance α = 0.01).

) shows that there is a statistically significant relationship (r_xy = 0.653) between the narrative and educational categories, finding a slight correlation (r_xy = 0.301) between the gameplay and educational categories and very similar (slightly higher than r_xy = 0.364) between gameplay and narrative. This shows that although the three dimensions considered contribute to conform the integrated concept of game quality, the object of evaluation, the narrative and educational dimensions have more weight in terms of correlated dimensions that give meaning to the possible learning processes triggered by these games. This aspect acquires special relevance in the case of serious games in general, and in our research whose thematic content is water, which are not intended for the sole purpose of entertainment and fun.

Aware of the importance of the gameplay dimension and its different integrating features for young people, as the main recipients of these games, it seems appropriate to note the interest of this result as a scenario for future research: can serious games not be fun? It will be necessary to analyse, in greater detail to what extent this lack of correlation reflects the slogan of seriousness with which video games designed with more pedagogical intent than playful have been classified, commonly called “serious games”, is a relevant issue that deserves more attention. In any case, the gameplay dimension should not be subordinated to the exciting opportunity to transform learning processes into situations of amusing inquiry and construction and discovery of rigorous scientific knowledge. This justifies the fact that some scholars prefer the term games for training or learning [89], because they provide learners with an authentic learning experience where the entertainment and learning are seamlessly integrated [90,91], while others prefer to refer to these games as serious games [63,92].

The games located in a scenario of medium quality encompass an endless variety of typologies such as simulation games, adventure games and questions, with different objectives ranging from sustainable fisheries management, awareness of the importance of water and pollution as a problem, the fight against floods and other natural disasters, and the survival of refugees, among others. Depending on the game to be dealt with, some of them get a higher or lower score depending on the aspect being analyzed, although most are in the following medium range: narrative (32–41 points) being the maximum 50 points; gameplay (14–17 points) being the maximum 20 points; and didactic (14–28 points) being the maximum 30 points (

Table 7. Standard scores by dimensions.

Quartil	Standard Score Narrative	Standard Score Gameplay	Standard Score Education	Standard Score Global Integrated Ranking
Q1	41,75	18	29	83,75
Q2	36,50	16,5	18	70
Q3	31,75	14	14	62,5
Q4	X ≤ 31	X ≤ 13	X ≤ 13	X ≤ 62

).

For example, “Project Wet” gets 34 points in narrative, 12 points in didactic, being the lowest scores in that range, but it is excellent in gameplay (18 points), because it contains different dynamics and profiles. By contrast, it can be observed that “Darfur is Dying” obtains 36 points in narrative and 18 points in education, which means a good score in that rank, and obtains 14 points in gameplay, which means quite a poor score in the dynamics that offers the game, the feedback or the possible profiles to develop.

In relation to low-quality games, you will find games such as “Fluvi”, “Kingfisher”, “Pipe Dreams”, “SAIH Ebro” and “The Water Cycle”. These games are of different nature as platform game and quiz, and include only a few narrative elements of poor content, mainly generalist, as well as a few dynamics, mechanics profiles, and feedback between the game and the player. Finally, if we analyse the educational aspect, we find that it promotes few competences, skills and learning is mainly theoretical.

The resulting dendogram of the cluster hierarchical analysis, according to the quadratic averages estimation model, shows two large clusters that represent the two main levels of polarization of the quality of the videogames on water analysed: the first group groups the games classified as excellent and good against the second group that includes games of medium and low quality (Figure 3).

5. Discussion

5.1. Narrative Dimension

The results of the study in relation to the narrative aspect indicate the existence of a narrative of medium relevance, being the majority with 40%, followed by a high relevance (35%) and characterized by a low relevance (25%) of the sample. Some games with high relevance are: “Simbiocity”, “Water Alert”, “Where the River Meets the Sea”, “Citizen Science” and “Moving through the Water”. The latter uses a narrator who accompanies the main character through different settings (home, school, city), showing the importance of water for daily life and different economic sectors (industry, agriculture, tourism), as well as showing different problems and proposing different measures (ecological agriculture, purification of the waste water, good practices of the use of the water in the home). In contrast, there are games of low relevance such as: “The Kingfisher”, “Dive in the Guadiana”, “Pipe Dreams” and “Fluvi”, the latter the narration is almost non-existent because it is a platform game. On the other hand, the existence of a narrator appears in 55% of the gamers. In those games in which a narrator appears, we encounter the figure of a citizen (35%), followed by mayor (15%) and manager of the urban water cycle (15%) and other diverse figures such as farmer, fisherman, animal and scientific. In this study, the space and time where the game was played were also analysed: results show that in 65% of the games the setting was fictitious, compared to 35% in a real setting (Africa, Sweden, Spain, England, etc.). In this sense, the story ran on a local scale in 70% of games (city, home, town) compared to 20% on a local and global scale and 10% on a regional hydrographic scale). In relation to the time scale, the stories took place in the present (50%) and in several scalespast, present and future at once (50%). For example, this is the case of “Climatica”, which begins in 2015 and the game evolves until the year 2034. In this game as in others, connections between past, present and future are established in order to teach how to depend of the management model of a territory, problems can be solved or worsened.

5.2. Contents Dimension

The main generic contents of the games deal with water management (55%), aquatic ecosystems (40%), water cycle (35%) and ecosystem services (35%). In relation to the use of misconceptions, only the game “Moving through the Water” refers to water as scarce (we understand from the prism of the new water culture, that water is not scarce, but is given according to the climate in each territory); the use of scientific concepts appear in 60% of the games as in the case of “Citizen Science” with terms like eutrophication. On the other hand, only 35% of the games use information sources, such as UN reports, compared to 65% that do not indicate any source. More striking is that only one game, “Plant it Green”, converges with social networks. If we look at the more specific contents, the following are listed in order of importance: water management (50%), urban planning and biodiversity (35%), aquatic ecosystems and water consumption (30%), citizen participation and water as economic resource (25%), water and water cycle properties (20%) and natural disasters, river basin management and water as a human right (10%). Regarding the problems, the games show the following: water pollution (45%, examples: “Where the River Meets the Sea”), loss of biodiversity (35%, ex.: “Fish Game” and “Citizen Science”), loss of ecosystem services (25%, ex.: “Simbiocity”, “Catchment”, “Climantica”), diseases and threats to “Darfur is Dying” (20%), natural disasters, rising temperatures and desertification (15% examples: “Floodsim” and “Stop Disaster”), change in species behaviour (10%, ex.: “Project Wet”) and sea level rise, social and political crises (5%, “Water Alert”). In this sense, the study reveals that games, to a large extent, focus the origin of problems on human causes (75%), followed by both human and natural causes (15%), only natural causes (5%) and No cause (5%). In relation to action proposals developed by the games: 40% of the games do not show any example, 30% propose individual actions, 20% show individual and collective actions, and just 10% only show collective actions.

Some examples of individual actions would be those related to good practices in the use of water in the home (“Project Wet”), to recycle or to donate used clothes (“Plant it Green”); examples of collective actions would be: wastewater treatment, energy efficiency in the city, more sustainable transport models, re-vegetation of urban areas, among others (“Simbiocity”). Finally, the tone used in the games is characterized by being essentially informative (95%), proactive (55%), alarmist and ethical moral (30%), and also protest (15%).

5.3. Gameplay Dimension

The sample of games is characterized by a high degree of interactivity (which allows the player a great capacity for intervention) (40%), medium grade (35%) and low grade (25%). In relation to the dynamics of the game prevails progression, discovery, challenge, and skill; and with respect to the mechanics of the game, it is mainly the decision making in 15 of the 20 games, as well as the collection of resources, the pressure of time, and the retention of memory. In this sense, we can highlight games in which decisions are taken repeatedly in different aspects in order to sustainably manage a city or territory or solve an environmental problem through the analysis and evaluation of the information given in a particular setting and time. On the other hand, if the type of player is observed, the most popular profile is that of explorer (45%), followed by the creator (35%), and equal collaborator and competitor (10% each).

In relation to the feedback system, which evaluates the player’s actions and their impact, the study reveals that 60% of the games have a mixed response system, sending both positive and negative messages; and with respect to the rewards system, 60% of the games do not have any system that rewards the actions of the player compared to 40% that values the actions through different mechanisms like extra points or new resources. Finally, the number of players is one in all games, mainly for individual use (14), with a duration of less than 30 minutes (50%), followed by games lasting more than 30 mt (35%) and one hour (15%).

5.4. Educational Dimension

The development of games mainly favours educational competences such as: knowledge and interaction with the physical world (70%), social and citizenship (55%) and linguistics (50%), among others. In relation to the promoted skills they emphasize understanding, analysis and application. On the other hand, the level of difficulty of the games is characterized by being mainly middle level (45%, e.g., “Moving through the Water”), low level (35%, e.g., “Water cycle”) and high level (20%, e.g., “Simbiocity”). It is noteworthy that 14 of the 20 games promote interdisciplinary elements (economic, ecological, social, cultural, etc.); by contrast, only 4 games provide educational guidelines or an educational guide to help the teacher in the development of the objectives and contents of the game, such as: “Where the River Meets the Sea”. Finally, most of the games do not favour mechanisms to evaluate the learning acquired by the students, nor do they facilitate group work.

6. Conclusions

This paper provides a useful tool for teachers and serious game designers to carry out a quality assessment based on evidence from narrative, gameplay and education. The validation process by consensus and traits evaluated in each of the dimensions through a triple round cycle with a panel of experts provides methodological rigor to the decisions that a teacher can adopt on the resources to be used in learning scientific-ecological content about themes like water. The final normalization of scores carried out with this sample of games allows us to follow a procedure to classify the games separating the excellent ones from those of lowest quality according to the learning purposes pursued by the teacher when integrating this type of resource in their classes.

6.1. Evaluation of the Dimensions

Despite some authors acknowledge that the existing common game formats lack clarity as well as consistency and thus cannot serve as a solid reference to inform research on digital educational games which are increasingly used as learning tools, the study of the characteristics of educational videogames on water has allowed us to confirm that the format of simulation supports to a greater extent learning about water: it facilitates reflection, development of critical thoughts, and the contribution of solutions through the analysis of information and creativity in decision making. The format of simulation, understood here as those games simulating aspects of a real or fictional reality, has been found to be an appropriate tool for learning by other authors [89,93]. However, the literature on simulation for teaching emphasizes that “the effect of gaming exercises crucially depends on a subsequent debriefing, as processing of experience is necessary to provide insight” [93].

It is still necessary to make an effort in relation to how to favour the change of attitudes and behaviours through the virtual game. This is a great challenge, since the change of attitudes and behaviours is a gradual process that requires time and scenarios that allow taking actions in the daily sphere or in the social sphere. On the other hand, it seems that protagonist figures with responsibility in managing a city or territory could favour a deeper learning about water (from a more global and complex perspective), dealing with real and everyday situations in the subject of water. In the world of narrative construction, whether it is in books or in games, characters are seen as “key” to identification and immersion [94]. Psychologically, immersion can be explained as “a state where the primary driver to engage is intrinsic motivation; this state is extended where the player’s attention to stimuli is exclusive, and awareness of the other stimuli in the environment loses awareness” [95]. Intrinsic motivation is, therefore, a key element in engaging players to learn the contents of the game. Other aspects to be improved would be to incorporate further the use of scientific terms and official sources of information, and to place greater emphasis on solutions or possible measures to be carried out to face the challenges and problems related to water in the different thematic areas addressed by the game (city, territory, watershed, ecosystem).

We can also conclude that in relation to the gameplay, those games whose main dynamics are decision making and the profile of the explorer or creator involve more interactivity, allowing the player a greater incidence in the capacity of action and modification of the reality or the history of the game. Regarding the educational aspect of the games, it is important to emphasize the lack of educational orientations that favour the achievement of the objectives. It would also be interesting to advance in games that promote group work, since this method of work supports the exchange of knowledge between a diversity of students, promoting values of mutual support, solidarity and understanding. Lastly, the study shows the limited capacity of games to incorporate cultural and artistic competence, fundamental to foster the development of creativity, in order to face current and future challenges.

6.2. Quality Indicator

With the ranking of games, we have noticed that the format of the game (simulation, adventures, platforms or questions) does not determine the quality of the game, although generally speaking, simulations and adventure games are placed in a range of medium or high quality. In relation to the theme, it is not possible to clarify if a certain theme can obtain a higher score, although it could be concluded that games that pursue objectives related to design and management of a territory in a sustainable way are also located between medium- and high-quality scenarios. In this sense, it deserves special attention that those games that support participatory learning versus theoretical learning are those games that have obtained a better score, as we saw in the four games that obtained a high-quality ranking. In relation to narrative, it can be determined that those games placed in a high-quality scenario must include almost all the elements of the matrix identified as high scores, as occurs with the examples like Simbiocity or Citizen Science, which obtained the maximum score. Regarding the curricular integration of the games, according to the typology of teaching methodologies it is observed that the games that encourage active/interactive learning seem to gain higher quality scores than those that have more traditional/passive methods.