Supporting Disaster Resilience Spatial Thinking with Serious GeoGames: Project Lily Pad

Abstract

The need for improvement of societal disaster resilience and response efforts was evident after the destruction caused by the 2017 Atlantic hurricane season. We present a novel conceptual framework for improving disaster resilience through the combination of serious games, geographic information systems (GIS), spatial thinking, and disaster resilience. Our framework is implemented via Project Lily Pad, a serious geogame based on our conceptual framework, serious game case studies, interviews and real-life experiences from 2017 Hurricane Harvey survivors in Dickinson, TX, and an immersive hurricane-induced flooding scenario. The game teaches a four-fold set of skills relevant to spatial thinking and disaster resilience, including reading a map, navigating an environment, coding verbal instructions, and determining best practices in a disaster situation. Results of evaluation of the four skills via Project Lily Pad through a “think aloud” study conducted by both emergency management novices and professionals revealed that the game encouraged players to think spatially, can help build awareness for disaster response scenarios, and has potential for real-life use by emergency management professionals. It can be concluded from our results that the combination of serious games, geographic information systems (GIS), spatial thinking, and disaster resilience, as implemented via Project Lily Pad and our evaluation results, demonstrated the wide range of possibilities for using serious geogames to improve disaster resilience spatial thinking and potentially save lives when disasters occur.

1. Introduction

Recent hurricanes impacting the Caribbean and the United States’ Atlantic coastlines—such as Maria, Irma, Sandy, and Harvey—have proven the need for at-risk community disaster resilience. The challenge of disaster resilience will only be greater in the coming years—scholars of disaster policy and climatologists predict increases in storm frequency and subsequent costs in the near future [1]. Exponential growth in populations, increased population density along coastal areas, and rapid infrastructure development all exacerbate the impacts of already intensifying natural disasters from climate change [2,3]. Impacts from such disasters continue to reveal disaster preparedness gaps. These gaps could potentially be addressed through increased use of spatial methodologies available via geographic information systems (GIS), real-world simulations, and serious games [4,5]. In this paper, we investigate these problems via three specific research questions:

• Which spatial thinking abilities are relevant to disaster resilience?
• Which GIS tools and visual representations best connect disaster resilience with spatial thinking?
• What are the best practices for serious game design that incorporate spatial thinking, GIS tools, and visual representations for disaster resilience to achieve learning outcomes?

Our research questions are addressed through a combination of concepts and ideas from (1) serious games, (2) geographic information systems (GIS), (3) spatial thinking, and (4) disaster resilience as a means to improve disaster resilience practices amongst community members, policymakers, and disaster responders alike.

Our broader motivation for doing so is to empower communities directly impacted by disasters through cultivating an ability to utilize spatial thinking skills in a wide range of real-world contexts. The combination of spatial information with an immersive serious gaming experience provides a real-world, multistaged problem solving simulation necessary for making informed decisions [4].

To unite these four concepts, we developed a conceptual framework using a scoping literature review. A scoping literature review is a methodology that can used to define concepts and related research questions and can be used as a precursor to a systematic literature review [6,7]. The following list is a non-exhaustive outline of the sources used for the scoping literature review that was used to develop the conceptual framework discussed further in Section 2, Section 3 and Section 4, and graphically represented in Figure 1.

• Serious Games

  a. 

  Representative Sources: Academic Journals

  i. 

  Games and Culture

  ii. 

  GeoGames and Geoplay: Game-Based Approaches to the Analysis of Geo-Information

  iii. 

  Various Serious Games: Peer-Reviewed Conferences

  b. 

  Representative Sources: Government Research

  i. 

  US National Research Council

  c. 

  Representative Sources: “Gray literature”

  i. 

  Serious Games Case Studies

• Geographic Information Systems (GIS)

  a. 

  Representative Sources: Academic Journals

  i. 

  Journal of Geography

  ii. 

  GIS for Disaster Management

  iii. 

  International Journal of Geographic Information Science

  iv. 

  International Society for Photogrammetry and Remote Sensing (ISPRS) International Journal of Geo-Information

  b. 

  Representative Sources: Government Research

  i. 

  US National Research Council

  c. 

  Representative Sources: “Gray literature”

  i. 

  Harvey Disaster Responder Accounts

• Spatial Thinking

  a. 

  Representative Sources: Academic Journals

  i. 

  GeoJournal

  ii. 

  Journal of Geography

  iii. 

  Journal of Geography in Higher Education

  b. 

  Representative Sources: Government Research

  i. 

  US National Research Council

  c. 

  Representative Sources: “Gray literature”

  i. 

  Harvey Survivor Accounts

  ii. 

  News reports

• Disaster Resilience

  a. 

  Representative Sources: Academic Journals

  i. 

  Journal of Homeland Security and Emergency Management

  ii. 

  International Journal of Emergency Management

  iii. 

  Global Environmental Change-Human and Policy Dimensions

  b. 

  Representative Sources: Disaster Resilience Practitioner Literature

  i. 

  United Nations Office for Disaster Risk Reduction (UNDRR) Reports

  ii. 

  US Federal Emergency Management Agency (FEMA)

  c. 

  Representative Sources: Government Research

  i. 

  US National Research Council

  d. 

  Representative Sources: “Gray literature”

  i. 

  Harvey Survivor Accounts

  ii. 

  News Reports

Concepts were added to the conceptual framework, using the representative sources outlined above, in terms of their combined relevance to both the practical problems of disaster resilience we outline in this section and our research questions. Often, we found sources that overlapped between the four focused concepts, such as map-based game uses for disaster response [8,9].

We then addressed the three research questions through an implementation and evaluation of our conceptual framework in the form of an original serious geogame, called Project Lily Pad, which is based on empirical evidence gathered from 2017 Hurricane Harvey survivors and the emergency response focused on rescuing flood victims (Figure 2)

In the following sections we (a) situate each research question with the context of our conceptual framework presented in Figure 1 and (b) identify the relevant concepts from the framework that we integrated into the development of Project Lily Pad.

2. Question 1: Which Spatial Thinking Abilities are Relevant to Disaster Resilience?

The aforementioned anthropogenic influences necessitate a new perspective on action taken in disaster scenarios. Congress provided roughly $120 billion in emergency aid in response to Hurricane Katrina in 2005, nearly $60 billion for Hurricane Sandy in 2012 [1], and approximately $125 billion for Hurricane Harvey in 2017 [10]. The top-down funding for relief efforts is immense, but there can be a disconnect between the survivors and responders, either real or imagined, as those impacted feel they are not getting the recovery help they need. For example, in the disaster case study used for this research, Hurricane Harvey victims in Dickinson, Texas, expressed their frustrations with the Federal Emergency Management Agency’s (FEMA) efforts, stating that “some people that actually needed (help) didn’t get what they needed (and that) people that are in desperate need (are left) in a worse situation” (Anonymous, personal communication, 20 November 2017). Victims said that they received no “concrete information from FEMA”, and that it seemed as if “EMA did not have a plan”. These feelings of abandonment and mismanagement combined with the poor allocation of resources from the Federal government caused many victims to be skeptical of trickle-down aid. From the Harvey survivors we interviewed, it was evident that there was a clear disconnect, which again was either real or imagined, between the federal government and the communities that they served during Hurricane Harvey. With this problem context in mind, we next discuss the two areas our conceptual framework related to this problem domain and research question 1: (1) disaster resilience; and (2) spatial thinking, skills, and ability.

2.1. Disaster Resilience

Disaster resilience is the ability to adapt to and recover from hazards, disturbances, or strains placed on an individual, community, or organization without compromising long-term potential for development [11]. It is an integrated approach to disaster management. As outlined by Aldunce, Beilin, Howden, and Howden [12], experiential and social learning are key to this approach. Empowerment and education are essential for the continued well-being of communities, as the frequency and intensity of serious natural disasters exceeds the capacity of current preventative actions. Our research aims to examine how to support community-driven, bottom-up disaster resilience within at-risk communities through the lens of serious games.

Disaster resilience is also an important tool for building sustainable communities, ensuring economic growth in both pre- and post-disaster situations; disaster resilience is key to both the recovery and preparedness stages of the disaster management cycle [13]. Given the inherently spatial nature of disasters, spatial thinking is a key piece of disaster resilience. It enables people to understand the multiple geographic scales disasters operate on and enables victims to navigate a quickly changing and dangerous environment, observe and analyze patterns, and understand the new limitations of their surroundings. We argue that one critical method for developing community resilience is through the development of spatial thinking skills.

2.2. Spatial Thinking, Skills, and Ability

Figure 3 is a conceptual framework that outlines the relationships between spatial thinking, skills, and ability, which we then discuss in the remainder of this section.

Spatial thinking (top middle of Figure 3) is a set of skills used across disciplines to support an understanding of the world through processes of visualization, perception, and rationalization [14]. The National Research Council (NRC) identifies three dimensions of spatial thinking: cognition in space, cognition about space, and cognition with space. Cognition in space is centered around how we locate ourselves in space. The ability to navigate, follow directions, or put together a puzzle exemplifies cognition in space. Cognition about space is primarily about how we can understand how the world and its physical space is structured. Cognition with space is how we relate to things that are not necessarily spatial but can be spatialized, such as the words in a sentence that convey different meanings depending on how they are arranged [14]. Together, these dimensions enable people to solve spatial problems, such as navigating without a GPS or using a paper map.

Spatial thinking is improved through the development of spatial abilities (bottom middle of Figure 3) or the capacity of an individual to process their physical surroundings through a set of specific spatial skills [15]. Spatial abilities are also subdivided into three types: spatial visualization, spatial orientation, and spatial relations [16]. These abilities are related to the fourteen core geographic concepts identified and synthesized by Ahlqvist and Schlieder [17]. Examples of these abilities include orienteering, mental rotation and manipulation of spatial objects, and recognizing spatial relationships or patterns between objects [16].

Spatial abilities can be taught and reinforced throughout a person’s lifetime through personal experience or in targeted educational programs [18]. When synthesizing spatial literacy emphasis in school curricula, Bartoschek, Schwering, Li, Münzer, and Carlos [19] identified several common threads among desired spatial abilities and underlying skills. They found that these fell into one of two categories: orientation or map comprehension. Using their system of classification, the orientation domain includes things such as the ability to locate oneself on a map (cognition in space), while map comprehension involves understanding basic cartographic principles that make up spatial characteristics of an environment (cognition about space).

Additional research examining spatial thinking training exercises found that the developed spatial skill proficiencies were also transferable across disciplines [20]; furthermore, since these skills are developed as permanent qualities, they can be applied in a variety of real-world situations [21]. Research seeking to understand the relationship between spatial thinking and disaster resilience, for example, highlights the applicability of cognition in space and about space within the field of disaster management [22]. This further supports the importance of spatial skill training regimes to bolster spatial thinking, and by extension contributes to the motivation of this project research.

3. Question 2: Which GIS Tools and Visual Representations Best Connect Disaster Resilience with Spatial Thinking?

Geographic information systems (GIS) and related technologies are utilized in various educational settings to facilitate the development of spatial thinking skills, both at institutional and localized levels [23]. This is seen within the growing field of GIS careers for community betterment, and the utilization of GIS within planning and policy efforts. Relevant spatial information supplements these efforts by observing geographic relationships that go otherwise unnoticed; various economic, environmental, and societal patterns can overlap to put certain communities at higher levels of risk [24]. Geospatial data and GIS technologies are used to identify these patterns and inform effective disaster response regimens [25,26]. Some approaches to GIS, such as public participatory GIS (PPGIS) and volunteered geographic information (VGI), enable students or independent users to address real-world issues, such as “analyzing the patterns of an invasive weed species, mapping the patterns of crime in an urban school neighborhood, and mapping the dangers presented by abandoned landfills in the school neighborhood” [14]. This can be done by people within the programs gathering information from their community or town, while simultaneously breaking “down boundaries and barriers between (institutions) and local communities” to help educate residents on their surroundings [14]. These applications encourage experienced locals to engage in spatial reasoning for the betterment of their community by facilitating mutual agreements and building trust between institutions and community members [11,14,27].

Although GIS requires users to think spatially, GIS technologies are disputed as educational tools. GIS software tools are limited by accessibility and training, which inhibits untrained personnel from comfortably navigating the system and understanding the outputs [28,29]. In addition, GIS programs focus on the “later stages of the inquiry process”, as outlined by the National Research Council(NRC), including analysis, presentation, and communication. This prevents the user from engaging in the beginning and intermediate stages of inquiry thatinvolve exploration, identification, and decision–making skills necessary for disaster resilience [30], as cited in [14].

Moreover, GIS software functions as a linear sequence of decisions; multiple variables cannot be assessed, nor can multiple actions be performed simultaneously. Disasters operate on numerous levels and adequate management requires a comprehensive understanding of these constantly changing elements [26].

Thus, based on this background, we emphasized the need for easy-to-use GIS tools not based on professional GIS software and realistic environmental visual representations [10] that community members or non-GIS professionals can understand. Additionally, GIS tools and visual representations demonstrate a clear need to promote an immersive, interactive user interface that “supports multistage inquiries” to connect disaster resilience with spatial thinking [14]. With this motivating perspective of traditional GIS in mind, we looked to the growing field of serious geogames to provide a unique learning experience to connect GIS tools and visual representations with disaster resilience spatial thinking skills [4].

4. Question 3: What are the Best Practices for Serious Game Design that Incorporate Spatial Thinking, GIS Tools, and Visual Representations for Disaster Resilience to Achieve Learning Outcomes?

Serious games are primarily used as educational support tools. Concepts taught via a serious game are abstracted through gamification, where concepts are combined with game elements to create immersive scenes for experiential learning—hence a relevant connection with GIS for our research. Competition, goals, rules, choices, challenges, and fantasy are the foundation of game development [31]. Furthermore, guidance and feedback—provided through “reflective wizard” interfaces—allow the user to assess their progress and reflect on their decisions, the outcomes, and their progress [14]. Together, these game elements dictate player motivation, engagement, learning, and ultimately, their ability to transfer in-game skills to real-life applications. Building on this, work by Boyle, Connolly, and Hainey [32] also suggests that serious games are useful in facilitating “learning that is active, experiential, situated, problem-based, and provides immediate feedback”.

Ashfield et al. [33] further argue for the use of these games as training tools, highlighting their usefulness in making the user comfortable with scenario-specific problem solving, as well as contributing to associative learning. Uttal et al. [20] make a compelling arguments for the use of video games as spatial literacy aids. Their research found that spatial thinking is tied to working memory (e.g., remembering landmarks, navigating by memory), which is actively exercised through video game use. One explicit iteration of this is the genre of geogames.

Serious geogames are an emerging subset of serious games that exhibit a spatial focus, while still maintaining educational purposes [5]. Bartoschek et al. [19] argues that many geogames build on orientation and map comprehension abilities, aligning with contemporary educational goals that encourage spatial literacy. Critical knowledge gaps remain in identifying methods for building, teaching, and learning how to combine skills pertinent to disaster resilience within communities who need these skills the most. By combining GIS with education tools for disaster resilience, we can attempt to fill this knowledge gap.

4.1. Serious Game Case Studies

In addition to looking at larger trends across serious games, we chose to dive deeper into specific serious game examples using the aforementioned spatial and gameplay elements as guidance. Examining the strengths and limitations of select past projects helped inform development of the Project Lily Pad game discussed later. Discussion of these relevant serious games will further illuminate our design process and provide some rationalization for our game development choices, as well as contribute to the Project Lily Pad framework.

A few key questions guided our inquiry when examining contemporary serious games:

• Which spatial thinking or disaster resilience skills does this game teach?
• How does this game create a sense of place (or situational awareness)?
• Which attributes of this game contribute to game-based learning?
• How does the player receive feedback?
• Which strengths and limitations of this game are relevant to our own goals?

4.1.1. YouPlaceIt!

YouPlaceIt! is a digital serious game set in Dharavi, a low-income neighborhood in Mumbai that contains around three million people [34]. The game is an urban planning scenario with a focus on consensus building and negotiation among stakeholders in urban development. The central objective involves the construction of a new road. The players’ use critical thinking and GIS tools GIS to analyze and overcome both topographic and economic obstacles.

A main focus of YouPlaceIt! is the idea of community. It was very important to developers that the scenario be specific to and representative of actual problems faced by citizens of Dharavi. Portraying the neighborhood in a visually authentic way was key. To accomplish that goal, the developers decided to use satellite imagery of Dharavi to create the game environment.

The gameplay is primarily centered around planning and negotiation. The game depends heavily on collaboration among players to resolve issues related to road planning and budgeting. In terms of actual mechanics, there is a chat function to allow players to communicate, as well as a map interface with familiar GIS features, such as the buffer tool, which players can use to work with the environment.

The game does an excellent job of representing the setting as not a mere backdrop but as an actual location. These general ideas were influential in the development of our own Project Lily Pad game, as we wanted to give players a strong sense of place.

4.1.2. OriGami

OriGami is a mobile game that was designed to help school-age children cultivate spatial thinking skills [19]. Specifically, OriGami was created to teach concepts such as georeferencing, symbology basics, and orienting. The game’s creators also placed significant emphasis on making the technology itself more accessible to novice users. The gameplay is not overly complicated, and essentially involves moving from waypoint to waypoint based on given directions.

In terms of feedback, an emoticon is displayed in the top left corner during gameplay, which changes color and expression based upon the player’s performance. Hints and tips are also displayed on the map as necessary, so the player receives feedback the entire time. As a training tool, one thing that OriGami succeeds in is providing immediate and consistent feedback on a player’s performance, which is critical in facilitating game-based learning [32].

More than anything else, route-finding is something we were deeply interested in incorporating into Project Lily Pad, so it was interesting to see just how OriGami explored this ability and how players were encouraged to approach these kinds of tasks. In this regard, Bartoschek et al. [19] took a look at allocentric, egocentric, and landmark-based methods of navigation, and the differences among conditions in the amount of time it took participants to complete a route and number of errors they made while doing so. Given the importance of aligning, orienting, and visualizing one’s own position in relation to points of interest to navigate in a disaster scenario, the inclusion of landmark-based navigation was something that we thought might be incorporated into Project Lily Pad in terms of spatial thinking skills and ability.

4.1.3. Hazagora

Hazagora is a serious analog game that focuses on geohazards and disaster risk reduction [35]. The game is cooperative and meant to be played by 5–10 players, who take on one of several roles. Each character occupies a different zone on the game board and has different objectives. Gameplay takes place over the course of several simulated “years” in-game, allowing players to prepare for and experience several different disasters.

Hazagora hits upon the complex nature of disaster mitigation and response, as well as explores how some populations are uniquely vulnerable to certain threats due to their locality. Players are given the opportunity to practice different risk reduction strategies by taking different protective actions and can see which tactics are effective and which are not. There is certainly a spatial aspect to the placement of risk reduction measures and land development, with some locations on the map more susceptible to hazards than others. Successful campaigns are ones in which players advance development thoughtfully and with consideration given to the spatial nature of disasters.

Each disaster is prefaced by a video explaining the circumstances and specifics of that kind of disaster. A game master is also responsible for leading discussions on relevant topics throughout the game and making sure that the information the game seeks to convey is fully communicated to players; that is, the impact of disasters, with an emphasis on four key factors, namely hazards, spatial exposure, vulnerability, and disaster risk reduction (DRR) capacities [35]. Additionally, players are given handouts at the end of the game that provide supplemental information about the disasters experienced in the game and different risk reduction strategies.

The concepts from Hazagora most relevant to Project Lily Pad is the unification of spatial thinking and disaster resilience concepts in a serious game context. Seeing how aspects of disaster resilience can be successfully gamified in great detail was especially informative.

4.1.4. Code Red: Triage

Code Red: Triage is a digital serious game that was designed to assess and train triage competency [36]. Broadly speaking, it is an exercise in disaster response, intended for the training of emergency response personnel. The player takes on the role of a “crisis management officer” who must locate and properly triage individuals injured after a terrorist attack in a subway station. At the level of skills acquisition, the player is meant to not only gain a greater comprehension of triage protocols, but to also become better at translating general knowledge to procedural knowledge. In other words, a player should learn how to effectively mobilize factual information in order to perform a particular task.

First-person perspective games such as Code Red: Triage oftentimes have an edge when it comes to immersiveness. This, in turn, can contribute to greater learning potential, as players who feel more immersed in a game often experience greater motivation and engagement, which are in turn associated with higher order learning [31]. While developers initially feared that players who had never experienced a first-person style game before would struggle with Code Red: Triage, accessibility in this regard was not an issue. New players had little trouble understanding and operating the game controls, and navigation was easy enough. The game’s strengths are its clear-cut goal, defined rules and expectations, and provision of direct and consistent feedback. After every triage attempt, the player receives feedback in the form of a score. Players are awarded more points if they follow the proper protocols exactly and fewer points if they take too long to assess a victim. In this way, feedback is immediate and constant throughout the game.

Unlike the other two games discussed that deal with disaster response, Code Red: Triage is strictly single player and focuses on the experience of one individual. While it is certainly true that disaster response is a “team” effort, this allows for greater concentration on facilitators of learning besides cooperation—namely immersiveness, direct feedback, and active and experiential qualities.

4.1.5. Case Study Summary

In summary, each of these case studies represents at least one aspect relevant to the development of Project Lily Pad. Hazagora depicts environmental hazards and how these hazards disproportionately impact vulnerable populations. Code Red: Triage emphasizes the immediacy of disaster response and the utility of first-person perspective in creating an immersive experience. YouPlaceIt! exemplifies a sense of place and identifies several important considerations for authenticity of representation. OriGami demonstrates route finding and landmark-based navigation in a clear and concise way.

While these examples were all useful, we did not identify a game that incorporated all of the relevant concepts cohesively. Drawing on past investigation to bridge these gaps we identified in the literature, we propose a novel and comprehensive methodology for designing a serious geogame to teach spatial thinking concepts for disaster resilience. Based on this, we developed a serious geogame that can satisfy the need for a holistic approach to disaster resilience.

5. Methods and Materials: Project Lily Pad, a Serious GeoGame

Project Lily Pad is intended as an educational support tool for users to develop foundational spatial thinking skills through integrated, applied problem solving. Project Lily Pad encourages spatial thinking through the navigation of a three-dimensional game environment. The game is set in Dickinson, Texas, in the aftermath of Hurricane Harvey. Figure 4a–h graphically outline the games’ missions.

5.1. Applying the Framework

The conceptual framework discussed in Section 1 was used to develop Project Lily Pad. As previously discussed, when developing the conceptual framework, we found that there were often overlaps between each of the four areas that were the basis for the conceptual framework. Thus, the framework was applied to the design of Project Lily Pad through concept combinations best suited to address our research questions and the practical disaster resilience problem domain discussed in the beginning of Section 1. To illustrate these points further, the following are three specific ways we applied the framework. First, for ideas from serious games, spatial thinking and GIS were used to (a) highlight certain game aspects that would create an immersive user experience, as well as (b) the importance of sense of place; abstractions of reality; and the incorporation of challenges, fantasy, and rules [31]. Second, and related to the first point, ideas from serious games and GIS provided insight into the effectiveness of serious games for educational purposes, focused on the abstraction of reality and the creation of a sense of place [37,38,39]. The former allows the player to empathize with the emotionally charged realities of the scenario, while the latter allows the player to experience the geography and culture of the specific location. Together, these elements create an immersive user experience. Finally, disaster resilience and GIS ideas were the technical backbone for our implementation of the framework in Project Lily Pad with actual street data and building footprints, with the goal of realistically modeling Dickinson, Texas. In addition, we incorporated post-Harvey survivor interviews from experiences (as seen in Figure 2) collected during field research in Dickinson into our game’s script and missions. By setting our game in a real disaster scenario, we were able to focus on the hurricane’s unique qualities. This allowed us to establish a deeper sense-of-place than traditional, location non-specific geogames [40].

Four Spatial Abilities

Beyond the disaster scenario, Project Lily Pad: A Serious GeoGame aims to establish a set of core abilities in the user. In seeking to answer our first research question, we selected a set of spatial skills that are useful in navigating a disaster situation. As established by Tomaszewski, Schwartz, and Szarzynski [41], serious geogames have a demonstrated capacity to teach such skills. We specifically leveraged this research, as it demonstrated how using a specific, real-life disaster response scenario in a GIS-based serious game tied to specific map-based spatial thinking tasks within the game, such as buffer, overlay, choosing spatial representations, and distance measurement, can be used to teach spatial thinking skills [41]. For Project Lily Pad, the set of spatial thinking skills were selected based on an analysis of spatial abilities suggested and highlighted in school curricula, and subsequently their applications in disaster resilience [12,42]. As identified by Bednarz and Bednarz [18] in their examination of spatial thinking applied during Hurricane Rita in Houston, such applications include reading maps, using geospatial data to create mental maps, and locating oneself in space. Noting the need for such skills in disaster situations, the following list of spatial abilities were chosen for incorporation into the Project Lily Pad gaming experience and are graphically summarized in Figure 5.

A. 

Reading and interpreting a map;

1. 

Understanding how the two-dimensional environment of a map translates to the three-dimensional environment of a city;

B. 

Navigating an environment;

1. 

Visualizing the street network of the environment;

2. 

Using distinct landmarks to orient oneself in space;

3. 

Finding detours around obstacles and rerouting on the fly;

C. 

Translating verbal instructions into physical navigation;

D. 

Good practices in a disaster;

1. 

Preparing for impending disasters by distributing supplies;

2. 

Determining why some places are better than others in terms of seeking refuge - determining good lily pads;

3. 

Understanding the wide-impact a disaster can have on a particular environment by comparing maps.

The four skills listed in Figure 5 are developed in the user through the different game stages. The first part of Project Lily Pad, for example, is intended to acquaint the player with their surroundings, introduce them to the process of map reading (as shown in Figure 4b), and help them learn how to translate verbal directions into actions. Competency in these skills is tested in the second part of the game. The player is asked to travel through flooded sections of the city without a clear street network (as shown in Figure 4e–f). They must rely on landmarks, street signs, and their mental and physical maps established in stage one to navigate. The lack of an in-game tracking beacon also challenges the visualization and orientation competency of the user. The reference and shaded digital elevation model (DEM) maps, in turn, encourage players to rationalize the spatial information presented to them and gain an understanding of to how the topography and environment can influence a disaster situation (as shown in Figure 4g). The user’s grasp of these concepts is evident through their ability to identify good lily pads based on the extrapolated information (as shown in Figure 4f).

6. Results: Evaluating Project Lily Pad

6.1. Student Evaluations

We were primarily interested in measuring changes in (1) players’ opinions of the importance of spatial thinking skills and (2) their confidence in their own spatial abilities when evaluating the game. The two changes were assessed via a pre- and post-game experience questionnaire, a common technique used in game evaluation [43]. The study group was comprised of 10 students, of which 50% was male and 50% was female, who were graduate or undergraduate students. This sample size that has been used in past GIS disaster management task assessments [41,44]. The pre- and post-game experience questionnaire allowed participants to self-rate specific spatial thinking skills using a five-point Likert scale of 1–5 (1 = strongly disagree; 2 = disagree; 3 = neutral; 4 = agree; 5 = strongly agree) via four questions:

Q1: I am confident in my ability to navigate an unfamiliar area.

Q2: I am confident in my ability to navigate without my phone or using a paper map.

Q3: I am confident in my ability to translate verbal directions into visual directions.

Q4: I am confident in my ability to find an alternate route or to reroute when a path is obstructed.

Figure 6 graphically represents the summary results of the pre- and post-game questionnaire related to these four questions, which we discuss next along with other results.

As seen in Figure 6, results from the preliminary questionnaire generally indicated a lack of confidence in navigations skills in terms of responses ranging from strongly disagree to neutral, as particularly seen is responses to question 2 (I am confident in my ability to navigate without my phone or using a paper map) and question 4 (I am confident in my ability to find an alternate route or to reroute when a path is obstructed). In addition, 90% of the study group self-identified that they had no previous experience with spatial thinking skills before participating in the study.

Qualitative assessment of the game was done through play testing and “think aloud” method, a technique that has been used in past serious game evaluation research [26]. Players were debriefed about the project goals and given an overview of the gameplay in terms of the Dickinson disaster response scenario during Hurricane Harvey. Then, they were asked to play through the game and verbalize their thoughts about what they were experiencing. A facilitator was present to answer any questions and encourage users to reflect on their reasoning as they carried out the missions. The game play and think aloud dialogs were screen-recorded for follow-up analysis.

Qualitative assessment of the recorded game play and spatial think aloud data indicated that the game helped players become aware of their spatial awareness skills, but that the level of spatial thinking challenges actually needed to increase. The qualitative assessment also revealed insight into accessibility issues, game mechanics, and the extent to which players used spatial thinking. For example, one player commented that the lack of beacons made it difficult to determine their location within Dickinson.

Despite these challenges, however, most of the users were able to orient themselves within the game environment using the provided map, as seen in Figure 4b. One player completed all of the missions on the second day by cutting through the fields, using the map to keep track of how many roads they had travelled. They expressed that the timer drove them to take shortcuts with the boat in order to reach people in time, but felt that there should be more obstacles to really engage the player with the gameplay. Conversely, another player claimed to have “no sense of direction” and that “our culture” is not really concerned with teaching people navigation skills. They had trouble navigating the scenario, and took longer than other players to complete the missions.

Various play testers expressed their appreciation of the landmarks, a common item used for navigation [45,46]. They noted that the big, recognizable buildings offered them a sense of direction. Some of them used the high school and the highway as points of reference, looking for them whenever they got lost. Other strategies included using the stickers as placeholders for the character’s location and destination to help guide them in the right direction, as seen in Figure 4b. They also expressed their concerns with navigating in the boat through areas of higher elevation to reach lily pads.

Finally, we gained insight into how the players viewed their ability to think spatially from the results of the post-questionnaire. The post-questionnaire asked the same questions as the preliminary questionnaire, as well as three additional questions about gameplay and three questions about the game’s relationship with disaster resilience. As seen in Figure 6, player confidence in their ability to navigate an unfamiliar area after the test increased (question 1) from 60% indicating that they agree or strongly agree that they are confident to 70%. Additionally, player confidence in their ability to navigate without their phone or using a paper map (question 2) increased from just 20% indicating that they agree or strongly agree that they are confident to 50%. The feedback from question 2 was useful because players indicated that they felt the weakest in this area of spatial thinking on the pre-questionnaire. Player confidence in their ability to translate verbal directions into visual directions (question 3) showed an interesting result of a slight decrease from 55% indicating that they agree or strong agree that they are confident to 40%, although one respondent indicated a strong increase in confidence after conducting the evaluation. Question 4 indicated that confidence increased in a player’s ability to find an alternate route or reroute when a path is obstructed (question 4), with 60% indicating that they agree or strongly agree that they are confident after playing Project Lily Pad. The increase in self-reported spatial ability confidence, as seen in changes to responses to questions 1, 2, and 4, reflect results from previous research on how providing visual, interactive, and geographic environments (games or otherwise) can improve spatial reasoning and decision making generally, but also for disaster management tasks specifically [23,41]. The slight dual decrease and increase in confidence in translating verbal directions into visual direction (question 3) could be explained by issues with design of the verbal directions themselves or the general challenge of navigation derived from spatial cognition of environments based on external representations, such as language, maps, and text [47,48].

The additional post-game experience questions also indicated that 50% of players thought that the game made them think more about how they navigated their environment. Furthermore, 90% of the players indicated the game made them think more about what they would do in a disaster and 70% of the game players indicated the game made them think more about how disaster response actually works in practice, results that generally reflect the established benefits serious games have shown for increasing disaster management domain knowledge [9,49]. Additionally, as discussed in Section 2, Harvey survivors we interviewed felt skeptical and abandoned by official response agencies. The fact that the Project Lily Pad game gave players a deeper understanding of disaster response could indicate how Project Lily Pad can potentially be used as a learning device to empower local communities to make learning about disaster response accessible and more realistic, and how they can use these training experiences if disasters were to occur again in their communities.

6.2. Practicioner Evaluations

The Project Lily Pad game was also evaluated by the emergency management professionals from the Galveston County Office of Emergency Management, who were actual 2017 Hurricane Harvey responders. These evaluations were more broadly focused on the utility of Project Lily Pad for emergency management practice, as serious games have a well-established record of being used for emergency management practitioner training [9]. Two emergency management professionals spent one hour each playing Project Lily Pad and provided feedback using the think aloud method (Figure 7).

Although the small sample size was small, the ecological validity of Project Lily Pad being reviewed by the actual people who were the motivation for the game’s creation shed valuable insight into the utility of the game for emergency management practice. Specific game design feedback included introducing more realism based on actual disaster response experiences, for example making it more difficult to navigate the environment in terms of less reference point such as street signs; and better game interface controls and visibility for non-gamers, such as emergency management professionals, who would use the game.

The emergency management professionals also indicated that the game would be valuable to their practice in terms of having a game as a device to teach spatial thinking and awareness of a given environment for people that are new to a given disaster response area. For example, during Hurricane Harvey, several National Guard units were deployed to the Galveston County Emergency Operations center before being deployed into the community. Many of the National Guard members did not have any previous spatial knowledge of Galveston County or Dickinson in terms of the neighborhoods, streets, and overall spatial layout of the emergency response area. A game such as Project Lily Pad could have been used to teach National Guard and others about the area that they were going to operate in as they were waiting to be deployed from the emergency operations center.

7. Future Work

Based on both the student and practitioner feedback, improvements can be made by increasing the difficulty of the game and implementing more realistic scenarios. For example, players could receive less locational information with less time to complete tasks. Future research could focus on creating a game environment that parallels the real world regarding both the built environment and the disaster itself. Project Lily Pad presents an important stepping stone that could become a useful training tool and educational device if developed in the future.

In terms of evaluation and assessment, although it was a relatively small sample size, we believe the results discussed in Section 6.1 provide promising implications for the future of Project Lily Pad and subsequent works. It is suggested that the game succeeds at getting players to apply spatial thinking to disaster scenarios. The exact extent to which it does this, however, cannot be extrapolated from the sample size used in the research reported here. Future evaluations should be carried out to measure development in spatial ability, whether this is maintained over time, and how spatial ability changes in terms of game performance. For this latter point, game performance could be measured based on the time in it takes a player to complete a game scenario. These evaluations should also encompass a more varied sample (overall larger sample size; diverse in age, gender, education) for validity’s sake, as well as to determine the playability of the game among different populations.

The accessibility of the final game product will also be an important topic for future research. While various GIS software tools have different levels of complexity in terms of use and learnability, serious games, such as the examples given previously, generally have an intuitive user interface and can be utilized in full capacity by untrained individuals. They also require less complex hardware and often have mobile phone or web applications. However, Project Lily Pad is currently only accessible through download on computers. Related to these points, natural disasters disproportionately affect those with less resources [50]. Under-resourced people, such as those we worked with in Dickinson, have the most need for disaster resilience spatial thinking tools due to a variety of underlying social vulnerability factors, such as income and education levels [51]. Increasing the accessibility of Project Lily Pad and other serious games outside of computer play can help provide greater assistance to these vulnerable populations. We hope to have Project Lily Pad running on mobile platforms in the future, and we highly encourage other disaster resilience serious games to consider access and audience when contemplating the impact and dissemination of the game. In time, we hope to develop other platforms for Project Lily Pad, and we strongly recommend that game format decisions are made with public use at the forefront.

Project Lily Pad was also developed as a single-player game. Disaster preparedness, response, and resilience ultimately depend upon strong communication and coordination between various actors in the scenario. Related to this point, there are aspects of the game that are not as reflective of the emergency management experience as we wished, as reflected by feedback from our practitioner evaluations discussed previously. The Project Lily Pad game focuses on the first responders roles during the preparedness and recovery efforts of Hurricane Harvey, however by focusing more on the role of an emergency manager our team hopes to add an additional part of the game that takes place in an emergency operations center (EOC). We have begun to address these issues via a new game we call “Project Emergency Operations Center (EOC)”, which (a) examines communication between different players with different roles in emergency response and (b) allows the player to experience the game as an emergency manager and educate them on how to make quick decisions in respect to disaster management (Figure 8).

The player would have the opportunity to learn how to handle disaster emergencies and face the consequences of making the right or wrong decision. Project EOC is still in an early prototype stage and it is beyond the scope of this paper to discuss Project EOC further; please see the website for more information on Project EOC. (The Project Lily Pad game is available online at https://www.rit.edu/gccis/geoinfosciencecenter/nsf-reu-outreach).

Project EOC also points to how the overall approach of GIS-based serious games for disaster resilience can allow for development of different types of game scenarios. In the present paper, we presented a hurricane disaster response scenario (Project Lily Pad) based on our approach, followed by new preliminary work using our approach to a disaster planning scenario (Project EOC). Endless varieties of scenarios within the disaster management cycle of response, recovery, mitigation, and planning could follow given the fundamental spatial nature of disasters and specific disaster management scenarios [52]. For representative examples of developing different game scenarios based on the disaster management scenarios, see [53,54].

Furthermore, although GIS technologies inspired characteristics of our game—such as the hand-held map, placing landmarks, and toggling layers—we did not integrate GIS analytical functions such as buffers and overlays directly into the game. This approached allowed users to rationalize different kinds of spatial information without the training required to use a GIS program. Where GIS technologies differ in complexity, serious games generally have an intuitive user interface and can be utilized at full capacity by untrained individuals [16]. Future work that offers GIS analytical features in the gaming experience could enable a wider range of spatial thinking skill development, but will also need to balance game-like entertainment with creative, stimulating experiences.

Project Lily Pad did, however, use GIS to develop a sense of place using real-life data for Dickinson, Texas. The data acquisition process, as expected, was challenging, requiring GIS skills that are not widely available among game developers, which could be a potential barrier for large-scale replication of our design in different geographic regions. To bolster the transferability of the spatial thinking skills, we used CityEngine to develop a sense of place. Zoning and OpenStreetMap data enabled accurate modelling of the city, but obtaining the data was very labor intensive. Much of the open source data publicly available are collaboratively contributed and often incomplete. This must be considered for future projects, as lower-resource areas are likely to have less publicly available spatial data. Within our game, road networks, some selected landmarks, and topography are generally accurate but building modelling does not necessarily capture the location. To overcome some of these technical challenges, our team developed work flows to acquire OpenStreetMap data for use in Unity software. We hope that this step will encourage subsequent game designers to utilize actual geographic data in their play environment. Furthermore, development of APIs that would enable direct input of open datasets directly into the game environment could help overcome the aforementioned technical challenges.

8. Discussion and Conclusions

Project Lily Pad and the theoretical framework it is based on is a first attempt to integrate spatial thinking, disaster resilience, and GIS within serious games. We make a preliminary identification of which spatial thinking skills are relevant to disaster resilience, how these can be aided by GIS, and whether serious games are effective as training tools to foster spatial thinking. Evaluation of the theoretical framework via Project Lily Pad, as discussed in Section 6, suggests that users are being challenged to use spatial thinking, but the extent to which these skills are maintained over time or contribute to disaster resilience is yet to be determined. The evaluation of Project Lily Pad by emergency response professionals who were actual responders to Hurricane Harvey, as discussed in Section 6.2, indicates the utility of Project Lily Pad for spatial awareness training during emergency response situations. Based on these results, our conceptual framework and its in-game application offer a foundation for more work of this kind, as discussed previously.

Ultimately, we must recognize the changing nature of disaster response and the important role that geographic and situational awareness plays within it. Analysis of the failures of the response for Hurricane Katrina, for example, demonstrated that while accurate predictions of risk were available, there were serious problems with communication of these to the public, which resulted in a lack of preparedness [55]. These gaps in communication could be addressed through serious games. Building a common foundation for spatial problem solving could be a first step towards improving this communication. This would encourage resilience in citizens’ daily lives, as well as motivate people to get involved in other GIS-based projects.

The response to Hurricane Harvey set a precedent, in that response officials called on the aid of civilian groups with boats to help with the evacuation of people and communities that could not be reached by emergency responders in a timely manner [56]. This process necessitated maps, navigation, and an ability to cross-analyze different kinds of spatial data to assess the best ways to reach people and get them to safety; these are the skills the Project Lily Pad game focuses on. Even if the user does not end up commandeering a boat to rescue neighboring communities, a general grasp of these skills might be the deciding factor in spatial decision making regarding whether a citizen chooses to evacuate or not, which route to take, or how to best anticipate risk and interpret the hazard data coming from official sources.

It can be concluded that the combination of serious games, geographic information systems (GIS), spatial thinking, and disaster resilience implemented via Project Lily Pad, along with our evaluation of these results, demonstrated the wide range of possibilities in using serious geogames to improve disaster resilience spatial thinking, and ultimately to save lives when disasters occur.