Transforming a Theoretical Framework to Design Cards: LEAGUE Ideation Toolkit for Game-Based Learning Design

Abstract

Educational game design is a complex process demanding multi-dimensional focus in a heterogeneous team to balance multiple aspects. The existing Game-based learning (GBL) frameworks detail the required knowledge but are hard to use in design practice. Conversely, card-based design tools are a lightweight approach used to assist the early design phase. While several game design cards exist, none is specific for informing GBL knowledge. There is a lack of operationalizable approaches for designing learning games that integrate research based GBL knowledge into the actual ideation process. This paper presents a card-based GBL ideation toolkit to reduce the complexity of framework application and introduction of key GBL concepts in the design process as a tangible reference point to facilitate multi-dimensional focus, supporting idea generation, critical reflection, and creation of a shared understanding in the collaborative design process. The paper describes a ten-step process of transforming the LEAGUE framework into the LEAGUE toolkit (GBL ideation cards), an evaluation of the toolkit with design workshop participants, and design lessons detailing strengths and limitations to support GBL design practices.

1. Introduction

Game-based learning (GBL) offers a rich design space encompassing multiple dimensions (such as learning, game factors, usability, affective reactions, user, and environment), which are interrelated, and GBL design teams need to consider these as they influence the design of learning games [1,2,3,4,5]. The interdisciplinary teams for designing educational games consists of experts in different domains (e.g., game design, learning domain, technology, and human factors), and it is essential to involve them in the design process to explore the design from different perspectives to create appealing and successful solutions [6,7].

Several GBL frameworks structure the design concepts of learning games and can be used to justify and reason the design decisions. However, most of the GBL frameworks are not used in learning game design practice because it is hard to apply these theoretical principles [4,8,9,10,11], and they lack tool-support and guidance for the practical application [8,10,12,13]. This points to the lack of an operationalizable approach that is more accessible and lightweight to integrate theoretical GBL design knowledge into actual ideation and hands-on practice of learning game design to improve the collaborative process of designing a learning game and guide the GBL design team [4,6,11,12,14,15,16]. To bridge the gap between theory and practice, Hornecker [9,15] proposed transforming a framework into a design tool by converting the theoretical concepts into easy questions that designers can relate to, which can be introduced in the design process.

Researchers have found many useful characteristics of cards: they provide a common vocabulary to designers for use in design, enable the transfer of knowledge from academia to design practice, help kick off the design discussion, assist in different viewpoints aiding multi-dimensional focus, help refine ideas, structure design discussions to guide the design process, facilitate shared understanding and communication, and offer a playful approach to involve in design tasks [4,17,18]. These characteristics can address GBL design practice challenges, making them a viable and affordable tool (instead of other approaches such as design patterns or guidelines) to provide the intermediate-level knowledge [4,5]. They offer an approachable way to introduce information as part of the collaborative design process, and their abstraction level has enabled researchers to successfully use them in a wide variety of fields [4,17,19,20]. However, the existing game design cards are not specific for GBL knowledge and cannot be used to ideate learning game design to reinforce GBL key concepts and facilitate the required multi-dimensional focus. We found only two card-based tools for ideation and design of learning games: Tango cards [4] (for tangible learning games) and Exertion cards [15] (for exergames). However, both these tools cannot be generalized to other design situations because of their specific nature and focus on these particular areas. One limitation of existing card-based design tools is that researchers have not always articulated the design knowledge embedded in their cards [4]. Therefore, in order to make the GBL design knowledge easily accessible in the early design process (addressing the gap between theory and practice), we transformed the game-based learning framework (LEAGUE framework [2]) into a card-based GBL design tool (LEAGUE ideation toolkit) to examine if this approach is useful and valuable for educational game design practice. The LEAGUE framework [2] is selected because it was developed to support game-based learning’s multi-dimensional nature. It emphasizes the key GBL components (with detailed hierarchy) and their interrelations, informing the design knowledge for learning games.

An educational game is different from an entertainment game as it requires a multi-dimensional focus (involving different aspects) in a collaborative design process with different stakeholders working together [1,2,6,14,16]. Therefore, retaining and balancing the different aspects of GBL is challenging for team members, as they often have limited knowledge beyond their expertise and, consequently, not the same interpretation of the overall design space [17,21]. Therefore, our LEAGUE ideation cards’ objective was to make theoretical knowledge about designing learning games easily accessible to team members involved in the GBL design process by informing them during their work and providing inspiration. Thus, the LEAGUE toolkit’s target audience would mostly be the GBL design teams, including researchers, students, and practitioners in the industry from the GBL community. We intended for a GBL specific yet a generic tool that can be used to design educational games for diverse learning domains and game genres, supporting GBL designers in the initial ideation phase. The developed toolkit consists of four types of card decks (primary cards, trigger cards, reflection cards, and custom cards), scaffolding GBL design, and five design activities (idea generation, idea development, idea refinement, idea illustration, and idea documentation), scaffolding collaborative ideation process, carried out in a workshop technique for learning game design ideation. To investigate the toolkit’s effectiveness in supporting the GBL design ideation process, the toolkit was employed and evaluated in three design workshop sessions with 34 participants. For this study, we focused on participants’ experience (perceived usefulness, understandability, level of fun, and satisfaction) using the toolkit and not examining produced design artifacts and team dynamics. This paper contributes by describing a 10-step process for transforming a framework into an ideation toolkit, providing tool support for applying theoretical GBL framework knowledge in design practice (support for GBL design team), and discussing design lessons by highlighting the strengths and limitations of the developed toolkit that can serve as guidelines for other researchers intending to do a similar task.

2. Background

This section presents challenges in the GBL design practice identified by relevant research studies, introduces the LEAGUE framework (used for transformation into card-based GBL toolkit), highlights the general characteristics of cards found effective for the design practice and the use of card-based tools in different domains that inspired the use of cards as an operationalizable approach for our work. Further, it underlines the limitations of existing design cards for GBL design practice.

2.1. Challenges in the GBL Design Practice

GBL is a complex multi-dimensional phenomenon, and several key factors influence the design of learning games such as learning, game factors, usability, affective reactions, user, and environment [2,3]. Designing learning games is complicated because it includes embedding learning content into gameplay, selecting game features that motivate the learner to repeat learning cycles within the game context, considering user characteristics to ensure proper usability, context requirements, and technical conditions for selecting appropriate technology [2,17,22,23]. Many researchers have explored the essential elements of educational games, including theoretical concepts and design knowledge of GBL. However, different researchers focus on different GBL design and evaluation elements producing a scattered picture [1]. The research work by Tahir and Wang [1] identified a lack of a holistic view and identification of the core dimensions of GBL. They addressed this problem by introducing the LEAGUE framework [2]. Although several frameworks can provide techniques to structure the design concepts of learning games and justify their design decisions, there is still a lack of research on improving the process of designing a learning game [6,14,16]. Most design frameworks/models are not used in the learning game industry because they lack tool-support and guidance for practical applications [8,10,12,13]. Furthermore, learning games require to be developed in interdisciplinary teams involving experts in different areas (technology, game design, pedagogy, and usability). These stakeholders sometimes have limited knowledge (of other areas) beyond their expertise and often do not have the same interpretation of the design space [17]. However, it is crucial to involve a variety of stakeholders in the design process [6]. Therefore, there is a need to translate the intricate multidisciplinary theoretical knowledge of GBL into some easily accessible design practice that can support the hands-on practice and guide the designers to develop effective learning games in a playful collaborative manner [4].

2.2. The LEAGUE Framework

The LEAGUE framework [2] provides a holistic view of GBL design outlining the core components in a hierarchy by defining four conceptual levels (dimensions, factors, sub-factors, and metrics) for comprehensive GBL design. The dimension level is at the highest abstraction and metric the lowest. The framework focuses on six dimensions, and each has factors (22 in total) and sub-factors (total 74). The GBL dimensions in the framework are related to each other in terms of cause and effect.

Table 1. LEAGUE framework components.

Conceptual Level	Elements
Dimensions	Learning, Environment, Affective-Cognitive Reactions (ACR), Game Factors, Usability, and User
Factors	Learning objectives, learning strategy, learning content, learning outcome, technical aspects, context, enjoyment, engagement, motivation, flow, game definition, game narrative, game mechanics, game resources, game aesthetics, gameplay, interface, learnability, satisfaction; learner profile, cognitive needs, and psychological needs
Interrelations	Learning (integrate) Game Factors; Game Factors (generate) ACR; Usability (address/cater) User; Usability (address/cater) Environment; Environment (map) Use; User (influence) Learning & Game factors, ACR; User (influence) Learning, Game factors, ACR

presents the dimensions, factors, and interrelations (between dimensions).

Such detailed frameworks are complex to use in design practice, as reported by other researchers [10,12,15,24,25]. Additionally, there is uncertainty applying such frameworks for educational game design in practice [8,9,10,11,14]. Currently, the LEAGUE framework [2] does not provide any step-by-step guidance for a GBL design process. Therefore, a more accessible and lightweight approach than the existing framework is needed to effectively transfer knowledge between theory and design practice [4], and tool support is needed.

2.3. Cards as Design Tools: Useful Characteristics for Design Practice

Design researchers have found many important characteristics of cards that make them effective for design practice [4,26]. According to Lucero et al. [19], cards are great for collaborative design owing to their general characteristics (i.e., triggers of combinatorial creativity, tangible idea containers, and collaboration enablers). Cards facilitate the design process by functioning as conversation-starters, orienting devices, and pacemakers by structuring the creative sessions [15,17]. Cards facilitate the ideation process [15,26] as they are used to bookmark ideas [4], rate, or evaluate those ideas, thus enabling critical reflection [26]. Furthermore, they make design practice more playful and engaging by providing a source of inspiration through provocative questions or triggers and preventing the discussion from becoming unproductive [26]. Researchers in different areas [4,18,26,27] have created card-based design tools to make domain concepts and knowledge easily accessible to designers in their design practice as they provide a tangible representation of abstract concepts that are easy to use. Design cards can support focus shifts, as evident from the work of [15,26], which is essential for multidisciplinary domains like GBL that require multi-dimensional focus. Cards act as physical props during design discussions that help articulate thoughts and make arguments tangible, thus aiding shared understanding and communication [4,15,26].

These cards’ characteristics provide a low-tech and approachable way to communicate the LEAGUE framework categories in learning game design practice, leading us to develop a card-based toolkit to make GBL design concepts easily accessible to the design team, facilitate collaboration, guide the ideation process and stimulate design ideas.

2.4. Card-Based Tools in Various Domains

Card-based tools have previously been used by researchers to put together knowledge from diverse areas (such as tangible interfaces, IoT, playfulness, and eco information) into an easily accessible form to stimulate design thinking and aid in design practice [7,24,26,28]. Many researchers focused on game design or gamification. Relevant examples include the Verbs, Nouns, and Adjectives (VNA) cards [29], card-based toolset for gamification design [18], three brainstorming games for game designers [30], ideation cards for mixed-reality game design [17], Playful Experiences framework (PLEX) cards [25], and a deck of lenses [31]. We found only two design card toolkits that focused on educational games, but these are limited to the specific domain’s knowledge. Deng et al. [4] developed “Tango cards” for designing tangible learning games. Tango cards summarize the design knowledge in two areas: “tangibles” and “games”. The focus is more on tangible and games rather than the GBL approach itself. Mueller et al. [15] developed a design tool specific to sports (exertion games), focusing perspectives on the body, and designing exertion experiences. Therefore, these cards do not inform complete GBL design knowledge.

For this study, we developed a card-based tool for GBL design to facilitate the ideation of learning game design in practice. The development of our toolkit focused on customizable and context-specific design patterns [32]. Our LEAGUE toolkit shares some core aspects with the previously discussed design cards. They communicate domain knowledge using provocative questions and tasks similar to [4,15,26,31], utilizing the characteristic of cards as tangible containers; use different card decks, each serving a specific purpose similar to [17,24,26,28]; we also structure the ideation process and organize participation using playful design activities similar to [17,26,33]; provide inspiration using examples (as triggers) from ad hoc external sources as means of supplementing and developing design concepts similar to [26,28]; facilitate critical thinking using reflection criteria similar to [17,26] and, supporting multi-dimensional focus using categories similar to [7]. However, our toolkit differs from existing card-based tools by extending the ideation process to include illustration and documentation (creating a small version Game design document) to support shared understanding and in-depth discussion and a log sheet along with ideation sheets for recording and tracking design decisions, thus supporting awareness and traceability on the design process [34]. The next section will focus on the toolkit development.

3. Toolkit Development Process: Turning the Framework into Ideation Cards

This section presents the process of turning the framework concepts into ideation cards. As discussed earlier, such transformation is valuable for bridging the gap between theory and practice [9,15] and several researchers followed this approach, e.g., PLEX cards based on the PLEX framework [25], Eco information individualization design toolkit based on the conceptual framework of Eco Information Individualization [24], exertion cards based on exertion framework [15]. However, except Mueller [15], none of the others explicitly detail the steps of transforming the framework into a design tool, and not many well-defined processes exist. The five-stage process described by Mueller [15] is not validated beyond their work. Therefore, we took their work as a starting point and further adapted and extended the process based on our experience to validate and extend the prior work that might be useful for other researchers for converting framework into design cards.

Our extended process of transforming the framework into ideation cards consisted of the following ten steps:

• Define goals/objectives: The following objectives were defined for the toolkit: (1) Summarize and communicate GBL design knowledge (LEAGUE framework [2] categories): making GBL concepts easily accessible to learning game designers in practice; (2) Support collaborative design process: fostering multidisciplinary focus shift by focusing on different dimensions; (3) Inspire designers: supporting the initial generation of ideas (brainstorming) by providing triggers to facilitate the creative thinking; (4) Support in-depth reflection of ideas: providing criteria to enable critical thinking and a trade-off between different aspects; and (5) Structure and guide the ideation process: orienting the ideation process from start to end with structured design activities.
• Establish target boundaries: We decided to aim for a relatively large number of cards (ultimately 176) to provide a comprehensive tool but targeted to keep the main cards (GBL concepts) to a limited number (28 in total) in order to minimize the chances of designers feeling overwhelmed.
• Scrutinize framework to extract concepts: The LEAGUE framework [2] provides the GBL design space. As described by [35], the design space is the set of decisions and choices that need to be made about the designed product, and it captures the essential elements that the design product must-have. We looked at the components of the LEAGUE framework and picked 6 key dimensions, 22 factors, and relations (see Table 1) for converting to ideation cards, as these components can fully communicate the GBL concepts required by designers to make design decisions in the learning game design process without overwhelming them with detailed sub-factors and metrics.
• Decide the type of cards: The extracted dimensions, factors, and relations were translated into a set of ideation cards. The main traits we wanted in LEAGUE cards are (i) informative and collaborative: to define and inform GBL design concepts and support multi-dimensional focus, (ii) inspirational: to support brainstorming, (iii) reflective: to support the refinement of ideas, and (iv) customizable: to facilitate the creative thinking. Therefore, we decided on four different decks of cards (primary, trigger, reflection, and custom) to focus on a particular task. In addition to the four card types, primary and trigger cards also belong to a sub-type. The two sub-types are dimensions and factors.
• Formulate the content: For primary cards, we focused on extracted dimensions and factors (see Table 1) from the LEAGUE framework [2]. The goal here was to translate the framework components into directive yet colloquial questions/tasks. For trigger cards, the goal was to provide some example answers/ideas to exemplify the possible design choices to stimulate brainstorming. Triggers were collected from ad-hoc external sources, existing educational games, and GBL literature [2,36]. For reflection cards, we focused on extracted interrelations (see Table 1) from the LEAGUE framework and translated them into critical thinking questions. The goal was to emphasize the trade-offs that need to be negotiated. Custom cards were blank cards to leave room for custom choices and support creativity.

  Table 2. Translation of framework concepts into primary cards questions.

  Framework Elements	Conceptual Level	Definition of GBL Element	Primary Card ID	Translated Primary Card Task/Question	Related Trigger Cards
  Learning (domain)	Dimension	The learning area(s) focus in an educational game to promote and facilitate learning.	DL	Decide the learning domain for the game.	Math; Climate change; Smart city; Dance
  Learning strategy	Factor	Pedagogical theories or approaches used to achieve learning objectives.	FL2	What strategy should be used to enable learning through the game?	Drill and Practice; Organize; Compare/contrast; Judge

  and

  Table 3. Translation of framework interrelations into reflection cards questions.

  Interrelated Dimensions in Framework	Identified Relation	Translated Question for Reflection Cards
  Learning & Game Factors	Integration/Balance	Are game elements (game objectives, narrative, etc.) and learning elements (learning objectives, strategy, content, etc.) well integrated into this game?
  Game Factors & ACR	Generate	Are selected game elements (narrative, mechanics, play, etc.) effective in generating user reactions (engagement, enjoyment, etc.) in this game?

  illustrate the translation of framework concepts into primary and reflection cards questions.

6.

Reduce items: The translation of framework dimensions and factors resulted in 28 primary cards (one question for each GBL element), and the translation of framework relations resulted in 7 reflection cards (focusing on questions that could challenge designers to reflect); thus, 35 question cards (primary and reflection cards) in total. To reach our target boundary, we limited the number of triggers (possible choices/examples) for each GBL element. This resulted in 113 examples called trigger cards.

7.

Define rules/process: The LEAGUE toolkit uses structured design activities to guide the ideation process (one of the defined objectives). We defined five design activities. Each design activity had a required output and used a different set of cards and ideation sheets. We also imposed time limitations for each activity to make participants active and prevent them from being unproductive.

8.

Visualize: All cards have a standard “playing card” size approximately 2.5 × 3.5 inches (64 × 89 mm). All cards are color-coded by deck (type) and category (six dimensions) to be distinct. Figure 1 and Figure 2 shows an example of developed cards. Each of the six categories has a different color (taken from the LEAGUE framework [2]). For Trigger cards, the categories are defined by the border color of each card. All cards have a consistent graphical layout and information architecture. We made sure to keep the card design minimal and easy to follow, not overcrowded with too much text, and balance text and images [4]. The text on trigger cards (presenting example answers or triggers) is limited to only a few words, as they are intended for inspirational use and should only provide a hint and not a concrete design [4,26]. The card’s backside consists of four elements (see Figure 2): type of card deck, card title, an image icon to visualize card type, and a short description of the role of the card or the definition of the GBL-concept. The card’s frontside consists of five main elements: a unique ID, card name, the sub-type, the main question/concept/ idea, and graphics (icon or image) illustrating the question/concept/idea. However, the custom cards are blank. We also developed a board with a playbook to make the process easy to understand and structured.

9.

Gather feedback: The feedback from the co-author was incorporated iteratively at each stage of the process of developing the LEAGUE toolkit. After the completion, the toolkit was discussed in detail with fellow researchers to verify that cards were understood without much explanation. They mainly provided feedback on improving the wording and presentation of cards and playboard. Afterward, the toolkit was employed in three design workshop sessions to explore the toolkit’s potential through feedback from participants and inspect the workshop session, design outcomes, and team dynamics. We used a questionnaire, focus group, observation, and video recording for the data collection.

10.

Refine and improve: The toolkit was iteratively refined with feedback from fellow researchers and design workshops. In the first iteration, definitions and questions on the cards were rephrased for clarity, preciseness in meaning, and their presentation based on fellow researchers’ feedback. In the second iteration, in addition to these changes, the design activities were adjusted and re-organized by changing the allocated time and rearranging debriefing sessions based on feedback from the first workshop session. In the third iteration, we plan to improve the cards’ searchability using accessories and precisely define the criteria for reflection cards to facilitate critical thinking based on collective results from three design workshop sessions. The toolkit is not ultimate and will still be improved based on future studies.

4. The LEAGUE Ideation Toolkit: Developed Card-based Tool for GBL Design

This section presents the developed LEAGUE ideation Toolkit. The toolkit consists of four decks of cards, five design activities (each with an idea sheet collectively called ideation sheets), a board with a playbook, and a log sheet. They are all used together in a workshop format to ideate learning game design (see Figure 1). The LEAGUE toolkit uses ideation cards with structured design activities to make theoretical and conceptual knowledge of GBL design (from LEAGUE framework) accessible to the GBL design team and guide the ideation process.

The toolkit is intended to facilitate and guide the collaborative ideation of learning game design and thus is designed to be played in a group (with a recommended group size of four to six players). The target audience of the LEAGUE toolkit is both academia and industry. Currently, the toolkit has only been used with researchers/students.

4.1. LEAGUE Cards

This section describes the LEAGUE cards and the rationale behind the different decks of cards. The toolkit consists of four distinct card types (Decks): Primary, Trigger, Custom, and Reflection cards. Figure 2 shows the different card types. The complete box of LEAGUE cards includes 28 primary cards, 113 trigger cards, 28 custom cards, and 7 reflection cards (176 cards in total). However, the main question cards are only 35 (primary and reflection). Each deck has a specific purpose in the overall ideation and design process and is introduced in a design activity to serve the goal set for that specific activity.

The primary and trigger cards are grouped into six main categories (based on six dimensions in LEAGUE framework): Learning, Game, Reaction, Usability, User, and Environment. Each of the two decks (Primary and Trigger) consists of two sub-decks (sub-types): dimensions and factors (see Table 1), used in different design activities (explained in Section 4.2).

The four card types are described as follows:

• Primary Cards (Present GBL design concepts): The Primary cards are the main deck of cards that are the building blocks for GBL design. Each primary card presents one particular GBL concept. The card poses a question, or a task related to that concept, which should be discussed in a team to develop a design idea (using either custom, trigger, or any combination of these cards). The team successively answers these tasks/questions to gradually build the learning game design idea through collaborative team discussion. There are 28 primary cards in total posing 28 different tasks/questions, out of which six are primary-dimension cards (focused on framework dimensions), and twenty-two primary-factor cards (focused on framework factors).
• Trigger Cards (Support for brainstorming): Trigger cards are examples of possible design ideas or hints for primary cards’ tasks or questions. These cards trigger the thinking process by giving a direction to think. Each primary card has multiple trigger cards (with the same name as the primary cards). For example, for the primary-dimension card “reaction”, there are three different trigger-dimension cards (emotional, behavioral, and cognitive) with the same name “reaction”, as shown in Figure 3. There are 113 trigger cards in total, out of which twenty-two are trigger-dimension cards (for primary-dimension cards), and ninety-one are trigger-factor cards (for primary-factor cards). We do not claim that the trigger cards are absolute and complete. However, we believe that they cover a range of different domains and areas of the GBL design space, which are enough to trigger the brainstorming and ideation.
• Custom Cards (Allow out-of-the-box thinking): This deck consists of blank cards used by the participants to write their creative design ideas. This provides an opportunity for out-of-the-box thinking and provides room for the creative impulses of participants.
• Reflection Cards (Aid refinement of generated ideas): Reflection cards present seven evaluation criteria to reflect on the generated ideas and design choices to refine them. Each reflection card contains a question pointing to a critical relation between different GBL dimensions that can negatively impact learning games’ effectiveness if not considered. It encourages the team to critically think about the trade-off and look for design iterations if problems exist.

Depending on the deck they belong to, the cards are either informative, inspirational, or reflective. Primary cards are informative, presenting GBL design elements and used as building blocks for the learning game design. The Trigger cards have an inspirational role and help trigger brainstorming by providing many provocative ideas as creative triggers. The Reflection cards provide a critical lens to validate or improve the developed design ideas by reflecting on them.

4.2. Design Activities

The LEAGUE toolkit play procedure is divided into five ideation design activities (see Figure 4). The five design activities are (1) Idea generation, (2) Idea development, (3) Idea refinement, (4) Idea illustration, and (5) Idea documentation.

Each design activity has a different goal and uses different toolkit items (cards and/or idea sheets). Each of the five design activities involves a separate idea sheet(s), which the team uses to produce the intended design outcome for that activity. A log sheet is used for logging the order of use of cards in the first three activities. The idea sheet has the same name as the design activity, e.g., the idea sheet for idea generation activity is called idea generation sheet. Collectively idea sheets of all five activities will be referred as “ideation sheets”. The idea sheet(s) for the first three design activities provides a layout for the placement of cards used as design ideas in that activity. They provide a space for annotating how the cards have been used to support discussion and also record the team’s decisions. The idea sheet for the fourth activity is a blank sheet to draw and visualize the design. Finally, the idea sheet for the fifth design activity provides a template for documenting the learning game design. Each used idea sheet is the design artifact/outcome of that specific design activity and provides the visual display of generated ideas, which helps the team summarize each design activity’s outcome. They are also crucial for data collection and recording not only the ideas but the complete ideation process.

The five design activities are as follows:

• Idea generation (Coming up with initial ideas): This activity aims to generate an initial concept for a learning game design. For this activity, the team uses sub-deck dimensions (see Section 4.1) and has six primary-dimension cards (6 dimensions), to solve using 22 trigger-dimension cards and 6 custom cards. Solving different primary cards (using trigger or custom cards) gradually generates an initial game idea. There is no right or wrong order of using the cards. Participants can shuffle through cards and pick one. The id of used primary cards is logged in the log sheet (in the order of use). The idea generation sheet is used to stick the trigger and custom cards to compose the initial idea.
• Idea development (Expanding the idea): The goal here is to expand and further develop the initial ideas from the first activity into more detailed and concrete ones. For this activity, the team uses sub-deck factors (see Section 4.1) and has 22 primary-factor cards (22 factors), to solve using 91 trigger-factor cards and 22 custom cards. The team can select and use the cards in any order. The idea development sheet is used to stick the trigger and custom cards to develop the design idea. The id of used primary cards is recorded in the log sheet in the order of use.
• Idea refinement (Reflecting on the idea): The goal is to improve or refine the developed ideas by reflecting on the design choices made using the reflection cards to identify the limitations and uncover questionable decisions. A team has seven reflection cards for this activity, and similar to the first two activities, they can shuffle through the cards and select in any order. The idea refinement sheet is used to add or replace the trigger and custom cards used to refine the developed idea. The idea refinement sheet has two sections for the placement of trigger/custom cards: one for rejected/replaced cards and one for new/added cards. In this activity, the team can use the ideation sheets from activity one and two to get an overview of design choices and stimulate reflection on what needs improvement. Both used and unused trigger and custom cards from the previous two activities can improve the design choices by discarding previously used cards and adding new adds. A log sheet is used for logging the order of the use of reflection cards.
• Idea illustration (Visualizing the game idea): This activity aims to plan the overall flow of the game in terms of how users will play the game from launching the game to quitting it. The idea illustration sheet is used to sketch the flow, and the team can choose from different ways (such as flow diagram, user scenarios, or screen prototypes) to illustrate the overall picture of a refined design idea. This activity allows for sketching the user experience and enables a transition from a static representation of ideas to a more dynamic view of how game players will play or interact with the learning game.
• Idea documentation (Archiving the final idea): This last activity aims to document the final state of the learning game design idea, producing a short version of a game design document (GDD). The idea documentation sheet is used that provides a format to fill in details of the final idea.

4.3. Board and Playbook

The board is provided as scaffolding and comprises two main parts: a layout structure (card deck placeholders for design activities) and a playbook (describing how to play along with required toolkit artifacts (cards and idea sheets), intended goal, and outcome). Figure 1 illustrates a portion of the board. It provides visual affordance and describes the play sequence to guide the ideation process, reducing the need for supervision. The playbook explains the card decks required for each design activity, and the layout structure provides the space for placing these card decks.

4.4. Workshop Technique

The workshop format (inspired by [17,26]) provides support for a collaborative design process. A workshop session is approximately 2 h, where the participants work in teams (four to six participants) to generate and develop learning game design ideas, reflect on them, and finally illustrate and document their design ideas using the LEAGUE toolkit. One or two facilitators organize the design workshop to lead the team(s) through the ideation session by sequentially presenting the design activities. Each workshop starts with a short introduction (10 min) of GBL concepts and the LEAGUE toolkit description. Afterward, all the teams are provided with the LEAGUE toolkit and are asked to start the five-step ideation session. First, one of the organizers presents each activity individually, and the other organizer simultaneously provides each team with the toolkit artifacts (cards and idea sheets) required for that activity (see Section 4.2). Subsequently, the teams start working on that specific activity. Each design activity is time-bound (activity 1 is of 10 min, activity 2 of 30 min, activity 3 of 10 min, activity 4 and 5 of 15 min each) and must be completed following certain rules specifying the use of particular cards in each design activity and required output (see Section 4.2). A time constraint is added to avoid getting stuck or reaching the game idea too early before exploring the different cards. After activities 2 and 3, teams very briefly present their ideas in 5 min. One team member takes the role of a logger and records the order of use of cards in activity 1–3 using a log sheet. The log sheet is useful for both data collection and for the team to reflect on their design strategy to make improvements in the future. In the end, teams summarize their learning game designs with group presentations. The workshop ends by collecting participants’ feedback.

5. Toolkit Evaluation: The User Study

We conducted design workshops with 34 participants (21 males and 13 females) in three different sessions to understand the value and utility of the LEAGUE toolkit in informing and guiding the ideation of learning games design in practice, which would, in turn, strengthen the argument for the transformation of the framework and also enabled the refinement of the design toolkit and the process. The participants were a convenient sample of university students and researchers (25 to 45 years old) at the faculty of computer science and engineering. The sample included 16 master students and 18 researchers (Ph.D./postdoc) that formed seven teams (comprising 4 to 6 members). Two teams had four participants each, four teams had five participants each, and one team had six participants. All participants’ primary subject of study was computer science except one researcher from electrical engineering. It should be mentioned that 24 participants had no background in learning game design, 3 had minimal experience, and 7 had some experience. We selected participants with no to less experience to fully explore the toolkit’s support in informing GBL design and not coming from their previous experience and knowledge to ensure the validity of data. None of the participants had previous experience with the LEAGUE toolkit. Each session had different participants. All participants were asked to sign a consent form and were informed that their participation was voluntary. Each session was approximately two hours long and was supervised by two organizers. The workshops were organized as described in Section 4.4. At the end of the workshop, data regarding participants’ experience with the LEAGUE toolkit was collected using a questionnaire (with a 3-point rating scale) and a short focus group session to get feedback on their collaborative design process using the toolkit and suggestions for improvement. The questionnaire consisted of 23 questions (inspired by [15,24,26]) related to understandability, satisfaction, fun, and usefulness. Data was also collected through observation (researchers taking notes during workshop sessions) and video recording. The focus group session was recorded, and data analysis was guided by the Corbin and Strauss process [37].

We focused on three evaluation goals:

• Participants experience using the toolkit: How did participants experience learning game design using the toolkit in terms of fun, satisfaction, understandability, and usefulness?
• Roles (defined objectives) of the toolkit in the GBL design practice: Were the five defined objectives (i) inform GBL design knowledge, (ii) support collaborative design process, (iii) brainstorming, (iv) reflection, and (v) guidance for GBL ideation for the toolkit achieved?
• Refinement of the toolkit: How to further improve the toolkit?

6. Results and Analysis

This section presents the design workshops’ results using the toolkit through five stages of ideating an educational game design. First, we very briefly describe a few of the educational game design ideas participants came up with during the ideation sessions to exemplify the different types (range of learning domain) of ideas participants could achieve in ~70-min ideation session using the toolkit. Some of the ideas developed include: Team 1) “My swinging 20’s (or Die)”: An augmented reality dance class where the elderly with mobility issues learn to dance with their famous idol by following the indicated move patterns shown by colored areas using a dance pad otherwise, they will die. Team 2) “Smart city simulator (SCS)”: A 3D simulation VR game for young adults to understand smart cities by developing and organizing a smart city to increase the inhabitants’ happiness level. Team 3) “University runners!!!”: A campus-based location-enabled game (using sensors installed at a campus that are linked to GPS location used in the web game) for all students at the university with assignments to learn to work as a team (teamwork skills) to achieve a common objective that is deadline extension by running away. The design outputs of these teams for each activity are shown in Appendix A.

Next, we will elaborate on the results from the questionnaire, focus group, and observation, which will be presented in three categories focusing on evaluation goals. An overview of the participants’ responses to statements on fun, satisfaction, understandability, and usefulness is presented in

Table 4. Participants’ feedback from the questionnaire.

Aspects	Key Concepts of the Questions	Agree	Neutral	Disagree
Fun	Interacting with cards was fun	74%	21%	6%
	Fun to do different activities	88%	9%	3%
	First activity (idea generation)	76%	18%	6%
	Second activity (idea development)	85%	9%	6%
	Third activity (idea refinement)	62%	32%	6%
	Fourth activity (idea illustration)	62%	29%	9%
	Fifth activity (idea documentation)	47%	35%	18%
Satisfaction	Visual design of cards	85%	12%	3%
	Time given for each activity	41%	32%	26%
	Sequence of use-primary cards	71%	26%	3%
Understandability	Cards	79%	12%	9%
	First activity (idea generation)	68%	12%	21%
	Second activity (idea development)	88%	9%	3%
	Third activity (idea refinement)	76%	24%	0%
	Fourth activity (idea illustration)	82%	18%	0%
	Fifth activity (idea documentation)	88%	12%	0%
Usefulness	Informing GBL design concepts (Primary Cards)	74%	18%	9%
	Supporting brainstorming (Trigger Cards)	76%	24%	0%
	Reflecting on ideas (reflection cards)	50%	41%	9%
	Information on card	74%	26%	0%
	Easy to ideate educational game design	62%	29%	9%
	Process provided guidance for GBL design	85%	12%	3%
	Considered elements I would not have without cards.	71%	26%	3%

(rating scale is, 1 = Agree, 2 = Neutral, 3 = Disagree).

6.1. Participants Experience Using the Toolkit

The aspects fun, satisfaction, understandability, and usefulness (shown in Figure 5) presents the participants’ responses about their experience using the toolkit. The results reveal that overall, 71% agreed (only 7% disagree) that using the toolkit was fun, 66% agreed (11% disagree) with overall satisfaction, 80% agreed (only 5% disagree) that overall toolkit was easy to understand and 70% agreed (only 5% disagree) that toolkit was useful for the defined roles. The results for individual questions of these aspects are shown in Table 4. The questions for “usefulness” are linked to the toolkit’s roles (defined objectives) and are discussed in the next section.

The results for specific questions related to “fun” show that most participants had fun using cards and activities (between 74 and 88% agreeing to the statements). The responses to how fun each design activity was show that most workshop participants agree that the first four design activities were fun to do (only 6 to 9% disagreed). However, only approximately half of the participants agreed that the fifth design activity (idea documentation) was fun. This can also be justified by the nature of the documentation task being time-consuming and tedious. Despite that, only 18% disagreed, which means the toolkit made it more accessible and fun to some extent, at least. Idea development was regarded as the most fun, followed by idea generation, which was the most creative thinking tasks. The results for specific questions related to “satisfaction” show that participants were satisfied with the visual design of cards and the sequence of use of primary cards (only 3% disagreed). The latter was asked to know if additional guidance was required for the sequence of use of cards to further scaffold ideation. Fewer participants agreed that sufficient time was given for each activity (41% agreed). This implies that some refinements are required in the workshop technique to readjust time distribution for design activities. The specific questions related to "understandability" show the participants’ response concerning how easy it was to understand cards and each of the five design activities. In general, most respondents (68% to 88% agreed) thought that cards and design activities were easy to understand. Nobody thought the three activities idea refinement, idea visualization, and idea documentation were hard to understand. However, 21% of the respondents disagree that idea generation was easy to understand and 3% for idea development. One reason for idea generation being slightly tricky to understand compared to others is because it was the first activity right after the introduction and also time-bound (only 10 min), participants felt rushed as they needed some time to understand the complete picture together as a team before they get started with the activity. From the results, we can conclude that the second activity (idea development) was the most fun and easy to understand. On the contrary, the fifth activity was also the easiest to understand. However, it was considered the least fun. Therefore, the nature of the task also affects the experience (fun). The feedback from focus group sessions also supported this. The following comments were received when asked about the most and least fun activity: “Second activity was the most fun to develop the idea more”, “Nobody likes documentation”.

Observations and feedback from focus group sessions support the results from the questionnaires. We received positive feedback from the participants highlighting that the toolkit was fun to use. “The sense of time diminished, time passed quicker than it actually does. It didn’t feel like a 2-h workshop”, “All group members were engaged”, “fun to play and engaging”. Overall, there were very few questions and misunderstandings during workshop sessions, suggesting that it was not difficult to understand the toolkit artifacts (cards and activities). However, in terms of time provided for each activity, participants sometimes felt rushed, although it kept them motivated and engaged. The participants’ feedback from the focus group include: “More time please! sometimes we felt a bit rushed”, “Since it is a group work, I think even if you give more time everyone is going to use it anyway, so it is good that it was time-restricted”. It was observed that the first activity was most difficult to understand. Some teams needed additional time to explore and read the playbook to understand the process before starting the activity, while others jumped too soon on the details about the game idea in activity 1.

6.2. Roles (Defined Objectives) of the Toolkit in the GBL Design Practice

This section focuses on investigating the five objectives (see Section 3, step 1) defined for the LEAGUE toolkit. The questions for usefulness in Table 4 are related to the roles (defined objectives) of the toolkit in GBL design ideation. Overall, 70% of the participants agree (only 5% disagree) that the toolkit was useful for the roles presented in questions. We will discuss the results for each role (objective) of the toolkit below.

Role 1 (Inform GBL design concepts): Only 9% disagree that the toolkit was useful in informing GBL design concepts through primary cards (achieving objective one). Figure 6 shows the percentage of teams’ usage for individual primary cards. Interestingly, all primary cards were used at least by one team, indicating that not a single primary card can be considered irrelevant. The feedback from focus group sessions includes: “Good discussions about the game design and what is important to make a good educational game”.

Role 2 (Support collaborative design process): The observation and focus group session provided some useful insights regarding play strategy emphasizing the toolkit’s role to support a collaborative design process (achieving objective two). Most teams played collaboratively, selected one card with consensus, and then discussed as a group. One team used a turn-taking strategy and divided the cards to have a better flow. In both cases, the cards were selected through voting and debating. Most teams did not define any roles based on the area of expertise (although all teams overall focused all six GBL dimensions in the ideation process) but developed roles for practical work such as logging, drawing, documenting, taping cards, or to spare time such as finding triggers. The feedback from focus group participants includes: “It was very collaborative, and it is fun to discuss ideas in a team and build on them”, “All group members were engaged”, “It was a good approach for initiating team discussion”.

Role 3 (Support brainstorming/idea generation): None of the participants disagreed that trigger cards supported brainstorming and information on cards was useful (achieving objective three). Based on observations, the teams used trigger cards in three different ways: (1) Use a trigger card as a design idea, (2) Use trigger cards to extend their ideas by combining different trigger card(s) and/or a custom card, and (3) Use trigger card as an example or inspiration to understand the concept and come up with their own ideas. However, all the teams almost always browse through the trigger cards either before initiating or finalizing their ideas based on their approach. Combining different cards (mixing trigger and custom cards) sparked the potential to generate creative design ideas. The feedback from focus group participants includes: “Trigger cards help to come up with ideas”, “Trigger cards also work to confine the idea”, “As a first-time user they worked really well but I wonder if it could be a bit restricting when you use the toolkit multiple times, but it is good the you are able to write your own ideas as well using custom cards”.

Role 4 (Support reflection): The toolkit’s role in providing support for reflection was agreed by only 50% of participants. However, only 9% disagreed with the statement (achieving objective four at least partially). Based on observations, most teams made only one or two changes to their designs using the reflection cards which is in line with the questionnaire results. The feedback from focus group participants include: “We refined based on reflection cards, but the questions in reflection cards were overlapping so we found only one problem”, “It set our purpose for the whole design, but they were not imposing any new idea changes”, “We did not change anything using reflection cards but we refined the idea further”, “Maybe reflection should go after idea illustration because if you haven’t visualized, you cannot change anything”.

Role 5 (Guidance for GBL ideation): The majority of the participants agreed that the toolkit provided guidance for educational game design (only 3% disagree), the cards prompted to consider new elements (only 3% disagree), and it was relatively easy to ideate and design an educational game using the toolkit (only 9% disagree) thus achieving objective five. The feedback from focus group sessions includes: “For a novice in learning game design like me, it was very helpful because it was not difficult, and the cards were guiding me on what to do. Otherwise, I do not know what to do in game design and how a game is designed”, “It is ‘meta-game’, a game to design a game”, “Ideation sheets are very useful to visualize because you don’t remember everything”. When asked if more guidance was required for primary card selection, most teams were satisfied with their selected order of use of primary cards and thought that open choice is better as the order may vary in different games and the team should decide what is important for their game idea, guiding order would constraint the process. However, few participants thought it would be helpful to guide the sequence. The feedback from focus group participants includes: “Since we already defined the purpose of our game in activity one it was easier for us to follow that path and select the cards that satisfy our purpose”, “We had many options in the second activity, so we were picking the concepts we thought were more important for our game idea”, “We can just browse through the cards to select the ones that base on our initial idea to further develop it”.

6.3. Refinement of the Toolkit

The analysis of the questionnaire, observation, and focus group data revealed that some elements played a role in hindering the ideation process of learning game design. This section highlights the issues in the developed toolkit (reflecting on the features that limited its use) and discuss the refinements that would improve its effectiveness. We identified four challenges that are presented along with the recommended refinements in the next two subsections.

6.3.1. Challenges in the Workshop Format

Challenge 1 (Introduction phase of the workshop): The observations during design workshop sessions and participants’ feedback from focus groups revealed that the workshop’s introduction phase could impact the toolkit’s understandability. For example, few teams (in the first session of the design workshop) felt that activity 1 was a bit difficult to understand due to which they focused too soon on the details of the initial game concept in design activity 1 and were also unclear that trigger cards are for inspiration and not the definite answer which made them rely more on trigger cards (instead of using custom cards) for generating their initial game idea. This is also evident from the questionnaire results where the participants least agreed (68%) with the understandability of the first activity compared to the other four activities where they agreed between 76 and 88%. The introduction phase was short (only 10 min) and used PowerPoint slides to introduce GBL concepts and the LEAGUE toolkit, after which the teams immediately started working on activity 1 without giving any time for free exploration of the toolkit.

Recommended refinement 1 (Use of video/demo and free play): Therefore, to address this issue, the introduction must be more focused and include a demo/video explaining and visualizing the process instead of just slides. One participant suggested similar improvements: “Show the cards when the slides presentation is happening so that there is an easier translation of knowledge from talk to gameplay”. It is also critical to give some time for free play after introduction, so team members can familiarize themselves with the toolkit components and understand the desired goal and outcome for each activity using the board and playbook before starting the ideation process.

Challenge 2 (Time-bounded design activities): We added a time constraint to the process similar to [26] in an attempt to avoid participants’ converging on an idea too early (without exploring different cards) or getting stuck. However, the study results revealed that time distribution for design activities is not an easy undertaking. Participants acknowledged the benefit of time-bounded activities. However, it was vital for them that it did not get in the way of the creative process. For example, some participants thought they could use more time in idea development (second activity) to further elaborate their ideas and make it more concrete. Similarly, some teams required more time for activity one as participants needed time to explore the play process as a team before getting started. For activity 3, a difference in opinion was observed, where some teams needed more time, and the others needed less, depending on their reflective thinking and improvements. This is also evident from the questionnaire results, where only 41% of the participants were satisfied with the time given for each activity.

Recommended refinement 2 (Flexible format to introduce relaxation in the time constraint): Therefore, to address this issue, it helps to make the workshop format more flexible and accommodating by adding room for relaxation in time constraint if and when needed. This can be done by adding rapid group debriefing slots after each activity, which can also be used for extending the time of the activity if needed. Another solution could be to run two parallel activities, for example, running the activities 4 and 5 in parallel (where team members can divide the task and simultaneously work on illustration and documentation), leaving more time for the first three activities. Based on our experience, it is also necessary to plan some extra time for setup and practical arrangements in case of minor setbacks.

6.3.2. Challenges in Working with the Cards

Challenge 3 (High number of trigger cards): Some participants thought that there were too many cards in the second activity, as evident from the focus group feedback, which includes: “They are many”, “yeah! Quite a lot”. Although, this was not a problem since the second activity was considered the most fun (the majority of the participants (85%) enjoyed) and easy to understand (88% participants agreed) according to the questionnaire results. However, due to the high number of cards provided, it is vital to make the cards highly searchable.

Recommended refinement 3 (Make cards highly searchable using accessories): Some teams were observed splitting the card decks among all the participants who browsed through the cards and then collectively selected the relevant cards to discuss (also evident from Role 2 in Section 6.2). This was a way to speed up and simplify the card-selection process by dividing tasks and can be further facilitated by introducing mechanics such as turn-taking, defending and attacking, etc. Previous card research has recommended applying visual design (such as color-coding) to make cards highly searchable [4,26]. The LEAGUE cards are also color-coded by the six categories (six GBL dimensions). The use of the same card-name and initial letters of card-id for linked cards (primary and trigger cards) worked effectively as an identifier for the cards. However, some participants suggested that working with cards should be simplified further by having accessories such as a card division box (for dividing each category or using alphabetical order) that would improve the searchability of cards during design activities. Deng et al. [4] proposed a similar approach to introduce accessories (such as clothes pegs) for designers to mark important cards. Another solution could be to reduce the number of cards.

Challenge 4 (Facilitate critical reflection): A reflection card in the LEAGUE toolkit presents a question concerning interrelation between GBL dimensions to uncover the questionable decisions in the game design idea. Therefore, each reflection card focused on two dimensions to encourage the team to critically think about the trade-off by urging and attesting the generated ideas against these cards’ criteria. However, this did not work very well, as some participants thought that questions in reflection cards were overlapping. Therefore, although they set the rationale for the learning game design but were not inflicting any new idea changes. This was also in line with the questionnaire results, where only 50% of the participants agreed with the usefulness of reflection cards in refining ideas.

Recommended refinement 4 (Precisely define criteria with examples imposing design change): Therefore, to address this issue, reflection cards need to be more directive to impose new design changes and challenge designers to reflect on developed design ideas. This can be done by offering clear guidance about what is required for critical reflection, for example, using additional directive questions guiding how to judge (similar to [26]), or providing some examples of possible refinements concerning the interrelation. Another approach can be to focus on one specific factor of each dimension (rather than a high-level concept) to make the question more specific, imposing a design change.

7. Discussion and Conclusions

In summary, the toolkit was found useful for the GBL design practice and contributed to informing and introducing the GBL concepts to the participants during the ideation process. In a short period of time, teams could ideate, develop, refine, illustrate, and document their educational game design ideas using the toolkit artifacts.

This section discusses some of our approach’s strengths and limitations by reflecting on the results from user studies and identifying which elements facilitated the ideation process of learning game design using the toolkit. These reflections can serve as useful design lessons and guidelines for designing similar GBL ideation tools, and finally, we conclude the paper with directions for future work.

7.1. Strengths of the Design Toolkit

There are five main successful aspects of transforming the framework into such a design tool that facilitated the collaborative design process of learning games.

• Easy to use in practice: The structured design activities systematically break down the creative process into individual steps that are easier to understand and operate. The cards, on the other hand, supported users to carry out the individual tasks. This is consistent with results from previous card-based tools, e.g., [15,26]. The cards helped the participants recognize that several elements combine to make an effective learning game and further helped them identify these essential elements. The team can shuffle through cards (owing to their tangible form [4,19]) to select them to cover the important aspects until they feel satisfied with their idea.
• Stimulate brainstorming and creative thinking: All participants found trigger cards useful (none disagreed) for stimulating creative thinking and as a kick start for brainstorming. They not only provided the existing ideas but also helped generate new ones. These results are in line with previous research on design cards [19,26]. Some teams would select a trigger card to elaborate on the idea with team discussion and end up combining the trigger card with a custom card to generate a new idea.
• Creative elements in the toolkit generate fun: The majority of participants considered that trigger and primary cards were more useful than the reflection cards, which can also be explained with the results for reflection activity that was considered comparatively less fun than idea generation and development (see Table 4). The fun element was led by the creativity involved in the design activities. The design activities which required more creativity were considered more fun (even if they were lengthy or less easy to understand at first) as compared to activities like reflection and illustration, which were comparatively less creative and required more critical and analytical thinking, were comparatively less fun (although they were fun for more than 60%). Lastly, the documentation activity was the least fun part, although it was the easiest to understand.
• Guide the design process in a playful manner: Cards and design activities together provided a structured path that offered guidance on how to proceed with the design process. They give a clear direction and order by providing guidelines to follow five steps (design activities) and building blocks to use (different card decks). The use of different types of cards was successful for individually supporting each design activity, introducing new elements specific to that step not only guided that activity but also added newness and individuality avoiding them to become boring. Participants were engaged in exploring new cards to achieve a new goal. Each card type was useful for their specific design task, and the card content was useful and easy to understand. Therefore, the results confirmed that the cards were useful for idea generation, development, and refinement, which is in line with the previous finding [17,26]. The majority of the participants enjoyed using the cards (74%) and thought that design activities were useful and fun (85–88%).
• Inform and encapsulate theoretical concepts: The primary cards were useful for informing and encapsulating theoretical GBL design concepts (only 9% participants disagreed). Such an assessment is similar to previous findings by [24]. The majority of the respondents (71–74%) thought that they considered elements they might have overlooked otherwise, and the information on the cards was useful. The cards’ information acts as a quick reminder for designers to the related knowledge/experience, which helps them focus on “all GBL aspects” during idea generation, development, and refinement resulting in a more concrete design. Using all six primary cards in the first activity resulted in a strong foundation, as the initial design idea comprised all six GBL aspects to expand on in the next activity. One of the participants praised the potential of the toolkit for academia: “This can be used by the teachers in the learning game design course since it explains all the important dimensions”.

7.2. Some Design Decisions that Proved Helpful

Unlike other design tools, we combined the playbook within the board to simplify its use and provide a structured step by step guidance in combination with the board layout. The board size was kept moderate. These design alternations proved helpful as it reduced the effort of handling two artifacts and managing large space and provided one point of reference for both layout and play procedure. The moderate board size encouraged participants to keep cards in place, which limited other projects [26]. Further, we used ideation sheets for each activity that provided the layout for used/selected cards and kept them organized in one place. The use of ideation sheets and a log sheet is a novel feature of this toolkit, which is not present in previous work and was found very useful for tracking each team’s design process and capturing the design decision rationales. These artifacts can be used to facilitate awareness and traceability of the design process which is vital for the design practice [34]. It is helpful to use all idea sheets from previous activities to form a comprehensive and meaningful description of the discussed ideas. These idea sheets serve as a useful visual reminder and help form a story around the overall game idea that ensures that all team members share the same understanding of the game idea. Each design activity was ordered and time-bound, so a team cannot skip to the next activity without completing the prior one, and also, the order was well though the following activity required outcomes from the preceding activity to work on. This created not only the sequence but also motivation for the next step. It was also useful to have a debrief session between different design activities. It made it possible to follow the game idea’s progress and change, and motivated teams to do better.

7.3. Limitations of the Study

There are a few limitations to this work. The toolkit has only been evaluated with researchers and students and not with design practitioners and learning game experts. Therefore, results are not a representation of the overall design community. There were some issues with time management and workshop organization that affected the understandability of a few activities and working with the cards. It is evident from the results in Table 4 that the first activity was most difficult to understand. The observed issue can be mitigated by simple modification in the workshop technique, such as letting participants explore the toolkit for five minutes instead of directly jumping into the design process and giving breaks between [4]. The first author has predominantly led the workshop. Therefore, we cannot conclude about the level of supervision needed when other researchers use the toolkit. However, the second facilitator was different in the three conducted sessions, and they presented different design activities indicating that the knowledge is in the toolkit and not the person introducing them, which suggests that the toolkit can also be used in settings with someone other than the lead author as the main facilitator. Moreover, although workshop sessions were recorded, this paper only focused on the participants’ perception of using the toolkit collected from a questionnaire, focus group, and an observation. We triangulated the questionnaire data by confirming statements from focus group feedback and/or observations to minimize the limitations of questionnaires. However, the quality of the generated ideas was not ranked or evaluated. Hence, no conclusions can be drawn in terms of their novelty. The further work will examine video recording and toolkit artifacts focusing on generated ideas, team dynamics, and multi-dimensional focus to explore the full potential of the toolkit.

7.4. Conclusion and Future Work

This paper points to a lack of operationalizable approaches for designing learning games that integrate the research based conceptual GBL design knowledge in educational games’ practical design process [38]. GBL design frameworks provide theoretical design knowledge but are challenging to use in practice without tool support [12]. Design cards are a well-accepted form of intermediate-level knowledge facilitating effective transfer from theory to practice [4,5]. However, none of the existing design cards entails complete GBL design knowledge. Therefore, to bridge this gap, we transformed the LEAGUE framework [2] into a card based GBL ideation toolkit to support the learning game design team in early design practice. The toolkit contains a set of four card deck types (Primary, Trigger, Custom, and Reflection cards) containing GBL design concepts and a workshop technique with five structured design activities that provide step-by-step guidance for the ideation process enabling team members to design learning games in a collaborative and playful manner. The cards are grouped into six key categories, each focusing on one dimension of GBL design. The results from three design workshops illustrated the toolkit’s value and utility in informing and guiding educational game design in practice. The toolkit can function as both a practitioner tool and a research instrument to further the domain of GBL design. Researchers in other domains can also learn from transforming the theoretical knowledge of the framework into a lightweight card-based tool.

Future work focuses on multiple directions. We would like to revise the toolkit and workshop format based on the findings from the user study. Future work will also focus on exploring the toolkit’s in-depth potential, examining how the toolkit supports multi-dimensional focus and collaboration by exploring design outcomes (toolkit artifacts and generated ideas) and video recordings (team dynamics) of ideation sessions. Future studies will involve industry practitioners to evaluate toolkit in industry-based projects with real-life constraints and more extended periods, investigate team dynamics, and track design decisions to identify design patterns leading to effective educational games. Furthermore, we also plan to use the toolkit with different workshop techniques, mechanics, and game rules to explore if it further facilitates the ideation process. We are also interested in investigating the feasibility of the developed game ideas by complementing ideation with prototyping to develop digital prototypes of the learning game design ideas generated in the workshops.