*2.6. Presentation of Results*

Qualitative results from the rapid thematic analysis are presented first, then qualitative and quantitative results are presented together to reflect the convergent design. Table 3 combines ranked themes with examples and important topics of discussion to illustrate how group processes affected participants' choices. We used these results to suggest ways that multiple stakeholder groups could help promote healthy gaming (Figure 1). This report follows standards for reporting qualitative, quantitative, and mixed methods research established by the American Psychological Association [27,28].




**Figure 1.** Suggestions for regulating video game play.



#### *2.7. Methodological Integrity*

To ensure fidelity and utility of our qualitative findings and address our research objectives, we address the adequacy, grounding, and context of data and how we used it to develop meaningful and coherent findings. First, we collected data from cultural insiders—those who had an interest in video gaming and self-regulation—grounding our data in their perspective. We used an emic approach to data collection, free listing, to elicit their cultural understanding and insider knowledge. We further grounded findings in evidence by using free-listed responses to illustrate themes. We included all audience members in our analysis, since all members identified as gamers. However, audience members also identified as members of other stakeholder groups. Although our sample was small, we aimed to gather insights/knowledge from gamers and other stakeholders; we feel the collected data are therefore adequate to capture a broad range of perspectives. Researchers' perspectives were used to determine an initial set of themes, facilitate rapid data analysis and guide group discussion.

We demonstrate consistency in our analysis by describing consensus development and discussion and when and how they were used in the analytic process. We used data displays (tables), author consensus on a priori themes, group discussion with participants, formal quantitative consensus, and participant feedback as checks to promote rigor in the qualitative analysis. We discuss the context of data collection and present findings coherently, reflecting on discrepancies and reporting according to mixed methods standards. Our results further the understanding of how gamers and other stakeholders can make insightful and meaningful contributions to a clinical- and policy-relevant research.

#### *2.8. Ethics*

The study protocol was approved by the institutional review board of the Johns Hopkins Bloomberg School of Public Health (IRB No. 00006931) and carried out in accordance with the Declaration of Helsinki. The need for written informed consent was waived by the institutional review board. Participants were fully informed about the study at the beginning of each study session and offered the opportunity not to participate.

#### **3. Results**

Six people participated in the free listing round and submitted between one and three answers per question. Seven people participated in the ranking round and six people completed demographic and gameplay questionnaires. Four questionnaires were complete, including rankings, while two were partially complete.

#### *3.1. Qualitative*

We initially identified nine themes from our brief review of several articles related to gaming engagement and design [8,9,29–32]. Our selection of themes was not meant to be exhaustive of all themes in the gaming literature, but rather a starting point for our discussion-based study design. Themes were chosen based on the authors' knowledge of game engagement and game design and how these factors contribute to problematic or self-regulated gaming. Categorization of responses to our four questions resulted in three to six themes for each question (Table 2). Some responses transcended themes, such as pop-up windows being a behavioral game mechanic that was "game-breaking", thus primarily having to do with the theme of immersion (Q4). Through discussion, some submissions categorized by the authors as having to do with one theme were reassigned to a different theme and one novel theme, analyzing moves, was suggested but did not end up being ranked highly by participants. Another theme, competition, was suggested as being different from achievement. The pre-determined theme of habit was not seen to match any of the responses. The full database of responses and assigned themes are available as Supplementary Materials.

#### *3.2. Mixed Methods Synthesis*

Table 3 presents results of nonparametric quantitative analysis that describe consensus on themes as they were ranked by participants. Immersion (e.g., *exploration of dynamic universe*) and achievement (e.g., *rewards*) emerged as important features of games that motivated playing (Q1), but there was less agreement on game type, mood and competition. Social factors (e.g., *online interactions with terrible people*) and novelty (e.g., *repetition*/*grinding with little variety*) were considered equally important reasons to stop playing (Q2). Immersion (e.g., *games that are crashy or slow*) was also agreed to be somewhat important, but achievement (*games that are not balanced*), environment (*out of time*), and mood (*not fun anymore*) were considered less important.

Social factors showed up again in Q3 as a strategy people could use to self-regulate (e.g., *play with others in a game that requires everyone to be present*). One response categorized as social described how online group play could limit gaming time. Participants debated this initially, but after deliberation agreed that choosing to play games online in groups might be an effective way to reduce time as other group members' leave, disrupting the group. Self-regulation strategies were also considered important. Responses focused on setting firm goals (*just three quests*) or limiting time (*setting an alarm, set a schedule for gaming time that is after your other work*/*responsibilities have been finished*) or money spent on gaming.

For Q4, there was strong agreement that the game industry could make it easier for players to self-regulate by changing or adding behavioral game mechanics (e.g., *clear break points, rewards for short sessions, negative reinforcement like XP decay*). Participants also offered social (*taking away features of online multiplayer and have local play only*) or environment-related (e.g., *integrated health tips into game that use health points that players can use themselves*) strategies, but these were less often ranked. The group also suggested benefits to new technology such as augmented reality games (e.g., Pokémon GO) that would offer novelty (*local multiplayer fun with new tech*), but there was low consensus on this theme. The group discussed this with regard to the benefits of staying grounded in the real world, especially when playing with others, which was thought to be useful for reducing the feeling of immersion that could lead to time loss.

#### **4. Discussion**

This study used a systematic, consensus-developing participatory approach with members of the gaming community to explore the importance gamers assigned to themes about self-regulating gaming. We found that even a small group (n = 7) was able to offer many specific suggestions that could form the basis of preventive interventions (Figure 1). These results support the emphasis on game mechanics and self-regulation identified in previous research [8,9,19], but add an additional dimension, social factors, that presents a new potential focus of prevention research. The findings also contribute to the importance of incorporating cultural insider knowledge about gaming and self-regulation into research and intervention development across disciplines.

The fact that gamers endorse the theme of self-regulation so strongly suggests that integrating specific techniques for promoting self-regulation (Figure 1) into preventive interventions may be a worthwhile approach. These strategies may be more palatable when endorsed by the gaming community itself. One possibility would be to have celebrity gamers (e.g., popular Twitch streamers) suggest or even discuss such strategies with their gaming audience. Hearing about risk from trusted individuals is important, and recent research shows that the general public may place greater trust in social media personalities than experts who may offer conflicting or confusing messages [37]. Streamers do address mental health topics in Twitch [38], so this may become an important mode for delivering preventive interventions relating to video game play.

Our small sample had many specific suggestions about features to include or leave out of games, such as exit points—places where they could leave the game without feeling that more had to be done. Exit points have been specifically mentioned as a factor of ethical game design that can be promoted in the video game industry [39]. Exit points may be unique to games and were not included in a recent

expert-developed list of recommended tools for the prevention of problematic gambling that could be revised to address gaming disorder [14].

Another specific suggestion was to incentivize taking breaks. Our sample mentioned how games sometimes punish players for limiting their play, for example temporarily blocking players from team play if the leave a team abruptly. While games may do this to foster engagement, this also makes it more difficult for gamers to leave when competing opportunities or demands arise, such as a friend's request for a visit or the dawning recognition that time has passed and the dishes still need to be washed. Games can also reward breaks by manipulating experience gains, either by allowing players to progress faster when they return from a break or limiting progress after long sessions. While interventions related to experience gains have been implemented in several countries (experience decay), this approach may be less appealing to game developers [12]. In contrast, allowing faster progress when returning from a break (e.g., rested experience) may be a more palatable way to reward healthy amounts of gaming.

Social factors are an area brought up by our participants that seems to have little mention in suggestions for prevention interventions. Our sample found them important as a driver of disengaging (negative online interactions) and self-regulating (playing with others in a game that requires everyone to be present). In fact, this theme is a good example of the strength of a group reflection process as it reflected a single participant's suggestion. At the beginning of the discussion, most participants didn't understand or agree with the suggestion that choosing to "play with others in a game that requires everyone to be present ... since the other players can effectively force you to stop" would be a good way to self-regulate, but after a few minutes of discussion, six out of seven people ranked that as one of the most effective ways to limit time. This reveals the unique benefit to conducting a Delphi method approach with gamers as stakeholders—the perspective of an individual can be clarified within a group and ultimately recognized as a ground truth: Yes, if we choose to play games with that require a group, our time may be limited whether we choose to limit it (or are able to effectively limit it) or not.

Our findings suggest that focusing research and prevention efforts without involving insights from community members may unnecessarily limit the range of potential interventions decision-makers consider and impede progress toward implementation. The current epistemological approach toward research on gaming disorder places greater emphasis on clinical expertise rather than multiple perspectives [40]. The findings here challenge this traditional approach by considering gamers as vital co-creators of knowledge. For example, loot boxes have been the focus of much public health activity related to gaming disorder to date [13,41], yet loot boxes or other gambling-like mechanics were not mentioned by our sample. Other targets may be more palatable to multiple stakeholders, potentially easier to achieve, or more effective at preventing problematic gaming. Gamers are also useful in pointing out nuances of how we think about interventions. For example, while research may concentrate on immersion as a motivating factor for problematic gaming, our applied research here allowed gamers themselves to make a clear link between an example intervention (pop-ups) and the experience of losing immersion (pop-ups as game-breaking).

While this study was limited in sample size, it does reflect the gender (male) and age of gamers in the United States [42,43]. The small size does limit generalizability, and other sample makeups may suggest different responses. For example, our sample was self-selected for attending a science fiction and education convention and participating in a panel focused on gaming research. Other samples may prioritize different themes when ranking questions related to self-regulation. Our sample also felt the theme of Immersion to be more important than achievement—had we used a sample of gamers at a gaming tournament, for example, this would likely be different. In addition, our sample was English-speaking and based in the US, so findings may not transfer to other countries or cultures. However, one goal of this study was to demonstrate that the combination of perspective seeking, deliberation and systematic quantitative consensus analysis provides additional information beyond that of a survey, interview or focus group. Further research would benefit from systematic recruitment

of a large sample of various stakeholders and a formal, multi-round consensus development technique in an online format to encourage transparency.

### **5. Conclusions**

Formal consensus development techniques with gamers produced many specific examples of ways gamers can self-regulate and also emphasized the importance of game mechanics in fostering healthy gaming and preventing gaming disorder. These examples are supported by previous research and could serve as the basis for interventions to promote healthy gaming (Figure 1). Going forward, decision makers could increase the value of research on the prevention of problematic or disordered gaming by involving gamers, developers, clinicians and other stakeholders in co-learning about self-regulation strategies, game mechanics, and other features of game engagement.

**Supplementary Materials:** The following are available online at http://www.mdpi.com/1660-4601/17/11/3846/s1, data and themes; analysis code.

**Author Contributions:** Conceptualization, M.C.C. and A.B.L.; data curation, M.C.; formal analysis, M.C.C., M.C. and A.B.L.; funding acquisition, M.C.C.; investigation, M.C.C.; methodology, M.C.C., M.C. and A.B.L.; project administration, M.C.C., M.C. and A.B.L.; resources, M.C. and A.B.L.; supervision, M.C.C. and A.B.L.; validation, M.C.C.; visualization, M.C.; writing—original draft, M.C.C.; writing—review and editing, M.C.C., M.C. and A.B.L. All authors have read and agreed to the published version of the manuscript.

**Funding:** This work was supported by the National Institute of Mental Health Training Grant 5T32MH014592-39.

**Acknowledgments:** The authors wish to thank Nico Pandi of the Museum of Science Fiction in Washington, D.C. for his extensive support.

**Conflicts of Interest:** M.C.C. has spoken to the World Health Organization about this topic at the request of the video game industry but has received no funding, honorarium, fees, meals, accommodations, donations or reimbursement. She is currently seeking funding from industry sources to conduct game research. She has acted as a consultant with a nonprofit organization that provides mental health and suicide prevention support through online video game play. She is the CEO and founder of the Gaming and Wellness Association, Inc., a nonprofit video game research organization. Author M.C. has worked for a video game developer as a game quality tester and heads an IT and research consulting company. Author A.B.L. declares no conflicts of interest.
