**1. Introduction**

Research on problems with video game play has led to a proposed diagnosis of Internet gaming disorder (IGD) in the Diagnostic and Statistical Manual 5 [1] and inclusion of gaming disorder in ICD-11 [2]. Representative surveys usually show that only a small percentage of the general population has clinical problems related to excessive gaming [3]. Systematic reviews link gaming disorder to correlates such as impulsivity, male gender, impaired mental health and low psychosocial well-being [3–5]. However, the Goldilocks hypothesis (based on the fairytale character Goldilocks, who struggles to find sizes of chair, porridge and bed that are "just right) suggests that engaging with technology in a moderate way is not harmful [6]. The WHO notes that most people are able to play video games without problems [7]. As this suggests that moderate video game play is a reasonable public health goal, investigating how people can play in a healthy way is vital to support the public health objective of preventing gaming disorder.

While many studies examine motivations to play, less is known about how gamers self-regulate their gaming. One qualitative study in children uncovered several reasons for engaging (e.g., motives, habit, and contextual factors) or disengaging (e.g., self-regulation, parental intervention, or disruption/frustration of gaming outcomes or psychological needs; [8]). Contextual factors of games or the environment also contribute to the length of gaming sessions: gaming sessions may be prolonged through losing track of time [9], achievement, winning or game mechanics, including gambling-like mechanisms such as loot boxes [8,10]. Disengaging from play may also be easier with support from real-life friends [11] or when using strategies like setting an alarm, taking regular breaks, or setting goals in the game [9]. The above findings show a range of individual, game and contextual factors that can promote healthy gaming and prevent disorder.

However, policy responses have focused largely on limiting time gaming [12] or regulating loot boxes as gambling [13]. Commentaries suggest that existing tools for self-regulation of gambling, such as self-exclusion, mandatory breaks, or pop-up messaging, might be useful for gaming [14]. A few technology platforms have already implemented feedback mechanisms to alert players about time or allow them to limit their time [15,16]. Games also provide such mechanisms in the form of parental controls. Some also remind players to take breaks, but forcing shutdown or exclusion periods as a preventive mechanism has so far been limited [12]. The current focus on a few limited policy aspects also neglects the need to take into account cultural factors, individual player factors, and how games change over time [12].

With the recent inclusion of gaming disorder in ICD-11, calls for collaboration with industry stakeholders have increased. Many researchers have suggested that the game industry has a responsibility to protect the health of gamers [14,17,18]. Fewer researchers support including gamers as partners in research and policy efforts to understand or prevent gaming disorder [19,20].

Participatory and multi-stakeholder approaches in health research allow for better translation of research into practice [21,22]. Stakeholders are members of diverse groups, including experts and the general public, who can provide unique perspectives on an issue. In the case of gaming disorder, this could mean not just clinicians, researchers, and policy makers, but also gamers, their families, and people who create games. Stakeholder-engaged research that involves group deliberation and decision-making promotes the type of co-learning that allows researchers to better examine all aspects of a problem or question, encourages decision-makers to consider multiple perspectives, and makes science more transparent [23,24]. Using systematic methods that integrate perspectives of those in the know—in this case video game players and developers—with research and theory will be vital to allow research on problematic gaming to address insider perspectives and cultural differences and keep up with changing technologies.

This approach has been used with gamers to explore their unique insights into what it means to be "addicted" [19]. As an illustration of the usefulness of this approach, we discovered that a small sample of gamers at a gaming fan convention were not only interested in discussing game "addiction", their ideas overlapped only partially with criteria for IGD. In fact, their suggestions leaned toward less strict criteria for social impairment and they suggested appointment mechanics (a game feature promoting engagement) as a primary contributor to loss of control. These findings suggest that including gamers in discussions of research into gaming disorder may improve research. However, a systematic, perspective-integrating approach that examines factors associated with gaming self-regulation would better contribute to intervention development.

To address this gap and provide clinicians, decision-makers, the game industry and the general public with additional insights about gaming self-regulation, we conducted participatory, mixed-methods, convergent-design research with gamers to discuss the challenges of regulating video game play. This research builds on our prior study regarding gamers' insights on game "addiction" [19]. The qualitative objectives of our study were to elicit a wide variety of knowledge from the video gaming stakeholder community about why gamers play, why they stop playing, and how they self-regulate gaming time and to map these to known themes in the literature. The quantitative goal was to numerically rank the consensus on importance of themes. The overall mixed methods goal was to demonstrate a rapid and transparent consensus-development approach that can be used with multiple stakeholders. Such an approach would foster knowledge development and research co-creation in order to develop and prioritize research and prevention interventions for problematic video game play.

#### **2. Materials and Methods**

#### *2.1. Overview*

Mixed methods research combines qualitative and quantitative data to answer research questions that require a deep understanding of multiple perspectives, contexts, and/or communities/cultures [25]. We used a convergent mixed methods design, i.e., we combined qualitative and quantitative data simultaneously, to develop a rapid consensus on the relative importance of themes related to video gaming engagement and self-regulation. We used a pragmatic approach toward study design and analysis, as these allow researchers to combine diverse approaches to best answer the research question and place equal value on both objective and subjective knowledge [26]. In our previous study [19], for example, we used an emic, free-listing approach, working with our gamer audience at a panel to generate categories and themes for free-listed responses. As this proved difficult within the one-hour time frame of a panel at a convention, we opted to combine deductive (pre-determined themes from the literature) and inductive (free-listing) approaches in the qualitative part of our study design here.

As qualitative research benefits from researchers' self-reflection, we describe our relevant background here. All authors are researchers with experience in conducting mixed methods research in technology and health, particularly in video games. We have conducted a similar study in another setting to address gamers' insights into game "addiction" [19]. The first two authors, who conducted the panel, also identify as members of the gaming community, regularly attend gaming fan conventions, and have personal experience with regulating their gaming time. They use this personal experience to connect with stakeholders and facilitate meaningful discussion of healthy and unhealthy video game play. Author MCC is a researcher specializing in video game play, technology use and mental health; author MC is an information technology specialist with game industry experience; and author AL is a professor specializing in digital health.

To address our mixed method goal of demonstrating a rapid and transparent consensusdevelopment technique, we first used a deductive approach among authors to identify prominent themes around self-regulation and gaming from the literature (Section 2.2). We then used an emic, inductive, free-listing approach to gather data about self- and other-regulation of video game play from audience members at a presentation (Section 2.4). We then facilitated a group decision-making procedure (Delphi process) with the audience to brainstorm, discuss/debate, and finally quantitatively rank concepts related to gaming self-regulation and steps the gaming industry could take to facilitate healthy play. We present the qualitative, quantitative and mixed methods results to better understand self-regulation as gamers see it. Our presentation here was guided by American Psychological Association standards for mixed methods and qualitative research reporting [27,28].

#### *2.2. Selection of Themes for Framework*

Prior to the presentation, the authors first briefly reviewed relevant literature on game engagement [8,9,29] and design [29–32]. We extracted themes and frameworks from these into tables and bulleted notes and discussed and combined results to condense themes surrounding game engagement and self-regulation. This deductive approach was not meant to be exhaustive but to present the most salient themes for discussion; other themes could be incorporated during data collection if necessary. This condensed list of themes (Table 1) was used to facilitate rapid thematic analysis during presentation and discussion with audience stakeholders. Themes were transcribed to index cards placed on the table in front of the presenters and used in a deductive, top-down theoretical thematic analysis approach [33] to rapidly classify participant responses during the presentations.



#### *2.3. Participants*

A panel was presented at a science fiction and education convention, Escape Velocity, held in Washington, D.C. in 2017. The convention featured topics and events relevant to science and technology, STEAM education, video games, and pop culture. The conference is sponsored by the Museum of Science Fiction and science organizations like NASA, nonprofit organizations and industry groups, and in its first year was attended by over 2000 people [34].

Participants were recruited passively through panel descriptions in the convention schedule and through posters on the door to the panel room. Our panel on Day 1 was entitled "Insights Needed: Video Gaming and the Goldilocks Principle" and our panel on Day 2 was entitled "Insights Needed Wrap-Up: Give Us Your Feedback on Video Games and Engagement Preliminary Analysis." The authors had no prior relationship with the research participants. Each session began with the reading of a consent document where participants were offered the opportunity to leave if they did not want to participate; no audience members left. Participants did not receive compensation for participation. As the pragmatic study design was dependent on the setting of a panel at a convention, we did not identify a needed number of participants ahead of time and our sample was one of convenience. One participant arrived late and two participants did not complete all portions of the questionnaire.

Six respondents completed at least some demographic information and percentages are reported out of those who answered the question. Age ranged from 27 to 49 (Mean = 35, SD 7.9) and one participant was female. Most respondents (66.7%) were white, one identified as mixed race, and one declined to answer. Half had a graduate degree and another 33.3% had graduated from college. All respondents identified as members of the gamer stakeholder group, but also identified with other groups such as family/friend of a gamer (n = 3), researcher (n = 3), educator (n = 1), and industry

(n = 1). In response to the question "Do you see yourself as a gamer", all responded either Yes or "Kind of (intermediate/casual)" but one participant also answered "No" to the question "Do you play video games, including mobile games (e.g., Candy Crush, Pokémon GO)?". (This participant identified primarily as a member of the game industry.) All worked full time and the majority (80%) earned over \$40,000/year. The group played on average 3.13 h a day (SD 1.31) and 5.33 days per week (SD 1.15). Half played about 1–2 h per session while the other half played 2–3 h per session.

#### *2.4. Data Collection and Study Procedure*

Questionnaire responses, sticky notes used in free listing, and field notes made up the qualitative and quantitative data as described here. This approach described is a combination of nominal group technique and Delphi approach that the authors used in a previous study [19]. Prior to data collection the authors printed out questionnaires with questions about demographics, stakeholder group membership, gaming experience and ranking of themes and added a number to each questionnaire. We then created a set of sticky notes with matching participant numbers. Each set of sticky notes contained five sticky notes for each of our four questions. Each note was labeled with a participant number and Q1, Q2, Q3, or Q4.

Data were collected over the course of an hour at a panel discussion. At the beginning of the panel, the presenters (the first two authors) introduced themselves as researchers and gamers and described the presentation as a study using group decision-making techniques to help answer questions about how to achieve just the right amount of engagement with video games. We then asked participants to list ideas for promoting healthy gaming by asking four open-ended questions (Table 2) about gaming engagement and regulation strategies. Participants recorded answers on sticky notes, then completed paper questionnaires asking about demographics and gaming experience. We allowed 10 min for data collection. Participants were asked not to complete the questions about ranking the importance of themes until told to do so.



As data on sticky notes was collected, responses were categorized by the presenters by placing each sticky note under a pre-determined theme's index card. Disagreements about how to categorize responses were resolved through discussion between presenters. Once all responses were categorized (about 10 min after all responses were received) they were transcribed into the slideshow and displayed to all participants. We asked for clarifications of some responses and, as a participant check, discussed whether we had categorized responses for each question appropriately based on our own understanding as gamers and researchers. Field notes were incorporated into the response examples at times for further clarification. Participants elaborated on some responses, disagreed with one another or provided additional explanation, and a robust discussion of specific responses and themes ensued. The authors' goal was to ensure clarity of themes and their examples; this was accomplished with about three minutes of discussion for each question.

Participants were then asked to rank the top three themes they felt best answered each question by listing their choices for most important themes in order on the last page of the questionnaire, e.g., Question 1: #1 = immersion, #2 = game type, #3 = achievement. The numeric ranking participants gave to themes for a particular question constituted the quantitative data. Participants were invited to return in two days for the results. After the panel, one author transcribed questions, responses, and themes into an Excel spreadsheet to record qualitative data and the results of thematic analysis. Another author entered rankings and questionnaire responses directly into a Stata do-file to create and analyze a quantitative analysis database. To validate data quality, data entry was double-checked and analyses re-run prior to manuscript revision.

At a second panel two days later, we discussed the study's organizing frameworks and procedures and presented the results of the ranking. We again asked for clarifications of some responses and further discussed difficult-to-categorize responses. Finally, we presented the limitations and strengths we felt the process provided and invited participants to provide additional feedback. Data are available as Supplementary Materials.

#### *2.5. Qualitative and Quantitative Analysis*

#### 2.5.1. Categorization of Responses

We categorized responses as the unit of analysis during the presentation using a deductive approach based on the a priori themes. We chose theory-based thematic analysis as a practical way to facilitate rapid categorization in keeping with our pragmatic approach. Initial thematic analysis was performed at the panel and final categorizations and themes for each question were arrived at after discussion with participants; we used participant checks to facilitate categorization of some responses. Themes and examples (Table 2) were then projected onscreen to allow participants to rank them.

#### 2.5.2. Calculation of Ranks

We used an observational design to gather participants' perspectives on the importance of themes for each question. Ranked themes were assigned a numerical value from one to eleven to represent the eleven total themes (the nine a priori themes from Table 1 and two themes emerging during group discussion). We validated data entry against collected data by listing database results for each question in a table and checking against each participant's questionnaire.

For each question, we transformed a participant's ranked themes into coded rankings by replacing a placeholder variable for each theme with the numeral 1, 2, or 3 if a participant ranked it among their top three choices. This resulted in a unique subset of themes as the domain for each question. If a respondent did not rank a particular theme for that subset, the rank for that theme for that participant was coded as 0. We conducted a quantitative consensus analysis of participants' ranking of the most important themes for each question by calculating the weighted sum of centered ranks (WSCR) in Stata 13 IC [35] using the Skillings–Mack statistic (skilmack.ado, [36]). This flexible approach aggregates rankings across participants by combining the ranking for a specific question provided by each participant with the consensus on ranking for that question across all participants. With this approach, high consensus themes (those that are ranked more often or ranked very highly) will have a high and positive WSCR while low consensus themes, i.e., those that are ranked fewer times, ranked as

less important, or show greater disagreement among respondents will have a low or negative WSCR. As we used rankings rather than raw numerical values in the analysis and replaced non-ranked themes with 0, standard errors are the same for all resulting WSCR for each question.
