**The More You Take It, the Better It Works: Six-Month Results of a Nalmefene Phase-IV Trial**

**Pablo Barrio 1,\*, Carlos Roncero 2, Lluisa Ortega 1, Josep Guardia 3, Lara Yuguero <sup>4</sup> and Antoni Gual <sup>1</sup>**


Received: 19 February 2019; Accepted: 3 April 2019; Published: 6 April 2019

**Abstract:** Background: Alcohol use disorders remain a major health problem. Reduced drinking has been increasingly recognized as a valuable alternative to abstinence. Nalmefene has shown in previous, experimental studies to be a useful tool to aid reduced drinking. However, more data from routine practice settings are needed in order to obtain evidence with high external validity. The aim of this study was to conduct a single-arm phase-IV study with alcohol-dependent outpatients starting with nalmefene for the first time. Here, we present the main effectiveness analysis, scheduled at six months. Methods: This was an observational, multisite, single-arm, phase-IV study conducted among adult alcohol-dependent outpatients who received nalmefene for the first time. The study consisted of four visits: Baseline, 1 month, 6 months, and 12 months. At each visit, drinking variables were obtained from the time-line follow-back regarding the previous month. Satisfaction with medication was also assessed from both patients and professionals with the Medication Satisfaction Questionnaire. A repeated measures mixed model was performed for effective analysis regarding drinking outcomes (reduction in total alcohol consumption and the number of heavy drinking days). Regression analyses were performed in order to find predictors of responses to nalmefene. Results: From a total of 110 patients included, 63 reported data at the six-month visit. On average, patients took nalmefene 69% of days during the month previous to the 6-month assessment. Compared to the one month results, the number of heavy drinking days and total alcohol consumption increased. Still, they were significantly lower than baseline values (outcome evolution over time was from 13.5 to 6.8 to 9.4 days/month, and from 169 to 79 to 116 units/month). A total of 23 patients were considered medication responders. The number of days of taking nalmefene was significantly associated in the regression analysis. Satisfaction was globally high for both professionals and patients and, overall, nalmefene was well-tolerated with no serious adverse events reported. Conclusion: The data provided by this phase-IV study suggest that nalmefene is an effective, well-tolerated treatment for alcohol-dependence in real world, clinical settings.

**Keywords:** drinking reduction; nalmefene; phase-IV trial; 6 months; observational

#### **1. Introduction**

Alcohol remains a first-order global health problem, with 15 million affected people in the EU [1]. Recent publications in the US also warn about an increase in the prevalence of alcohol use disorders

during the last decade [2]. The impact it has on both individuals and society is of an enormous dimension, both medically and economically [3,4]. Several strategies have been applied to decrease the burden of the problem, ranging from public health intervention to individualized psychosocial and pharmacological treatment.

In the arena of pharmacological treatments, one of the latest incorporations has been nalmefene. The lack of real-world data, the need to assess adverse events in clinical populations, and the critiques surrounding the drug since its approval led to the need of a phase-IV trial. A single-arm, observational, phase-IV study of nalmefene, including 110 patients, was started in four different sites in Barcelona, Spain, in 2015.

Baseline and one month results [5] suggested nalmefene was effective in reducing alcohol consumption. They also showed that it was well-tolerated with no serious adverse events, and the satisfaction with the drug of both professionals and patients was high.

The study, which was designed to last 12 months, consisted of four assessment points in time (baseline, 1, 6 and 12 months). In this paper we report the results of the main effectiveness analysis which was scheduled at 6 months.

#### **2. Methods**

A full description of the methods has been described elsewhere [5]. As a summary, this was an observational, multisite, single-arm, phase-IV study, conducted among adult alcohol-dependent outpatients taking nalmefene for the first time. The study consisted of 4 visits: Baseline, 4 weeks, 6 months, and 12 months.

The main outcome variables of the study were:

(1) Reduction in drinking parameters, measured as a change from baseline in heavy drinking days and total alcohol consumption (units in the previous 28 days). The data were derived from the previous month's timeline follow-back. As the study was conducted in Spain, one drink was considered to contain 10 g of pure ethanol.

(2) Patient and clinician satisfaction, as measured by the Medication Satisfaction Questionnaire (MSQ) [6,7]. Secondary outcomes included changes in drinking risk-level according to the WHO definitions (very high-risk: More than 100 g of alcohol per day in men and more than 60 in women; high-risk: 60–100 g per day in men, 40–60 g per day in women; medium-risk: 40–60 g per day in men, 20–40 g per day in women; low-risk: 1–40 g per day in men, 1–20 g per day in women). Liver enzymes were also analyzed.

Other collected variables at baseline included previous history of drug use, psychiatric history, family history of drug and alcohol use, and concomitant or changes in psychiatric medication during the study period. At the study visit, the number of days that patients took nalmefene was also recorded.

Responders to medication were defined in the same manner as in the 1-month publication (a reduction in daily alcohol consumption of at least 70% or downshift of two categories in the drinking risk-level, according to the World Health Organization (WHO), or a shift to low-risk category)

For effectiveness analysis, the repeated measures linear mixed procedure was used for both heavy-drinking days and total alcohol consumption as main outcomes. Age, sex, and number of days taking nalmefene were entered as fixed effects. We also included the presence of any psychiatric or addictive comorbidity as covariates. For each main outcome, regression coefficients (b), *t*-values and *p*-values were calculated. Statistical significance was set at 0.05. Missing data at 6 months for outcome variables was addressed with the conservative approach of baseline observation carried forward (BOCF). Days of medication intake were imputed to 0 for missing values. A descriptive analysis of the Medication Satisfaction Questionnaire (MSQ) was conducted. A satisfaction analysis and a logistic regression analysis for the detection of significant predictors of medication responders were conducted. Included variables were sex, age, number of days taking nalmefene, presence of comorbid drug use, and presence of psychiatric comorbidity. A descriptive analysis of adverse events was also conducted.

The study protocol, final approved informed consent document, and all supporting information were submitted to and approved by the institutional review boards of all participating centers. All participants provided written informed consent before taking part in study procedures. The study was conducted in accordance with the International Conference on Harmonization and Good Clinical Practice and the principles of the Declaration of Helsinki.

#### **3. Results**

#### *3.1. Sample*

From a total of 110 patients included at baseline, 47 were lost to the follow-up at six months, leaving a total of 63 patients reporting drinking outcomes at six months. Of these, 34 patients were still taking nalmefene. Accordingly, drug interruption was reported by 29 patients (eight due to low efficacy, four due to the achievement of the reduction aim, one due to adverse reactions, two due to the change to an abstinence aim, and 14 due to other reasons). No overdoses were reported. Of the 34 patients still on nalmefene, complete abstinence was reported by six subjects.

In patients still on nalmefene, the mean number of days of patients taking the drug over the previous month was 19.3 (SD = 11.6). More than half of the patients (57%) took it on a daily basis. Basal characteristics of participants can be observed in Table 1.

**Table 1.** Baseline sociodemographic and clinical characteristics of included patients.


a: Data are expressed as *n* (%) for categorical variables and mean (SD) for continuous variables; b: defined as the presence of diabetes, hypertension, high blood cholesterol or any other significant medical condition; c: defined as any substance use disorder ( except nicotine dependence), past or current, as clinically evaluated in the first visit of the study.

#### *3.2. Efficacy*

Both drinking outcomes increased, compared to month 1, but were still significantly reduced when compared to baseline values. The mean number of heavy-drinking days over the last 4 weeks was 9.4 (SD = 10.8) and the total alcohol consumption in units, also over the last 4 weeks, was 116.4 (SD = 171.8). Evolution over time of these parameters can be seen in Figure 1. As a sensitivity analysis, evolution over time, according to per protocol analysis, is provided in Figure 2. Per protocol analysis was conducted using only available data, that is, patients lost to follow-up were not included in this analysis.

The repeated measures linear mixed model revealed a significant effect in both outcomes at time 1 (first month). Given the increase in both outcomes at time 2 (month 6), no further significant changes were observed. The rest of the covariates were not statistically significant.

A total of 23 patients (21%) were considered medication responders. The only significant predictor found was the number of days with medication intake (OR = 1.058, CI 95% 1.001–1.118).

**Figure 1.** The change over time of study outcomes according to intention-to-treat analysis. The left axis shows alcohol units over the last 4 weeks. The right axis shows heavy-drinking days over the last 4 weeks. HDD: heavy drinking days.

**Figure 2.** The change over time of study outcomes, according to per protocol analysis. The left axis shows alcohol units over the last 4 weeks. The right axis shows heavy-drinking days over the last 4 weeks. HDD: heavy drinking days.

#### *3.3. Satisfaction*

Figure 3 displays the satisfaction of professionals and patients with nalmefene at six months, as recorded by the MSQ. In the logistic regression analysis, the the number of days taking study medication revealed a significant effect upon patient satisfaction (OR = 1.074, CI 95% 1.021–1.130).

**Figure 3.** The satisfaction with treatment according to the Medication Satisfaction Questionnaire. 1: Extremely dissatisfied; 2: Very dissatisfied; 3: Somewhat dissatisfied; 4: Neither satisfied nor dissatisfied; 5: Somewhat satisfied; 6: Very satisfied; 7: Extremely satisfied.

#### *3.4. Safety*

At six months, no new drug-related adverse events were notified. That left a total of 29 patients with medication-related adverse events during the first month of treatment. Most events were mild, and no serious adverse events were recorded. Additionally, no overdose was observed or notified.

#### **4. Discussion**

The main 6 month effectiveness analysis of this phase-IV trial suggests that nalmefene is effective in reducing alcohol-use when used in real-world, clinical settings. Similar to the 1 month results, nalmefene was well-tolerated and no significant, severe, or life-threatening reactions were observed. There are, however, relevant observations to be made in comparison with previous 1 month results.

Both heavy-drinking days and total alcohol consumption increased for the whole sample, suggesting nalmefene loses some efficacy over time. It is important to bear in mind that this is, in fact, a common phenomenon to many addiction treatments, whether pharmacological or psychosocial. On the other hand, we took the conservative approach of baseline observation carried forward (BOCF) to deal with missing data, a fact that could have decreased our statistical power.

It is important to note the relatively small number of patients still taking nalmefene at six months (31% approximately). While low-efficacy and adverse reactions might explain an important share of medication drop-outs, it is also possible that patients who are offered nalmefene are more prone to treatment abandonment, since it has been shown that patients still drinking, and those who aim at reduction objectives, are at greater risk of treatment drop-outs [8–10].

Taken together, we believe these observations should remind professionals that patients who are prescribed nalmefene are especially prone to abandon treatment, and that efforts should also be directed toward increasing treatment retention.

Worth mentioning is a similar, recent study [11] conducted among outpatients in routine settings that showed a significant decrease of drinking outcomes at 24 weeks. Similar to our sample, and other previous experimental studies with nalmefene [12], psychiatric comorbidity was high. Interestingly, and contrary to our findings, improvements were seen over the six-month period. All taken together, recent evidence suggests that nalmefene is indeed effective for alcohol use disorder patients in routine settings, where comorbidity is frequent.

In trying to find differential characteristics between responders and non-responders to treatment, as measured by reductions in alcohol consumption parameters and changes in drinking risk categories, only the number of days taking nalmefene yielded significant effects.

Regarding satisfaction data, a slight decrease in comparison to one month results was observed, probably mimicking the decreased effectiveness at six months. Interestingly, the number of days taking medication was the only covariate associated with increased satisfaction. While it could be interpreted as a consequence, rather than a cause of increased satisfaction, and similar to the regression analysis, which was conducted in order to find predictors of treatment response, we believe this finding suggests that nalmefene might work better over the long-run if taken daily or with a high degree of frequency, rather than sporadically. Another hypothesis worth considering when analyzing this data could be that a higher degree of medication intake is, indeed, a reflection of higher motivation in patients. Therefore, the results obtained in this study are probably not to be entirely attributed to pharmacological effects. It is also fundamental to comment on the fact that we imputed as 0 the days of medication intake in the cases where this variable was missing. While we consider this imputation not unlikely, especially given that nalmefene requires ongoing medical prescription in Spain, it is also true that, in combination with the BOCF imputation for missing drinking outcomes, it could have biased the results obtained, regarding days of medication intake as a significant predictor of both medication response and satisfaction.

Finally, several other limitations apply to this study, such as its observational design and lack of control group, the reduced sample size, and the limited geographical area where the study was conducted.

#### **5. Conclusions**

Nalmefene seems to provide further effectiveness at 6 months, in spite of it being reduced, as compared to the first month, after initiating treatment. Our results suggest that a more frequent intake might be related to better outcomes, both in terms of satisfaction and reduced drinking.

**Author Contributions:** Conceptualization, A.G. and P.B.; methodology, A.G. and P.B.; formal analysis, P.B.; investigation, P.B., C.R., J.G., L.Y. and L.O.; writing—original draft preparation, P.B.; writing—review and editing, P.B.,C.R.,L.O., L.Y. and J.G.; funding acquisition, A.G.

**Funding:** This study was funded by Lundbeck. The sponsor was involved in the study design, but not in data collection, analysis, manuscript writing, or decision of submitting the article for publication.

**Conflicts of Interest:** P.B., C.R., J.G., and A.G. have received honoraria from Lundbeck. P.B. has also received honoraria from Pfizer. C.R. has also received honoraria from Janssen-cilag, Otsuka, Server, GSK, Rovi, Astra, MSD and Sanofi. L.Y. L.O. have no conflict of interest to declare.

#### **References**


© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## *Article* **Application of Diagnostic Interview for Internet Addiction (DIA) in Clinical Practice for Korean Adolescents**

#### **Hyera Ryu 1, Ji Yoon Lee 1, A Ruem Choi 1, Sun Ju Chung 1, Minkyung Park 1, Soo-Young Bhang 2, Jun-Gun Kwon 3, Yong-Sil Kweon 4,\* and Jung-Seok Choi 1,5,\***


Received: 4 January 2019; Accepted: 2 February 2019; Published: 6 February 2019

**Abstract:** The increased prevalence of Internet Gaming Disorder (IGD) and the inclusion of IGD in DSM-5 and ICD-11 emphasizes the importance of measuring and describing the IGD symptoms. We examined the psychometric properties of the Diagnostic Interview for Internet Addiction (DIA), a semi-structured diagnostic interview tool for IGD, and verified the application of DIA in clinical practice for Korean adolescents. The DIA is conducted in a manner that interviews both adolescents and their caregivers, and each item has a standardized representative question and various examples. It consists of 10 items based on the DSM-5 IGD diagnostic criteria, which is cognitive salience, withdrawal, tolerance, difficulty in regulating use, loss of interest in other activities, persistent use despite negative results, deception regarding Internet/games/SNS use, use of Internet/games/SNS to avoid negative feelings, interference with role performance, and craving. The study included 103 adolescents divided into three subgroups (mild risk, moderate risk, and addicted group) based on the total score of DIA. Demographic and clinical characteristics were compared among the DIA subgroups using the chi-square test and analysis of variance (ANOVA), and correlation analysis was used to examine the associations of IGD symptoms with clinical variables (e.g., impulsivity, aggression, depression, anxiety, self-esteem). The DIA total score was significantly correlated with Internet and smartphone addiction, depression, state anxiety, self-esteem, impulsivity, aggression, and stress. Furthermore, the moderate risk and addicted group showed significantly higher levels of Internet and smartphone addiction, anxiety, depression, impulsivity, aggression, stress, and lower self-esteem compared with the mild risk group. The Junior Temperament and Character Inventory (JTCI), which measures temperament and character traits, revealed that the mild risk group had higher levels of persistence and self-directedness than did the addicted group. Our findings confirmed the psychometric properties of DIA and the application of the DIA classifications in Korean adolescents.

**Keywords:** internet gaming disorder; semi-structured diagnostic interview; psychometric properties; adolescents

#### **1. Introduction**

The prevalence of Internet addiction has steadily increased, from 10.4% in 2011 to 12.5% in 2014 in Korea [1], and the prevalence of Internet gaming disorder (IGD) is about 6% in Korean adolescents [2]. The American Psychiatric Association (APA) included IGD as a condition worthy of future study in Section III of the 5th edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-5), and the draft of 11th revision of the International Classification of Diseases (ICD-11), released in 2018, included the definition of gaming disorder (GD) [3,4]. The growing prevalence of IGD and its recognition as a possible behavioral addiction has increased the importance of describing and measuring the symptoms and severity of the condition. In this context, several researchers have noted that unifying the terminology and developing measurement tools based on the DSM-5 IGD diagnostic criteria are necessary to integrate concepts related to IGD [5–8].

As part of this effort, two self-report questionnaires, the Internet Gaming Disorder Test (IGD-20) [9] and the Internet Gaming Disorder Scale-Short Form (IGDS9-SF) [10], were developed based on the DSM-5 IGD diagnostic criteria. Although self-report questionnaires are cost-effective and easy to administer, the tool has some limitations. Jeong et al. (2018) recently found a discrepancy between self-report data and the clinical diagnosis of IGD in adolescents due to underreporting as a result of social desirability effects, which is a limitation of self-report questionnaires [11]. Therefore, semi-structured diagnostic interviews tools are needed to measure IGD symptoms more accurately in adolescents.

Several structured and the semi-structured interviews have been developed to assess IGD symptoms. The checklist for the Assessment of Internet and Computer Game Addiction (AICA-C) is a semi-structured interview that assesses six criteria (craving, tolerance, withdrawal, loss of control, preoccupation, and negative consequences) [12]. The Structured Clinical Interview for Internet Gaming Disorder (SCI-IGD) is a 12-item structured interview based on DSM-5 criteria and IGD-related clinical experience [13]. However, these tools were developed for middle school students and are only interviewed for adolescents without including their caregivers. The Diagnostic Interview for Internet Addiction (DIA) was developed to evaluate internet, games, and SNS addiction according to the DSM-5 diagnostic criteria, both children/adolescents and their caregivers are interviewed. It consists of 10 items, as 'craving' was added to the 9 DSM-5 criteria for IGD (cognitive salience, withdrawal, tolerance, difficulty in regulating use, loss of interest in other activities, persistent use despite negative results, deception regarding Internet/games/social network site (SNS) use, the use of Internet/games/SNS to avoid negative feelings, and interference with role performance). Each item is rated on a 4-point scale (0: No information; 1: No symptoms; 2: Subthreshold; and 3: Threshold level), and the number of items with a score of 3 (threshold) is calculated as the DIA total score. Previous studies have classified the severity of IGD as mild risk, moderate risk, plain addicted, and severe addicted based on the total number of IGD criteria met [14]. Thus, we adopted the previous categories for the DIA subgroups and verified the application of the DIA classifications in clinical practice by comparing IGD-related psychiatric symptoms among the subgroups.

Several studies have shown that IGD is related to internalizing (e.g., depression, anxiety, stress, and self-esteem) and externalizing (e.g., impulsivity and aggression) problems in adolescents [15–17]. Depression is the most common symptom associated with IGD in all age groups, and several studies have found that individuals with IGD experienced more severe depression and anxiety symptoms than did non-addicted individuals [17–21]. A longitudinal study found that depression and anxiety were negative outcomes of Internet gaming [22]. Lam et al. (2009) found that stress-related variables such as family dissatisfaction and recent stressful events were associated with IGD symptoms in adolescents [23]. Moreover, low self-esteem, impulsivity, and aggression are risk factors for IGD in adolescents [22,24,25]. Several studies have found positive correlations between IGD symptoms and personality traits (e.g., sensation seeking, neuroticism, high impulsivity, and high aggressiveness) and negative correlations with extraversion, responsibility, reward dependence, complacency, and self-directedness [25–29]. Thus, we examined the relationship between DIA total scores and

internalizing/externalizing problems in adolescents, and compared the various characteristics among the DIA severity subgroups.

We examined the psychometric properties of the DIA in Korean adolescents and verified the application of the DIA classification in clinical practice by comparing the clinical characteristics among subgroups.

#### **2. Materials and Methods**

#### *2.1. Participants and Procedure*

We screened children and adolescents (aged 7–18 years) who used excessively internet games and/or smartphones in Clinic I-CURE Centers. All participants were screened using questionnaires pertaining to Internet and smartphone addiction (e.g., the Korean Scale for Internet Addiction for adolescents (K-scale); the Korean Smartphone Addiction Scale (S-scale); and the Internet Addiction Proneness Scale for Adolescents (O\_A), which is completed by caregivers). Subjects who scored above the cutoff for addiction on at least one screening questionnaire were enrolled in the study. We enrolled 166 participants between August 2015 and December 2017. Of these, 47 children were excluded because the sample size of children was small and there were differences between the questionnaires in children and adolescents (e.g., BDI versus CDI, etc.), and 16 adolescents with intelligence quotient scores <80 or missing data were excluded from the study. The final analysis included 103 subjects between the ages of 13 and 18 years. In this study, the interviewers consisted of a master's-level of clinical psychology and every interviewer was trained by addiction and/or child-adolescents psychiatry specialists. A flow chart of the study is shown in Figure 1.

**Figure 1.** Study flow chart. NOTE: The screening cut-off values shown are those for adolescents; children were excluded from this analysis, and their scores are not reported here. The study participants were divided into subgroups according to their total DIA score. K-scale = Korean Scale for Internet Addiction for adolescents; SAS-SV = Smartphone Addiction Scale-short form version; S-scale = Korean Smartphone Addiction scale; O\_A = Internet Addiction Proneness Scale for Adolescents checked by caregivers.

#### *2.2. Measurements*

#### 2.2.1. Diagnostic interview for Internet Addiction (DIA)

The DIA is a semi-structured diagnostic interview tool consisting of 10 items based on the DSM-5 Section III IGD diagnostic criteria; it is used to assess Internet, games, and SNS addiction symptoms (i.e., cognitive salience, withdrawal, tolerance, difficulty in regulating use, loss of interest in other activities, persistent use despite negative results, deception regarding Internet/gaming/SNS use, use of Internet/gaming/SNS to avoid negative feelings, interference with role performance, and craving). In this study, the internal consistency coefficient of DIA was 0.72.

DIA interviews take about 10–20 min for each subject and their caregivers. Each item has a standardized representative question and various examples, so clinicians can more easily evaluate the score. For example, to assess 'regulation difficulty in regulating use', there is a representative question: "Do you feel you should reduce the Internet/Games/SNS, but you can't reduce the time you spend doing Internet/Games/SNS?" Detailed examples are provided as follows: "I often do more Internet/Games/SNS than I originally thought", "I often don't do other activities I was planning because of the Internet/Games/SNS", "I try to stop the Internet/games/SNS but it is difficult to break". Table 1 shows the standardized interview script for each item. After interviewing the subjects and caregivers, the clinician calculated the total score to determine whether subjects were addicted to the Internet, games, and/or SNS. Each item was rated on a 4-point scale (0: No information; 1: No symptoms; 2: Subthreshold; 3: Threshold level), and the number of items with a score of 3 was calculated as the total DIA score (range, 0–10).


**Table 1.** Examples of standardized interview script in Diagnostic Interview for Internet Addiction.

#### 2.2.2. Internet and Smartphone Addiction Scales

The Korean Scale for Internet Addiction for adolescents (K-scale), Smartphone Addiction Scale-short form version (SAS-SV), Smartphone Addiction scale (S-scale), Young's Internet Addiction Test (YIAT), and the Internet Addiction Proneness scale for adolescents (O\_A) were used to measure Internet and smartphone addiction symptoms. The K-scale, developed by the National Information Society Agency [30], is a 40-item questionnaire with scores on each item ranging from 1 (not at all) to 4 (always). The SAS-SV is a 10-item scale in which each item is rated on a 6-point Likert scale. Scores above the cutoff values of 31 for males and 33 for females indicate high-risk use [31]. The S-scale is a 15-item questionnaire that measures the level of smartphone addiction on a 4-point Likert scale [32]. The YIAT, developed by Young (1998) [33] and validated in Korean by Kim et al. (2003) [34], consists of 20 items, with higher scores indicating more severe Internet addiction. The O\_A consists of 15 items [32]. The internal consistency coefficient of all scales was higher than 0.91 in our study.

#### 2.2.3. Clinical Measurements

We examined internalizing and externalizing problems associated with Internet and smartphone addiction using questionnaires to assess depression, anxiety, self-esteem, impulsivity, aggression, stress, temperament, and personality traits.

The Beck Depression Inventory-II (BDI-II), developed by Beck et al. [35], is a 21-item questionnaire that measures the severity of depression, with higher scores reflecting more severe symptoms. The BDI-II was validated for Korean adolescents by Lee et al. [36], and the Cronbach's alpha was 0.56 in this study. The State–Trait Anxiety Inventory-X1 (STAI-X1) is a 20-item tool that measures state anxiety [37]. The Cronbach's alpha was 0.98 in our study. The Rosenberg Self-Esteem Scale (RSES) [38], which is also translated in Korean, measures perceived self-esteem and self-acceptance, with higher scores reflecting high self-esteem. The Barratt Impulsiveness Scale-II (BIS-II), which consists of 23 items and three subscales (cognitive, motor, and non-planning impulsivity), was developed by Barratt and White [39] and translated into Korean by Lee [40]. The Cronbach's alpha was 0.99. The Korean version of the Aggression Questionnaire (AQ) consists of 27 items, as two of the original 29 items were excluded [41,42]. The AQ measures physical and verbal aggression, anger, and hostility. The Cronbach's alpha was 0.98. The Daily Hassles Questionnaire (DHQ) measures stress related to parents, family environment, friends, school, teachers, and school life. It was developed by Rowlison and Felner [43] and modified and validated in Korean adolescents by Han and Yoo [44]. Finally, we used the Junior Temperament and Character Inventory (JTCI) to assess four temperaments (novelty seeking, harm avoidance, reward dependence, and persistence) and three character traits (self-directedness, cooperativeness, and self-transcendence) in adolescents. The Korean version of the JTCI consists of 82 items, each with 'yes' or 'no' response options.

#### *2.3. Statistical Analysis*

The chi-square test and analysis of variance (ANOVA) including post-hoc test (Bonferronni method) were used to compare demographic and clinical characteristics among the DIA subgroups and to assess the application of the DIA subgroup classifications in clinical practice. The psychometric properties of the DIA were examined using internal consistency analysis and Pearson's correlation analysis to assess the association of IGD symptoms (DIA) with Internet and smartphone addiction (K, SAS-SV, S, YIAT, O\_A), impulsivity (BIS-11), aggression (AQ), depression (BDI), anxiety (STAI), self-esteem (RSES), stress (DHQ), and temperament/character traits (JTCI). All statistical tests were performed using SPSS software version 21.0 (SPSS, Inc., Chicago, IL, USA).

#### *2.4. Ethical Approval*

All subjects and their caregivers gave their informed consent for inclusion before they participated in the study. The study was conducted in accordance with the Declaration of Helsinki, and the protocol was approved by the Institutional Review Board (IRB) for human subjects of Uijeonbu St. Mary's Hospital (UC150NMI0072), Eulji University Eulji Hospital (EMCS2015-05-020-001) and SMG-SNU Boramae Medical Center (16-2016-4).

#### **3. Results**

#### *3.1. Demographic Characteristics*

The study included 103 adolescents (mean age, 14.35 ± 1.43 years; 70.9% males) and their caregivers (mean age 46.57 ± 7.69 years; 7.6% males). The Korean Wechsler Intelligence Scale for Children-fourth edition (K-WAIS-IV, 100.03 ± 13.52) was administered to 64 subjects under 16 years of age, and the Korean version of the Wechsler Adult Intelligence Scale, 4th edition (K-WISC-IV, 107.13 ± 12.51) was administered to the remaining subjects. Subjects with DIA total scores of 0–2 were categorized as mild risk, those with scores of 3–4 were moderate risk, and those with scores of 5–10 were classified as addicted. We found no significant differences in sex, age, caregiver's age, or IQ among the three subgroups (mild risk, moderate risk and addicted).

#### *3.2. Psychometric Properties of the Diagnostic Interview for Internet Addiction*

The convergent validity of the DIA was assessed by comparing total DIA scores with scores on other scales measuring Internet and smartphone addiction. The correlation coefficients between the DIA and the other scales were as follows: K-scale, 0.426 (*p* < 0.01); SAS-SV, 0.205 (*p* < 0.05); S-scale, 0.234 (*p* < 0.05); Y-IAT, 0.390 (*p* < 0.01); and O\_A, 0.343 (*p* < 0.01; Table 2).

**Table 2.** Correlation analysis between Diagnostic Interview for Internet Addiction and Internet and Smartphone related scale.


\* *p* < 0.05, \*\* *p* < 0.01. Note. *n* = 103. DIA = Diagnostic Interview for Internet Addiction; K = Korean Scale for Internet Addiction; SAS-SV = Smartphone Addiction Scale-Short form Version; S = Korean Smartphone Addiction scale; YIAT = Young's Internet Addiction Test; O\_A = Internet Addiction Proneness Scale for Adolescents.

Additionally, the correlation analysis performed to examine construct validity revealed significant relationships of the DIA score with the BDI-II (*r* = 0.285, *p* < 0.01), STAI\_X1 (*r* = 0.294, *p* < 0.01), RSES (*r* = −0.312, *p* < 0.01), BIS-II (*r* = 0.278, *p* < 0.01), AQ (*r* = 0.256, *p* < 0.05), and DHQ (*r* = 0.283, *p* < 0.01) scores (Table 3). These findings suggest that the presence of several IGD symptoms is associated with higher levels of depression, anxiety, impulsivity, aggression, stress, and low self-esteem.

**Table 3.** Correlation analysis between the Diagnostic Interview for Internet Addiction and Clinical symptoms.


\* *p* < 0.05, \*\* *p* < 0.01. Note. *n* = 103. DIA = Diagnostic Interview for Internet Addiction; BDI-II = Beck Depression Inventory-II; STAI\_X1 = State-Trait Anxiety Inventory X−1; RSES = Rosenberg Self-Esteem Scale; BIS = Barratt Impulsiveness Scale-II; AQ = Aggression Questionnaire; DHQ = Daily Hassles Questionnaire.

#### *3.3. Comparison of Clinical Variables among the Diagnostic Interview for Internet Addiction Subgroups*

The participants were divided into three subgroups (mild risk, moderate risk, addicted) according to their DIA total score. Comparisons of the clinical variables (internalizing/externalizing problems and temperament and character traits) among the DIA subgroups are shown in Table 4. We found significant differences in Internet and smartphone addiction, depression, anxiety, self-esteem, impulsivity, aggression, and stress among the subgroups on all of the scales except the SAS-SV. The moderate risk and addicted group had significantly higher levels of Internet and smartphone addiction, anxiety, depression, impulsivity, aggression, and stress and lower self-esteem compared with the mild risk group. Moreover, scores on the JTCI, which measures temperament and character traits, revealed that the mild risk group had significantly higher levels of persistence, self-directedness, and cooperativeness than the addicted group did.



\* *p* < 0.05, \*\* *p* < 0.01. Note. No. of mild risk group = 19; No. of moderate risk group = 26; No. of addicted group = 58. Bonferroni post-hoc test results are reported. DIA = Diagnostic Interview for Internet Addiction; K = Korean Scale for Internet Addiction; S = Korean Smartphone Addiction scale; YIAT = Young's Internet Addiction Test; O\_A = Internet Addiction Proneness Scale for Adolescents; BDI-II = Beck Depression Inventory-II; STAI\_X1 = State-Trait Anxiety Inventory X−1; RSES = Rosenberg Self-Esteem Scale; BIS = Barratt Impulsiveness Scale-II; AQ = Aggression Questionnaire; DHQ = Daily Hassles Questionnaire; JTCI\_P = Junior Temperament and Character Inventory\_Persistence; SD = Self-Directedness; C = Cooperativeness.

#### **4. Discussion**

In this study, we examined the psychometric properties of the DIA and verified the application of the DIA severity classifications in Korean adolescents. The main findings and implications of our study are as follows.

First, in this study, the reliability and validity of DIA were verified. The Cronbach alpha of DIA was 0.72, which means that the DIA meets the internal consistency reliability. Also, we found positive associations between the total DIA scores and scores on other Internet and smartphone addiction scales in Korean adolescents. These findings support the convergent validity of the DIA and are consistent with those of previous studies investigating the relationship between IGD symptoms and Internet/smartphone addiction scales [9,10,45]. In particular, Cho et al. (2014) found that the self-diagnostic Internet addiction scale based on the DSM-5 criteria for IGD was positively correlated with the K-scale, which measures Internet addiction, in Korean middle school students (ages 13 and 14 years) [45]. Kim et al. (2016) reported similar findings in adults [46].

We assessed the construct validity of the DIA by examining the relationship of DIA results with depression, anxiety, stress, impulsivity, aggression, and self-esteem. We found that the subjects who reported more severe IGD symptoms as measured by the DIA had higher levels of depression, anxiety, stress, impulsivity, and aggression. Several studies have shown a relationship between IGD symptoms and psychiatric comorbidity; in particular, one review paper found 92% studies described a significant correlation between IGD symptoms and anxiety and 89% studies with depression [47]. Depression is strongly associated with IGD symptoms, and is the most significant factor associated with the development of online gaming addiction in adolescents [18,48–50]. Gentile et al. (2011) reported that depression and anxiety levels increased in adolescents who became and remained problematic gamers, and Mehroof et al. (2010) found that state anxiety was significantly associated with online gaming addiction [16,22]. Moreover, Internet addiction may contribute to stress. Individuals who experience anxiety and stress have difficulty communicating and interacting with others in healthy and positive ways [51,52]. Furthermore, several studies have shown that impulsivity and aggression are related to Internet addiction in adolescents and increase vulnerability to IGD [22,53–55]. We found a negative correlation between the total DIA score and self-esteem, suggesting that more severe IGD symptoms are associated with lower self-esteem. Similarly, several studies have found a significant relationship between low self-esteem and IGD symptoms such that low self-esteem has been shown to predict the emergence of Internet addiction [25]. Therefore, in this study, the psychometric properties of the DIA were verified by examining the reliability and validity of DIA.

In the context of the application of the DIA classifications in clinical practice for Korean adolescents, we found that subjects in the moderate risk and addicted group had lower self-esteem and significantly higher levels of Internet and smartphone addiction, anxiety, depression, impulsivity, aggression, and stress than did the mild risk group. These findings are similar to those of previous studies mentioned above [16,18,22,47–55], suggesting that early intervention is required when the total DIA score is 3 or above. This is because adolescents who were included in the moderate risk group reported internalizing and externalizing problems similar to the addicted group in DIA.

Previous studies have found associations between Internet addiction and personality traits (e.g., sensation seeking, reward dependence, and self-directedness) [26–28]. Similarly, we found that persistence, self-directedness, and cooperativeness as measured by the JTCI were significantly higher in the mild risk group than in the addicted group. In previous studies of the relationship between personality traits and IGD symptoms, Montag et al. (2011) found that IGD scores were negatively correlated with self-directedness and cooperativeness, and Jimenez-Murcia et al. (2014) found that low self-directedness predicted high IGD scores in video game users [56,57]. These findings indicate that subjects who are in the mild risk group, as classified by the DIA, tend to persist in behavior without sustained reinforcement and are able to control their behavior, unlike those who are in the internet/games/SNS addicted group. Moreover, several studies have shown that previous reports of a relationship between IGD symptoms and novelty seeking, a personality feature linked to impulsivity [58], were inconsistent [16,59,60]. It may be that IGD symptoms are unrelated to novelty seeking, or they may be associated with both high and low novelty seeking, leading to the appearance of no relationship. We found no significant differences in novelty seeking among the DIA subgroups in the present study. These findings suggest that impulsive people may not derive fulfillment or enjoyment from games such as the massively multiplayer online role-playing games (MMORPGs), depending on their psychological profile. Indeed, Billieux et al. (2015) found that the behaviors of online gamers were heterogeneous; individuals with IGD had various psychological profiles, including with regard to novelty seeking, and IGD symptoms differed according to these profiles [61].

Our study has several limitations. First, our subjects were screened using an Internet/smartphone addiction scale. Thus, it may be difficult to generalize our findings to non-clinical populations because our study did not include a control group. In addition, the proportion of the addicted group was about 80% or more, because the internet/games/SNS high-risk users were recruited in this study. Therefore, additional analysis (e.g., factor analysis, ROC curve etc.) should be made to propose a cut-off score of DIA or to make it more useful in a clinical setting. Second, previous studies have shown that the severity of IGD differs among game genres [62,63]. Lope-Fernandez et al. (2014) reported that subjects who played MMORPGs spent more time playing and had significantly higher scores on the

IGD-20 [64]. We did not investigate differences in game genres; further research is needed to examine the DIA subgroups according to the game media type (e.g., internet, games, SNS etc.), various game genres, gaming patterns, and causes of use.

Despite these limitations, we examined the psychometric properties of the DIA, a semi-structured tool, and investigated the relationship between DIA total score and a wide range of clinical characteristics. In addition, although the DSM-5 diagnostic criteria for IGD require the presence of five or more symptoms, our findings suggest that early intervention and continuous observation are advisable for individuals with three or more symptoms according to the DIA.

**Author Contributions:** Study concept and design: H.R. and J.-S.C.; collection of data: H.R. and J.G.K.; analysis and interpretation of data: H.R., J.Y.L., A.R.C., S.J.C. and M.P.; statistical analysis: H.R.; writing–original draft: H.R., and J.-S.C.; study supervision: J.-S.C., Y.-S.K., and S.Y.B.; access to data: All authors.

**Funding:** This research was funded by the Korean Mental Health Technology R&D Project, Ministry for Health and Welfare, Republic of Korea (HM14C2603) and the National Research Foundation of Korea (2014M3C7A1062894).

**Conflicts of Interest:** The authors declare no conflicts of interest.

#### **References**


© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

#### *Article*
