**Arnold Alejandro Tafur-Mendoza 1, Julio César Acosta-Prado 2,3,\*, Rodrigo Arturo Zárate-Torres <sup>4</sup> and Duván Emilio Ramírez-Ospina <sup>3</sup>**


Received: 30 May 2020; Accepted: 24 July 2020; Published: 10 August 2020

**Abstract:** The use of the Internet has been gradually and unstoppably gaining ground in all areas of life, from recreational activities to how social relations are established. However, the existence of clinical cases indicates that the addictive use of the Internet is a problem that seriously affects some people. Among the instruments that measure this construct, the Internet Addiction Test (IAT) stands out. However, instrumental studies of this test are scarce in Latin America. The present study sought to analyze the psychometric properties of the IAT in a sample of 227 Peruvian undergraduate university students. Confirmatory factor analysis was used to provide validity evidence based on the internal structure, and evidence based on the relationship with other variables was also provided. Reliability was estimated through the ordinal alpha coefficient. The results indicated that the IAT adequately fits a bifactor model (with two specific factors, time/control and stress/compensate), obtaining good levels of reliability. Additionally, the IAT scores correlate significantly with the average number of hours per day on the internet and social skills. The results lead to the conclusion that the scores in the IAT have evidence of validity and reliability for its use.

**Keywords:** internet addiction test; university students; Peruvian sample; psychometric properties

#### **1. Introduction**

In the framework of a society in which both communication and the free flow of information are closely related to the development of the network, it is necessary to know to what extent reality and the virtual sphere intermingle. From the lowering of costs, the multiplicity of ways of accessing the network, and the advent of social networks, the Internet grows exponentially every day [1]. As studies in Sweden [2] and Spain [3] seem to indicate, the fierce proliferation of the network as a means of communication has brought negative consequences (cyberbullying, problematic Internet use, sexting, nomophobia, etc.) that have a stronger impact on the young population, having identified a series of problems among which stand out the addiction to this environment and that affect above all the social sphere of the individual.

However, the use of the Internet has expanded gradually and unstoppably in all areas of life [4]. The existence of clinical cases indicates that maladaptive use of the Internet is an existing problem that seriously affects some people, mainly those with special emotional needs and young people and adolescents [5]. The use or abuse of the Internet arises from disciplines, such as psychology or psychiatry [6]. In this sense, it is not surprising that the term Internet addiction was used for the first time by the psychiatrist Ivan Golberg in 1995 [7]. The literature on problematic use of the Internet shows a very varied terminology to describe the problems derived from the use of the Internet, among

which are: Computer addiction, excessive use of the Internet, pathological use of the Internet, Internet dependence, compulsive use of the Internet, disorder by impulsive use, and compulsive use of the Internet or Internet addiction [8,9].

Kimberly Young [10], after reviewing a series of investigations, which indicated that some online users became addicted to the Internet in the same way that others became addicted to drugs, alcohol, or gambling, suggests the need to empirically investigate the concept of addictive use of the Internet. In addition to the review, the study sought to identify whether excessive use of the Internet can be considered addictive and to know the magnitude of the problems created by these abuses. However, Young's approaches to Internet addiction sparked a controversial debate among doctors and academics at the time [11].

On the other hand, there are proposals [12] that suggested that Internet addiction is a disorder that should have been considered in the fifth edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-V) since it is an impulsive-compulsive spectrum disorder [13], which consists of the use of computers. However, these recommendations or proposals were not reflected in the DSM-V, since the term Internet addiction does not appear in the manual [14]. Among the diagnoses referenced in the DSM-V, Internet gaming disorder (IGD) is strongly related to the pathological nature of Internet use. The excessive use of the Internet not only involves the reproduction of online games (for example, it also implies the excessive use of social networks, such as Facebook or Twitter). Therefore, IGD is within the Internet addiction. Thus, research on other excessive uses of the Internet should follow guidelines analogous to those suggested with IGD [14].

Among the theoretical models proposed to understand Internet addiction, Davis' cognitive-behavioral model [15] is the one that has had the most development and has been empirically tested in different contexts [16]. Two parts are distinguished in this model, one specific and the other general. The first involves excessive use of an aspect of the internet, while generalized use encompasses multiple excessive uses of the Internet [15]. Generalized use is associated with social support and social service uses of the Internet, where maladaptive cognitions are a strong predictor of this component and to a lesser extent of specific use [17]. Under this model, symptoms of Internet addiction are primarily affective or behavioral and are usually preceded or caused by cognitive symptoms, mediated by specific and general pathological Internet use [15,16]. The value of the cognitive-behavioral model lies in that it contemplates a continuum of severity regarding Internet use, allowing a better understanding by mental health specialists of the way and degree to which the excessive use of the internet can affect the lives of people [18].

To measure Internet addiction, various tests have been developed, which, mainly, are based on the diagnosis of this disorder, including the Problematic Internet Use Questionnaire (PIUQ) [19], the Compulsive Internet Use Scale (CIUS) [20], Internet-Related Problem Scale [21], and the Internet-Related Experiences Questionnaire (IREQ) [22]. The PIUQ, made up of 18 items, was built for the general population and measures problems related to Internet use through three subscales: Obsession, neglect, and control disorder. The CUIS is made up of 14 items that measure the severity of compulsive Internet use. The Internet-Related Problem Scale has 20 items that address DSM-IV symptoms for substance abuse: Tolerance, escape from other problems, reduced activities, loss of control, negative effects, withdrawal, craving, and introversion. The IREQ was developed in middle-school students and is made up of 10 items that measure possible Internet addiction based on intrapersonal and interpersonal experiences. However, the test that has had the most development and that has been adapted to various contexts is the Internet Addiction Test (IAT) [23]. Thus, the objective of this study was to analyze the psychometric properties of the IAT in a sample of Peruvian undergraduate university students.

#### **2. Literature Review**

For this study, in a search conducted in Scopus and Web of Science (WoS) on studies of the psychometric properties of the IAT, 46 studies were found (April 2020). Table 1 presents a summary of 39 of the 46 studies found, which were those that analyzed the internal structure of the IAT, an aspect that does not have a consensus in all the studies carried out. Most of the studies were carried out in Europe and Asia; in North America, only one study was found in the United States and another in Canada, while, in Latin America, the IAT has been studied in Colombia and Chile. Regarding the populations where these studies have been carried out, the majority were in university students, probably due to the accessibility of the sample and that it is the age group that spends the most time on the Internet (young people between 18 and 25 years old).


**Table 1.** Previous studies on psychometric properties of the Internet Addiction Test (IAT).

Note. <sup>a</sup> Hierarchical model; <sup>b</sup> Bifactor model; <sup>c</sup> α coefficients of the factors; <sup>d</sup> Construct reliability; <sup>e</sup> Rasch model (person reliability); <sup>f</sup> ω coefficients of the factors; PCA = Principal Component Analysis; EFA = Exploratory Factor Analysis; CFA = Confirmatory Factor Analysis.

The reliability of the IAT has usually been estimated through the alpha coefficient, obtaining, in most cases, acceptable values above 0.70, and even exceeding 0.90. However, most of these studies have not considered the assumptions that this coefficient has, such as the one-dimensionality of the data, that the measurement model is tau-equivalent, and that there are no correlated errors [63]. Violation of any of these assumptions would distort the values reported in these studies. Additionally, there is no consideration of the ordinality of the items, which must be considered when working with Likert-type scales [64]. A meta-analysis of the internal consistency of the IAT (reliability generalization) indicates a combined alpha of 0.90 for samples of university students [65].

Regarding the internal structure of the IAT, the literature shows factorial models with one to six factors. Although, the diversity of models increases, because two studies may have the same number of factors, but they differ in the number of items used (not all studies end their analyses with the original 20 items but eliminate some of them because they do not fit their proposed model), in the addition of correlated errors (in many cases to improve the adjustment indices in the confirmatory factor analysis, without taking into account the substantive justification of the correlated errors [66]) or in the distribution of the items by the factor (two models can coincide in the number of factors and items, but the grouping of the items is different). This large number of models does not allow for a uniform IAT structure to be available, and for studies seeking to study this aspect in a particular sample, they have to resort to exploratory techniques or to test a large number of existing models.

An aspect linked to the above is the analysis technique used to find the factor structure in these studies. Almost half of the studies reported in Table 1 indicate that the principal component analysis (PCA) has been performed as the first or only technique, which is not suitable for psychological constructs, since it is a variable reduction technique; therefore, it seeks to explain the greater proportion of explained variance, considering both the variance shared by the items and the error variance. On the contrary, the factor analysis only considers the extraction of the factors the variance that the items share among themselves [67]. The use of the PCA could lead to obtaining structures that do not fit the data, especially since the measurement nature of the data is not considered.

In Peru, only one study of the psychometric properties of IAT [68] in middle-school students between the ages of 13 and 19 years was found. Item 1 was removed from the scale because it presented a low discriminatory capacity; therefore, this version had 19 items. Reliability was estimated through the alpha coefficient, obtaining a value of 0.870. On the other hand, the internal structure was evaluated by PCA, finding a factorial solution of four components. This structure was corroborated in the same sample through confirmatory factor analysis (CFA). It is important to note that the use of middle-school students and university students is different. University students use the Internet primarily for information search; also, they spend more time online and have a wider range of Internet uses than middle-school students [69].

Another source of validity evidence that has been explored in these background studies is evidence based on the relationship with other variables, specifically convergent evidence, where the hours of daily Internet use was used as a measure, in addition to the IAT [38,41,45,50]. In these studies, correlation coefficients between 0.29 and 0.48 were found. Additionally, Internet use has an impact on a person's social life, and studies have shown that Internet addiction is negatively related to social skills [70–72].

Considering all the above, the IAT is one of the most used instruments for measuring Internet addiction and that, in Peru, it has only worked with middle-school students in the adolescence stage but not in other samples, such as university students, which is precisely in this group where most adaptations have been made in other countries. Furthermore, the statistical procedures used in previous studies may be questionable and, in some cases, inadequate. Thus, the IAT does not have a version for Peruvian university students, so its psychometric properties are not known in this population and the extrapolation of results in other contexts would not be appropriate, since validity and reliability are not inherent in a test but correspond to the specific interpretations and uses of the scores obtained in a test [73]. Therefore, the objective of this study was to analyze the psychometric properties of the IAT in a sample of Peruvian undergraduate university students.

#### **3. Materials and Methods**

#### *3.1. Design*

According to the classification system of research designs in psychology [74], the objective of the study corresponds to an instrumental investigation, since the psychometric properties of a psychological instrument are analyzed in a specific sample. For the development of the study, various standards, guidelines, good practices, and recommendations were followed in instrumental studies in the behavioral and health sciences [73,75].

#### *3.2. Participants*

The selection of the participants was carried out through an intentional non-probability sampling [76], where the individuals in the sample were directly and selectively chosen, seeking to obtain similar proportions in each of the categories of the variables sex, year of study, and academic discipline. Regarding the sample size, a priori statistical power analysis was performed to determine the minimum recommended sample size. Statistical power is the ability of a statistical test (for example, U Mann–Whitney or H Kruskal–Wallis) to reject a null hypothesis when it is false; in other words, it is the probability of not committing the type II error [77]. The input parameters for this analysis were based on a simple two-tailed correlational model (similar to the one that will be carried out in the collection of validity evidence based on the relationship with other variables), the significance level (α) being 0.05, the expected effect size equal to 0.20 (recommended minimum value representing practical significance in social science data [78]), and an expected statistical power of 0.80 (recommended minimum in behavioral science [79]). The recommended minimum sample size was 191 (Figure 1).

**Figure 1.** A priori statistical power analysis to determine the minimum recommended sample size.

The final sample was made up of 227 Peruvian undergraduate university students from a public university located in Metropolitan Lima (Peru). The ages of the participants were between 18 and 40 years (M = 20.81, SD = 2.92). The highest proportion of students were females (57.30%), in their second year of studies (30.00%), and belonged to professional careers in the engineering area (19.80%). Table 2 presents a more detailed description of the sample characteristics.


**Table 2.** Sociodemographic characteristics of participants (*n* = 227).

#### *3.3. Instruments*

#### 3.3.1. IAT—Internet Addiction Test

The Spanish version of the IAT was used (Appendix A), adapted to a sample of Colombian university students [32]. The IAT is made up of 20 items, all in a positive sense, on a five-point Likert-type scale (1 = rarely; 2 = occasionally; 3 = frequently; 4 = usually; 5 = always), the minimum score being 20 and the maximum score 100, where the student was asked to choose the alternative that best suits their reality. The IAT items were reviewed by the authors of this study, pointing out that it was not necessary to adjust their wording. To corroborate this assumption, a pilot study was carried out with 10 university students, who indicated that they had no problems in understanding the items.

In the adaptation study, the total scale had a good level of reliability (α = 0.89). Likewise, through an analysis of the main components, they found an internal structure of three factors (consequences for the use of the Internet, cognitive-emotional dimension, and time control), which together had an explained variance of 47.80%, and correlated positively with the number of hours of daily Internet access, although the magnitudes were low [32].

For the application of this study, in addition to the IAT, a sociodemographic and Internet usage file was added to collect information on the career that the participants studied, as well as the year of studies they were studying, age, gender, and the average daily hours spent on the Internet.

#### 3.3.2. EHS—Social Skills Scale

The Social Skills Scale, developed by Gismero [80], is made up of 33 items. Five items are written in the direct sense, while 28 items are in an inverse sense, seeking to detect the lack of assertion or deficit in social skills. The original instrument is composed of six factors: (1) Self-expression in social situations, eight items; (2) defense of one's rights as a consumer, five items; (3) expression of anger or disagreement, four items; (4) assertiveness, six items; (5) making requests, five items; and (6) starting interactions with the opposite sex, five items. Each item is answered using a four-point response format (A = I do not identify myself at all; B = Rather it does not have to do with me, even if it ever happens to me; C = It describes me approximately, although I do not always act or feel like this; and D = Strongly agree, and I would feel or act like this in most cases). A higher global score indicates that the person has a higher level of social skills and better insertion in various contexts or situations.

This instrument has been studied in various countries, presenting adequate psychometric properties, good levels of reliability, adequate adjustment to the six-factor structure, and discriminant evidence [81]. In the present study, the scores on the items showed good internal consistency at the global level (ω = 0.890). Likewise, in the factors, the omega coefficient (ω) varied from 0.500 to 0.748.

#### *3.4. Procedure*

The data collection process began with the request of permits, both to the directors of the schools of each academic discipline and the teachers in charge of some courses for the application of the instrument in classrooms. Before the application of the IAT, the students were given an informed consent form that contained the objective of the evaluation and where they were guaranteed the confidentiality and anonymity of their answers, as well as the possibility of withdrawing from their participation at any time of the evaluation without consequence. Only those students who voluntarily signed the informed consent participated in the study.

The application was collective, with an average duration of 15 min. Data collection lasted approximately five weeks. At the end of each evaluation, the examiners reviewed the application protocols to verify that there are no unanswered items or items with more than one marked answer option. If one or both situations described occurred, the evaluated person was asked to correct and provide their definitive answer. After data collection, each evaluation protocol was coded for the elaboration of the database in a Microsoft Excel 2016 spreadsheet.

During the evaluation process, the ethical guidelines for working with humans outlined in the code of ethics of the American Psychological Association (APA) were followed. Additionally, the ethical principles of the Declaration of Helsinki were respected, including its recent updates and regulations for research on human beings.

#### *3.5. Data Analysis*

The statistical analysis was divided into four stages: (1) Item analysis; (2) validity evidence based on the internal structure of the test through CFA; (3) validity evidence based on the relationship with other variables from the convergent evidence; and (4) reliability analysis using the internal consistency method.

The descriptive analysis of the items was performed using the mean and standard deviation, as measures of central tendency and dispersion, respectively. As descriptive measures of the distribution of the items, the skewness and kurtosis coefficients were used, where values between −2.00 and 2.00 indicate that the items follow a normal distribution [82]. Likewise, item discrimination was estimated through the item-rest polyserial correlation, which considers the ordinality of the items, considering values above 0.30 as adequate [83]. Finally, possible floor and ceiling effects in the items were analyzed, that is, to identify the proportion of participants who chose the lowest alternative (floor effect) and the highest (ceiling effect), taking as acceptable effects those that were between 1% and 15% [84].

The CFA was carried out using the 23 models presented in Table 1, considering only those models made up of the 20 original items and whose factors had at least three items. The estimation method used was the weighted least squares with mean and variance adjusted (WLSMV), appropriate for observable ordinal variables and that performs well against slightly non-normal underlying distributions [85]. The WLSMV involves the use of the diagonal weighted least squares (DWLS) estimator with robust standard errors and a statistical test with adjusted mean and variance (using a scale-shifted approach). To assess the level of adequacy of the models, the following fit indices were used [86,87]: SSχ2/*df* < 2.00, root mean square error of approximation (RMSEA) < 0.08, comparative fit index (CFI) > 0.90, Tucker–Lewis index (TLI) > 0.90, standardized root mean square residual (SRMR) < 0.08, and weighted root mean square residual (WRMR) < 1.00.

The convergent evidence involved correlating the scores obtained in the IAT with another measure that seeks to evaluate the same construct or other construct with which it is expected is correlated; in this case, these measures were the average of daily hours that the study participants spent on the Internet and a social skills scale. Before selecting the correlation statistic to be used, the presence of outliers in these two variables with which the IAT scores were correlated was examined. Univariate analysis of outliers was visually inspected through boxplots for each variable (including the six dimensions of the social skills scale). A few outliers were found in two factors of the social skills scale and in the average of daily hours that the study participants spent on the Internet. Therefore, a robust statistic was chosen for the correlation between variables, the skipped correlation coefficient. This coefficient is robust to slight changes in any distribution and has the advantage of treating outliers in a way that considers the general structure of a data set [88]. As a measure of the effect size, the coefficient of determination was used, considering its interpretation values of 0.04, 0.25, and 0.64 as the minimum, moderate, and strong effect, respectively [78].

The internal consistency of the items was estimated using the ordinal alpha coefficient [89] since it works with the inter-item polychoric correlation matrix. Values above 0.70 were considered acceptable [90].

A priori statistical power analysis to determine the minimum sample size required for the study was performed using the G\*Power 3.1.9.7 software [91]. The other analyses were performed in the R software version 4.0.2 (R Foundation for Statistical Computing, Vienna, Austria, 2016) [92], using the following packages: pacman 0.5.1 [93], readxl 1.3.1 [94], tidyverse 1.3.0 [95], psych 1.9.12.31 [96], lavaan 0.6–6 [97], BifactorIndicesCalculator 0.2.0 [98], semTools 0.5–2.925 [99], and WRS 0.36 [100].

#### **4. Results**

#### *4.1. Item Analysis*

Table 3 presents the results of the item analysis. The means of the items were between 1.352 (item 20) and 3.084 (items 1 and 7), indicating that the participants' responses were concentrated on the lowest alternatives (rarely, occasionally, and frequently). Likewise, the standard deviation fluctuated between 0.658 (item 20) and 1.211 (item 7), showing low variability in the data. Regarding the skewness and kurtosis measures, most of the items presented values between −2.00 and 2.00. However, items 9, 15, 19, and 20 showed an excess of kurtosis, and these last two also indicated an excess of skewness. Therefore, these four items showed a deviation from a normal distribution.


**Table 3.** Item analysis for the Internet Addiction Test (IAT).

Note. M = Mean; SD = Standard Deviation; Sk = Skewness; Ku = Kurtosis.

#### *Int. J. Environ. Res. Public Health* **2020**, *17*, 5782

Concerning the item-rest polyserial correlation, all the items were above the threshold of 0.30, ranging from 0.371 (item 7) to 0.684 (item 15), indicating good discrimination by the IAT items. On the other hand, in the analysis of the response options, a soil effect was found in all the items, except for items 1 and 7. However, regarding the ceiling effect, most of the items had acceptable values (between 1% and 15%). These effects reflect the tendency of the participants to choose the lowest response options, being the most prominent in items 19 and 20, where more than 70% of the sample chose the response option "rarely". On the contrary, the "always" alternative was selected in a low percentage, even reaching 0% in some items (items 6, 8, 10, 11, 15, and 20).

#### *4.2. Validity Evidence Based on the Internal Structure*

The models tested were taken from Table 1. However, of the 39 studies presented, the studies selected were those where the factor structure was configured by all the items of the IAT (20 items) and that the proposed factors have at least three items in its composition, which is recommended to achieve an adequate representation of a factor [86]. Twenty-four studies met the two requirements and their factor models were tested. Four studies had the same unifactorial structure [29,45,51,53]. Whereas, five models [26,32,38,46,59] presented problems in the analysis: Covariance matrix of latent variables was not positive definite [59], some estimated latent variable variances were negative [26,38,46], or the model did not converge [32].

Table 4 presents the fit indices of the models tested. The bifactor model of Watters et al. (2013) [34] was the one that obtained the best results (SSχ2/*df* < 2.00, RMSEA < 0.08, CFI > 0.90, TLI > 0.90, SRMR < 0.08, and WRMR < 1.00). Figure 2 shows the factor structure of the bifactor model, made up of a general factor that measures Internet addiction through 20 items and two specific factors, time/control and stress/compensate, the first factor consisting of five items (1, 2, 7, 16, and 17) and the second factor of 11 items (3, 4, 9, 10, 11, 12, 13, 15, 18, 19, and 20). Figure 2 also presents the factor loadings of the items in the general factor and the corresponding specific factors.


**Table 4.** Confirmatory factor analysis for the IAT.

Note. RMSEA = Root Mean Square Error of Approximation; CI = Confidence Interval; CFI = Comparative Fit Index; TLI = Tucker–Lewis Index; SRMR = Standardized Root Mean Square Residual; WRMR = Weighted Root Mean Square Residual.

#### *Int. J. Environ. Res. Public Health* **2020**, *17*, 5782

**Figure 2.** Factorial structure of the bifactor model.

Additionally, complementary statistical indices were calculated that allowed assessment of the robustness of the general factor, as well as the contribution of specific factors [101]. The omega hierarchical was used for the general factor and the specific factors, expecting values above 0.70 (ωH) for the first and greater than 0.30 (ωHS) for the seconds. The results indicate a value of 0.704 for the general factor, 0.234 for time/control, and 0.478 for stress/compensate. The H coefficient was also calculated, considering values greater than 0.70 to be adequate. The general factor obtained a coefficient of 0.888: For the time/control factor it was 0.785 while, for the stress/compensate factor, the H coefficient was equal to 0.513.

The explained common variance (ECV) for the general factor was 0.612, while the factors (ECV of a specific factor concerning itself) showed an ECV of 0.368 for time/control and 0.562 for stress/compensate. On the other hand, the proportion of uncontaminated correlations (PUCs) was equal to 0.658. At the item level, the ECV index (I-ECV) was between 0.126 and 1.000. Thus, the results allow the use of this bifactor model [102].
