*4.3. Validity Evidence Based on Relations to Other Variables*

The convergent evidence was collected from the correlation between the two specific factors of IAT and the general factor with the average hours per day that study participants spent on the Internet and social skills. The minimum time of hours per day was less than one hour and the maximum 12 h (M = 3.368, SD = 2.277). The skipped correlation coefficients between the average daily hours on the Internet with Internet addiction (and time/control factor) were statistically significant (Table 5). Likewise, the effect size (skipped correlation coefficient squared) in the time/control factor and total Internet addiction was greater than 0.04, indicating a recommended minimum effect size that represents practical significance in social science data [78]. Regarding social skills, the correlations between Internet addiction (and its two factors) with social skills (and the defense of rights factor) were statistically significant (Table 5). Similar results were obtained for correlations between Internet

addiction (and stress/compensate factor) with self-expression, disagreement, and assertiveness (Table 5). The size of the effect between Internet addiction and social skills presented practical significance. These findings provide convergent evidence to the IAT.


**Table 5.** Interfactorial matrix, convergent evidence, and reliability of the IAT.

Note. \* *p* < 0.05; \*\* *p* < 0.01; \*\*\* *p* < 0.001.

On the other hand, the correlation between the specific factors of the IAT presented a minimum effect size (skipped correlation coefficient squared > 0.04). For the correlation between time/control and stress/compensate with Internet addiction, the effect size was strong (skipped correlation coefficient squared > 0.64). In all three cases, the correlations were statistically significant (Table 5).

#### *4.4. Reliability*

Reliability was evaluated by the internal consistency method, using for this purpose the ordinal alpha coefficient, which considers the ordinal nature of the items for its calculation. The specific factors and the general factor obtained satisfactory values, above 0.70 (Table 5). Likewise, the coefficients obtained were not remarkably high (0.90 or higher), indicating that the IAT, in the sample studied, does not include redundant items [103].

#### **5. Discussion**

The present study analyzed the psychometric properties of the IAT in a sample of Peruvian university students. The 20 items that make up the instrument presented adequate levels of discrimination, although the analysis of the response options indicated the presence of a floor effect in 18 items. Likewise, the levels of skewness and kurtosis were acceptable in most of the items. Regarding the validity evidence based on the internal structure, different models were tested through CFA, being the bifactor model with two specific factors (time/control and stress/compensate), the one that presented the best indexes of adjustment. Another source of validity evidence that was used was that based on the relationship with other variables, specifically the convergent evidence, finding statistically significant correlations between the two specific factors and the general factor with the average number of hours per day on the Internet and social skills. Finally, the reliability, estimated through the ordinal alpha coefficient, was acceptable for the general factor and the specific factors.

The items showed discrimination indexes above 0.30, which implies that each item is related to the other items taken together, which would also justify the presence of an underlying general factor. These results agree with a previous study [43], where all the items were higher than the cut-off point used in this study. However, other studies showed problems only with item 7 ("How often do you check your email before something else that you need to do?"), finding values of 0.170 [42], 0.250 [41], −0.098 [104], and 0.195 [32], while all the other items were greater than 0.30. In this study, item 7 had the lowest item-rest correlation (0.371).

Item 7 is problematic in the literature because it would be mainly relevant for university students or people whose jobs involve communication by this means. In university students, virtual communication occurs mainly with their professors by email, for sending papers, receiving corrected papers, or notifying activities on virtual platforms, as well as with their peers to exchange information (e.g., articles or books) or share working documents. It is important to highlight that, in this study, the item-rest correlation was estimated using the polyserial correlation, unlike previous studies that worked with the Pearson correlation coefficient. The polyserial correlation coefficient considers the ordinality of the items, being more precise in estimating the degree of item discrimination.

The presence of a floor effect in the items, together with low averages in these, shows the low level of Internet addiction of the participants. This result may be due to the characteristics of the study participants, who belong entirely to a public sector university, where most of the population belongs to a medium or medium-low socioeconomic level, having some limitations regarding the Internet accessibility as it involves spending on devices (cell phones, tablets, laptops, etc.) and mobile data for connectivity. On the other hand, Internet addiction is a clinical construct, so its presence in a non-clinical population (university students) should be low under normal conditions.

Regarding the internal structure of the IAT, this study recollected a large part of the models reported in previous studies to test them and find out how the IAT is structured in the Peruvian sample. The bifactor model (one general factor and two specific factors) [34] presented the best fit. In the reviewed literature, the other study that reported a bifactor model found a general factor and three specific factors [56]. Comparing the results obtained in this study with those reported by Watters et al. [34], many points of agreement were observed, both at the level of fit indices and in factor loadings. In both studies, the factor loadings had higher values in the general factor than in the specific factors. In both studies, several items presented factor loadings below 0.30 or 0.40, which are the usual cut-off points in this type of study. However, the results of the present bifactor model indicate that those items that had low factor loadings in the general factor, had higher factor loadings in the specific factors, and vice versa.

The characteristic described above is typical of bifactor models, which allow the simultaneous evaluation of the influence of the general factor and specific factors on the variability of each item. The evaluation of the bifactor model through the omega hierarchical (ω<sup>H</sup> and ωHS) and H coefficients (also known as construct reliability), ECV, I-ECV, and PUC, provided evidence regarding the relevance of the model. The remaining 15 models reviewed and tested presented convergence or adjustment problems. The difference in fit between the previous studies and the present study is probably due to the different estimation methods used. In this study, the WLSMV estimator was used that considers the categorical nature of the items. Furthermore, many of the previous studies had problems in choosing the appropriate statistical technique or methods.

The use of the PCA for work with psychological variables is not appropriate since in its conception it considers formative models, useful in other disciplines (economics, marketing, among others), where it seeks to group indicators or reduce the number of variables. By contrast, factor analysis works with reflective models, where an underlying variable (factor) causes certain behaviors (indicators or items). For the factor analysis, the exploratory or non-restrictive version involves making a series of decisions during the analysis, the most critical being the determination of the number of factors. Additionally, the choice of the rotation method should be justified by how the factors are related. In the reviewed antecedents, the use, in most of the studies, of the "Little Jiffy" was observed, which supposes a routine of analysis in the three mentioned aspects: PCA, eigenvalues greater than one (method to choose the number of factors) and Varimax rotation (consider that the factors are not correlated). The use of Little Jiffy has been heavily criticized and its use is not recommended, as it may lead to the acceptance of erroneous factor models [67].

From a theoretical point of view, the time/control factor is related to behavioral symptoms (e.g., neglect household chores to spend more time online), while the stress/compensate factor is made up of cognitive and affective symptoms (e.g., block out disturbing thoughts about your life with soothing

thoughts of the Internet or snap, yell, or act annoyed if someone bothers you while you are online). This bifactor model is framed within the cognitive-behavioral perspective, explaining the symptoms of Internet addiction (of varied nature) from specific and generalized uses, which simultaneously influence the symptoms [15]. The model obtained in this study theoretically differs from the models presented in the literature review, because a component (behavioral, cognitive, or affective) is not emphasized, but rather, the components are worked on simultaneously.

Convergent evidence from the IAT was also provided, finding statistically significant correlations with the hours of daily Internet use, and time/control and total Internet addiction presented a recommended minimum effect size that represents practical significance in social science data. These results were like those obtained by other researchers [38,41,45,50]. Additionally, the IAT negatively correlates with social skills measures (total score and self-expression, disagreement, and assertiveness factors). Previous studies also report these relationships with similar degrees of correlation. Internet addiction is associated with greater difficulties in social skills, probably since the emotional burden produced by being connected to the Internet interferes with social aspects. In this way, the development of social skills is left aside due to the few social interactions that the subject experiences, since most of his time is online [2,71].

Regarding the reliability of the scores on the IAT, the ordinal alpha coefficient showed acceptable levels for the general factor and the specific factors. On this point, most of previous studies coincide, including meta-analytical studies [65].

Regarding the limitations of the study, the main one focuses on the size and variety of the sample. Regarding the first aspect, although globally, the sample size is justified in an a priori statistical power analysis, the number of participants within the groups of sociodemographic variables is small, which limits the possibility of carrying out additional analyses in the items. For example, knowing the differential functioning of items or knowing the factorial invariance of the IAT. Regarding the variety of the sample, the students belonged to a public university; therefore, they share various characteristics that make it a homogeneous group, and therefore, the variability in the responses to the items was low.

To know and deepen other characteristics of the IAT, future studies should focus their objectives on analyses that provide evidence of its clinical utility. In this way, the appropriate cut-off points should be determined to be able to classify people addicted to the Internet and, in turn, evaluate the intensity of this addiction. Likewise, it is necessary to previously know how the instrument works in people with a presumption of Internet addiction, as well as to what extent it is related to other tests that measure clinical constructs (e.g., depression or anxiety), being relevant measures to obtain validity evidence based on relations to other variables. Therefore, working with clinical samples is a necessity, since the IAT could have different uses in the diagnosis and treatment of Internet addiction.

Likewise, the study of the IAT in non-academic populations must be accompanied by a review of the content of the items, since some of them may only be valid for the population of university students, for example, item 6 "How often do your grades or school work suffer because of the amount of time you spend online? ", being, in this case, the rewriting of the item or its exclusion from the test. Additionally, given the complexity of the IAT structure, other multivariate techniques could be tried to corroborate what was found here or to propose more stable structures. Techniques, such as network analysis, exploratory structural equation modeling (ESEM), or Bayesian approaches to factor analysis, would help in this regard. Regarding reliability, it is relevant to obtain evidence on temporal stability (test-retest reliability), particularly useful in the IAT, due to the high variability that scores in this type of test can have.

#### **6. Conclusions**

This study represents a contribution to the study of the IAT in Latin America, where it has been little studied, unlike other contexts, such as Europe or Asia. The findings indicate that the IAT, in the sample of Peruvian university students, is made up of a general factor and two specific factors (time/control and stress/compensate), or a bifactor model. Likewise, added to the validity evidence based on the internal structure, the IAT showed evidence based on the relationship with other variables (average hours per day on the Internet). On the other hand, the reliability of the scores was acceptable.

The results lead to the conclusion that the scores in the IAT have evidence of validity and reliability for its use. This has implications for both researchers and those who are primarily involved in the patient practice. For researchers, the IAT constitutes an instrument that would allow studies on Internet addiction to be carried out, for example, knowing its prevalence in certain groups, identifying the factors associated with its genesis and evolution in people, or knowing the degree of sensitivity and specificity with which one can diagnose a person with Internet addiction. For professionals, the IAT is a tool that would help diagnose Internet addiction, and it would also allow evaluation of the effects produced by a treatment or therapy that seeks to decrease the level of Internet addiction.

**Author Contributions:** Conceptualization, A.A.T.-M., J.C.A.-P., R.A.Z.-T. and D.E.R.-O.; methodology, A.A.T.-M., J.C.A.-P., R.A.Z.-T. and D.E.R.-O.; software, A.A.T.-M., J.C.A.-P., R.A.Z.-T. and D.E.R.-O.; validation, A.A.T.-M., J.C.A.-P., R.A.Z.-T. and D.E.R.-O.; formal analysis, A.A.T.-M., J.C.A.-P., R.A.Z.-T. and D.E.R.-O.; investigation, A.A.T.-M., J.C.A.-P., R.A.Z.-T. and D.E.R.-O.; resources, A.A.T.-M., J.C.A.-P., R.A.Z.-T. and D.E.R.-O.; data curation, A.A.T.-M., J.C.A.-P., R.A.Z.-T. and D.E.R.-O.; writing—original draft preparation, A.A.T.-M., J.C.A.-P., R.A.Z.-T. and D.E.R.-O.; writing—review and editing, A.A.T.-M., J.C.A.-P., R.A.Z.-T. and D.E.R.-O.; visualization, A.A.T.-M., J.C.A.-P., R.A.Z.-T. and D.E.R.-O.; supervision, A.A.T.-M., J.C.A.-P., R.A.Z.-T. and D.E.R.-O.; project administration, A.A.T.-M., J.C.A.-P., R.A.Z.-T. and D.E.R.-O. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Conflicts of Interest:** The authors declare no conflict of interest.
