*2.3. Results and Discussion*

#### 2.3.1. Validation of the Three Scales

The results of item analysis, CFA, and tests of reliability and convergent validity are shown in Table 2. Item-sum correlations were computed to examine item functioning. The PPCS and PPUS yielded higher coefficients, and both these scales also yielded good fit indices (i.e., CFA) and stronger reliability coefficients. PPCS, PPUS, and s-IAT-sex significantly positively related with SCS, PCQ, OSAs and usage time severally, and PPCS demonstrated stronger convergent validity.


**Table 2.** Reliability and validity of the three scales.

<sup>1</sup> PPCS = Problematic Pornography Consumption Scale, <sup>2</sup> PPUS = Problematic Pornography Use Scale, <sup>3</sup> s-IAT-sex = Short Internet Addiction Test Adapted to Online Sexual Activities, <sup>4</sup> CFI = comparative fit index, <sup>5</sup> TLI = Tucker-Lewis index, <sup>6</sup> RMSEA = root mean square error of approximation, <sup>7</sup> CI = confidence interval, <sup>8</sup> SCS = Sexual Compulsivity Scale, <sup>9</sup> PCQ = Pornography Craving Questionnaire, <sup>10</sup> OSAs = online sexual activities, <sup>11</sup> UT = usage time. \*\*\* *p* < 0.001.

#### 2.3.2. LPA

The results of LPA are shown in Table 3. For PPCS, the Lo-Mendell-Rubin adjusted likelihood ratio test (LMRT) results were significant when the number of classes was 4, and the entropy value was lower. Thus, the classification accuracy was not as high as that of the three-class solution; accordingly, the three-class solution was selected. For PPUS, when the model consisted of three classes, the LMRT results were significant; furthermore, the entropy value was evidently higher than that of the four-class solution. With regard to the s-IAT-sex, the nonsignificant *p*-value that emerged for the LMRT results suggested that the three- and four-class solutions should be rejected in favor of the two-class solution.

**Table 3.** Fit indices for latent profile analysis of the three scales assessing problematic internet pornography use.


<sup>1</sup> PPCS = Problematic Pornography Consumption Scale, <sup>2</sup> PPUS = Problematic Pornography Use Scale, <sup>3</sup> s-IAT-sex = Short Internet Addiction Test Adapted to Online Sexual Activities, <sup>4</sup> classes = number of latent classes, <sup>5</sup> AIC = Akaike information criterion, <sup>6</sup> BIC = Bayesian information criterion, <sup>7</sup> SSABIC = sample-size-adjusted Bayesian information criterion, <sup>8</sup> LMRT = Lo-Mendell-Rubin adjusted likelihood ratio test, *p* = *p*-value associated with the LMRT results. Bold text is the finally selected models. \* *p* < 0.05, \*\* *p* < 0.01, \*\*\* *p* < 0.001.

With regard to the three groups that emerged for the PPCS and PPUS, the first class obtained the lowest averages across all the scale dimensions; thus, this group was referred to as nonproblematic consumption. The second class obtained moderate scores on all the scale dimensions; therefore, these group members were referred to as low-risk pornography users. The third class obtained the highest scores on all the scale dimensions; thus, this group was referred to as at-risk users. As shown in Table 4, with regard to the two classes that emerged for the s-IAT-sex, class 1 obtained lower scores than class 2 on both the scale dimensions; therefore, they were referred to as the nonproblematic and at-risk groups, respectively (group differences in scores on the specific dimensions are shown in Appendix A).


**Table 4.** Comparisons of the accuracy of the three scales.

<sup>1</sup> PPCS = Problematic Pornography Consumption Scale, <sup>2</sup> PPUS = Problematic Pornography Use Scale, <sup>3</sup> s-IAT-sex = Short Internet Addiction Test Adapted to Online Sexual Activities, <sup>4</sup> Sensitivity = the proportion of persons with positive symptoms and members of the at-risk group that was identified through LPA, <sup>5</sup> Specificity = the proportion of persons with negative symptoms and the nonproblematic group.

#### 2.3.3. Sensitivity and Specificity Analysis

The results showed that the sensitivity of the PPCS was 89.66%, which is higher than the values that emerged for the PPUS (i.e., 81.25%) and the s-IAT-sex (i.e., 71.72%). There were differences in the specificity of the three scales, and the values ranged from 85.86% to 94.95%. The PPCS demonstrated greater sensitivity (89.66%), and its specificity was 85.86%. This indicates that approximately 10% of problematic users had been classified as nonproblematic users and that approximately 14% of nonproblematic users had not been identified. In general, the PPCS and PPUS performed better than the s-IAT-sex. Since this study aimed to identify the scale with greater sensitivity in detecting problematic IPU, the PPCS was investigated in greater detail.

### **3. The Qualitative Part: Identification of the Most Accurate Scale**

#### *3.1. Methods*

#### 3.1.1. Sample

We interviewed 22 (20 men; mean age = 27.2) problematic IPU service volunteers (who provide online services on the following website: http://www.ryeboy.org/; average service time = 3.3 years) and 11 therapists (who have worked with individuals with problematic IPU and had more than 3 years of clinical experience).

#### 3.1.2. The Interview Outline

Since the used scales were easy to administer and consisted of close-ended questions, interviews were conducted to examine participants' perspectives more deeply and comprehensively. The interview guide primarily sought to explore interviewees' understanding of problematic IPU/addiction and their evaluations of the dimensions of the selected scale. The interviewees were required to rate the importance of the dimensions on a scale that ranged from 1 (not at all important) to 7 (very important).

#### 3.1.3. Procedure

In this study, we primarily explored their understanding of the concept of problematic IPU and the dimensions of the recommended scale. Two psychology graduate students served as the interviewers. At the beginning of the interview, the interviewees were informed about the purpose and significance of the interview and assured of the anonymity and strict confidentiality of their interview data; the interviews were recorded with their permission.

#### *3.2. Analysis*

The interview recordings were transcribed into verbatim scripts, and participants' identifying information was concealed. Next, we undertook thematic analysis of the text; in other words, we collated different interviewees' responses to the same question to create new text. Tree Nodes were established based on the dimensions of the selected scale, and interviewees' original statements were identified and summarized as a named code. Through this process, NVivo automatically generated statistics for all the references of the texts.

#### *3.3. Results*

With regard to the characteristics of problematic IPU, we generated a total of 20 codes by analyzing the interview data. Among these features, preoccupation with IPU (22 mentions), IPU to escape or avoid a negative emotional state (21 mentions), interpersonal conflict (22 mentions), and physiological and psychological symptoms (45 mentions) were most commonly mentioned. Furthermore, the 20 codes were summarized into the six dimensions of the PPCS (see Figure 1).

**Figure 1.** Volunteers and therapists' frequency of mentioning the dimensions of the Problematic Pornography Consumption Scale, features, and importance ratings for the six dimensions (average scores across 33 interviewees). Note: the numbers in the color blocks represent the frequency of mentions, whereas the polyline represents importance ratings for the six dimensions (range = 1–7).

Instance of the interview:

Interviewer: According to your service experience, what do you think is problematic internet pornography use? In other words, what are the expressions/symptoms of problematic internet pornography use?

Interviewee (service volunteer): They (problematic users) show difficulty controlling the craving for internet pornography (code: pornography carving), they are unable to control their own behavior, for instance, browsing pornographic websites, masturbating while watching porn frequently (code: difficulties in control). Their brains are constantly bombarded with sexual materials (code: preoccupation). If they are not exposed to internet pornography, they will feel uncomfortable, or feel that their heart is empty (code: depression resulting from unsuccessful withdrawal).

After presenting interviewees with the definitions of the six components of problematic IPU and further clarifying their meaning using examples, we presented them with questions "Based on your service experience, do you endorse this structure? Which dimension or dimensions do you think are particularly central to IPU?" Most (>95%) participants endorsed the six dimensions. It also can be inferred from Figure 1 that both volunteers and therapists emphasized the centrality of conflict, relapse and withdrawal in IPU (basing the frequency of mentions); at the same time, they weighted the mood modification, relapse and withdrawal as more important features in the problematic use (basing the important rating).

#### **4. General Discussion**

Problematic IPU is still a controversial issue; notably, it appears that no real consensus exists regarding the conceptualization and screening tool of problematic IPU. Several scales are available; thus, the assessment of problematic IPU is inconsistent, indicating that findings in this area are not readily comparable. The present study aimed to selected a more sensitive scale to screen problematic IPU, because higher sensitivity implies lower rate of missed diagnosis (i.e., problematic users who have been incorrectly screened as nonproblematic users). Basing on a systematic literature review, three scales were retained. Considering that research with mixed methods combining quantitative and qualitative analyses can enrich and improve our understanding of complicated phenomena [38,39], a quantitative method was used to identify a "more accurate" analysis from the three retained scales. Results of CFA showed that all three scales have good applicability in the wide range of adult groups (age in this case ranged from 18 to 45 years) in three highly homogeneous samples; compared to the other two scales, the PPCS demonstrated greater sensitivity and comparative specificity among samples drawn from the general population (results of the QUAN). Considering that the expression of questionnaire survey is brief and closed, and that the interview can understand the participants' undefined views more deeply and comprehensively, subsequently, results of QUAL showed that symptoms of problematic IPU proposed by the servers (volunteers and therapists) can be grouped into the six dimensions of PPCS and most of the servers supported the six-factor structure of PPCS.

Among the three scales, the PPCS score was most robustly related to the duration of usage, frequency of engagement in OSAs, and pornography cravings. Problematic IPU can appear under the umbrella of hypersexuality similarly to frequent engaging in various forms of cybersex, intense craving for pornography, and compulsive sexual behaviors [40], insofar that the robust relationship not only demonstrated a higher criterion validity, but also implied that co-screening instruments (i.e., pornography craving, frequency and duration of use, compulsive use) are expected to work as auxiliary screening indicators. Recent studies have revealed that for some people, pornographic use gave rise to their feeling of discord and shame contributing to their conflict of actual sexual materials consumption and their belief; in turn, these feelings of distress and shame may drive a morbid self-perception that they are addicted, but this may not be a real behavioral disorder [41,42]. In order to avoid misjudgment due to the self-perceived problematic use, it is more advisable to combine other supporting scales, and the combination diagnosis indexes of the diversity were selected to screen the prevalence of problematic IPU. In this study, with the higher correlation of PPCS with frequency of OSAs, the PCQ showed that combined with other indicators, it can better screen out problematic use and is more likely to avoid the misjudgment caused by subjective self-perceived addiction.

The more robust psychometric properties and higher recognition accuracy of the PPCS may be attributable to the fact that it has been developed in accordance with Griffiths's six-component structural theory of addiction (i.e., in contrast to the PPUS and s-IAT-sex). The PPCS has a very strong theoretical framework, and it assesses more components of addiction [11]. In particular, tolerance and withdrawal are the important dimensions of problematic IPU that are not assessed by the PPUS and s-IAT-sex; PPCS is the only instrument that explicitly assesses the "tolerance" component [11,14]. According to the "two-phased" internet pornography addiction model, in which the first step is characterized by an excessive use to internet pornography, and the second functions as a marker by repeated failures to break free from excessive use, despite negative consequences [43]. Items related to information about salience, carving, and tolerance reflect the engagement in internet pornography, corresponding to the first step, whereas items related to withdrawal, relapse, and conflict measure addiction more, corresponding to the second step. Obviously, components of PPCS includes both engagement in pornography and addiction of IPU, which has an intact theoretical framework of addiction.

The PPCS appears to be a more valid instrument for assessing problematic pornography use, has potential application in detecting prevalence concerning problematic IPU or cybersex addiction, and may be useful in assessing treatment outcomes. Our findings indicate that individuals who score high on the PPCS also report frequent engaging in various forms of online sexual activities, intense craving for pornography, and compulsive sexual behaviors. Thus, it appears important for clinicians to be aware of problematic pornography use and its related associations such as pornography craving, compulsive use. Moreover, it is important to note that the scale PPCS is recommended as a screening instruments to identify problematic users in the public and assess the prevalence rather than a diagnostic tool; future studies should further research its validity and cutoff in clinical sample; we also encourage individuals to visit a clinical therapist after being identified with problematic IPU by the use of PPCS.

This study has several limitations. First, data were collected using self-report measures; therefore, the reliability of the results depends on the respondents' honesty and accuracy of their comprehension of the scale items. Second, the study sample was recruited through an online survey company; therefore, the participants of this study may have been more educated and affluent than the average Chinese person. Furthermore, the study participants primarily lived in the capital/provincial capital, cities, and towns. Third, because the sample consisted of only a small number of non-heterosexual subjects, it was not possible to examine whether the factor structure and meaning of the contents of the PPCS differed across individuals with different sexual orientations.

#### **5. Conclusions**

The present study showed that the PPUS, PPCS, and s-IAT-sex are promising measures of problematic IPU. However, when sensitivity and specificity were simultaneously examined, the PPCS emerged as a more suitable measure of problematic IPU. The qualitative findings further confirmed that service providers endorsed the underlying structure of the PPCS.

**Author Contributions:** Conceptualization, L.C.; Data curation, L.C.; Formal analysis, X.J.; Funding acquisition, L.C.; Investigation, X.J.; Methodology, L.C.; Project administration, L.C.; Resources, L.C.; Supervision, L.C.; Visualization, X.J.; Writing—original draft, L.C.; Writing—review and editing, L.C. and X.J. All authors have read and agreed to the published version of the manuscript.

**Funding:** This work was supported by the National Social Science Foundation of China (Grant No. CEA150173 and 19BSH117) and the education reform project of Fujian province (FBJG20170038). The funding agencies did not have input into the content of the manuscript and the views described in the manuscript reflect those of the authors and not necessarily those of the funding agencies.

**Acknowledgments:** We would like to acknowledge Bin Wu and Yan Zhao (the founders of the "*Reyboys*", a non-governmental organization focusing on helping the problematic internet pornography users) for their help to recruit volunteers who serviced the addicts in the qualitative step, and pay tribute to them for their effort in helping problematic users.

**Conflicts of Interest:** The authors report no conflict of interest with respect to the content of this manuscript.

#### **Appendix A**

**Figure A1.** The average scores of the three latent classes based on the dimensions of PPCS. Note: PPCS = Problematic Pornography Consumption Scale, range = 1–7; \*\*\* *p* < 0.001 indicate that the score of at-risk group was significantly higher than that of low-risk group; *p* < 0.001 indicate that the score of low-risk group was significantly higher than that of non-problematic group; -- *p* < 0.001 indicate that the score of at-risk group was significantly higher than that of non-problematic group. The same below.

**Figure A2.** The average scores of the three latent classes based on the dimensions of PPUS. Note: PPUS = Problematic Pornography Use Scale, range = 0–5.

**Figure A3.** The average scores of the tow latent classes based on the dimensions of s-IAT-sex. Note: s-IAT-sex = short version of the Internet Addiction Test adapted to online sexual activities, range = 1–5.

#### **References**


© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

International Journal of *Environmental Research and Public Health*
