Next Article in Journal
Overlooking of Individuals with Cardiometabolic Risk by Evaluation of Obesity Using Waist Circumference and Body Mass Index in Middle-Aged Japanese Women
Next Article in Special Issue
Smartphone Use and Addiction among Pharmacy Students in Northern Thailand: A Cross-Sectional Study
Previous Article in Journal
Convergent Validity of the Edinburgh Postnatal Depression Scale and the Patient Health Questionnaire (PHQ-9) in Pregnant and Postpartum Women: Their Construct Correlations with Functional Disability
Previous Article in Special Issue
The Mediating Role of Selfitis in the Associations between Self-Esteem, Problematic Social Media Use, Problematic Smartphone Use, Body-Self Appearance, and Psychological Distress among Young Ghanaian Adults
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Healthcare
Volume 11
Issue 5
10.3390/healthcare11050700
healthcare-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

The Psychometric Properties of the DASS-21 and Its Association with Problematic Internet Use among Chinese College Freshmen

by
Cui-Hong Cao
1,
Chang-Yan Dang
1,
Xia Zheng
2,
Wang-Guang Chen
3,
I-Hua Chen
4,* and
Jeffrey H. Gamble
5,*
1
School of Foreign Languages, Shandong Women’s University, Jinan 250300, China
2
Mental-Health Education Center, Nanchang University, Nanchang 330031, China
3
School of Administration, Guangdong Polytechnic Normal University, Guangzhou 510665, China
4
Chinese Academy of Education Big Data, Qufu Normal University, Qufu 273165, China
5
Department of English, National Changhua University, Changhua 50007, Taiwan
*
Authors to whom correspondence should be addressed.
Healthcare 2023, 11(5), 700; https://doi.org/10.3390/healthcare11050700
Submission received: 12 December 2022 / Revised: 17 February 2023 / Accepted: 23 February 2023 / Published: 27 February 2023
(This article belongs to the Special Issue The Correlates of Technology-Based Addictive Behaviors and Their Impacts on Wellbeing)
Browse Figure
Review Reports Versions Notes

Abstract

:
During transitional periods, college freshmen may experience mental health issues. The Depression, Anxiety, and Stress Scale—21-item version (DASS-21) is commonly used for mental health assessment in China. However, evidence is lacking regarding its applicability with freshmen as a demographic. Debates also exist regarding its factor structure. This study aimed to evaluate the DASS-21′s psychometric properties with Chinese college freshmen and investigate its association with three kinds of problematic Internet use. A convenience sampling method was used to recruit two samples of freshmen—one of 364 (female 248; mean age 18.17 years) and the other of 956 (female 499; mean age 18.38 years) participants. McDonald’s ω and confirmatory factor analysis were conducted to evaluate both the scale’s internal reliability and construct validity. The results indicated acceptable reliability, with a one-factor structure inferior to a three-factor structure in terms of model fit. Furthermore, it was demonstrated that problematic Internet use was significantly and positively associated with depression, anxiety, and stress among Chinese college freshmen. Based on the prerequisite of measurement equivalence across two samples, the study also found that freshmen’s problematic Internet use and psychological distress were likely to be affected by the strict measures put in place during the COVID-19 pandemic.

1. Introduction

Adolescence is a key period when many psychological disorders can occur, and, broadly speaking, college life can be considered the final stage of adolescence for students [1]. Previous studies have demonstrated that, due to feeling frustrated by homesickness, academic performance, pressure to succeed, and environmental changes, as well as being in a transitional period of life, college students are likely to experience more negative emotions and mental disorders in their freshman years [2,3,4,5,6]. It was reported that college freshmen have a higher level of psychological distress, most likely stemming from experiencing the unknown, newly moving away from home, over-expectations of embarking on a new life path, or feeling overwhelmed by coursework and unfamiliar teaching methods [7]. For all these reasons, college students’ psychological distress deserves special attention [1], as it is not only associated with their struggles in academic performance, dropout rate, and excessive alcohol intake, but also negatively affects their professional development [8]. Therefore, early detection, prevention programs, and mental health interventions are necessary and can have a significant, long-term impact. As such, it is necessary and important to be able to use a reliable and valid instrument to assess the levels of freshmen’s psychological distress.

2. Literature Background

2.1. The Depression, Anxiety, and Stress Scale

The Depression, Anxiety, and Stress Scale (DASS), is a commonly utilized instrument for screening mental health problems, the full version consisting of 42 items, and the short version consisting of 21 items [9]. Developed by Lovibond et al., this scale takes the form of a self-report questionnaire, with the original purpose of having a consistent measurement system which could be used to distinguish between and define depression, anxiety, and stress, in order to assist with clinical diagnosis using additional psychometric indicators and provide a rapid and accurate tool for screening research subjects [10]. The Depression, Anxiety, and Stress Scale—21-item version (DASS-21) adopted in the present study has the advantage of being able to be completed and scored more quickly.
The DASS-21 has been empirically validated with college students from diverse cultures with high internal consistency [2,11,12,13,14,15]. In China, several studies have been conducted to assess its psychometric properties. Gong et al. first introduced the Chinese version of this scale. They conducted a survey of college students, finding that the DASS-21 had stable psychometric properties and could successfully reflect the psychological distress status of Chinese college students [10]. Another study also confirmed the satisfactory internal consistency indices for the three subscales—depression, anxiety, and stress (with Cronbach’s αs of 0.83, 0.80, and 0.82, respectively)—and a total internal consistency of 0.92 [16].
Although this evidence shows that the Chinese DASS-21 can be a useful tool for measuring the psychological distress of college students, there has not been a psychometric evaluation of this scale specifically for college freshmen. Due to the fact that this demographic in particular may be suffering from mental health difficulties, it is important to be able to screen their mental health situations with a reliable scale in order to suggest effective intervention measures [17]. Furthermore, even though there have been many consistent findings regarding the scale’s internal consistency regarding college students, many inconsistent findings have also been found regarding its factor structure (i.e., the use of a one-factor or a three-factor structure) when used with the same demographic [11,18,19]. Based on this gap, our primary aim was to evaluate the psychometric properties of this commonly used instrument—the DASS-21—for Chinese college freshmen.

2.2. The Association of the DASS-21 with Problematic Internet Use

In addition to validating the DASS-21 in the context of Chinese college freshmen, the association of this scale with problematic Internet use (PIU) was also investigated. Existing studies have found that PIU is strongly linked to depression, anxiety, and stress [20]. Recently, Internet and mobile technology have developed rapidly and become more widespread, as modern life has become increasingly inseparable from the Internet. Based on the 50th China Statistical Report on Internet Development released by the China Internet Network Information Center, the number of Chinese Internet users reached 1.05 billion in June 2022, with cellphone users comprising 99.6 percent of this number [21]. Despite the many advantages of using the Internet (e.g., increased accessibility for communication, information searching, and recreation) [22], the fact that people can now easily obtain access to the Internet and can afford to use it globally contributes to the rising incidence of PIU [23,24,25], a mental disorder manifested as a result of being excessively engaged in out-of-control Internet use which is detrimental to one’s well-being [26]. Adolescents experiencing negative emotions and mental disorders have a tendency to resort to the Internet in order to escape from the anxieties or difficulties they face in their daily life [23]. However, being absorbed by the Internet makes it more likely for them to experience both physical and psychological disorders [27,28], including insomnia, low self-esteem, depression, anxiety, and stress [20,29]. These physical and mental health problems are more often found among students experiencing problematic smartphone use (PSU) [30], problematic social media use (PSMU) [31,32], and problematic Internet gaming (PG) [33,34].
As technological devices have become more prevalent over the past few decades, many college students are now regularly using Internet-based applications and portals [35]. Currently, smartphones are the most commonly used device for accessing the Internet due to its multi-functionality and consequent global popularity [36]. Middle school and college students are the primary age groups of Internet users [37]. Smartphones are a convenient tool that allows people to communicate, socialize, seek entertainment, and enhance their productivity [38]. However, using smartphones excessively may have adverse effects, such as the development of PSU [36]. An analysis of 117 studies concluded that PSU is linked to psychological distress symptoms [39]. Currently, there is a debate within the literature regarding whether this is a result of being addicted to the device itself (e.g., a smartphone) or to the content and applications available and accessible on these devices (e.g., websites, applications, and social networks) [38]. One study has suggested that applications on smartphones are the source of the problematic behavior [40]. Meanwhile, using a single cross-sectional study, Stuart et al. concluded that the level of addiction was greater for the device as a whole than for each particular service it provided [38]. Therefore, the influence of PSU on the emotions of college freshmen should be studied.
In addition to smartphones, social media also exerts a great influence on people’s living habits. Social media is defined as “Internet-based channels that allow users to opportunistically interact and selectively self-present, either in real-time or asynchronously, with both broad and narrow audiences who derive value from user-generated content and the perception of interaction with others” [41]. PSMU, however, is characterized by the compulsive or poorly controlled use of social media that negatively impacts personal and professional functioning, with growing concern about PSMU among adolescents [42]. Several previous studies found that the level of PSMU is positively associated with the level of psychological distress experienced [32,43].
Besides smartphones and social media, Internet gaming is another relatively popular form of entertainment about which several public health concerns have been raised. Internet gaming disorder [44] was first treated as a mental health disorder in 2013 by the American Psychiatric Association [45]. Evidence has demonstrated that levels of anxiety, depression, and stress are significantly correlated with PG tendency [46,47].
Considerable research has been conducted aiming to detect the adverse effects of PSU, PSMU, and PG on college students’ psychological well-being. One study with Hong Kong University students as research subjects found that PSMU and PG had positive associations with psychological distress [23]. To the best of our knowledge, only one research study has been conducted to investigate the correlation among three PIUs and psychological distress, and this study concerned Malaysian university students [29]. Therefore, we note a research gap in that PSU, PSMU, and PG have rarely been examined and compared within a single study, specifically concerning college freshmen in a Chinese context. To address this gap, the second aim of this work was to assess and compare the associations between the three PIUs and psychological distress among Chinese college freshmen.

2.3. The Impact of COVID-19

The impact of the novel coronavirus (COVID-19) pandemic on the mental health of college freshmen cannot be ignored. Since the pandemic broke out in December 2019, the Chinese Government has enacted strict prevention and control measures. An early notice issued by the Ministry of Education of the People’s Republic of China clarified that, in the case of an infection, campuses were expected to respond quickly to follow emergency plans, which included the quick implementation of isolation protection, on-site disinfection, dormitory lockdown, and online teaching [48]. Two years later, the Omicron variant emerged, with increased transmissibility. To care for the lives and health of teachers and students, as well as to increase campus safety and the overall stability of the education system, as a general policy, China’s national education system has unwaveringly adhered to the “dynamic zero COVID” and “external defense import, internal defense rebound” policies [49]. At the time of writing, many universities continue to be under lockdown conditions and students are asked to stay on campus and even remain in their dorms in order to prevent infection and limit the spread of the pandemic. Given that various studies have found that the COVID-19 pandemic and social isolation policies may have resulted in the increased prevalence of serious mental health problems [50,51], it is important to investigate whether differences exist between freshmen’s psychological states in 2020 and those of freshmen in 2022.
To summarize, the present preliminary study had two research questions (RQs):
RQ1: What are the DASS-21’s psychometric properties regarding Chinese college freshmen, including its internal reliability and construct validity (i.e., factorial validity, convergent validity, and discriminant validity)?
In order to find the answer to this question, the present study conducted a cross-sectional study of two groups of college freshmen (i.e., freshmen in 2020, the early stage of the pandemic; and freshmen in 2022, a more recent stage of the pandemic). In addition, a multiple-group analysis was carried out to determine whether or not there was measurement invariance across these two samples.
RQ2: What is the association between the DASS-21 and PSU, PSMU, and PG?
The present study hypothesized that associations would exist between psychological distress and the three PIUs in Chinese college freshmen. Furthermore, the study expected to find different results regarding mental health between the two freshmen groups as they had experienced different stages of the pandemic.
It was decided that the combination of validating the instrument, analyzing the influencing factors (i.e., PSU, PSMU, and PG), and comparing freshmen’s psychological states between the 2020 and 2022 cohorts could provide useful insights to both policymakers and university staff to help them better understand the psychological states of those freshmen confronted with public health emergencies, empowering them to develop effective prevention and intervention measures. The study results can also be used as a reference for the formulation and adjustment of campus pandemic prevention and control policies.

3. Materials and Methods

3.1. Participants and Procedure

A total of two sets of data were collected, the first in October 2020, and the second in October 2022. The study participants were recruited utilizing convenient sampling. The selection criteria for participants included (1) being a freshman student, (2) understanding simplified Chinese, and (3) owning a smartphone with Internet access for more than three months. The demographic characteristics of the two samples are presented in Table 1. A background questionnaire was included in the survey, as well as the Chinese versions of the following three questionnaires: the DASS-21, the Smartphone Application-Based Addiction Scale (SABAS), the Bergen Social Media Addiction Scale (BSMAS), and the nine-item Internet Gaming Disorder Scale—Short Form (IGDS-SF9). The background information section included items relating to age, gender (male or female), department (science, literature, management, and other), and study program level (undergraduate or junior college). The study obtained approval from the Institutional Review Board of the Jiangxi Psychological Consultant Association (IRB ref: JXSXL-2021-J99).
The first set of data, collected in October 2020, was primarily from the city of Nanchang, in the province of Jiangxi. The authors contacted psychology teachers in colleges and universities, letting them know about the purposes of the research and the estimated time needed to answer all the questions on the questionnaire by telephone. After receiving a positive response, the researcher sent the psychology teachers a hyperlink and QR code to access the measurement materials. These teachers shared the link and QR code with their students in the classroom, either before or after class, and students completed the questionnaire immediately after receiving it. During the data collection period, universities were busy adjusting teaching after experiencing the sudden impact of the pandemic in the first half of the year, so the data collection process encountered difficulties, and only 364 participants were recruited, of which 116 (31.9%) were male and 248 (68.1%) were female, the latter being more than double the number of the former. The participants’ average age was 18.17 (SD = 0.42) years. All were from undergraduate colleges, studying one of the four categories of majors: literature (33%), management (14%), engineering (39.3), or “other” (13.7%).
To collect the second set of data, the author sent an invitation message to research assistants and instructors, distributed an online link to the survey, and also provided information regarding the purposes of the research and the estimated time needed to complete the questionnaire. Several research assistants and faculty members sent the hyperlink and QR code onwards, providing access to the questionnaire to potential participants. The second set of data was collected in October 2022, from schools in a total of 13 provinces, including Shanxi, Guangdong, Guizhou, and Shandong, among others. A final total of 956 participants were recruited in the second round of data collection, of which 457 (47.8%) were male and 499 (52.2%) were female, representing a more balanced distribution. The participants’ average age was 18.38 (SD = 1.11) years, and two thirds of the participants came from undergraduate colleges, with one third coming from junior colleges. The students reported that they majored in science (15.8%), literature (15.9%), management (16.3%), engineering (28.7%), or “other” (23.3%).

3.2. Measures

3.2.1. The Depression, Anxiety, and Stress Scale—21-Item Version (DASS-21)

Developed by Lovibond and Lovibond, the DASS-21 is a shortened version of the original 42-item scale [9]. The scale consists of three self-reported subscales for depression, anxiety, and stress that are intended to measure the psychological distress experienced by the participant over the previous week. Each subscale comprises seven items. An example of a depression subscale item is “I found it difficult to work up the initiative to do things”. An example of an anxiety subscale item is “I was worried about situations in which I might panic and make a fool of myself”. An example of a stress subscale item is “I felt that I was using a lot of nervous energy”. The 21 items are rated on a Likert scale with four points, ranging from 0 (“did not apply to me at all”) to 3 (“applied to me very much or most of the time”). An overall higher score is indicative of a higher severity of psychological distress. Previous research has already proven that the DASS-21 possesses satisfactory psychometric properties coupled with a high degree of consistency and a confirmed unidimensional structure [7,29,52,53,54,55]. Moreover, it has been demonstrated that the Chinese version of this instrument has good internal consistency (α = 0.89) among Chinese college students [10]. Please refer to the Results section, below, for information on the scale’s reliability and validity for the college freshmen participating in this study.

3.2.2. The Smartphone Application-Based Addiction Scale (SABAS)

The SABAS was adopted to gauge the severity of problematic smartphone use by college freshmen. Developed by Csibi, Demetrovics, and Szabo [56], the scale contains six items. The six items are rated on a six-point Likert scale ranging from 1 (“strongly disagree”) to 6 (“strongly agree”). An example of an SABAS item is “If I cannot use or access my smartphone when I feel like, I feel sad, moody, or irritable”. The maximum score is 36, with a higher score indicating that the participant is more severely addicted to smartphone use. Acceptable psychometric properties for this scale have been demonstrated in different languages in several previous studies, including English [56], Hungarian [57], Persian [58], and Italian [59]. The unidimensional structure SABAS’s Chinese translation has good internal reliability when used for Hong Kong University students (α = 0.75) [35] and mainland Chinese primary school students (α = 0.81) [22]. In the current study, the internal consistency of the SABAS had a McDonald’s ω of 0.745 for the first set of data and 0.884 for the second set of data.

3.2.3. The Bergen Social Media Addiction Scale (BSMAS)

The BSMAS was used to assess the severity of college freshmen’s PSMU. Developed by Andreassen et al. [31], this scale contains six items, and each evaluates a person’s experience of using social media during the past year. An example item is “How often during the last year have you become restless or troubled if you have been prohibited from using social media?” The six items are rated on a five-point Likert scale with a maximum score of 30, with categories ranging from 1 (“very rarely”) to 5 (“very often”). A higher total score means that the respondent’s risk of PSMU is more severe [60]. The Chinese version of the BSMAS has been previously validated for use on Chinese primary school students with acceptable psychometric properties [22]. In the present study, the internal consistency of the BSMAS had a McDonald’s ω of 0.782 for the first set of data and 0.866 for the second set of data.

3.2.4. The Internet Gaming Disorder Scale—Short Form (IGDS-SF9)

The current study adopted the IGDS-SF9 to assess the severity of PG in college freshmen. This measure is a self-report scale developed by Pontes and Griffiths based on DSM-5 criteria [61]. It comprises nine items evaluating how an individual has recently experienced gaming activity. An example item is “Do you feel the need to spend an increasing amount of time engaged in gaming in order to achieve satisfaction or pleasure?” A five-point Likert scale is used to rate the nine items, ranging from 1 (“never”) to 5 (“very often”). The sum of the individual item scores, with a minimum potential score of 9 and a maximum possible score of 45, is used to determine the overall result, with higher total scores indicating that the respondent has a more severe degree of gaming disorder. Previous research has proven that the IGDS-SF9 is a valid and reliable scale which can be used to measure PG [62]. Previous studies have been conducted to assess the Chinese version among Hong Kong adults [63], Hong Kong university students [35], and mainland Chinese primary school students, all with acceptable psychometric properties [22]. In the current study, according to the preliminary analysis, the internal consistency of the IGDS-SF9 had a McDonald’s ω of 0.906 for the first set of data and 0.930 for the second set of data.

3.3. Data Analysis Strategy

The current study adopted descriptive statistics and Pearson correlations for the purpose of analyzing the mean (SD) of the included variables and the correlations among them. Additionally, to compare how much the variables varied between the participants of the 2020 and 2022 cohorts, an independent t-test method was adopted in order to determine whether the discrepancies between the variables were at a statistically significant level. Subsequently, McDonald’s ω and confirmatory factor analysis (CFA) were both utilized to assess the DASS-21’s internal consistency and factorial validity. Regarding its factorial validity, two fits of structures were compared: one was a one-factor structure and the other was a three-factor structure, in addition to evaluating the individual factor structure used. It is noteworthy that we conducted an Exploratory Factor Analysis (EFA) on two separate samples. This was done to address the ongoing controversy regarding the dimensional structure of DASS-21; the EFA, which allows for the cross-loading of items, provided additional support for the use of Confirmatory Factor Analysis (CFA). However, considering the extensive testing that has been performed on the DASS-21, as mentioned previously, and that EFA is usually employed in the scale development stage [64], the EFA results are presented in the Supplementary Information file which accompanies the present work.
We further tested the convergence, discriminant validity, and measurement invariance using CFA. Based on the standardized factor loadings, the composite reliability (CR) and average variance extracted (AVE) were computed to investigate convergent and discriminant validity. In order to evaluate DASS-21’s measurement invariance across the two samples, a comparison of several models was conducted, which included (a) a configural model (i.e., no constrained model, as a baseline model); (b) a factor-loading constrained equal model (less constrained model); and (c) a factor-loading and item-intercept constrained equal model (more constrained model). When the differences of the fit indices from different models do not violate the criterion, the equivalence of the DASS-21 across distinct samples can be determined. Finally, for the examination of the associations between PIU and the DASS-21, structural equation modeling (SEM) was utilized in the current study to test the influence of the three PIUs on freshmen’s DASS-21 score, with gender as the controlled variable, considering potential differences in the prevalence of PIU [19,33] between males and females.
In terms of the criteria of the above-mentioned statistics, several indices were used. For factorial validity and SEM, indices including the comparative fit index (CFI), non-normed fit index (NNFI), root mean square error of approximation (RMSEA), and standardized root mean square residual (SRMR) were selected. In order to meet the criteria, it was essential to meet the following values: the values of CFI and NNFI should not be lower than 0.95; the values of RMSEA should not be higher than 0.06; and the values of SRMR should not be higher than 0.08 [65]. As for the model selection of the factor structure, the Akaike information criterion (AIC) was adopted by the current study to compare the models with acceptable fit indices to determine which had the best model fit (i.e., smaller AIC indicates a better fit) [66]. For better convergent and discriminant validity, convergent validity is considered to be satisfactory when the CR is greater than 0.70 and the AVE is greater than 0.50 for each construct [65]. Additionally, the square root of the AVE needs to be greater than the correlations between the constructs, thus supporting the discriminant validity of the construct [61]. Regarding measurement invariance, a comparison of the models using CFI, RMSEA, and SRMR was performed to verify whether measurement invariance could be supported, with the following requirements: an ΔCFI higher than −0.01, an ΔRMSEA lower than 0.015, and an ΔSRMR lower than 0.03 (for factor-loading constrained) or lower than 0.01 (for item-intercept constrained) [67].

4. Results

4.1. Descriptive Statistics and Pearson Correlations

The mean values of Sample 1 (collected in October 2020) and Sample 2 (collected in October 2022) for the observed variables and the correlations between these variables are presented in Table 2. The results show that college freshmen in 2022 had a significantly higher prevalence of depression than college freshmen in 2020 (t = 2.89, p < 0.01, Cohen’s d = 0.18, with the value of 0.18 indicating a small effect size). Moreover, freshmen in 2022 also showed significantly more severe PIU than freshmen in 2020, with the differences having either small effect size, small to medium effect size, or medium to large effect size, depending on the type of PIU (PSU: t = 2.52, p = 0.01, Cohen’s d = 0.16; PSMU: t = 4.66, p < 0.01, Cohen’s d = 0.29; PG: t = 7.88, p < 0.01, Cohen’s d = 0.49). In terms of the Pearson correlations, significant positive relationships were found among all variables. Of these coefficients, strong associations were found regarding the three emotional disorders (Sample 1: r = 0.59 to 0.74, all p < 0.01; Sample 2: r = 0.79 to 0.83, all p < 0.01). Despite the fact that the three types of PIU were mutually correlated in the two samples, we noticed that a relatively moderate magnitude of correlation was found for the three PIUs in Sample 1 (r = 0.29 to 0.47, all p < 0.01), with a relatively higher magnitude of correlation for Sample 2 (r = 0.50 to 0.61, all p < 0.01). Additionally, the three emotional disorders and three different types of PIU were all positively correlated, and paired correlations were higher in Sample 2 (r = 0.42 to 0.48, all p < 0.01) than in Sample 1 (r = 0.22 to 0.38, all p < 0.01), which is similar to the pattern seen in the association among the three PIUs.

4.2. Reliability, Factorial, Convergent, Discriminant Validity, and Measurement Invariance

For Sample 1 data, the internal consistency was acceptable, and the McDonald’s ωs for depression, anxiety, and stress were 0.789, 0.633, and 0.755, respectively. For the Sample 2 data, the internal consistency was excellent, and the McDonald’s ωs for depression, anxiety, and stress were 0.887, 0.825, and 0.861, respectively.
In terms of the factorial validity of the factor structure, the CFA results in the current study demonstrated that the scale fit the factor structure well in both samples (i.e., the fit indices mostly met the criteria (see Table 3)). Furthermore, the fit was significantly better for the three-factor structure than for the one-factor structure, due to a much lower AIC for the three-factor structure (Sample 1: one-factor structure = 652.67 and three-factor structure = 571.79; Sample 2: one-factor structure = 1043.83 and three-factor structure = 826.94).
As for convergent validity, the results demonstrated that the convergent validity of the DASS-21 was ideal in Sample 2, since the CR for each factor was higher than 0.70 (the CRs of depression, anxiety, and stress in the current study were 0.94, 0.78, and 0.90, respectively) and the AVE of each factor was higher than 0.50 (the AVEs of depression, anxiety, and stress in the current study were 0.70, 0.57, and 0.59, respectively). Compared to Sample 2, although the CRs of all three dimensions were also higher than 0.70 in Sample 1 (the CR of depression, anxiety, and stress in the current study was 0.89, 0.78, and 0.82, respectively), but the value of the AVEs (the AVE of depression, anxiety, and stress in the current study were 0.54, 0.34, and 0.41, respectively) was unsatisfactory. As for the discriminant validity, the results revealed poor discriminant validity for both samples for the three DASS-21 factors because the square root of the AVE for each factor was less than the correlation between the latent factors (see Table 4). These findings are further supported by the results of the EFA. The Scree plot from the EFA results shows that the number of factors extracted from both Sample 1 and Sample 2 does not align with the initial expectations of the three factors (as depicted in Figures S1 and S2). Additionally, the factor loadings reveal cross-loadings for some of the items (see Tables S1 and S2), indicating a potential issue with the overly high correlations among the latent variables in the DASS-21. These findings suggest that these three factors should be considered as an overall construct, namely, psychological distress, rather than three separate constructs.
Regarding the examination of measurement invariance across the two sets of data, the results indicated that the equivalence of the DASS-21 was supported across the two samples in terms of both factor loadings and the item thresholds in both the one-factor structure and three-factor structures (see Table 5). Specifically, the DASS-21 had generally acceptable fit indices in the configural model, with the exception of SRMR, which had a value slightly higher than 0.80. Moreover, the results of the model comparison showed that the ΔCFI ranged from −0.006 to 0.001, ΔRMSEA ranged from 0 to 0.011, and ΔSRMR ranged from 0.003 to 0.011. All these values met the criteria for measurement invariance.

4.3. Associations between PIU and the DASS-21

For the purpose of examining the associations between PIU and the DASS-21 in SEM, we used the mean score of the three sub-scales (depression, anxiety, and stress) as the indicators of a general construct (i.e., psychological distress), given that the above results indicated that the score of the DASS-21 should be considered as an overall variable, due to poor discriminant validity for the three sub-factors. Using gender as the controlled variable, the results of the current study revealed that the model in both samples had generally satisfactory fit indices, despite the fact that RMSEA did not meet the criterion in Sample 1 (see Figure 1). Subsequently, the results of the path coefficients (also in Figure 1) indicated that higher PSU and PG contributed to a more severe level of psychological distress in both samples (Sample 1: PSU = 0.37, t = 3.83, p < 0.01; PG = 0.15, t = 2.55, p = 0.01) (Sample 2: PSU = 0.32, t = 6.74, p < 0.01; PG = 0.24, t = 5.42, p < 0.01). Finally, regarding the association between PSMU and psychological distress, the results showed that, although a higher PSMU was more indicative of poorer mental health in Sample 2 (β = 0.13, t = 2.66, p < 0.01), this association did not exist in Sample 1. Clearly, higher PSU and PG values reflected a more severe level of psychological distress in freshmen compared to PSMU.

5. Discussion

In consideration of the unique psychological status of college freshmen and the current lack of effective measurement scales for use regarding this demographic, the current investigation evaluated the reliability and validity of the DASS-21 for use among Chinese college freshmen using two sets of sample data (from October 2020 and October 2022). In addition, due to the fact that some debate still exists regarding its factorial validity, the DASS-21 factor structure was also examined in more detail. Furthermore, this study assessed the associations between three PIUs (PSU, PSMU, and PG) and freshmen’s psychological distress. The results indicated that the DASS-21 possesses robust psychometric properties for use among Chinese college freshmen and had reasonable factorial validity using either a three-factor structure or a one-factor structure, which is inconsistent with the results of previous studies [10,12,68,69]. Moreover, it was also found that PIUs had a significant and positive relationship with Chinese college freshmen’s psychological distress: the higher the PIU level, the more severe their psychological distress. PSU and PG, two out of the three PIUs, contributed more to harming the mental health of freshmen than PSMU, the third type of PIU.

5.1. Psychometric Properties of the DASS-21 for Chinese College Freshmen

As for the evaluation of psychometric properties, the reliability of the Chinese version of the DASS-21 was deemed to be acceptable for both samples of college freshmen used. All three scales and the total scores displayed an acceptable degree of internal consistency, which means that the DASS-21 can be reliably used to assess freshmen’s psychological distress. The results obtained in the current study are in agreement with those obtained in previously conducted studies using the Chinese and other versions of the measure for college students [10,11,14]. As for the validity evaluation, the present study adopted a CFA approach to assess factorial validity, and it was demonstrated that the scale fit the one-factor and three-factor structures well in both samples; however, the three-factor structure was a better fit than the one-factor structure. This finding was in line with that of previous studies [10,69]. Moreover, comparing the convergent validity of both samples, the DASS-21 was found to have unsatisfactory discriminant validity. Consequently, we recommend that the DASS-21 be treated as an overall construct of psychological distress, rather than three separate constructs.

5.2. Significant and Positive Association between the DASS-21 Results and PSU, PSMU, and PG

Regarding the association between the three PIUs (PSU, PSMU, PG) and freshmen’s psychological distress, the present study adopted SEM analysis. The findings demonstrated that PSU, PSMU, and PG all had a significant and positive influence on freshmen’s psychological distress, which also echoes the results of previous studies [23,29,46,61]. These findings affirm the association in problematic smartphone, social media, or Internet gaming use, in terms of increased likelihoods of suffering from psychological distress. We also found that PSU and PG caused more severe psychological distress for freshmen than PSMU, which is similar to that of prior findings [69]. Despite the fact that no cause-and-effect relationship was established in this study, we speculate that, to some degree, social media platforms can be used by freshmen to maintain communication with family and friends, and also to alleviate loneliness and boredom, both of which can reduce anxiety and long-term distress, and are therefore can be helpful to students, especially when in isolation, as a means of reducing psychological distress [70]. However, we should also note that once an individual has become accustomed to communicating online, rather than face-to-face, the individual may have difficulty communicating effectively offline, which may exacerbate their psychological problems.

5.3. Discrepancies between Two Samples Due to the Impact of the Pandemic

Furthermore, it should be pointed out that, based on the fact that the DASS-21 results were found to possess characteristics of measurement invariance across the two samples, t-tests could be utilized to evaluate discrepancies between these groups without confounding the interpretation of the items. The present study also found that the state of the mental health of freshman in 2022 was worse that of the 2020 cohort. Especially in terms of depression, freshmen in 2022 were found to be more severely affected. The findings regarding the relatively poorer state of the mental health of the freshmen in 2022 are consistent with that of previous studies conducted on university students [71,72]. Chinese college students were found to experience a higher prevalence of depression (26.0%) during the COVID-19 pandemic, while this prevalence was lower (23.8%) in the pre-COVID-19 era [71]. Peng et al. suggested that college students’ level of anxiety during lockdown conditions increased from June 2020 to June 2021 [72]. In line with these findings, this study revealed that COVID-19 prevention and control policies may serve as potential risk factors for the deterioration of the psychological well-being of college students.
At the end of 2019, the COVID-19 pandemic first broke out in Wuhan, China, but was largely under control by April 2020, across the country [72]. Chinese college students began to return to campus starting in May 2020 [73]. The freshmen in Sample 1 entered college and simultaneously started a new phase of their lives (i.e., in September 2020), after passing their college entrance examination. At that time, the Sample 1 participants did not experience much psychological pressure due to zero (or very few) pandemic cases in their cities. In comparison, Sample 2 participants took part in the current study in the second half of 2022, when the pandemic situation had begun to once again become severe in China. Many colleges and universities quickly launched pandemic prevention and control measures in response to this situation [74], and campuses implemented school closure management and launched fully online teaching (e.g., restricting students from leaving campus, strictly limiting campus activities, and even requiring students to stay in their dorms during emergency situations) [74]. Although these measures effectively protected students from the infection, their harm to the students’ mental health should not be ignored. Indeed, it was more challenging for freshmen, who had just entered university after experiencing an intense high school life and highly competitive college entrance examinations. The situation they faced was not the freedom they expected but, instead, strict lockdown and life in isolation in their dorms. These were huge obstacles for them to adapt to, which could have easily caused them to experience increased psychological distress. Consequently, the mental health statuses of these freshmen, who had recently left their parents, deserves more attention and the monitoring of freshmen students’ psychological distress.
A higher degree of correlation among PSU, PSMU, PG, and psychological distress (i.e., depression, anxiety, and stress) was noted in Sample 2 (October 2022) as compared to Sample 1 (October 2020). This is partly due to the changes in the pandemic situation and in the prevention and control measures. Compared with the second half of 2020, more Chinese colleges and universities had implemented campus closure policies in the second half of 2022 [75]. One prior study demonstrated how people’s daily lives can be adversely affected by prolonged pandemic lockdowns that limit their regular activities, including work and study habits [70]. Students were more susceptible to be afflicted by PIUs when universities were required to cease all campus activities and move all academic activities online as a result of the COVID-19 lockdown measures [76], and the higher severity and prevalence of PIUs, coupled with prolonged lockdowns, has led to poorer mental health.

5.4. Summary of Findings and Limitations

To summarize, the results of the present study have two similarities with previous studies. Firstly, as for the DASS-21 factor structure, both one-factor and three-factor structures were found to be acceptable, but the latter was found to be a better fit. This finding echoes that of previous studies [10,68,77]. Secondly, PSU, PSMU and PG were all confirmed to be positively correlated with the depression, anxiety, and stress levels of Chinese college freshmen, which is in line with the findings of previous studies [23,29,46,61,78]. The association was stronger for the data of Sample 2, which shows that college students’ mental health worsened due to the pandemic. This finding is inconsistent with that of a previous study [79]. In addition, the present study also found that the correlation among the three subscales of the DASS-21 was very high, which indicates a low discrimination rate. This finding is different from that of a previous study conducted with Chinese college students, which pointed out that the DASS-21 had good discriminant validity [10].
There are some limitations of this study which must be addressed. First, the method of convenience sampling was adopted in order recruit participants; hence, generalizing the results reported in this study to all Chinese college freshmen would be inappropriate. In addition, the first sample we recruited was smaller than the second. Larger and more representative samples are suggested for future research in order to confirm the preliminary results obtained by the current study. Second, even though we used measurement invariance as the basis for the comparison of the means of the two datasets, we found that there was a significant difference between the two. However, because we did not track the same group of freshmen with a longitudinal design, these conclusions must be taken with caution. More longitudinal studies are necessary in the future.

6. Conclusions

According to the findings of the current study, the Chinese DASS-21 is demonstrated to be valid and reliable for assessing the psychological distress of Chinese college freshmen. In addition, the study found that PIUs and psychological distress were significantly correlated, which further confirmed the current concerns centering around the problematic Internet use of Chinese college and university students, especially with regard to PSU and PG.
The findings of this study reiterate the importance of paying close attention to the online behaviors of college freshmen. Moreover, given that freshmen faced many unfamiliar situations while experiencing unexpected lockdowns after they entered their universities, university faculty and healthcare providers should provide proper psychological counseling, and the relevant departments should give more consideration to their mental health when formulating and adjusting pandemic prevention policies.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/healthcare11050700/s1, Table S1: Factor loadings in Exploratory Factor Analysis of the DASS-21 among sample 1 (1 October 2020); Table S2: Factor loadings in Exploratory Factor Analysis among sample 2 (1 October 2022). Figure S1: Scree plot of sample 1 (1 October 2020); Figure S2: Scree plot of sample 2 (1 October 2022).

Author Contributions

Conceptualization, I.-H.C. and C.-H.C.; methodology, I.-H.C.; software, X.Z.; validation, I.-H.C., C.-Y.D. and J.H.G.; formal analysis, C.-H.C.; investigation, I.-H.C.; resources, X.Z. and W.-G.C.; data curation, W.-G.C.; writing—original draft preparation, I.-H.C. and C.-H.C.; writing—review and editing, C.-Y.D. and J.H.G.; visualization, J.H.G.; supervision, I.-H.C.; project administration, I.-H.C. and J.H.G.; funding acquisition, I.-H.C. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by the 2022 Shandong Province Social Science Foundation Project “Research on teaching management innovation in rural primary schools in the post-pandemic era,” chaired by I-Hua Chen (Project No.: 22CJYJ16).

Institutional Review Board Statement

The study was conducted in accordance with the Declaration of Helsinki and approved by the Institutional Review Board of Jiangxi Psychological Consultant Association (IRB ref: JXSXL-2021-J99).

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

The data generated for the present study are available from the corresponding authors on reasonable request.

Acknowledgments

We thank all the participants to take part in the study.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Liu, X.; Ping, S.; Gao, W. Changes in Undergraduate Students’ Psychological Well-Being as They Experience University Life. Int. J. Environ. Res. Public Health 2019, 16, 2864. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  2. Beiter, R.; Nash, R.; McCrady, M.; Rhoades, D.; Linscomb, M.; Clarahan, M.; Sammut, S. The prevalence and correlates of depression, anxiety, and stress in a sample of college students. J. Affect. Disord. 2015, 173, 90–96. [Google Scholar] [CrossRef] [PubMed]
  3. Thurber, C.A.; Walton, E.A. Homesickness and adjustment in university students. J. Am. Coll. Health 2012, 60, 415–419. [Google Scholar] [CrossRef] [PubMed]
  4. Holland, D. College student stress and mental health: Examination of stigmatic views on mental health counseling. Mich. Sociol. Rev. 2016, 30, 16–43. [Google Scholar]
  5. Bewick, B.; Koutsopoulou, G.; Miles, J.; Slaa, E.; Barkham, M. Changes in undergraduate students’ psychological well-being as they progress through university. Stud. High. Educ. 2010, 35, 633–645. [Google Scholar] [CrossRef]
  6. Wu, Y.; Sang, Z.Q.; Zhang, X.C.; Margraf, J. The Relationship Between Resilience and Mental Health in Chinese College Students: A Longitudinal Cross-Lagged Analysis. Front. Psychol. 2020, 11, 108. [Google Scholar] [CrossRef]
  7. Moutinho, I.L.; Maddalena, N.C.; Roland, R.K.; Lucchetti, A.L.; Tibirica, S.H.; Ezequiel, O.D.; Lucchetti, G. Depression, stress and anxiety in medical students: A cross-sectional comparison between students from different semesters. Rev. Assoc. Med. Bras. 2017, 63, 21–28. [Google Scholar] [CrossRef]
  8. Dyrbye, L.N.; Thomas, M.R.; Shanafelt, T.D. Systematic review of depression, anxiety, and other indicators of psychological distress among US and Canadian medical students. Acad. Med. 2006, 81, 354–373. [Google Scholar] [CrossRef]
  9. Lovibond, S.H.; Lovibond, P.F. Manual for the Depression Anxiety Stress Scales, 2nd ed.; Psychology Foundation of Australia: Sydney, Australia, 1996. [Google Scholar]
  10. Gong, X.; Xie, X.-Y.; Xu, R.; Luo, Y.-J. Psychometric properties of the Chinese versions of DASS-21 in Chinese college students. Chin. J. Clin. Psychol. 2010, 18, 443–446. [Google Scholar] [CrossRef]
  11. Lee, B.; Kim, Y.E. Validity of the depression, anxiety, and stress scale (DASS-21) in a sample of Korean university students. Curr. Psychol. 2020, 41, 3937–3946. [Google Scholar] [CrossRef]
  12. Camacho, A.; Cordero, E.D.; Perkins, T. Psychometric Properties of the DASS-21 Among Latina/o College Students by the US-Mexico Border. J. Immigr. Minor Health 2016, 18, 1017–1023. [Google Scholar] [CrossRef] [PubMed]
  13. Ahmad, N.; Roslan, S.; Shamsuddin Othman, S.F.A. The Validity and Reliability of Psychometric Profile for Depression, Anxiety and Stress Scale (DASS21) Instrument among Malaysian Undergraduate Students. Int. J. Acad. Res. Bus. Soc. Sci. 2018, 8, 812–827. [Google Scholar] [CrossRef] [PubMed]
  14. Osman, A.; Wong, J.L.; Bagge, C.L.; Freedenthal, S.; Gutierrez, P.M.; Lozano, G. The Depression Anxiety Stress Scales-21 (DASS-21): Further examination of dimensions, scale reliability, and correlates. J. Clin. Psychol. 2012, 68, 1322–1338. [Google Scholar] [CrossRef] [PubMed]
  15. Antúnez, Z.; Vinet, E.V. Depression Anxiety Stress Scales (DASS-21): Validation of the abbreviated version in Chilean university students. Ter. Psicol. 2012, 30, 49–55. [Google Scholar] [CrossRef] [Green Version]
  16. Wang, K.; Shi, H.-S.; Geng, F.-L.; Zou, L.-Q.; Tan, S.-P.; Wang, Y.; Neumann, D.L.; Shum, D.H.; Chan, R.C. Cross-cultural validation of the depression anxiety stress scale–21 in China. Psychol. Assess. 2016, 28, e88. [Google Scholar] [CrossRef] [PubMed]
  17. Sauer-Zavala, S.; Tirpak, J.W.; Eustis, E.H.; Woods, B.K.; Russell, K. Unified Protocol for the Transdiagnostic Prevention of Emotional Disorders: Evaluation of a Brief, Online Course for College Freshmen. Behav. Ther. 2021, 52, 64–76. [Google Scholar] [CrossRef]
  18. Zanon, C.; Brenner, R.E.; Baptista, M.N.; Vogel, D.L.; Rubin, M.; Al-Darmaki, F.R.; Gonçalves, M.; Heath, P.J.; Liao, H.Y.; Mackenzie, C.S.; et al. Examining the dimensionality, reliability, and invariance of the Depression, Anxiety, and Stress Scale–21 (DASS-21) across eight countries. Assessment 2021, 28, 1531–1544. [Google Scholar] [CrossRef] [PubMed]
  19. Lee, D. The convergent, discriminant, and nomological validity of the Depression Anxiety Stress Scales-21 (DASS-21). J. Affect. Disord. 2019, 259, 136–142. [Google Scholar] [CrossRef]
  20. Younes, F.; Halawi, G.; Jabbour, H.; El Osta, N.; Karam, L.; Hajj, A.; Rabbaa Khabbaz, L. Internet Addiction and Relationships with Insomnia, Anxiety, Depression, Stress and Self-Esteem in University Students: A Cross-Sectional Designed Study. PLoS ONE 2016, 11, e0161126. [Google Scholar] [CrossRef] [Green Version]
  21. CGTN. Available online: https://news.cgtn.com/news/2022-08-31/China-hits-1-05-billion-internet-users-74-4-penetration-1cWikCwAMWk/index.html (accessed on 31 August 2022).
  22. Chen, I.H.; Ahorsu, D.K.; Pakpour, A.H.; Griffiths, M.D.; Lin, C.Y.; Chen, C.Y. Psychometric Properties of Three Simplified Chinese Online-Related Addictive Behavior Instruments Among Mainland Chinese Primary School Students. Front. Psychiatry 2020, 11, 875. [Google Scholar] [CrossRef]
  23. Wong, H.Y.; Mo, H.Y.; Potenza, M.N.; Chan, M.N.M.; Lau, W.M.; Chui, T.K.; Pakpour, A.H.; Lin, C.Y. Relationships between Severity of Internet Gaming Disorder, Severity of Problematic Social Media Use, Sleep Quality and Psychological Distress. Int. J. Environ. Res. Public Health 2020, 17, 1879. [Google Scholar] [CrossRef] [Green Version]
  24. Zhang, W.; Pu, J.; He, R.; Yu, M.; Xu, L.; He, X.; Chen, Z.; Gan, Z.; Liu, K.; Tan, Y.; et al. Demographic characteristics, family environment and psychosocial factors affecting internet addiction in Chinese adolescents. J. Affect. Disord. 2022, 315, 130–138. [Google Scholar] [CrossRef] [PubMed]
  25. Haug, S.; Castro, R.P.; Kwon, M.; Filler, A.; Kowatsch, T.; Schaub, M.P. Smartphone use and smartphone addiction among young people in Switzerland. J. Behav. Addict. 2015, 4, 299–307. [Google Scholar] [CrossRef] [Green Version]
  26. Weinstein, A.; Dorani, D.; Elhadif, R.; Bukovza, Y.; Yarmulnik, A.; Dannon, P. Internet addiction is associated with social anxiety in young adults. Ann. Clin. Psychiatry 2015, 27, 4–9. [Google Scholar] [CrossRef] [PubMed]
  27. Yen, J.Y.; Ko, C.H.; Yen, C.F.; Chen, S.H.; Chung, W.L.; Chen, C.C. Psychiatric symptoms in adolescents with Internet addiction: Comparison with substance use. Psychiatry Clin. Neurosci. 2008, 62, 9–16. [Google Scholar] [CrossRef] [PubMed]
  28. Chang, C.W.; Huang, R.Y.; Strong, C.; Lin, Y.C.; Tsai, M.C.; Chen, I.H.; Lin, C.Y.; Pakpour, A.H.; Griffiths, M.D. Reciprocal Relationships Between Problematic Social Media Use, Problematic Gaming, and Psychological Distress Among University Students: A 9-Month Longitudinal Study. Front. Public Health 2022, 10, 858482. [Google Scholar] [CrossRef] [PubMed]
  29. Tung, S.E.H.; Gan, W.Y.; Chen, J.S.; Kamolthip, R.; Pramukti, I.; Nadhiroh, S.R.; Chang, Y.L.; Lin, C.C.; Pakpour, A.H.; Lin, C.Y.; et al. Internet-Related Instruments (Bergen Social Media Addiction Scale, Smartphone Application-Based Addiction Scale, Internet Gaming Disorder Scale-Short Form, and Nomophobia Questionnaire) and Their Associations with Distress among Malaysian University Students. Healthcare 2022, 10, 1448. [Google Scholar] [CrossRef]
  30. Yang, J.; Fu, X.; Liao, X.; Li, Y. Association of problematic smartphone use with poor sleep quality, depression, and anxiety: A systematic review and meta-analysis. Psychiatry Res. 2020, 284, 112686. [Google Scholar] [CrossRef]
  31. Schou Andreassen, C.; Billieux, J.; Griffiths, M.D.; Kuss, D.J.; Demetrovics, Z.; Mazzoni, E.; Pallesen, S. The relationship between addictive use of social media and video games and symptoms of psychiatric disorders: A large-scale cross-sectional study. Psychol. Addict. Behav. 2016, 30, 252–262. [Google Scholar] [CrossRef] [Green Version]
  32. Huang, C. A meta-analysis of the problematic social media use and mental health. Int. J. Soc. Psychiatry 2022, 68, 12–33. [Google Scholar] [CrossRef]
  33. Yen, J.Y.; Lin, H.C.; Chou, W.P.; Liu, T.L.; Ko, C.H. Associations Among Resilience, Stress, Depression, and Internet Gaming Disorder in Young Adults. Int. J. Environ. Res. Public Health 2019, 16, 3181. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  34. Teng, Z.; Pontes, H.M.; Nie, Q.; Griffiths, M.D.; Guo, C. Depression and anxiety symptoms associated with internet gaming disorder before and during the COVID-19 pandemic: A longitudinal study. J. Behav. Addict. 2021, 10, 169–180. [Google Scholar] [CrossRef] [PubMed]
  35. Yam, C.W.; Pakpour, A.H.; Griffiths, M.D.; Yau, W.Y.; Lo, C.M.; Ng, J.M.T.; Lin, C.Y.; Leung, H. Psychometric Testing of Three Chinese Online-Related Addictive Behavior Instruments among Hong Kong University Students. Psychiatr. Q. 2019, 90, 117–128. [Google Scholar] [CrossRef] [Green Version]
  36. Yu, S.; Sussman, S. Does Smartphone Addiction Fall on a Continuum of Addictive Behaviors? Int. J. Environ. Res. Public Health 2020, 17, 422. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  37. Aljomaa, S.S.; Qudah, M.F.A.; Albursan, I.S.; Bakhiet, S.F.; Abduljabbar, A.S. Smartphone addiction among university students in the light of some variables. Comput. Hum. Behav. 2016, 61, 155–164. [Google Scholar] [CrossRef]
  38. Barnes, S.J.; Pressey, A.D.; Scornavacca, E. Mobile ubiquity: Understanding the relationship between cognitive absorption, smartphone addiction and social network services. Comput. Hum. Behav. 2019, 90, 246–258. [Google Scholar] [CrossRef] [Green Version]
  39. Elhai, J.D.; Dvorak, R.D.; Levine, J.C.; Hall, B.J. Problematic smartphone use: A conceptual overview and systematic review of relations with anxiety and depression psychopathology. J. Affect. Disord. 2017, 207, 251–259. [Google Scholar] [CrossRef]
  40. Chen, I.-H.; Strong, C.; Lin, Y.-C.; Tsai, M.-C.; Leung, H.; Lin, C.-Y.; Pakpour, A.H.; Griffiths, M.D. Time invariance of three ultra-brief internet-related instruments: Smartphone application-based addiction scale (SABAS), Bergen social media addiction scale (BSMAS), and the nine-item internet gaming disorder scale-short form (IGDS-SF9) (study Part B). Addict. Behav. 2020, 101, 105960. [Google Scholar] [CrossRef] [Green Version]
  41. Carr, C.T.; Hayes, R.A. Social media: Defining, developing, and divining. Atl. J. Commun. 2015, 23, 46–65. [Google Scholar] [CrossRef]
  42. Arrivillaga, C.; Rey, L.; Extremera, N. A mediated path from emotional intelligence to problematic social media use in adolescents: The serial mediation of perceived stress and depressive symptoms. Addict. Behav. 2022, 124, 107095. [Google Scholar] [CrossRef]
  43. Marino, C.; Musetti, A.; Vieno, A.; Manari, T.; Franceschini, C. Is psychological distress the key factor in the association between problematic social networking sites and poor sleep quality? Addict. Behav. 2022, 133, 107380. [Google Scholar] [CrossRef] [PubMed]
  44. Xiang, G.-X.; Gan, X.; Jin, X.; Zhang, Y.-H. The more developmental assets, the less internet gaming disorder? Testing the cumulative effect and longitudinal mechanism during the COVID-19 pandemic. Curr. Psychol. 2022, 10, 1–12. [Google Scholar] [CrossRef] [PubMed]
  45. Pontes, H.M.; Griffiths, M.D. A new era for gaming disorder research: Time to shift from consensus to consistency. Addict. Behav. 2019, 103, 106059. [Google Scholar] [CrossRef] [Green Version]
  46. Pontes, H.M. Investigating the differential effects of social networking site addiction and Internet gaming disorder on psychological health. J. Behav. Addict. 2017, 6, 601–610. [Google Scholar] [CrossRef] [PubMed]
  47. Wu, A.M.; Lai, M.H.; Zhang, M.; Yogo, M.; Yu, S.M.; Mao, S.; Chen, J.H. Effects of psychological distress and coping Resources on Internet gaming disorder: Comparison between Chinese and Japanese university students. Int. J. Environ. Res. Public Health 2022, 19, 2951. [Google Scholar] [CrossRef]
  48. China, Ministry of Education of the People’s Republic of China. Available online: http://www.moe.gov.cn/srcsite/A17/moe_943/s3285/202208/t20220826_655510.html (accessed on 25 August 2022).
  49. China, Ministry of Education of the People’s Republic of China. Available online: http://www.moe.gov.cn/jyb_xwfb/s5148/202203/t20220331_612411.html (accessed on 31 March 2022).
  50. Kim, H.H.; Laurence, J. COVID-19 restrictions and mental distress among American adults: Evidence from Corona Impact Survey (W1 and W2). J. Public Health 2020, 42, 704–711. [Google Scholar] [CrossRef]
  51. Guo, S.; Kaminga, A.C.; Xiong, J. Depression and Coping Styles of College Students in China During COVID-19 Pandemic: A Systemic Review and Meta-Analysis. Front. Public Health 2021, 9, 613321. [Google Scholar] [CrossRef]
  52. Norton, P.J. Depression Anxiety and Stress Scales (DASS-21): Psychometric analysis across four racial groups. Anxiety Stress Coping 2007, 20, 253–265. [Google Scholar] [CrossRef]
  53. Moya, E.; Larson, L.M.; Stewart, R.C.; Fisher, J.; Mwangi, M.N.; Phiri, K.S. Reliability and validity of depression anxiety stress scale (DASS)-21 in screening for common mental disorders among postpartum women in Malawi. BMC Psychiatry 2022, 22, 352. [Google Scholar] [CrossRef]
  54. Thapa, D.K.; Visentin, D.; Kornhaber, R.; Cleary, M. Psychometric properties of the Nepali language version of the Depression Anxiety Stress Scales (DASS-21). Nurs. Open 2022, 9, 2608–2617. [Google Scholar] [CrossRef]
  55. Henry, J.D.; Crawford, J.R. The short-form version of the Depression Anxiety Stress Scales (DASS-21): Construct validity and normative data in a large non-clinical sample. Br. J. Clin. Psychol. 2005, 44, 227–239. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  56. Csibi, S.; Griffiths, M.D.; Cook, B.; Demetrovics, Z.; Szabo, A. The Psychometric Properties of the Smartphone Application-Based Addiction Scale (SABAS). Int. J. Ment. Health Addict. 2018, 16, 393–403. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  57. Csibi, S.; Demetrovics, Z.; Szabo, A. Hungarian adaptation and psychometric characteristics of Brief Addiction to Smartphone Scale (BASS). Psychiatr. Hung. 2016, 31, 71–77. [Google Scholar] [PubMed]
  58. Lin, C.-Y.; Imani, V.; Broström, A.; Nilsen, P.; Fung, X.C.; Griffiths, M.D.; Pakpour, A.H. Smartphone application-based addiction among Iranian adolescents: A psychometric study. Int. J. Ment. Health Addict. 2019, 17, 765–780. [Google Scholar] [CrossRef] [Green Version]
  59. Soraci, P.; Ferrari, A.; Antonino, U.; Griffiths, M.D. Psychometric properties of the Italian version of the smartphone application-based addiction scale (SABAS). Int. J. Ment. Health Addict. 2021, 19, 1261–1273. [Google Scholar] [CrossRef] [Green Version]
  60. Chen, I.H.; Chen, H.-P.; Gamble, J.H.; Liao, X.l.; Chen, X.-M.; Yang, Y.-T.C.; Pakpour, A.H.; Griffiths, M.D.; Lin, C.-Y. Evaluating a cross-lagged panel model between problematic internet use and psychological distress and cross-level mediation of school administrator support on problematic internet use: The serial mediating role of psychological needs thwarting of online teaching and psychological distress. Front. Public Health 2022, 10, 987366. [Google Scholar] [CrossRef]
  61. Pontes, H.M.; Griffiths, M.D. Measuring DSM-5 internet gaming disorder: Development and validation of a short psychometric scale. Comput. Hum. Behav. 2015, 45, 137–143. [Google Scholar] [CrossRef] [Green Version]
  62. Poon, L.Y.; Tsang, H.W.; Chan, T.Y.; Man, S.W.; Ng, L.Y.; Wong, Y.L.; Lin, C.-Y.; Chien, C.-W.; Griffiths, M.D.; Pontes, H.M. Psychometric Properties of the Internet Gaming Disorder Scale–Short-Form (IGDS9-SF): Systematic Review. J. Med. Internet Res. 2021, 23, e26821. [Google Scholar] [CrossRef]
  63. Leung, H.; Pakpour, A.H.; Strong, C.; Lin, Y.-C.; Tsai, M.-C.; Griffiths, M.D.; Lin, C.-Y.; Chen, I.-H. Measurement invariance across young adults from Hong Kong and Taiwan among three internet-related addiction scales: Bergen social media addiction scale (BSMAS), smartphone application-based addiction scale (SABAS), and internet gaming disorder scale-short form (IGDS-SF9) (study Part A). Addict. Behav. 2020, 101, 105969. [Google Scholar] [CrossRef] [Green Version]
  64. Hair, J.F.; Black, W.C.; Babin, B.J.; Anderson, R.E. Multivariate Data Analysis (Ed. 8th), 2nd ed.; Cengage Learning EMEA: Hampshire, UK, 2019. [Google Scholar]
  65. Hu, L.T.; Bentler, P.M. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Struct. Equ. Model. A Multidiscip. J. 1999, 6, 1–55. [Google Scholar] [CrossRef]
  66. Korner-Nievergelt, F.; Roth, T.; Von Felten, S.; Guélat, J.; Almasi, B.; Korner-Nievergelt, P. Model selection and multimodel inference. In Bayesian Data Analysis in Ecology Using Linear Models with R, BUGS, and Stan; Korner-Nievergelt, F., Roth, T., Von Felten, S., Guélat, J., Almasi, B., Korner-Nievergelt, P., Eds.; Academic Press: Boston, MA, USA, 2015; pp. 175–196. [Google Scholar]
  67. Chen, F.F. Sensitivity of goodness of fit indexes to lack of measurement invariance. Struct. Equ. Model. A Multidiscip. J. 2007, 14, 464–504. [Google Scholar] [CrossRef]
  68. Moussa, M.T.; Lovibond, P.F.; Laube, R. Psychometric Properties of a Chinese Version of the 21-Item Depression Anxiety Stress Scales (DASS21); Transcultural Mental Health Centre: Parramatta, NSW, Australia; Cumberland Hospital: Sydney, NSW, Australia, 2001. [Google Scholar] [CrossRef]
  69. Chen, I.-H.; Pakpour, A.H.; Leung, H.; Potenza, M.N.; Su, J.-A.; Lin, C.-Y.; Griffiths, M.D. Comparing generalized and specific problematic smartphone/internet use: Longitudinal relationships between smartphone application-based addiction and social media addiction and psychological distress. J. Behav. Addict. 2020, 9, 410–419. [Google Scholar] [CrossRef] [PubMed]
  70. Brooks, S.K.; Webster, R.K.; Smith, L.E.; Woodland, L.; Wessely, S.; Greenberg, N.; Rubin, G.J. The psychological impact of quarantine and how to reduce it: Rapid review of the evidence. LANCET 2020, 395, 912–920. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  71. Luo, W.; Zhong, B.-L.; Chiu, H.F.-K. Prevalence of depressive symptoms among Chinese university students amid the COVID-19 pandemic: A systematic review and meta-analysis. Epidemiol. Psychiatr. Sci. 2021, 30, E31. [Google Scholar] [CrossRef]
  72. Zhan, H.; Zheng, C.; Zhang, X.; Yang, M.; Zhang, L.; Jia, X. Chinese college students’ stress and anxiety levels under COVID-19. Front. Psychiatry 2021, 915, 615390. [Google Scholar] [CrossRef]
  73. Province, People’s Government of Jiangxi Province. Available online: http://www.jiangxi.gov.cn/art/2020/5/14/art_412_1795192.html (accessed on 14 May 2020).
  74. China, Ministry of Education of the People’s Republic of China. Available online: http://www.moe.gov.cn/jyb_xwfb/s5147/202111/t20211102_577013.html (accessed on 2 November 2021).
  75. Peng, X.; Liu, L.; Liang, S.; Chen, J.; Zhao, J. Longitudinal changes in fear and anxiety among Chinese college students during the COVID-19 pandemic: A one-year follow-up study. Curr. Psychol. 2022, 7, 1–10. [Google Scholar] [CrossRef]
  76. Xia, Y.; Fan, Y.; Liu, T.-H.; Ma, Z. Problematic Internet use among residential college students during the COVID-19 lockdown: A social network analysis approach. J. Behav. Addict. 2021, 10, 253–262. [Google Scholar] [CrossRef]
  77. Gomez, R.; Stavropoulos, V.; Griffiths, M.D. Confirmatory factor analysis and exploratory structural equation modelling of the factor structure of the Depression Anxiety and Stress Scales-21. PLoS ONE 2020, 15, e0233998. [Google Scholar] [CrossRef]
  78. Wang, Q.; Mati, K.; Cai, Y. The link between problematic internet use, problematic gaming, and psychological distress: Does sleep quality matter? BMC Psychiatry 2021, 21, 103. [Google Scholar] [CrossRef]
  79. Gecaite-Stonciene, J.; Saudargiene, A.; Pranckeviciene, A.; Liaugaudaite, V.; Griskova-Bulanova, I.; Simkute, D.; Naginiene, R.; Dainauskas, L.; Ceidaite, G.; Burkauskas, J.; et al. Impulsivity mediates associations between problematic internet use, anxiety, and depressive symptoms in students: A cross-sectional COVID-19 study. Front. Psychiatry 2021, 12, 634464. [Google Scholar] [CrossRef]
Healthcare 11 00700 g001 550
Figure 1. Structural equation modeling of the associations between problematic smartphone use, problematic social media use, and problematic gaming, using the DASS-21. The former and latter values were calculated using the data sets from October 2020, and October 2022. * means p < 0.05, ** means p < 0.01.
Figure 1. Structural equation modeling of the associations between problematic smartphone use, problematic social media use, and problematic gaming, using the DASS-21. The former and latter values were calculated using the data sets from October 2020, and October 2022. * means p < 0.05, ** means p < 0.01.
Healthcare 11 00700 g001
Table
Table 1. Demographic characteristics of study participants.
Table 1. Demographic characteristics of study participants.
Sample 1
(October 2020)
n = 364		Sample 2
(October 2022)
n = 956
Age in years; mean (SD)18.17 (0.42)18.38 (1.11)
Gender; n (%)
  Male116 (31.9)457 (47.8)
  Female248 (68.1)499 (52.2)
Department; n (%)
  Science0151 (15.8)
  Literature120 (33.0)152 (15.9)
  Management51 (14.0)156 (16.3)
  Engineering143 (39.3)274 (28.7)
  Others50 (13.7)223 (23.3)
Study program level; n (%)
  Undergraduate364 (100)642 (68.1)
  Junior college 301 (31.9)
Table
Table 2. Descriptive and correlation values between the variables.
Table 2. Descriptive and correlation values between the variables.
Variable (Range)MSD123456
1. Depression (0–42)5.33/6.575.62/7.381
2. Anxiety (0–42)7.10/7.064.60/6.530.62/0.791
3. Stress (0–42)8.56/8.555.71/7.230.59/0.800.74/0.831
4. PSU (6–36)17.98/18.925.01/6.430.37/0.470.38/0.460.33/0.481
5. PSMU (6–30)12.85/14.073.71/4.460.28/0.420.30/0.420.31/0.440.47/0.611
6. PG (9–45)13.93/17.115.46/6.920.29/0.480.26/0.430.22/0.430.31/0.500.29/0.521
Notes: The former and latter values were calculated using data sets from October 2020 and October 2022, respectively. The severity of depression, anxiety, and stress was then assessed using the Depression, Anxiety, and Stress Scale—21-item version where the values were computed by multiplying the score by two; PSU = problematic smartphone use; PSMU = problematic social media use; PG = problematic gaming. All p < 0.01.
Table
Table 3. Model fit of the different factor structures.
Table 3. Model fit of the different factor structures.
χ2 (df)CFINNFIRMSEA
(90% Confidence Interval)		SRMRAIC
One-factor model
Sample 1 (October 2020)568.67 (189)0.9680.9650.074 (0.067–0.082)0.091652.67
Sample 2 (October 2022)959.83 (189)0.9890.9880.065 (0.061–0.070)0.0511043.83
Three-factor model
Sample 1 (October 2020)481.79 (186)0.9750.9720.066 (0.059–0.074)0.086571.79
Sample 2 (October 2022)736.94 (186)0.9920.9910.056 (0.052–0.060)0.046826.94
Notes: CFI = comparative fit index; NNFI = non-normed fit index; RMSEA = root mean square error of approximation; SRMR = standardized root mean square residual; AIC = Akaike information criterion.
Table
Table 4. Results of convergent and discriminant validity analysis of the DASS-21.
Table 4. Results of convergent and discriminant validity analysis of the DASS-21.
DepressionAnxietyStress
Depression0.73/0.84
Anxiety0.92/0.920.58/0.75
Stress0.82/0.930.99/0.980.64/0.76
Notes: Diagonal elements in bold are square root of averaged variance extracted. When these values were higher than the inter-latent factor correlations (off-diagonal elements), discriminant validity was supported for the respective latent variable.
Table
Table 5. Fit indices for measurement invariance across freshmen from two data sets.
Table 5. Fit indices for measurement invariance across freshmen from two data sets.
Configural ModelLoadings Constrained as EqualLoadings and Thresholds
Constrained as Equal
One-factor model
χ2(df) or Δχ2(Δdf)1452.03 (378)75.36 (20)518.78 (20)
CFI or ΔCFI0.987−0.001−0.006
RMSEA or ΔRMSEA0.06600.011
SRMR or ΔSRMR0.0910.0110.009
Three-factor model
χ2(df) or Δχ2(Δdf)1141.56 (372)61.86 (18)187.27 (18)
CFI or ΔCFI0.9890.001−0.002
RMSEA or ΔRMSEA0.05600.004
SRMR or ΔSRMR0.0860.0030.003
Notes: CFI = comparative fit index; RMSEA = root mean square error of approximation; SRMR = standardized root mean square residual. The bold values indicate invariance, i.e., ΔCFI > −0.01; ΔRMSEA < 0.015; ΔSRMR < 0.03 (for factor loading) or <0.01 (for item intercept).
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Cao, C.-H.; Dang, C.-Y.; Zheng, X.; Chen, W.-G.; Chen, I.-H.; Gamble, J.H. The Psychometric Properties of the DASS-21 and Its Association with Problematic Internet Use among Chinese College Freshmen. Healthcare 2023, 11, 700. https://doi.org/10.3390/healthcare11050700

AMA Style

Cao C-H, Dang C-Y, Zheng X, Chen W-G, Chen I-H, Gamble JH. The Psychometric Properties of the DASS-21 and Its Association with Problematic Internet Use among Chinese College Freshmen. Healthcare. 2023; 11(5):700. https://doi.org/10.3390/healthcare11050700

Chicago/Turabian Style

Cao, Cui-Hong, Chang-Yan Dang, Xia Zheng, Wang-Guang Chen, I-Hua Chen, and Jeffrey H. Gamble. 2023. "The Psychometric Properties of the DASS-21 and Its Association with Problematic Internet Use among Chinese College Freshmen" Healthcare 11, no. 5: 700. https://doi.org/10.3390/healthcare11050700

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop